File Exchange

image thumbnail

Spectral Visual Saliency Toolbox

version 1.20 (213 KB) by

Implementations of several state-of-the-art visual saliency detection algorithms.

60 Downloads

Updated

View License

The Spectral Visual Saliency Toolbox (SViST) provides implementations
of several spectral (FFT,DCT) saliency algorithms for images.

There are several methods to calculate the visual saliency:

'fft': by default the same as 'fft:whitening'
'fft:whitening' Uses spectral whitening to calculate the saliency of
each channel separately and then averages the result.
'fft:residual' Uses the spectral residual to calculate saliency of
each channel separately and then averages the result.
'dct' Uses DCT-based image signatures to calculate saliency
of each channel separately and then averages the
result.
'quat:fft': Converts the image into a quaternion-based
representation, uses quaternion FFT/IFFT operations.
'quat:fft:eigenpqft': Similar to 'quat:fft', but relies on the use of
the quaternion eigenaxis and eigenangle for
calculation.
'quat:fft:eigensr': The principle of 'fft:residual' transferred to
'quat:fft:eigenpqft'.
'quat:dct' Converts the image into a quaternion-based
representation, uses quaternion DCT/IDCT operations.
'quat:dct:fast' Same as 'quad:dct', but with a fixed image
resolution of 64x48 and uses optimized .mex files for
faster calculation.
[...] some others, e.g., Itti-Koch, Achanta, GBVS, region
contrast, and locally debiased region contrast

Usage examples:
- spectral_saliency_multichannel(imread(..image path...))
or as an example for other color spaces (e.g. ICOPP, Lab, ...)
- spectral_saliency_multichannel(rgb2icopp(imread(..image path...)))
Furthermore, 'example' and 'process_folder' are two usage examples.
The former makes it possible to visualizes the saliency maps of
different algorithms for a given test image. The latter calculates
and saves the saliency maps for all images in a given folder.

If you use any of this work in scientific research or as part of a
larger software system, you are requested to cite the use in any
related publications or technical documentation. The work is based
upon:

B. Schauerte, and R. Stiefelhagen, "Quaternion-based Spectral
Saliency Detection for Eye Fixation Prediction," in European
Conference on Computer Vision (ECCV), 2012

B. Schauerte, and R. Stiefelhagen, "Predicting Human Gaze using
Quaternion DCT Image Signature Saliency and Face Detection," in
IEEE Workshop on the Applications of Computer Vision (WACV),
2012.

Notes:
- The implementation of the quaternion-based approach requires the
quaternion toolbox for Matlab (QTFM).
- I kept the implementations as focused and simple as possible and
thus they lack more advanced functionality, e.g. more complex
normalizations. However, I think that the provided functionality is
more than sufficient for (a) people who want to get started in the
field of visual attention (especially students), (b) practitioners
who have heard about the spectral approach and want to try it, and
(c) people who just need a fast, reliable, well-established visual
saliency algorithm (with a simple interface and not too many
parameters) for their applications.
- GBVS and Itti require the original GBVS Matlab implementation by
J. Harel (see http://www.klab.caltech.edu/~harel/share/gbvs.php)

For more details on the method see:
[1] X. Hou and L. Zhang, "Saliency Detection: A Spectral Residual
Approach", in CVPR, 2007.
(original paper)
[2] C. Guo, Q. Ma, and L. Zhang, "Spatio-temporal saliency detection
using phase spectrum of quaternion fourier transform," in CVPR,
2008.
(extension to quaternions; importance of the residual)
[3] X. Hou, J. Harel, and C. Koch, "Image Signature: Highlighting
sparse salient regions," in PAMI, 2011.
(uses DCT-based "image signatures")
[4] B. Schauerte, and R. Stiefelhagen, "Predicting Human Gaze using
Quaternion DCT Image Signature Saliency and Face Detection," in
IEEE Workshop on the Applications of Computer Vision (WACV), 2012.
(extension to quaternions; spectral saliency and face detection;
evaluation of spectral saliency approaches on eye-tracking data;
achieved the currently best reported results on the CERF/FIFA
eye-tracking data set and Toronto/Bruce-Tsotsos data set)
[5] B. Schauerte, and R. Stiefelhagen, "Quaternion-based Spectral
Saliency Detection for Eye Fixation Prediction," in European
Conference on Computer Vision (ECCV), 2012
(integration and evaluation of DCT- and FFT-based spectral saliency
detection, quaternion component weights, and the use of multiple
scales; also introduces EigenPQFT and EigenSR; evaluated on the
Bruce-Tsotsos (Toronto), Judd (MIT), and Kootstra-Schomacker
eye-tracking data sets)

Comments and Ratings (3)

@Tallha: I would like to help you, but I need more information. I can successfully compile the code under Linux, Windows, and Mac OS X (Lion) using GCC, MSVC, and LLVM. First, you might want to try the pre-compiled binaries that you can download using "get_additional_files.m". Second, if that does not help, then please send me an e-Mail (you can find the address on my website) with a/the error message and some other information (compiler, platform, ...). Kind regards, Boris

Tallha Akram

unable to compile following files

mex -c dct_type2_48.cpp
mex -c dct_type2_64.cpp
mex -c dct_type3_48.cpp
mex -c dct_type3_64.cpp

Updates

1.20

- integrated (locally debiased) region contrast saliency, see https://github.com/bschauerte/region_contrast_saliency

1.19

- fixed automatic download URLs (broken after change on the server)
- gave the toolbox a real name

1.18

- added a simple GPU benchmark routine
- minor change in imconvert.m

1.17

- added reshape_spectral_image.m
- minor updates and fixes

1.16

- added multi-scale routines
- added imconvert.m
- added get_additional_files.m

1.15

- added EigenPQFT and EigenSR algorithms
- added two saliency algorithms by Achanta et al.
- added process_folder.m
- rewrote example.m
- QDCT .mex-binaries can now be found on my website (see README.txt)

1.14

- C/C++ implementation now compiles without problems under Windows (MS Visual Studio Compiler / Windows SDK 7.1)

1.12

- fixed an if-statement

1.11

- updated the documentation

1.10

- added an efficient C/C++ implementation of the QDCT image signatures
- added an example.m file for a simple example (including an example image)

1.8

- changed the default parameters

1.7

- fix

1.6

- fix

1.5

fix

1.3

added the quaternion support again; added quaternion DCT signatures and saliency based on DCT image signatures; added another ICOPP conversion; added a visualization for quaternion images; added a README.txt

1.2

- temporarily removed the Quaternion support (will be added again, soon)

1.1

- added the quaternion-based multichannel saliency calculation

MATLAB Release
MATLAB 7.10 (R2010a)

Download apps, toolboxes, and other File Exchange content using Add-On Explorer in MATLAB.

» Watch video

Win prizes and improve your MATLAB skills

Play today