Code covered by the BSD License  

Highlights from
Spectral Visual Saliency Toolbox

5.0

5.0 | 1 rating Rate this file 118 Downloads (last 30 days) File Size: 213 KB File ID: #32455
image thumbnail

Spectral Visual Saliency Toolbox

by

 

07 Aug 2011 (Updated )

Implementations of several state-of-the-art visual saliency detection algorithms.

| Watch this File

File Information
Description

The Spectral Visual Saliency Toolbox (SViST) provides implementations
of several spectral (FFT,DCT) saliency algorithms for images.

There are several methods to calculate the visual saliency:

'fft': by default the same as 'fft:whitening'
'fft:whitening' Uses spectral whitening to calculate the saliency of
                each channel separately and then averages the result.
'fft:residual' Uses the spectral residual to calculate saliency of
                each channel separately and then averages the result.
'dct' Uses DCT-based image signatures to calculate saliency
                of each channel separately and then averages the
                result.
'quat:fft': Converts the image into a quaternion-based
                representation, uses quaternion FFT/IFFT operations.
'quat:fft:eigenpqft': Similar to 'quat:fft', but relies on the use of
                the quaternion eigenaxis and eigenangle for
                calculation.
'quat:fft:eigensr': The principle of 'fft:residual' transferred to
                'quat:fft:eigenpqft'.
'quat:dct' Converts the image into a quaternion-based
                representation, uses quaternion DCT/IDCT operations.
'quat:dct:fast' Same as 'quad:dct', but with a fixed image
                resolution of 64x48 and uses optimized .mex files for
                faster calculation.
[...] some others, e.g., Itti-Koch, Achanta, GBVS, region
                contrast, and locally debiased region contrast

Usage examples:
- spectral_saliency_multichannel(imread(..image path...))
  or as an example for other color spaces (e.g. ICOPP, Lab, ...)
- spectral_saliency_multichannel(rgb2icopp(imread(..image path...)))
Furthermore, 'example' and 'process_folder' are two usage examples.
The former makes it possible to visualizes the saliency maps of
different algorithms for a given test image. The latter calculates
and saves the saliency maps for all images in a given folder.

If you use any of this work in scientific research or as part of a
larger software system, you are requested to cite the use in any
related publications or technical documentation. The work is based
upon:

    B. Schauerte, and R. Stiefelhagen, "Quaternion-based Spectral
    Saliency Detection for Eye Fixation Prediction," in European
    Conference on Computer Vision (ECCV), 2012

    B. Schauerte, and R. Stiefelhagen, "Predicting Human Gaze using
    Quaternion DCT Image Signature Saliency and Face Detection," in
    IEEE Workshop on the Applications of Computer Vision (WACV),
    2012.

Notes:
- The implementation of the quaternion-based approach requires the
  quaternion toolbox for Matlab (QTFM).
- I kept the implementations as focused and simple as possible and
  thus they lack more advanced functionality, e.g. more complex
  normalizations. However, I think that the provided functionality is
  more than sufficient for (a) people who want to get started in the
  field of visual attention (especially students), (b) practitioners
  who have heard about the spectral approach and want to try it, and
  (c) people who just need a fast, reliable, well-established visual
  saliency algorithm (with a simple interface and not too many
  parameters) for their applications.
- GBVS and Itti require the original GBVS Matlab implementation by
  J. Harel (see http://www.klab.caltech.edu/~harel/share/gbvs.php)

For more details on the method see:
[1] X. Hou and L. Zhang, "Saliency Detection: A Spectral Residual
    Approach", in CVPR, 2007.
    (original paper)
[2] C. Guo, Q. Ma, and L. Zhang, "Spatio-temporal saliency detection
    using phase spectrum of quaternion fourier transform," in CVPR,
    2008.
    (extension to quaternions; importance of the residual)
[3] X. Hou, J. Harel, and C. Koch, "Image Signature: Highlighting
    sparse salient regions," in PAMI, 2011.
    (uses DCT-based "image signatures")
[4] B. Schauerte, and R. Stiefelhagen, "Predicting Human Gaze using
    Quaternion DCT Image Signature Saliency and Face Detection," in
    IEEE Workshop on the Applications of Computer Vision (WACV), 2012.
    (extension to quaternions; spectral saliency and face detection;
     evaluation of spectral saliency approaches on eye-tracking data;
     achieved the currently best reported results on the CERF/FIFA
     eye-tracking data set and Toronto/Bruce-Tsotsos data set)
[5] B. Schauerte, and R. Stiefelhagen, "Quaternion-based Spectral
    Saliency Detection for Eye Fixation Prediction," in European
    Conference on Computer Vision (ECCV), 2012
    (integration and evaluation of DCT- and FFT-based spectral saliency
     detection, quaternion component weights, and the use of multiple
     scales; also introduces EigenPQFT and EigenSR; evaluated on the
     Bruce-Tsotsos (Toronto), Judd (MIT), and Kootstra-Schomacker
     eye-tracking data sets)

MATLAB release MATLAB 7.10 (R2010a)
Other requirements - Quaternion Toolbox for Matlab (optional) - GCC (Linux) or MSVC (Windows) (optional)
Tags for This File   Please login to tag files.
Please login to add a comment or rating.
Comments and Ratings (3)
04 Feb 2013 Boris Schauerte

@Tallha: I would like to help you, but I need more information. I can successfully compile the code under Linux, Windows, and Mac OS X (Lion) using GCC, MSVC, and LLVM. First, you might want to try the pre-compiled binaries that you can download using "get_additional_files.m". Second, if that does not help, then please send me an e-Mail (you can find the address on my website) with a/the error message and some other information (compiler, platform, ...). Kind regards, Boris

04 Feb 2013 Tallha Akram

unable to compile following files

mex -c dct_type2_48.cpp
mex -c dct_type2_64.cpp
mex -c dct_type3_48.cpp
mex -c dct_type3_64.cpp

19 Jan 2012 Brahim HAMADICHAREF  
Updates
10 Aug 2011

- added the quaternion-based multichannel saliency calculation

17 Aug 2011

- temporarily removed the Quaternion support (will be added again, soon)

02 Nov 2011

added the quaternion support again; added quaternion DCT signatures and saliency based on DCT image signatures; added another ICOPP conversion; added a visualization for quaternion images; added a README.txt

02 Nov 2011

fix

02 Nov 2011

- fix

02 Nov 2011

- fix

03 Nov 2011

- changed the default parameters

03 Nov 2011

- added an efficient C/C++ implementation of the QDCT image signatures
- added an example.m file for a simple example (including an example image)

09 Dec 2011

- updated the documentation

13 Dec 2011

- fixed an if-statement

15 Dec 2011

- C/C++ implementation now compiles without problems under Windows (MS Visual Studio Compiler / Windows SDK 7.1)

02 Jul 2012

- added EigenPQFT and EigenSR algorithms
- added two saliency algorithms by Achanta et al.
- added process_folder.m
- rewrote example.m
- QDCT .mex-binaries can now be found on my website (see README.txt)

04 Sep 2012

- added multi-scale routines
- added imconvert.m
- added get_additional_files.m

02 Oct 2012

- added reshape_spectral_image.m
- minor updates and fixes

25 Oct 2012

- added a simple GPU benchmark routine
- minor change in imconvert.m

04 Sep 2013

- fixed automatic download URLs (broken after change on the server)
- gave the toolbox a real name

06 Jan 2014

- integrated (locally debiased) region contrast saliency, see https://github.com/bschauerte/region_contrast_saliency

Contact us