Description |
The Spectral Visual Saliency Toolbox (SViST) provides implementations
of several spectral (FFT,DCT) saliency algorithms for images.
There are several methods to calculate the visual saliency:
'fft': by default the same as 'fft:whitening'
'fft:whitening' Uses spectral whitening to calculate the saliency of
each channel separately and then averages the result.
'fft:residual' Uses the spectral residual to calculate saliency of
each channel separately and then averages the result.
'dct' Uses DCT-based image signatures to calculate saliency
of each channel separately and then averages the
result.
'quat:fft': Converts the image into a quaternion-based
representation, uses quaternion FFT/IFFT operations.
'quat:fft:eigenpqft': Similar to 'quat:fft', but relies on the use of
the quaternion eigenaxis and eigenangle for
calculation.
'quat:fft:eigensr': The principle of 'fft:residual' transferred to
'quat:fft:eigenpqft'.
'quat:dct' Converts the image into a quaternion-based
representation, uses quaternion DCT/IDCT operations.
'quat:dct:fast' Same as 'quad:dct', but with a fixed image
resolution of 64x48 and uses optimized .mex files for
faster calculation.
[...] some others, e.g., Itti-Koch, Achanta, GBVS, region
contrast, and locally debiased region contrast
Usage examples:
- spectral_saliency_multichannel(imread(..image path...))
or as an example for other color spaces (e.g. ICOPP, Lab, ...)
- spectral_saliency_multichannel(rgb2icopp(imread(..image path...)))
Furthermore, 'example' and 'process_folder' are two usage examples.
The former makes it possible to visualizes the saliency maps of
different algorithms for a given test image. The latter calculates
and saves the saliency maps for all images in a given folder.
If you use any of this work in scientific research or as part of a
larger software system, you are requested to cite the use in any
related publications or technical documentation. The work is based
upon:
B. Schauerte, and R. Stiefelhagen, "Quaternion-based Spectral
Saliency Detection for Eye Fixation Prediction," in European
Conference on Computer Vision (ECCV), 2012
B. Schauerte, and R. Stiefelhagen, "Predicting Human Gaze using
Quaternion DCT Image Signature Saliency and Face Detection," in
IEEE Workshop on the Applications of Computer Vision (WACV),
2012.
Notes:
- The implementation of the quaternion-based approach requires the
quaternion toolbox for Matlab (QTFM).
- I kept the implementations as focused and simple as possible and
thus they lack more advanced functionality, e.g. more complex
normalizations. However, I think that the provided functionality is
more than sufficient for (a) people who want to get started in the
field of visual attention (especially students), (b) practitioners
who have heard about the spectral approach and want to try it, and
(c) people who just need a fast, reliable, well-established visual
saliency algorithm (with a simple interface and not too many
parameters) for their applications.
- GBVS and Itti require the original GBVS Matlab implementation by
J. Harel (see http://www.klab.caltech.edu/~harel/share/gbvs.php)
For more details on the method see:
[1] X. Hou and L. Zhang, "Saliency Detection: A Spectral Residual
Approach", in CVPR, 2007.
(original paper)
[2] C. Guo, Q. Ma, and L. Zhang, "Spatio-temporal saliency detection
using phase spectrum of quaternion fourier transform," in CVPR,
2008.
(extension to quaternions; importance of the residual)
[3] X. Hou, J. Harel, and C. Koch, "Image Signature: Highlighting
sparse salient regions," in PAMI, 2011.
(uses DCT-based "image signatures")
[4] B. Schauerte, and R. Stiefelhagen, "Predicting Human Gaze using
Quaternion DCT Image Signature Saliency and Face Detection," in
IEEE Workshop on the Applications of Computer Vision (WACV), 2012.
(extension to quaternions; spectral saliency and face detection;
evaluation of spectral saliency approaches on eye-tracking data;
achieved the currently best reported results on the CERF/FIFA
eye-tracking data set and Toronto/Bruce-Tsotsos data set)
[5] B. Schauerte, and R. Stiefelhagen, "Quaternion-based Spectral
Saliency Detection for Eye Fixation Prediction," in European
Conference on Computer Vision (ECCV), 2012
(integration and evaluation of DCT- and FFT-based spectral saliency
detection, quaternion component weights, and the use of multiple
scales; also introduces EigenPQFT and EigenSR; evaluated on the
Bruce-Tsotsos (Toronto), Judd (MIT), and Kootstra-Schomacker
eye-tracking data sets) |