A novel multiresolution spatiotemporal saliency detection model and its applications in image and video compression

IEEE Transactions on Image Processing

Volumn 19, Issue 1, 2010, Pages 185-198

  Guo, Chenlei ^a,b   Zhang, Liming ^a  

^a FUDAN UNIVERSITY   (China)

^b CARNEGIE MELLON UNIVERSITY   (United States)

Author keywords

Hierarchical selectivity (HS); Multiresolution wavelet domain foveation (MWDF); Phase spectrum of Fourier transform (PFT); Phase spectrum of quaternion fourier transform (PQFT); Receiver operating characteristic (ROC) curve; Spatiotemporal saliency map; Visual attention

Indexed keywords

FOVEATION; MULTIRESOLUTION WAVELET DOMAIN FOVEATION (MWDF); MULTIRESOLUTION WAVELETS; PHASE SPECTRA; PHASE SPECTRUM OF FOURIER TRANSFORM (PFT); RECEIVER OPERATING CHARACTERISTIC CURVES; SPATIOTEMPORAL SALIENCY; VISUAL ATTENTION;

COMPUTATIONAL COMPLEXITY; COMPUTER VISION; HYDRAULIC STRUCTURES; IMAGE CODING; IMAGE COMPRESSION; IMAGE SEGMENTATION; MULTIRESOLUTION ANALYSIS;

FOURIER TRANSFORMS;

EID: 72949100573     PISSN: 10577149     EISSN: None     Source Type: Journal    
DOI: 10.1109/TIP.2009.2030969     Document Type: Article

References (56)

1. R. Fergus, P. Perona, and A. Zisserman, "Object class recognition by unsupervised scale-invariant learning," presented at the CVPR, 2003.
2. T. Liu, J. Sun, N. Zheng, X. Tang, and H. Shum, "Learning to detect a salient object," presented at the CVPR, 2007.
3. W. James, The Principles of Psychology. New York: Holt, 1890.
4. A. Treisman and G. Gelade, "A feature-integration theory of attention," Cogn. Psych., vol.12, no.1, pp. 97-136, 1980.
5. J.Wolfe, "Guided search 2.0: A revised model of guided search," Psychonomic Bull. Rev., vol.1, no.2, pp. 202-238, 1994.
6. C. Koch and S. Ullman, "Shifts in selection in visual attention: Toward the underlying neural circuitry," Human Neurobiol., vol.4, no.4, pp. 219-227, 1985.
7. L. Itti, C.Koch, and E. Niebur, "A model of saliency-based visual attention for rapid scene analysis," IEEE Trans. Pattern Anal. Mach. Intell., vol.20, no.11, pp. 1254-1259, Nov. 1998.
8. L. Itti and C. Koch, "A saliency-based search mechanism for overt and covert shifts of visual attention," Vis. Res., vol.40, pp. 1489-1506, 2000.
9. L. Itti and P. Baldi, "Bayesian surprise attracts human attention," in Proc. NIPS, 2005, pp. 547-554.
10. L. Itti and P. Baldi, "A principled approach to detecting surprising events in video," in Proc. CVPR, 2005, pp. 631-637.
11. L. Itti, "Models of Bottom-Up and Top-Down Visual Attention," Ph.D. dissertation, California Inst. chnol., Pasadena, 2000.
12. D.Walther and C. Koch, "Modeling attention to salient proto-objects," Neural Netw., vol.19, pp. 1395-1407, 2006.
13. D. Walther, L. Itti, M. Riesenhuber, T. Poggio, and C. Koch, "Attentional selection for object recognition-A gentle way," Lecture Notes in Computer Science, vol.2525, no.1, pp. 472-479, 2002.
14. X. Hou and L. Zhang, "Saliency detection: A spectral residual approach," presented at the CVPR, 2007.
15. C. L. Guo, Q. Ma, and L. M. Zhang, "Spatio-temporal saliency detection using phase spectrum of quaternion Fourier transform," presented at the CVPR, 2008.
16. N. D. Bruce and J. K. Tsotsos, "Saliency based on information maximization," presented at the NIPS, 2005.
17. D. Gao, V. Mahadevan, and N. Vasconcelos, "The discriminant centersurround hypothesis for bottom-up saliency," presented at the NIPS, 2007.
18. J. Harel, C. Koch, and P. Perona, "Graph-based visual saliency," presented at the NIPS, 2006.
19. G. Backer, B. Mertsching, and M. Bollmann, "Data- and model-driven gaze control for an active-vision system," IEEE Trans. Pattern Anal. Mach. Intell., vol.23, no.12, pp. 1415-1429, Dec. 2001.
20. M. Z. Aziz and B. Mertsching, "Fast and robust generation of feature maps for region-based visual attention," IEEE Trans. Image Process., vol.17, no.5, pp. 5, 633-644, May 2008.
21. S. Frintrop, "OCUS: A Visual Attention System for Object Detection and Goal-Directed Search," Ph.D. dissertation, 2006.
22. O. L. Meur, O. L. Callet, D. Barba, and D. Thoreau, "A coherent computational approach to model bottom-up visual attention," IEEE Trans. Pattern Anal. Mach. Intell., vol.28, pp. 802-817, 2006.
23. J. K. Tsotsos, S. M. Culhane, W. Y. K.Wai, Y. H. Lai, N. Davis, and F. Nuflo, "Modelling visual attention via selective tuning," Artif. Intell., vol.78, no.1-2, pp. 507-545, 1995.
24. R. Rensink, "Seeing, sensing, and scrutinizing," Vis. Res., vol.40, no.10-12, pp. 1469-1487, 2000.
25. L. Itti, "Automatic foveation for video compression using a neurobiological model of visual attention," IEEE Trans. Image Process., vol.13, no.10, pp. 1304-1318, Oct. 2004.
26. K. Castleman, Digital Image Processing. New York: Prentice-Hall, 1996.
27. S. Engel, X. Zhang, and B. Wandell, "Colour tuning in human visual cortex measured with functional magnetic resonance imaging," Nature, vol.388, no.6,637, pp. 68-71, Jul. 1997.
28. S. J. Sangwine, "Fourier transforms of colour images using quaternion, or hypercomplex, numbers," Electron. Lett., vol.32, no.21, pp. 1979-1980, Oct. 1996.
29. T. A. Ell, "Hypercomplex Spectral Transforms," Ph.D. dissertation, Univ. Minnesota, Minneapolis, 1992.
30. T. A. Ell, "Quaternion-Fourier transforms for analysis of two-dimensional linear time-invariant partial-differential systems," in Proc. IEEE Conf. Decision and Control, San Antonio, TX, Dec. 15-17, 1993, vol.1-4, pp. 1830-1841.
31. S.-C. Pei, J.-J. Ding, and J.-H. Chang, "Efficient implementation of quaternion Fourier transform, convolution, and correlation by 2-D complex FFT," IEEE Trans. Signal Process., vol.49, no.11, pp. 2783-2797, Nov. 2001.
32. S. J. Sangwine and T. A. Ell, J. M. Blackledge and M. J. Turner, Eds., "The discrete Fourier transform of a colour image," in Proc. Image Processing II Mathematical Methods, Algorithms and Applications, Chichester, U.K., 2000, pp. 430-441.
33. T. Ell and S. Sangwin, "Hypercomplex Fourier transforms of color images," IEEE Trans. Image Process., vol.16, pp. 22-35, 2007.
34. B. J. Scholl, "Objects and attention: The state of the art," Cognition, vol.80, pp. 1-46, 2001.
35. M. I. Posner and Y. Cohen, "Components of visual orienting," in Attention and Performance, H. Bouma and D. G. Bouwhuis, Eds. Hillsdale, NJ: Erlbaum, 1984, vol.10, pp. 531-556.
36. Y. Sun and R. Fisher, "Object-based visual attention for computer vision," Artif. Intell., vol.146, no.1, pp. 77-123, May 2003.
37. B. W. Tatler, R. J. Baddeley, and I. D. Gilchrist, "Visual correlates of fixation selection: Effects of scale and time," Vis. Res., vol.45, pp. 643-659, 2005.
38. B. Horn and B. Schunck, "Determining optical flow," Artif. Intell., vol.17, no.1-3, pp. 185-203, 1981.
39. T. Brox, A. Bruhn, N. Papenberg, and J.Weickert, "High accuracy optical flow estimation based on a theory for warping," presented at the ECCV, 2004.
40. A. Treisman and S. Gormican, "Feature analysis in early vision: Evidence from search asymmetries," Psych. Rev., vol.95, pp. 14-58, 1988.
41. B. Wandell, Foundations of Vision. Sunderland, MA: Sinauer, 1995.
42. V. Sanchez, A. Basu, and M. K. Mandal, "Prioritized region of interest coding in JPEG2000," IEEE Trans. Circuits Syst. Video Technol., vol.14, no.9, pp. 1149-1155, Sep. 2004.
43. JPEG 2000 Part I: Final Draft International Standard (ISO/IEC FDIS15444-1), , Aug. 2000, ISO/IECJTC1/SC29/WG1 N1855.
44. P. T. Kortum and W. S. Geisler, "Implementation of a foveated imagecoding system for bandwidth reduction of video images," in Proc. SPIE, 1996, vol.2657, pp. 350-360.
45. N. Doulamis, A. Doulamis, D. Kalogeras, and S. Kollias, "Improving the performance of MPEG coders using adaptive regions of interest," IEEE Trans. Circuits Syst. Video Technol., vol.8, no.12, pp. 928-934, Dec. 1998.
46. S. Lee, A. C. Bovik, and Y. Y. Kim, "Low delay foveated visual communications over wireless channels," in Proc. Int. Conf. Image Processing, 1999, pp. 90-94.
47. U. Rauschenbach and H. Schumann, "Demand-driven image transmission with levels of detail and regions of interest," Comput. Graph., vol.23, no.6, pp. 857-866, 1999.
48. E. M. Reingold and L. C. Loschky, "Saliency of peripheral targets in gaze-contingent multiresolutional displays," Behavior Res. Meth., Instrum. Comput., vol.34, no.4, pp. 491-499, 2002.
49. D. J. Parkhurst and E. Niebur, "Variable-resolution displays: A theoretical, practical, and behavioral evaluation," Human Factors, vol.44, no.4, pp. 611-629, 2002.
50. W. Osberger and A. J. Maeder, "Automatic identification of perceptually important regions in an image using a model of the human visual system," in Proc. ICPR, Aug. 1998, pp. 701-704.
51. F. Stentiford, "An estimator for visual attention through competitive novelty with application to image compression," in Proc. Picture Coding Symp., Apr. 2001, pp. 101-104.
52. C. M. Privitera and L. W. Stark, "Algorithms for defining visual regions- of-interest: Comparison with eye fixations," IEEE Trans. Pattern Anal. Mach. Intell., vol.22, no.9, pp. 970-982, Sep. 2000.
53. Z.Wang, L. G. Lu, and A. C. Bovik, "Foveation scalable video coding with automatic fixation selection," IEEE Trans. Image Process., vol.12, no.2, pp. 243-254, Feb. 2003.
54. V. Navalpakkam and L. Itti, "An integrated model of top-down and bottom-up attention for optimizing detection speed," in Proc. CVPR, 2006, pp. 2049-2056.
55. U. Rutishauser,D.Walther, C.Koch, and P. Perona, "Is bottomup attention useful for object recognition?," in Proc. CVPR, 2004, pp. 37-44.
56. [Online]. Available: http://homepage.fudan.edu.cn/~clguo