Learning-based prediction of visual attention for video signals

IEEE Transactions on Image Processing

Volumn 20, Issue 11, 2011, Pages 3028-3038

  Lee, Wen Fu ^a   Huang, Tai Hsiang ^a   Yeh, Su Ling ^a   Chen, Homer H ^a  

^a NATIONAL TAIWAN UNIVERSITY   (Taiwan)

Author keywords

Eye tracking experiment; fixation distribution; human visual system; regression; saliency map; visual attention

Indexed keywords

EYE TRACKING EXPERIMENT; HUMAN VISUAL SYSTEM; REGRESSION; SALIENCY MAP; VISUAL ATTENTION;

IMAGE PROCESSING;

ADULT; ALGORITHM; ARTICLE; ARTIFICIAL INTELLIGENCE; FEMALE; HUMAN; MALE; PATTERN RECOGNITION; PHOTOSTIMULATION; VIDEORECORDING; VISUAL FIELD;

ADULT; ALGORITHMS; ARTIFICIAL INTELLIGENCE; FEMALE; HUMANS; MALE; PATTERN RECOGNITION, VISUAL; PHOTIC STIMULATION; VIDEO RECORDING; VISUAL FIELDS;

EID: 80052994754     PISSN: 10577149     EISSN: None     Source Type: Journal    
DOI: 10.1109/TIP.2011.2144610     Document Type: Article

References (36)

1. C. Maioli, I. Benaglio, S. Siri, K. Sosta, and S. Cappa, "The integration of parallel and serial processing mechanisms in visual search: Evidence from eye movement recordings," Eur. J. Neurosci., vol. 13, pp. 364-372, Jan. 2001.
2. J. M. Findlay, "Saccade target selection during visual search," Vis. Res., vol. 37, pp. 617-631, 1997.
3. G. Rizzolatti, L. Riggio, I. Dascola, and C. Umilta, "Reorienting attention across the horizontal and vertical meridians: Evidence in favor of a premotor theory of attention," Neuropsychologia, vol. 25, no. 1A, pp. 31-40, 1987.
4. Y. S. Wang, C. L. Tai, O. Sorkine, and T. Y. Lee, "Optimized scaleand- stretch for image resizing," ACM Trans. Graph, vol. 27, no. 5, pp. 118:1-118:8, 2008.
5. H. Li and K. N. Ngan, "Saliency model-based face segmentation and tracking in head-and-shoulder video sequences," J. Vis. Commun. Image Representation, vol. 19, no. 5, pp. 320-333, 2008.
6. C. W. Tang, C. H. Chen, Y. H. Yu, and C. J. Tsai, "Visual sensitivity guided bit allocation for video coding," IEEE Trans. Multimedia, vol. 8, no. 1, pp. 11-18, Feb. 2006.
7. S. M. Jin, I. B. Lee, J. M. Han, J. M. Seo, and K. S. Park, "Context-based pixelization model for the artificial retina using saliency map and skin color detection algorithm," in Proc. SPIE Human Vision and Electronic Imaging XIII, Feb. 2008, vol. 6806, pp. L:1-L:8.
8. A. P. Hillstrom and S. Yantis, "Visual motion and attentional capture," Perception Psychophys., vol. 55, no. 4, pp. 399-411, Apr. 1994.
9. J. K. Tsotsos, Y. Liu, J. C. Martinez-Trujillo, M. Pomplun, E. Simine, and K. Zhou, "Attending to visual motion," Comput. Vis. Image Understanding, vol. 100, no. 1-2, pp. 3-40, 2005.
10. K. R. Ansgar and Z. Li, "Feature-specific interactions in salience from combined feature contrasts: Evidence for a bottom-up saliency map in V1," J. Vis., vol. 7, pp. 1-14, 2007.
11. H. J. Müller and P. M. A. Rabbitt, "Reflexive and voluntary orienting of visual attention: Time course of activation and resistance to interruption," J. Exper. Psychol. : Human Perception Performance, vol. 15, no. 2, pp. 315-330, May 1989.
12. R. Carmi and L. Itti, "Visual causes versus correlates of attentional selection in dynamic scenes," Vis. Res., vol. 46, no. 26, pp. 4333-4345, Dec. 2006.
13. U. Engelke, H. J. Zepernick, and A. Maeder, "Visual attention modeling: Region-of-interest versus fixation patterns," in Proc. IEEE 27th Picture Coding Symp., 2009, pp. 521-524.
14. A. Torralba, A. Oliva, M. S. Castelhano, and J. M. Henderson, "Contextual guidance of eye movements and attention in real-world scenes: The role of global features in object search," Psycholog. Rev., vol. 113, pp. 766-786, 2006.
15. T. Liu, J. Sun, N. N. Zheng, X. Tang, and H. Y. Shum, "Learning to detect a salient object," in Proc. IEEE CVPR, 2007, pp. 1-8.
16. Y. F. Ma, X. S. Hua, L. Lu, and H. J. Zhang, "A generic framework of user attention model and its application in video summarization," IEEE Trans. Multimedia, vol. 7, no. 5, pp. 907-919, May 2005.
17. O. L. Meur, P. L. Callet, and D. Barba, "Predicting visual fixations on video based on low-level visual features," Vis. Res., vol. 47, pp. 2483-2498, Sep. 2007.
18. L. Itti, N. Dhavale, and F. Pighin, "Realistic avatar eye and head animation using a neurobiological model of visual attention," in Proc. SPIE 48th Annu. Int. Symp. Opt. Sci. Technol., Aug. 2003, vol. 5200, pp. 64-78.
19. H. Liu, S. Jiang, Q. Huang, and C. Xu, "A generic virtual content insertion system based on visual attention analysis," in Proc. 16th ACM Int. Conf. Multimedia, 2008, pp. 379-388.
20. Y. Zhai and M. Shah, "Visual attention detection in video sequences using spatiotemporal cues," in Proc. 14th Annu. ACM Int. Conf. Multimedia, Oct. 2006, pp. 815-824.
21. O. L. Meur, P. L. Callet, D. Barba, and D. Thoreau, "A spatio-temporal model of the selective human visual attention," in Proc. IEEE Int. Conf. Image Process., 2005, pp. 1188-1191.
22. S. Li and M. C. Lee, "An efficient spatiotemporal attention model and its application to shot matching," IEEE Trans. Circuits Syst. Video Technol., vol. 17, no. 10, pp. 1383-1387, Oct. 2007.
23. M. Z. Aziz and B. Mertsching, "Fast and robust generation of feature maps for region-based visual attention," IEEE Trans. Image Process., vol. 17, no. 5, pp. 633-644, May 2008.
24. R. Palermo and G. Rhodes, "Are you always on my mind? Areview of how face perception and attention interact," Neuropsychologia, vol. 45, no. 1, pp. 75-92, 2007.
25. N. Kanwisher, "Domain specificity in face perception," Nature Neurosci., vol. 3, no. 8, pp. 759-763, 2000.
26. J. Krauskopf, D. R. Williams, and D. W. Heeley, "Cardinal directions of color space," Vis. Res., vol. 22, no. 9, pp. 1123-1131, 1982.
27. R. L. De Valois, "Behavioral and electrophysiological studies of primate vision," in Contributions to Sensory Physiology, W. D. Neff, Ed. New York: Academic, 1965, vol. 1, pp. 137-138.
28. H. Senane, A. Saadane, and D. Barba, "The computation of visual bandwidths and their impact in image decomposition and coding," in Proc. Int. Conf. Signal Process. Applic. Technol., Santa Clara, CA, 1993, pp. 770-776.
29. O. L. Meur, P. L. Callet, D. Barba, and D. Thoreau, "A coherent computational approach to model bottom-up visual attention," IEEE Trans. Pattern Anal. Mach. Intell., vol. 28, no. 5, pp. 802-817, May 2006.
30. W. Li and E. Salari, "Successive elimination algorithm for motion estimation," IEEE Trans. Image Process., vol. 4, no. 1, pp. 105-107, Jan. 1995.
31. M. Fischler and R. Bolles, "Random sample consensus: A paradigm for model fitting with application to image analysis and automated cartography," in Commun. Assoc. Comp. Mach., 1981, vol. 24, pp. 381-395.
32. A. G. Leventhal, The Neural Basis of Visual Function: Vision and Visual Dysfunction. Boca Raton, FL: CRC, 1991, vol. 4.
33. M. Nilsson, J. Nordberg, and I. Claesson, "Face detection using local SMQT features and split up snow classifier," in Proc. IEEE Int. Conf. Acoust., Speech Signal Process., 2007, vol. 2, pp. 589-592.
34. C. C. Chang and C. J. Lin, "LIBSVM: A Library for Support Vector Machines," 2001. [Online]. Available: http://www. csie. ntu. edu. tw/cjlin/libsvm
35. R. O. Duda, P. E. Hart, and D. G. Stork, Pattern Classification. New York: Wiley, 2000.
36. T. Judd, K. Ehinger, F. Durand, and A. Torralba, "Learning to predict where humans look," in Proc. IEEE 12th Int. Conf. Comput. Vis., 2009, pp. 2106-2113.