Learning saliency-based visual attention: A review

Signal Processing

Volumn 93, Issue 6, 2013, Pages 1401-1407

  Zhao, Qi ^a   Koch, Christof ^b,c  

^a NATIONAL UNIVERSITY OF SINGAPORE   (Singapore)

^b CALIFORNIA INSTITUTE OF TECHNOLOGY   (United States)

^c ALLEN INSTITUTE FOR BRAIN SCIENCE   (United States)

Author keywords

Central fixation bias; Feature representation; Machine learning; Public eye tracking datasets; Visual attention

Indexed keywords

CENTRAL FIXATION BIAS; COMPUTATIONAL POWER; ECONOMIC IMPACTS; EYE-TRACKING; FEATURE REPRESENTATION; PROCESSING RESOURCES; SCIENCE AND ENGINEERING; VISUAL ATTENTION;

LEARNING SYSTEMS; MAMMALS;

BEHAVIORAL RESEARCH;

EID: 84875275420     PISSN: 01651684     EISSN: None     Source Type: Journal    
DOI: 10.1016/j.sigpro.2012.06.014     Document Type: Article

References (90)

1. T. Avraham, and M. Lindenbaum Esaliency (extended saliency): meaningful attention using stochastic image modeling IEEE Transactions on Pattern Analysis and Machine Intelligence 99 1 2009 693 708
2. R. Baddeley, and B. Tatler High frequency edges (but not contrast) predict where we fixate: a Bayesian system identification analysis Vision Research 46 18 2006 2824 2833
3. A. Bahill, D. Adler, and L. Stark Most naturally occurring human saccades have magnitudes of 15 degrees or less Investigative Ophthalmology & Visual Science 14 6 1975 468 469
4. N. Bruce, and J. Tsotsos Saliency, attention, and visual search: an information theoretic approach Journal of Vision 9 3 2009 1 24
5. C. Burges A tutorial on support vector machines for pattern recognition Data Mining and Knowledge Discovery 2 2 1998 121 167
6. A. Carbone, F. Pirri, Learning saliency. An ica based model using bernoulli mixtures, in: Proceedings of Brain Inspired Cognitive Systems, 2010.
7. R. Carmi, and L. Itti The role of memory in guiding attention during natural vision Journal of Vision 6 9 2006 898 914
8. R. Carmi, and L. Itti Visual causes versus correlates of attentional selection in dynamic scenes Vision Research 46 26 2006 4333 4345
9. M. Cerf, E. Frady, and C. Koch Faces and text attract gaze independent of the task: experimental data and computer model Journal of Vision 9 12 2009 10:1-15
10. S. Chikkerur, T. Serre, C. Tan, and T. Poggio What and where: a Bayesian inference theory of attention Vision Research 50 22 2010 2233 2247
11. F. Cristino, and R. Baddeley The nature of the visual representations involved in eye movements when walking down the street Visual Cognition 17 6 2009 880 903
12. J. Deng, W. Dong, R. Socher, L.-J. Li, K. Li, L. Fei-Fei, ImageNet: a large-scale hierarchical image database, in: IEEE Conference on Computer Vision and Pattern Recognition, 2009, pp. 248-255.
13. W. Einhauser, M. Spain, and P. Perona Objects predict fixations better than early saliency Journal of Vision 8 14 2008 18:1-26
14. S. Engmann, B. 't Hart, T. Sieren, S. Onat, P. König, and W. Einhäuser Saliency on a natural scene background: effects of color and luminance contrast add linearly Attention, Perception, & Psychophysics 71 6 2009 1337 1352
15. R. Fan, K. Chang, C. Hsieh, X. Wang, C. Lin, Liblinear: a library for large linear classification, Journal of Machine Learning Research 9 (2008) 1871-1874.
16. P. Felzenszwalb, D. McAllester, D. Ramanan, A discriminatively trained, multiscale, deformable part model, in: IEEE Conference on Computer Vision and Pattern Recognition, 2008, pp. 1-8.
17. D. Field What is the goal of sensory coding? Neural Computation 6 1994 559 601
18. T. Foulsham, and G. Underwood What can saliency models predict about eye movements? Spatial and sequential aspects of fixations during encoding and recognition Journal of Vision 8 2 2008 601 617
19. Y. Freund, R. Schapire, 1996. Game theory, on-line prediction and boosting, in: Conference on Computational Learning Theory, pp. 325-332.
20. J. Friedman, T. Hastle, and R. Tibshirani Additive logistic regression: a statistical view of boosting Annals of Statistics 38 2 2000 337 374
21. J. Fuller Eye position and target amplitude effects on human visual saccadic latencies Experimental Brain Research 109 3 1996 457 466
22. D. Gajewski, A. Pearson, M. Mack, F. Bartlett, and J. Henderson Human gaze control in real world search L. Paletta, J. Tsotsos, E. Rome, G. Humphreys, Attention and Performance in Computational Vision 2005 Springer-Verlag New York pp. 3368:83-99
23. D. Gao, V. Mahadevan, N. Vasconcelos, The discriminant center-surround hypothesis for bottom-up saliency, in: Advances in Neural Information Processing Systems, 2007, pp. 497-504.
24. D. Hansen, and Q. Ji In the eye of the beholder: a survey of models for eyes and gaze IEEE Transactions on Pattern Analysis and Machine Intelligence 32 3 2010 478 500
25. J. Harel, C. Koch, P. Perona, Graph-based visual saliency, in: Advances in Neural Information Processing Systems, 2007, pp. 545-552.
26. X. Hou, L. Zhang, Dynamic visual attention: searching for coding length increments, in: Advances in Neural Information Processing Systems, 2008.
27. Y. Hu, X. Xie, W. Ma, L. Chia, D. Rajan, Salient region detection using weighted feature maps based on the human visual attention model, in: IEEE Pacific-Rim Conference on Multimedia, 2004, pp. 993-1000.
28. A. Hyvarinen, and E. Oja Independent component analysis: algorithms and applications Neural Networks 13 4-5 2000 411 430
29. L. Itti Automatic foveation for video compression using a neurobiological model of visual attention IEEE Transactions on Image Processing 13 10 2004 1304 1318
30. L. Itti Quantifying the contribution of low-level saliency to human eye movements in dynamic scenes Visual Cognition 12 6 2005 1093 1123
31. L. Itti, P. Baldi, Bayesian surprise attracts human attention, in: Advances in Neural Information Processing Systems, 2006, pp. 547-554.
32. L. Itti, C. Koch, Comparison of feature combination strategies for saliency-based visual attention systems. In: Proceedings of SPIE Human Vision and Electronic Imaging, 1999, pp. 3644:473-82.
33. L. Itti, C. Koch, and E. Niebur A model for saliency-based visual attention for rapid scene analysis IEEE Transactions on Pattern Analysis and Machine Intelligence 20 11 1998 1254 1259
34. R. Jin, Y. Liu, L. Si, J. Carbonell, H. A.G., A new boosting algorithm using input-dependent regularizer, in: International Conference on Machine Learning, 2003.
35. T. Jost, N. Ouerhani, R. von Wartburg, R. Muri, and H. Hugli Assessing the contribution of color in visual attention Computer Vision and Image Understanding 100 1-2 2005 107 123
36. T. Judd, K. Ehinger, F. Durand, A. Torralba, Learning to predict where humans look, in: IEEE International Conference on Computer Vision, 2009.
37. P. Khuwuthyakorn, A. Robles-Kelly, J. Zhou, Object of interest detection by saliency learning, in: European Conference on Computer Vision, 2010, pp. 6312:636-649.
38. W. Kienzle, F. Wichmann, B. Scholkopf, M. Franz, A nonparametric approach to bottom-up visual saliency, in: Advances in Neural Information Processing Systems, 2006, pp. 689-696.
39. A. Koene, and L. Zhaoping Feature-specific interactions in salience from combined feature contrasts: evidence for a bottom-up saliency map in v1 Journal of Vision 7 7 2007 6:1-14
40. G. Krieger, I. Rentschler, G. Hauske, K. Schill, and C. Zetzsche Object and scene analysis by saccadic eye-movements: an investigation with higher-order statistics Spatial Vision 13 2-3 2000 201 214
41. J. Lafferty, A. McCallum, F. Pereira, Conditional random fields: probabilistic models for segmenting and labeling sequence data, in: International Conference on Machine Learning, 2001, pp. 282-289.
42. P. Lang, M. Bradley, B. Cuthbert, (iaps): Affective Ratings of Pictures and Instruction Manual, Technical Report, University of Florida, 2008.
43. O. Le Meur, P. Le Callet, D. Barba, and D. Thoreau A coherent computational approach to model the bottom-up visual attention IEEE Transactions on Pattern Analysis and Machine Intelligence 28 5 2006 802 817
44. Z. Li A saliency map in primary visual cortex Trends in Cognitive Sciences 6 1 2002 9 16
45. T. Liu, Z. Yuan, J. Sun, J. Wang, N. Zheng, X. Tang, and H. Shum Learning to detect a salient object IEEE Transactions on Pattern Analysis and Machine Intelligence 33 2 2011 353 367
46. T. Liu, N. Zheng, W. Ding, Z. Yuan, Video attention: learning to detect a salient object sequence, in: IEEE Conference on Pattern Recognition, 2008, pp. 1-4.
47. V. Mahadevan, and N. Vasconcelos Spatiotemporal saliency in highly dynamic scenes IEEE Transactions on Pattern Analysis and Machine Intelligence 32 1 2010 171 177
48. S. Mannan, C. Kennard, and M. Husain The role of visual salience in directing eye movements in visual object agnosia Current Biology 19 6 2009 247 248
49. C. Masciocchi, S. Mihalas, D. Parkhurst, and E. Niebur Everyone knows what is interesting: salient locations which should be fixated Journal of Vision 9 11 2009 25:1-22
50. S. Mihalas, Y. Dong, R. von der Heydt, and E. Niebur Mechanisms of perceptual organization provide auto-zoom and auto-localization for attention to objects Proceedings of the National Academy of Sciences 108 18 2011 75 83
51. V. Navalpakkam, and L. Itti Modeling the influence of task on attention Vision Research 45 2 2005 205 231
52. H. Nothdurft Salience from feature contrast: additivity across dimensions Vision Research 40 10-12 2000 1183 1201
53. A. Oliva, and A. Torralba Modeling the shape of the scene: a holistic representation of the spatial envelope International Journal of Computer Vision 42 2001 145 175
54. A. Oliva, A. Torralba, M. Castelhano, J. Henderson, Top-down control of visual attention in object detection, in: International Conference on Image Processing, 2003, pp. I:253-256.
55. S. Onat, K. Libertus, and P. König Integrating audiovisual information for the control of overt attention Journal of Vision 7 10 2007 11:1-6
56. M. Pare, and D. Munoz Expression of a recentering bias in saccade regulation by superior colliculus neurons Experimental Brain Research 137 3-4 2001 354 368
57. D. Parkhurst, K. Law, and E. Niebur Modeling the role of salience in the allocation of overt visual attention Vision Research 42 1 2002 107 123
58. J. Pelz, and R. Canosa Oculomotor behavior and perceptual strategies in complex tasks Vision Research 41 25-26 2001 3587 3596
59. R. Peters, A. Iyer, L. Itti, and C. Koch Components of bottom-up gaze allocation in natural images Vision Research 45 18 2005 2397 2416
60. C. Privitera, and L. Stark Algorithms for defining visual regions-of-interest: comparison with eye fixations IEEE Transactions on Pattern Analysis and Machine Intelligence 22 9 2000 970 982
61. R. Raj, W. Geisler, R. Frazor, and A. Bovik Contrast statistics for foveated visual systems: fixation selection by minimizing contrast entropy Journal of the Optical Society of America A 22 10 2005 2039 2049
62. P. Reinagel, and A. Zador Natural scene statistics at the center of gaze Network: Computation in Neural Systems 10 4 1999 341 350
63. B. Russell, A. Torralba, K. Murphy, and W. Freeman LabelMe: a database and web-based tool for image annotation International Journal of Computer Vision 77 1-3 2008 157 173
64. R. Schapire, and Y. Singer Improved boosting algorithms using confidence-rated predictions Machine Learning 37 3 1999 297 336
65. F. Schumann, W. Einhauser, J. Vockeroth, K. Bartl, E. Schneider, and P. Konig Salient features in gaze-aligned recordings of human visual input during free exploration of natural environments Journal of Vision 8 14 2008 12:1-17
66. H. Seo, and P. Milanfar Static and space-time visual saliency detection by self-resemblance Journal of Vision 9 12 2009 15:1-27
67. E. Simoncelli, W. Freeman, The steerable pyramid: a flexible architecture for multi-scale derivative computation. in: International Conference on Image Processing, 1995, pp. III:444-447.
68. M. Song, D. Tao, C. Chen, J. Bu, J. Luo, and C. Zhang Probabilistic exposure fusion IEEE Transactions on Image Processing 21 1 2012 341 357
69. M. Song, D. Tao, C. Chen, X. Li, and C. Chen Colour to grey: visual cue preservation IEEE Transactions on Pattern Analysis and Machine Intelligence 32 9 2010 1537 1552
70. M. Song, D. Tao, Z. Sun, and X. Li Visual context boosting for eye detection IEEE Transactions on Systems, Man, and Cybernetics, Part B: Cybernetics 40 6 2010 1460 1467
71. R. Subramanian, H. Katti, N. Sebe, M. Kankanhalli, T. Chua, An eye fixation database for saliency detection in images, in: European Conference on Computer Vision, 2010, pp. 6314:30-43.
72. B. Tatler The central fixation bias in scene viewing: selecting an optimal viewing position independently of motor biases and image feature distributions Journal of Vision 7 14 2007 1 17
73. B. Tatler, R. Baddeley, and I. Gilchrist Visual correlates of fixation selection: effects of scale and time Vision Research 45 5 2005 643 659
74. B. Tatler, M. Hayhoe, M. Land, and D. Ballard Eye guidance in natural vision: reinterpreting salience Journal of Vision 11 5 2011 5:1-23
75. A. Treisman, and G. Gelade A feature-integration theory of attention Cognitive Psychology 12 1 1980 97 136
76. I. van der Linde, U. Rajashekar, A. Bovik, and L. Cormack Doves: a database of visual eye movements Spatial Vision 22 2 2009 161 177
77. A. Vezhnevets, V. Vezhnevets, Modest adaboost - teaching adaboost to generalize better, in: Graphicon, 2005.
78. B. Vincent, R. Baddeley, T. Troscianko, and I. Gilchrist Optimal feature integration in visual search Journal of Vision 9 5 2009 15:1-11
79. P. Viola, M. Jones, Rapid object detection using a boosted cascade of simple features, in: IEEE Conference on Computer Vision and Pattern Recognition, 2001, pp. I:511-518.
80. F. Vitu, Z. Kapoula, D. Lancelin, and F. Lavigne Eye movements in reading isolated words: evidence for strong biases towards the center of the screen Vision Research 44 3 2004 321 338
81. D. Walther, T. Serre, T. Poggio, and C. Koch Modeling feature sharing between object detection and top-down attention Journal of Vision 5 8 2005 1041
82. W. Wang, Y. Wang, Q. Huang, W. Gao, Measuring visual saliency by site entropy rate, in: IEEE Conference on Computer Vision and Pattern Recognition, 2010, pp. 2368-2375.
83. J. Xu, Z. Yang, and J. Tsien Emergence of visual saliency from natural scenes via context-mediated probability distributions coding PLoS ONE 5 12 2010 e15796
84. V. Yanulevskaya, J. Marsman, F. Cornelissen, and J. Geusebroek An image statistics-based model for fixation prediction Cognitive Computation 3 1 2010 94 104
85. D. Zambarbieri, G. Beltrami, and M. Versino Saccade latency toward auditory targets depends on the relative position of the sound source with respect to the eyes Vision Research 35 23-24 1995 3305 3312
86. L. Zhang, M. Tong, G. Cottrell, Sunday: saliency using natural statistics for dynamic analysis of scenes, in: Proceedings of the 31st Annual Cognitive Science Conference, 2009, pp. 2944-2949.
87. L. Zhang, M. Tong, T. Marks, H. Shan, and G. Cottrell Sun: a Bayesian framework for saliency using natural statistics Journal of Vision 8 7 2008 1 20
88. Q. Zhao, and C. Koch Learning a saliency map using fixated locations in natural scenes Journal of Vision 11 3 2011 9:1-15
89. Q. Zhao, C. Koch, Learning visual saliency, in: Conference on Information Sciences and Systems, 2011, pp. 1-6.
90. Q. Zhao, and C. Koch Learning visual saliency by combining feature maps in a nonlinear manner using Adaboost Journal of Vision 12 6 2012 22:1-15