Learning visual saliency by combining feature maps in a nonlinear manner using AdaBoost

Journal of Vision

Volumn 12, Issue 6, 2012, Pages

  Zhao, Qi ^a,b   Koch, Christof ^a,c  

^a CALIFORNIA INSTITUTE OF TECHNOLOGY   (United States)

^b NATIONAL UNIVERSITY OF SINGAPORE   (Singapore)

^c ALLEN INSTITUTE FOR BRAIN SCIENCE   (United States)

Author keywords

AdaBoost; Computational saliency model; Feature integration

Indexed keywords

ARTICLE; ATTENTION; BIOLOGICAL MODEL; COMPARATIVE STUDY; EYE MOVEMENT; HUMAN; PATTERN RECOGNITION; PHOTOSTIMULATION; PHYSIOLOGY;

ATTENTION; EYE MOVEMENTS; HUMANS; MODELS, BIOLOGICAL; PATTERN RECOGNITION, VISUAL; PHOTIC STIMULATION;

EID: 84865657272     PISSN: None     EISSN: 15347362     Source Type: Journal    
DOI: 10.1167/12.6.22     Document Type: Article

References (73)

1. Avraham, T., & Lindenbaum, M. (2009). Esaliency (Extended Saliency): Meaningful attention using stochastic image modeling. IEEE Transactions on Pattern Analysis and Machine Intelligence, 99(1), 693-708.
2. Baddeley, R., & Tatler, B. (2006). High frequency edges (but not contrast) predict where we fixate: A Bayesian system identification analysis. Vision Research, 46(18), 2824-2833.
3. Bahill, A., Adler, D., & Stark, L. (1975). Most naturally occurring human saccades have magni-tudes of 15 degrees or less. Investigative Ophthal-mology & Visual Science, 14(6), 468-469, http:// www.iovs.org/content/14/6/468. [PubMed] [Article].
4. Bruce, N., & Tsotsos, J. (2009). Saliency, attention, and visual search: An information theoretic approach. Journal of Vision, 9(3):5, 1-24, http://www. journalofvision.org/content/9/3/5, doi:10.1167/9.3.5. [PubMed] [Article].
5. Carbone, A., & Pirri, F. (2010). Learning saliency. An ICA based model using Bernoulli mixtures. In Proceedings of Brain Inspired Cognitive Systems.
6. Cerf, M., Frady, E., & Koch, C. (2009). Faces and text attract gaze independent of the task: Experimental data and computer model. Journal of Vision, 9(12): 10, 1-15, http://www.journalofvision.org/content/ 9/12/10, doi:10.1167/9.12.10. [PubMed] [Article].
7. Chen, X., & Yuille, A. (2004). Detecting and reading text in natural scenes. In IEEE Conference on Computer Vision and Pattern Recognition (pp. 366- 373).
8. Chikkerur, S. S., Serre, T., Tan, C., & Poggio, T. (2010). What and where: A Bayesian inference theory of attention. Vision Research, 50(22), 2233- 2247.
9. Einhauser, W., Spain, M., & Perona, P. (2008). Objects predict fixations better than early saliency. Journal of Vision, 8(14):18, 1-26, http://www.journalofvision.org/content/8/14/18, doi:10.1167/8.14.18. [PubMed] [Article].
10. Engmann, S., 't Hart, B. M., Sieren, T., Onat, S., Konig, P., & Einhauser, W. (2009). Saliency on a natural scene background: Effects of color and luminance contrast add linearly. Attention, Percep-tion, & Psychophysics, 71(6), 1337-1352.
11. Field, D. (1994). What is the goal of sensory coding?. Neural Computation, 6, 559-601.
12. Foulsham, T., & Underwood, G. (2008). What can saliency models predict about eye movements? Spatial and sequential aspects of fixations during encoding and recognition. Journal of Vision, 8(2):6, 601-617, http://www.journalofvision.org/content/ 8/2/6, doi:10.1167/8.2.6. [PubMed] [Article].
13. Freund, Y., & Schapire, R. (1996). Game theory, on-line prediction and boosting. In Conference on Computational Learning Theory (pp. 325-332). New York: ACM.
14. Friedman, J., Hastie, T., Rosset, S., Tibshirani, R., & Zhu, J. (2004). Discussion of three boosting papers. Annals of Statistics, 32(1), 102-107.
15. Friedman, J., Hastie, T., & Tibshirani, R. (1998). Additive logistic regression: A statistical view of boosting. Annals of Statistics, 38(2), 337-374.
16. Fuller, J. (1996). Eye position and target amplitude effects on human visual saccadic latencies. Exper-imental Brain Research, 109(3), 457-466.
17. Gajewski, D.A., Pearson, A.M., Mack, M.L., Bartlett, F.N., & Henderson, J.M. (2005). Human gaze control in real world search. In L. Paletta, J. Tsotsos, E. Rome, & G. Humphreys (Eds.), Attention and Performance in Computational Vision (Vol. 3368, pp. 83-99). New York: Springer-Verlag.
18. Gao, D., Mahadevan, V., & Vasconcelos, N. (2007). The discriminant center-surround hypothesis for bottom-up saliency. In Advances in Neural Infor-mation Processing Systems (pp. 497-504). Cam-bridge, MA: MIT Press.
19. Harel, J., Koch, C., & Perona, P. (2007). Graph-based visual saliency. In Advances in Neural Information Processing Systems (pp. 545-552).
20. Hayhoe, M., & Ballard, D. (2005). Eye movements in natural behavior. Trends in Cognitive Sciences, 9(4), 188-194. Cambridge MA: MIT Press.
21. Hou, X., & Zhang, L. (2008). Dynamic visual attention: searching for coding length increments. In Advances in Neural Information Processing Systems (pp. 681-688). Cambridge, MA: MIT Press.
22. Hu, Y., Xie, X., Ma, W., Chia, L., & Rajan, D. (2004). Salient region detection using weighted feature maps based on the human visual attention model. In IEEE Pacific-Rim Conference on Multimedia (pp. 993-1000). Berlin, Heidelberg: Springer-Ver-lag.
23. Hyvarinen, A., & Oja, E. (2000). Independent compo-nent analysis: Algorithms and applications. Neural Networks, 13(4-5), 411-430.
24. Itti, L., & Baldi, P. (2006). Bayesian surprise attracts human attention. In Advances in Neural Informa-tion Processing Systems (pp. 547-554).
25. Itti, L., & Koch, C. (1999). Comparison of feature combination strategies for saliency-based visual attention systems. In Proc. SPIE Human Vision and Electronic Imaging (pp. 3644:473-82). Belling-ham, WA: SPIE.
26. Itti, L., Koch, C., & Niebur, E. (1998). A model for saliency based visual attention for rapid scene analysis. IEEE Transactions on Pattern Analysis and Machine Intelligence, 20(11), 1254-1259.
27. Jin, R., Liu, Y., Si, L., Carbonell, J., & Hauptmann, A. G. (2003). A new boosting algorithm using input-dependent regularizer. In International Conference on Machine Learning. Palo Alto, CA: AAAI Press.
28. Jost, T., Ouerhani, N., von Wartburg, R., Muri, R., & Hugli, H. (2005). Assessing the contribution of color in visual attention. Computer Vision and Image Understanding, 100(1-2), 107-123.
29. Judd, T., Ehinger, K., Durand, F., & Torralba, A. (2009). Learning to predict where humans look. In IEEE International Conference on Computer Vision. Washington, DC: IEEE Computer Society.
30. Khuwuthyakorn, P., Robles-Kelly, A., & Zhou, J. (2010). Object of interest detection by saliency learning. In European Conference on Computer Vision (pp. 636-649). Berlin, Heidelberg: Springer-Verlag.
31. Kienzle, W., Wichmann, F., Scholkopf, B., & Franz, M. (2006). A nonparametric approach to bottom-up visual saliency. In Advances in Neural Information Processing Systems (pp. 689-696). Cambridge, MA: MIT Press.
32. Koch, C., & Ullman, S. (1985). Shifts in selective visual attention: Towards the underlying neural circuitry. Human Neurobiology, 4(4), 219-227.
33. Koene, A., & Zhaoping, L. (2007). Feature-specific interactions in salience from combined feature contrasts: Evidence for a bottom-up saliency map in V1. Journal of Vision, 7(7):6, 1-14, http://www. journalofvision.org/content/7/7/6, doi:10.1167/7.7. 6. [PubMed] [Article].
34. Krieger, G., Rentschler, I., Hauske, G., Schill, K., & Zetzsche, C. (2000). Object and scene analysis by saccadic eye movements: An investigation with higher-order statistics. Spatial Vision, 13(2-3), 201- 214.
35. Lafferty, J., McCallum, A., & Pereira, F. (2001). Conditional random fields: Probabilistic models for segmenting and labeling sequence data. In International Conference on Machine Learning (pp. 282-289). San Francisco: Morgan Kaufmann Publishers, Inc.
36. Land, M. (2006). Eye movements and the control of actions in everyday life. Progress in Retinal and Eye Research, 25(3), 296-324.
37. Lang, P., Bradley, M., & Cuthbert, B. (2008). (IAPS): Affective ratings of pictures and instruction man-ual. In Technical report, University of Florida.
38. Le Meur, O., Le Callet, P., Barba, D., & Thoreau, D. (2006). A coherent computational approach to model the bottom-up visual attention. IEEE Transactions on Pattern Analysis and Machine Intelligence, 28(5), 802-817.
39. Li, Z. (2002). A saliency map in primary visual cortex. Trends in Cognitive Sciences, 6(1), 9-16.
40. Liu, T., Yuan, Z., Sun, J., Wang, J., Zheng, N., Tang, X., et al. (2011). Learning to detect a salient object. IEEE Transactions on Pattern Analysis and Ma-chine Intelligence, 33(2), 353-367.
41. Liu, T., Zheng, N., Ding, W., & Yuan, Z. (2008). Video attention: Learning to detect a salient object sequence. In IEEE Conference on Pattern Recogni-tion (pp. 1-4) Washington, DC: IEEE Computer Society.
42. Masciocchi, C., Mihalas, S., Parkhurst, D., & Niebur, E. (2009). Everyone knows what is interesting: Salient locations which should be fixated. Journal of Vision, 9(11):25, 1-22, http:// www.journalofvision.org/content/9/11/25, doi:10. 1167/9.11.25. [PubMed] [Article].
43. Navalpakkam, V., & Itti, L. (2005). Modeling the influence of task on attention. Vision Research, 45(2), 205-231.
44. Nothdurft, H. (2000). Salience from feature contrast: Additivity across dimensions. Vision Research, 40, 1183-1201.
45. Oliva, A., Torralba, A., Castelhano, M., & Henderson, J. (2003). Top-down control of visual attention in object detection. In IEEE International Conference on Image Processing (pp. I:253-256). Washington, DC: IEEE Computer Society.
46. Onat, S., Libertus, K., & Konig, P. (2007). Integrating audiovisual information for the control of overt attention. Journal of Vision, 7(10):11, 1-16, http:// www.journalofvision.org/content/7/10/11, doi:10. 1167/7.10.11. [PubMed] [Article].
47. Ouerhani, N., Bur, A., & Hugli, H. (2006). Linear vs. nonlinear feature combination for saliency compu-tation: A comparison with human vision. In Lecture Notes in Computer Science (pp. 4174:314- 323). Berlin, Heidelberg: Springer-Verlag.
48. Parkhurst, D., Law, K., & Niebur, E. (2002). Modeling the role of salience in the allocation of overt visual attention. Vision Research, 42(1), 107-123.
49. Pelz, J. B., & Canosa, R. (2001). Oculomotor behavior and perceptual strategies in complex tasks. Vision Research, 41(25-26), 3587-3596.
50. Pelz, J. B., Canosa, R., Kucharczyk, D., Babcock, J., Silver, A., & Konno, D. (2000). Portable eye tracking: A study of natural eye movements. In Proceedings of Human Vision and Electronic Imaging V (pp. 3959:566-582). Bellingham, WA: SPIE.
51. Peters, R., Iyer, A., Itti, L., & Koch, C. (2005). Components of bottom-up gaze allocation in natural images. Vision Research, 45(18), 2397-2416.
52. Privitera, C., & Stark, L. (2000). Algorithms for defining visual regions-of-interest: Comparison with eye fixations. IEEE Transactions on Pattern Analysis and Machine Intelligence, 22(9), 970-982.
53. Raj, R., Geisler, W., Frazor, R., & Bovik, A. (2005). Contrast statistics for foveated visual systems: Fixation selection by minimizing contrast entropy. Journal of the Optical Society of America A, 22(10), 2039-2049.
54. Reinagel, P., & Zador, A. (1999). Natural scene statistics at the centre of gaze. Network, 10(4), 341-350.
55. Rubner, Y., Tomasi, C., & Guibas, L. (2000). The Earth Mover's Distance as a metric for image retrieval. International Journal of Computer Vision, 40(2), 99-121.
56. Schapire, R., & Singer, Y. (1999). Improved boosting algorithms using confidence-rated predictions. Ma-chine Learning, 37(3), 297-336.
57. Schumann, F., Einhauser, W., Vockeroth, J., Bartl, K., Schneider, E., & Konig, P. (2008). Salient features in gaze- aligned recordings of human visual input during free exploration of natural environments. Journal of Vision, 8(14):12, 1-17, http://www. journalofvision.org/content/8/14/12, doi:10.1167/8. 14.12. [PubMed] [Article].
58. Seo, H., & Milanfar, P. (2009). Static and space-time visual saliency detection by self-resemblance. Journal of Vision, 9(12):15, 1-27, http://www. journalofvision.org/content/9/12/15, doi:10.1167/9. 12.15. [PubMed] [Article].
59. Subramanian, R., Katti, H., Sebe, N., Kankanhalli, M., & Chua, T. S. (2010). An eye fixation database for saliency detection in images. In European Conference on Computer Vision (pp. 6314:30-43). Berlin, Heidelberg: Springer-Verlag.
60. Tatler, B. (2007). The central fixation bias in scene viewing: Selecting an optimal viewing position independently of motor biases and image feature distributions. Journal of Vision, 7(14):4, 1-17, http://www.journalofvision.org/content/7/14/4, doi:10.1167/7.14.4. [PubMed] [Article].
61. Tatler, B., Baddeley, R., & Gilchrist, I. (2005). Visual correlates of fixation selection: Effects of scale and time. Vision Research, 45(5), 643-659.
62. Torralba, A., Oliva, A., Castelhano, M., & Henderson, J. M. (2006). Contextual guidance of eye move-ments and attention in real-world scenes: The role of global features in object search. Psychological Review, 113:766-786.
63. Treisman, A. M., & Gelade, G. (1980). A feature-integration theory of attention. Cognitive Psychol-ogy, 12(1), 97-136.
64. Vezhnevets, A., & Vezhnevets, V. (2005). Modest AdaBoost-Teaching AdaBoost to generalize bet-ter. In Graphicon.
65. Vincent, B. T., Baddeley, R. J., Troscianko, T., & Gilchrist, I. D. (2009). Optimal feature integration in visual search. Journal of Vision, 9(5):15, 1-11, http://www.journalofvision.org/content/9/5/15, doi:10.1167/9.5.15. [PubMed] [Article].
66. Viola, P., & Jones, M. (2001). Rapid object detection using a boosted cascade of simple features. In IEEE Conference on Computer Vision and Pattern Recognition (pp. I:511-518). Washington, DC: IEEE Computer Society.
67. Vitu, F., Kapoula, Z., Lancelin, D., & Lavigne, F. (2004). Eye movements in reading isolated words: Evidence for strong biases towards the center of the screen. Vision Research, 44(3), 321-338.
68. Walther, D., Serre, T., Poggio, T., & Koch, C. (2005). Modeling feature sharing between object detection and topdown attention. Journal of Vision, 5(8): 1041-1041, http://www.journalofvision.org/ content/5/8/1041, doi:10.1167/5.8.1041. [Abstract].
69. Wang, W., Wang, Y., Huang, Q., & Gao, W. (2010). Measuring visual saliency by site entropy rate. In IEEE Conference on Computer Vision and Pattern Recognition (pp. 2368-2375). Washington, DC: IEEE Computer Society.
70. Zambarbieri, D., Beltrami, G., & Versino, M. (1995). Saccade latency toward auditory targets depends on the relative position of the sound source with respect to the eyes. Vision Research, 35(23-24), 3305-3312.
71. Zhang, L., Tong, M., & Cottrell, G. (2009). SUNDay: Saliency using natural statistics for dynamic anal-ysis of scenes. In Proceedings of the 31st Annual Cognitive Science Conference (pp. 2944-2949).
72. Zhang, L., Tong, M., Marks, T., Shan, H., & Cottrell, G. (2008). SUN: A Bayesian framework for saliency using natural statistics. Journal of Vision, 8(7):32, 1-20, http://www.journalofvision.org/ content/8/7/32, doi:10.1167/8.7.32. [PubMed] [Article].
73. Zhao, Q., & Koch, C. (2011). Learning a saliency map using fixated locations in natural scenes. Journal of Vision, 11(3):9, 1-15, http://www.journalofvision. org/content/11/3/9, doi:10.1167/11.3.9. [PubMed] [Article].