Reconciling Saliency and Object Center-Bias Hypotheses in Explaining Free-Viewing Fixations

IEEE Transactions on Neural Networks and Learning Systems

Volumn 27, Issue 6, 2016, Pages 1214-1226

  Borji, Ali ^a   Tanner, James ^b  

^a University of Wisconsin   (United States)

^b University of Southern California   (United States)

Author keywords

Bottom up attention; center bias; eye movements; free viewing; object center bias; object saliency; object based attention; saliency; space based attention; Visual attention

Indexed keywords

COMPUTER VISION; SEMANTICS;

COMBINED MODEL; COMPUTATIONAL NEUROSCIENCE; HIGH LEVEL SEMANTICS; IMAGE SALIENCIES; INDIVIDUAL COMPONENTS; LINE DRAWINGS; NATURAL SCENES; SEMANTIC IMAGES;

EYE MOVEMENTS;

EID: 84943411248     PISSN: 2162237X     EISSN: 21622388     Source Type: Journal    
DOI: 10.1109/TNNLS.2015.2480683     Document Type: Article

References (78)

1. J. M. Henderson, "Eye movement control during visual object processing: Effects of initial fixation position and semantic constraint," Can. J. Experim. Psychol., vol. 47, no. 1, pp. 79-98, 1993.
2. A. Nuthmann and J. M. Henderson, "Object-based attentional selection in scene viewing," J. Vis., vol. 10, no. 8, p. 20, 2010.
3. J. Xu, M. Jiang, S. Wang, M. S. Kankanhalli, Q. Zhao, "Predicting human gaze beyond pixels," J. Vis., vol. 14, no. 1, p. 28, 2014.
4. A. Borji and L. Itti, "State-of-the-art in visual attention modeling," IEEE Trans. Pattern Anal. Mach. Intell., vol. 35, no. 1, pp. 185-207, Jan. 2013.
5. M. Hayhoe and D. Ballard, "Eye movements in natural behavior," Trends Cognit. Sci., vol. 9, no. 4, pp. 188-194, 2005.
6. L. Itti and C. Koch, "Computational modelling of visual attention," Nature Rev. Neurosci., vol. 2, no. 3, pp. 194-203, 2001.
7. M. F. Land and M. Hayhoe, "In what ways do eye movements contribute to everyday activities" Vis. Res., vol. 41, nos. 25-26, pp. 3559-3565, Nov./Dec. 2001.
8. V. Navalpakkam and L. Itti, "Modeling the influence of task on attention," Vis. Res., vol. 45, no. 2, pp. 205-231, 2005.
9. A. C. Schütz, D. I. Braun, K. R. Gegenfurtner, "Eye movements and perception: A selective review," J. Vis., vol. 11, no. 5, p. 9, 2011.
10. B. W. Tatler, M. M. Hayhoe, M. F. Land, D. H. Ballard, "Eye guidance in natural vision: Reinterpreting salience," J. Vis., vol. 11, no. 5, p. 5, 2011.
11. F. Baluch and L. Itti, "Mechanisms of top-down attention," Trends Neurosci., vol. 34, no. 4, pp. 210-224, 2011.
12. A. Borji, "Boosting bottom-up and top-down visual features for saliency estimation," in Proc. IEEE Conf. Comput. Vis. Pattern Recognit. (CVPR), Providence, RI, USA, Jun. 2012, pp. 438-445.
13. A. Borji, M.-M. Cheng, H. Jiang, J. Li. (Nov. 2014). "Salient object detection: A survey." [Online]. Available: http://arxiv.org/abs/1411.5878
14. A. Borji, D. N. Sihite, L. Itti, "Quantitative analysis of human-model agreement in visual saliency modeling: A comparative study," IEEE Trans. Image Process., vol. 22, no. 1, pp. 55-69, Jan. 2012.
15. M. Cerf, E. P. Frady, C. Koch, "Faces and text attract gaze independent of the task: Experimental data and computer model," J. Vis., vol. 9, no. 12, p. 10, Nov. 2009.
16. T. Judd, K. Ehinger, F. Durand, A. Torralba, "Learning to predict where humans look," in Proc. IEEE 12th Int. Conf. Comput. Vis. (ICCV), Kyoto, Japan, Sep./Oct. 2009, pp. 2106-2113.
17. K. Humphrey and G. Underwood, "The potency of people in pictures: Evidence from sequences of eye fixations," J. Vis., vol. 10, no. 10, p. 19.
18. H.-C. Wang and M. Pomplun, "The attraction of visual attention to texts in real-world scenes," J. Vis., vol. 12, no. 6, p. 26, 2012.
19. B. W. Tatler, "The central fixation bias in scene viewing: Selecting an optimal viewing position independently of motor biases and image feature distributions," J. Vis., vol. 7, no. 14, p. 4, 2007.
20. P.-H. Tseng, R. Carmi, I. G. M. Cameron, D. P. Munoz, L. Itti, "Quantifying center bias of observers in free viewing of dynamic natural scenes," J. Vis., vol. 9, no. 7, p. 4, 2009.
21. J. P. Ossandón, S. Onat, P. König, "Spatial biases in viewing behavior," J. Vis., vol. 14, no. 2, p. 20, 2014.
22. A. D. Hwang, H.-C. Wang, M. Pomplun, "Semantic guidance of eye movements in real-world scenes," Vis. Res., vol. 51, no. 10, pp. 1192-1205, 2011.
23. A. Torralba, A. Oliva, M. S. Castelhano, J. M. Henderson, "Contextual guidance of eye movements and attention in real-world scenes: The role of global features in object search," Psychol. Rev., vol. 113, no. 4, pp. 766-786, 2006.
24. R. Subramanian, D. Shankar, N. Sebe, D. Melcher, "Emotion modulates eye movement patterns and subsequent memory for the gist and details of movie scenes," J. Vis., vol. 14, no. 3, p. 31, 2014.
25. J. A. Droll, M. M. Hayhoe, J. Triesch, B. T. Sullivan, "Task demands control acquisition and storage of visual information," J. Experim. Psychol., Human Perception Perform., vol. 31, no. 6, pp. 1416-1438, 2005.
26. R. Carmi and L. Itti, "The role of memory in guiding attention during natural vision," J. Vis., vol. 6, no. 9, p. 4, 2006.
27. M. S. Castelhano, M. Wieth, J. M. Henderson, "I see what you see: Eye movements in real-world scenes are affected by perceived direction of gaze," in Attention in Cognitive Systems. Theories and Systems From an Interdisciplinary Viewpoint. Berlin, Germany: Springer-Verlag, 2007, pp. 251-262.
28. A. Borji, D. Parks, L. Itti, "Complementary effects of gaze direction and early saliency in guiding fixations during free viewing," J. Vis., vol. 14, no. 13, p. 3, 2014.
29. H. F. Chua, J. E. Boland, R. E. Nisbett, "Cultural variation in eye movements during scene perception," Proc. Nat. Acad. Sci. USA, vol. 102, no. 35, pp. 12629-12633, 2005.
30. K. J. Friston, G. Tononi, G. N. Reeke, Jr., O. Sporns, G. M. Edelman, "Value-dependent selection in the brain: Simulation in a synthetic neural model," Neuroscience, vol. 59, no. 2, pp. 229-243, 1994.
31. J. Shen and L. Itti, "Top-down influences on visual attention during listening are modulated by observer sex," Vis. Res., vol. 65, pp. 62-76, Jul. 2012.
32. A. L. Yarbus, Eye Movements and Vision. New York, NY, USA: Plenum, 1967.
33. M. F. Land and D. N. Lee, "Where we look when we steer," Nature, vol. 369, pp. 742-744, Jun. 1994.
34. D. H. Ballard, M. M. Hayhoe, J. B. Pelz, "Memory representations in natural tasks," J. Cognit. Neurosci., vol. 7, no. 1, pp. 66-80, 1995.
35. A. Borji, D. N. Sihite, L. Itti, "What/where to look next Modeling top-down visual attention in complex interactive environments," IEEE Trans. Syst., Man, Cybern., Syst., vol. 44, no. 5, pp. 523-538, May 2014.
36. A. Borji and L. Itti, "Defending Yarbus: Eye movements reveal observers' task," J. Vis., vol. 14, no. 3, p. 29, 2014.
37. A. Borji, A. Lennartz, M. Pomplun, "What do eyes reveal about the mind Algorithmic inference of search targets from fixations," Neurocomputing, vol. 149, pp. 788-799, Feb. 2015.
38. S. N. Hajimirza, M. J. Proulx, E. Izquierdo, "Reading users' minds from their eyes: A method for implicit image annotation," IEEE Trans. Multimedia, vol. 14, no. 3, pp. 805-815, Jun. 2012.
39. A. M. Treisman and G. Gelade, "A feature-integration theory of attention," Cognit. Psychol., vol. 12, no. 1, pp. 97-136, 1980.
40. C. Koch and S. Ullman, "Shifts in selective visual attention: Towards the underlying neural circuitry," Human Neurobiol., vol. 4, no. 4, pp. 219-227, 1985.
41. L. Itti, C. Koch, E. Niebur, "A model of saliency-based visual attention for rapid scene analysis," IEEE Trans. Pattern Anal. Mach. Intell., vol. 20, no. 11, pp. 1254-1259, Nov. 1998.
42. P. Reinagel and A. M. Zador, "Natural scene statistics at the center of gaze," Netw., Comput. Neural Syst., vol. 10, no. 4, pp. 341-350, 1999.
43. D. Parkhurst, K. Law, E. Niebur, "Modeling the role of salience in the allocation of overt visual attention," Vis. Res., vol. 42, no. 1, pp. 107-123, 2002.
44. A. Borji and L. Itti, "Exploiting local and global patch rarities for saliency detection," in Proc. IEEE Conf. Comput. Vis. Pattern Recognit., Providence, RI, USA, Jun. 2012, pp. 478-485.
45. M. M. Hayhoe, A. Shrivastava, R. Mruczek, J. B. Pelz, "Visual memory and motor planning in a natural task," J. Vis., vol. 3, no. 1, pp. 49-63, 2003.
46. W. Einhaüser, M. Spain, P. Perona, "Objects predict fixations better than early saliency," J. Vis., vol. 8, no. 14, p. 18, 2008.
47. A. Borji et al., "Objects do not predict fixations better than early saliency: A re-analysis of Einhaüser et al.'s data," J. Vis., vol. 13, no. 10, p. 18, 2013.
48. H. A. Trukenbrod and R. Engbert, "Oculomotor control in a sequential search task," Vis. Res., vol. 47, no. 18, pp. 2426-2443, 2007.
49. M. Pajak and A. Nuthmann, "Object-based saccadic selection during scene perception: Evidence from viewing position effects," J. Vis., vol. 13, no. 5, p. 2, 2013.
50. R. A. Rensink, "The dynamic representation of scenes," Vis. Cognit., vol. 7, nos. 1-3, pp. 17-42, 2000.
51. K. Rayner, S. P. Liversedge, A. Nuthmann, R. Kliegl, G. Underwood, "Rayner's 1979 paper," Perception, vol. 38, no. 6, pp. 895-906, 2009.
52. L. Elazary and L. Itti, "Interesting objects are visually salient," J. Vis., vol. 8, no. 3, p. 3, Mar. 2008.
53. A. Belardinelli and M. V. Butz, "Gaze strategies in object identification and manipulation," in Proc. 35th Annu. Meeting Cognit. Sci. Soc., Berlin, Germany, 2013, pp. 319-341.
54. T. Liu et al., "Learning to detect a salient object," IEEE Trans. Pattern Anal. Mach. Intell., vol. 33, no. 2, pp. 353-367, Feb. 2011.
55. M. Dziemianko, A. Clarke, F. Keller, "Object-based saliency as a predictor of attention in visual tasks," in Proc. 34th Annu. Conf. Cognit. Sci. Soc., Berlin, Germany, 2013, pp. 2237-2242.
56. A. Borji, D. N. Sihite, L. Itti, "What stands out in a scene A study of human explicit saliency judgment," Vis. Res., vol. 91, pp. 62-77, Oct. 2013.
57. K. Yun, Y. Peng, D. Samaras, G. J. Zelinsky, T. L. Berg, "Exploring the role of gaze behavior and object detection in scene understanding," Frontiers Psychol., vol. 4, p. 917, Dec. 2013.
58. Y. Sun, R. Fisher, F. Wang, H. M. Gomes, "A computer vision model for visual-object-based attention and eye movements," Comput. Vis. Image Understand., vol. 112, no. 2, pp. 126-142, 2008.
59. K.-Y. Chang, T.-L. Liu, H.-T. Chen, S.-H. Lai, "Fusing generic objectness and visual saliency for salient object detection," in Proc. IEEE Int. Conf. Comput. Vis., Barcelona, Spain, Nov. 2011, pp. 914-921.
60. B.M. Hart, H. C. E. F. Schmidt, C. Roth, W. Einhaüser, "Fixations on objects in natural scenes: Dissociating importance from salience," Frontiers Psychol., vol. 4, p. 455, Jul. 2013.
61. Y. Kavak, E. Erdem, A. Erdem. (Jul. 2013). "Visual saliency estimation by integrating features using multiple kernel learning." [Online]. Available: http://arxiv.org/abs/1307.5693
62. V. Yanulevskaya, J. Uijlings, J.-M. Geusebroek, N. Sebe, A. Smeulders, "A proto-object-based computational model for visual saliency," J. Vis., vol. 13, no. 13, p. 27, 2013.
63. J. Stoll, M. Thrun, A. Nuthmann, W. Einhaüser, "Overt attention in natural scenes: Objects dominate features," Vis. Res., vol. 107, pp. 36-48, Feb. 2015.
64. A. Garcia-Diaz, V. Leborán, X. R. Fdez-Vidal, X. M. Pardo, "On the relationship between optical variability, visual saliency, eye fixations: A computational approach," J. Vis., vol. 12, no. 6, p. 17, 2012.
65. R. J. Peters, A. Iyer, L. Itti, C. Koch, "Components of bottom-up gaze allocation in natural images," Vis. Res., vol. 45, no. 8, pp. 2397-2416, 2005.
66. M. O. Hartendorp, S. Van der Stigchel, I. Hooge, J. Mostert, T. de Boer, A. Postma, "The relation between gaze behavior and categorization: Does where we look determine what we see" J. Vis., vol. 13, no. 6, p. 6, 2013.
67. G. Kootstra, B. de Boer, L. R. B. Schomaker, "Predicting eye fixations on complex visual stimuli using local symmetry," Cognit. Comput., vol. 3, no. 1, pp. 223-240, 2011.
68. M. R. Greene, "Statistics of high-level scene context," Frontiers Psychol., vol. 4, p. 777, Oct. 2013.
69. M.-M. Cheng, N. J. Mitra, X. Huang, S.-M. Hu, "SalientShape: Group saliency in image collections," Vis. Comput., vol. 30, no. 4, pp. 443-453, 2014.
70. Y. Li, X. Hou, C. Koch, J. M. Rehg, A. L. Yuille, "The secrets of salient object segmentation," in Proc. IEEE Conf. Comput. Vis. Pattern Recognit., Columbus, OH, USA, Jun. 2014, pp. 280-287.
71. J. Duncan, "Selective attention and the organization of visual information," J. Experim. Psychol., General, vol. 113, no. 4, pp. 501-517, 1984.
72. R. Egly, J. Driver, R. D. Rafal, "Shifting visual attention between objects and locations: Evidence from normal and parietal lesion subjects," J. Experim. Psychol., General, vol. 123, no. 2, pp. 161-177, 1994.
73. S. P. Vecera and M. J. Farah, "Does visual attention select objects or locations" J. Experim. Psychol., General, vol. 123, no. 2, pp. 146-160, 1994.
74. L. Drummond and S. Shomstein, "Object-based attention: Shifting or uncertainty" Attention, Perception, Psychophys., vol. 72, no. 7, pp. 1743-1755, 2010.
75. J. Gottlieb and P. Balan, "Attention as a decision in information space," Trends Cognit. Sci., vol. 14, no. 6, pp. 240-248, 2010.
76. J. Harel, C. Koch, P. Perona, "Graph-based visual saliency," in Proc. Adv. Neural Inf. Process. Syst., Vancouver, BC, Canada, 2007, pp. 545-552.
77. K. Koehler, F. Guo, S. Zhang, M. P. Eckstein, "What do saliency models predict" J. Vis., vol. 14, no. 3, p. 14, 2014.
78. M. Everingham, L. Van Gool, C. K. I. Williams, J. Winn, A. Zisserman, "The PASCAL visual object classes (VOC) challenge," Int. J. Comput. Vis., vol. 88, no. 2, pp. 303-338, 2010.