Temporal eye movement strategies during naturalistic viewing

Journal of Vision

Volumn 12, Issue 1, 2012, Pages 1-27

  Wang, Helena X ^a   Freeman, Jeremy ^a   Merriam, Elisha P ^a   Hasson, Uri ^a   Heeger, David J ^a  

^a NEW YORK UNIVERSITY   (United States)

Author keywords

Eye movement; Naturalistic viewing; Point of interest

Indexed keywords

ADULT; ARTICLE; ATTENTION; AUDIOVISUAL EQUIPMENT; BIOLOGICAL MODEL; EYE MOVEMENT; HUMAN; METHODOLOGY; MIDDLE AGED; PHOTOSTIMULATION; PHYSIOLOGY; REACTION TIME; REPRODUCIBILITY; VISION; YOUNG ADULT;

ADULT; ATTENTION; EYE MOVEMENTS; HUMANS; MIDDLE AGED; MODELS, NEUROLOGICAL; MOTION PICTURES AS TOPIC; PHOTIC STIMULATION; REACTION TIME; REPRODUCIBILITY OF RESULTS; VISUAL PERCEPTION; YOUNG ADULT;

EID: 84862066639     PISSN: None     EISSN: 15347362     Source Type: Journal    
DOI: 10.1167/12.1.16     Document Type: Article

References (100)

1. Andrews, T. J., & Coppola, D. M. (1999). Idiosyncratic characteristics of saccadic eye movements when viewing different visual environments. Vision Research, 39, 2947-2953.
2. Angelone, B. L., Levin, D. T., & Simons, D. J. (2003). The relationship between change detection and recognition of centrally attended objects in motion pictures. Perception, 32, 947-962.
3. Ballard, D. H., & Hayhoe, M. M. (2009). Modelling the role of task in the control of gaze. Visual Cognition, 17, 1185-1204.
4. Birmingham, E., Bischof, W. F., & Kingstone, A. (2008). Gaze selection in complex social scenes. Visual Cognition, 16, 341-355.
5. Bordwell, D., & Thompson, K. (2001). Film art: An introduction (6th ed.). New York: McGraw-Hill.
6. Brainard, D. H. (1997). The Psychophysics Toolbox. Spatial Vision, 10, 433-436.
7. Buswell, G. T. (1935). How people look at pictures. Chicago: University of Chicago Press.
8. Butko, N. J., Zhang, L., Cottrell, G. W., & Movellan, J. R. (2008). Visual saliency model for robot cameras. Proceedings of the 2008 International Conference on Robotics and Automation (ICRA), 2398-2403.
9. Carmi, R., & Itti, L. (2006a). The role of memory in guiding attention during natural vision. Journal of Vision, 6(9):4, 898-914, http://www.journalofvision. org/content/6/9/4, doi:10.1167/6.9.4.
10. Carmi, R., & Itti, L. (2006b). Visual causes versus correlates of attentional selection in dynamic scenes. Vision Research, 46, 4333-4345.
11. Carroll, J. M., & Bever, T. G. (1976). Segmentation in cinema perception. Science, 191, 1053-1055.
12. Caspi, A., Beutter, B. R., & Eckstein, M. P. (2004). The time course of visual information accrual guiding eye movement decisions. Proceedings of the National Academy of Sciences of the United States of America, 101, 13086-13090.
13. Dmytryk, E. (1986). On filmmaking. London: Focal Press. Dorr, M., Martinetz, T., Gegenfurtner, K. R., & Barth, E. (2010). Variability of eye movements when viewing dynamic natural scenes. Journal of Vision, 10(10):28, 1-17, http://www.journalofvision.org/content/10/10/28, doi:10.1167/10.10.28.
14. d'Ydewalle, G., Desmet, G., & Van Rensbergen, J. (1998). Film perception: The processing of film cuts. In G. Underwood (Ed.), Eye guidance in reading and scene perception (pp. 357-367). Oxford, UK: Elsevier.
15. d'Ydewalle, G., & Vanderbeeken, M. (1990). Perceptual and cognitive processing of editing rules in film. In R. Groner, G. d'Ydewalle, & R. Parhani (Eds.), From eye to mind: Information acquisition in perception, search, and reading (pp. 129-139). Amsterdam, The Netherlands: Elsevier.
16. Eckstein, M. P., Drescher, B. A., & Shimozaki, S. S. (2006). Attentional cues in real scenes, saccadic targeting, and Bayesian priors. Psychological Science, 17, 973-980.
17. Einhauser,W.,&Konig, P. (2003). Does luminance-contrast contribute to a saliency map for overt visual attention? European Journal of Neuroscience, 17, 1089-1097.
18. Einhäuser, W., Spain, M., & Perona, P. (2008). Objects predict fixations better than early saliency. Journal of Vision, 8(14):18, 1-26, http://www.journalofvision. org/content/8/14/18, doi:10.1167/8.14.18.
19. Fenton, L. F. (1960). The sum of log-normal probability distributions in scatter transmission systems. IRE Transactions on Communications Systems, 8, 57-67.
20. Friesen, C. K., & Kingstone, A. (1998). The eyes have it! Reflexive orienting is triggered by nonpredictive gaze. Psychonomic Bulletin & Review, 5, 490-495.
21. Frith, U., & Robson, J. E. (1975). Perceiving the language of films. Perception, 4, 97-103.
22. Garsoffky, B., Huff, M., & Schwan, S. (2007). Changing viewpoints during dynamic events. Perception, 36, 366-374.
23. Garsoffky, B., Schwan, S., & Hesse, F. W. (2002). Viewpoint dependency in the recognition of dynamic scenes. Journal of Experimental Psychology: Learning, Memory, and Cognition, 28, 1035-1050.
24. Germeys, F., & d'Ydewalle, G. (2007). The psychology of film: Perceiving beyond the cut. Psychological Research, 71, 458-466.
25. Goldstein, R. B., Woods, R. L., & Peli, E. (2007). Where people look when watching movies: Do all viewers look at the same place? Computers in Biology and Medicine, 37, 957-964.
26. Grimes, J. (1996). On the failure to detect changes in scenes across saccades. In K. A. Akins (Ed.), Perception, Vancouver studies in cognitive science (vol. 5, pp. 89-110). New York: Oxford University Press.
27. Hasson, U., Landesman, O., Knappmeyer, B., Vallines, I., Rubin, N., & Heeger, D. (2008). Neurocinematics: The neuroscience of film. Projections, 2, 1-26.
28. Hasson, U., Malach, R., & Heeger, D. J. (2010). Reliability of cortical activity during natural stimulation. Trends in Cognitive Sciences, 14, 40-48.
29. Hasson, U., Nir, Y., Levy, I., Fuhrmann, G., & Malach, R. (2004). Intersubject synchronization of cortical activity during natural vision. Science, 303, 1634-1640.
30. Hasson, U., Yang, E., Vallines, I., Heeger, D. J.,& Rubin, N. (2008). A hierarchy of temporal receptive windows in human cortex. Journal of Neuroscience, 28, 2539-2550.
31. Hayhoe, M., & Ballard, D. (2005). Eye movements in natural behavior. Trends in Cognitive Sciences, 9, 188-194.
32. Henderson, J. M., Brockmole, J. R., Castelhano, M. S., & Mack, M. (2007). Image salience versus cognitive control of eye movements in real-world scenes: Evidence from visual search. In R. van Gompel, M. Fischer, W.Murray,&R. Hill (Eds.), Eye movement research: Insights into mind and brain (pp. 537-562). Oxford, UK: Elsevier.
33. Henderson, J. M., & Hollingworth, A. (1999). High-level scene perception. Annual Review of Psychology, 50, 243-271.
34. Henderson, J. M., & Hollingworth, A. (2003). Eye movements and visual memory: Detecting changes to saccade targets in scenes. Perception & Psychophysics, 65, 58-71.
35. Henderson, J. M., Weeks, P. A., Jr., & Hollingworth, A. (1999). The effects of semantic consistency on eye movements during complex scene viewing. Journal of Experimental Psychology: Human Perception and Performance, 25, 210-228.
36. Hirose, Y., Kennedy, A., & Tatler, B. W. (2010). Perception and memory across viewpoint changes in moving images. Journal of Vision, 10(4):2, 1-19, http://www.journalofvision.org/content/10/4/2, doi:10.1167/10.4.2.
37. Hochberg, J., & Brooks, V. (1978). Film cutting and visual momentum. In J. W. Senders, D. F. Fisher, & R. A. Monty (Eds.), Eye movements and the higher psychological functions (pp. 293-313). Hillsdale, NJ: Erlbaum.
38. Hollingworth, A., & Henderson, J. (2002). Accurate visual memory for previously attended objects in natural scenes. Journal of Experimental Psychology: Human Perception and Performance, 28, 113-136.
39. Irwin, D. E., & Zelinsky, G. J. (2002). Eye movements and scene perception: Memory for things observed. Perception & Psychophysics, 64, 882-895.
40. Itti, L. (2005). Quantifying the contribution of low-level saliency to human eye movements in dynamic scenes. Visual Cognition, 12, 1093-1123.
41. Itti, L., & Baldi, P. (2005). A principled approach to detecting surprising events in video. Proceedings of the 2005 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 1, 631-637.
42. Itti, L., & Baldi, P. (2009). Bayesian surprise attracts human attention. Vision Research, 49, 1295-1306.
43. Itti, L., & Koch, C. (2001). Computational modelling of visual attention. Nature Reviews Neuroscience, 2, 194-203.
44. Itti, L., Koch, C., & Niebur, E. (1998). A model of saliency-based visual attention for rapid scene analysis. IEEE Transactions on Pattern Analysis and Machine Intelligence, 20, 1254-1259.
45. Kanan, C., Tong, M., Zhang, L., & Cottrell, G. (2009). SUN: Top-down saliency using natural statistics. Visual Cognition, 17, 979-1003.
46. Kienzle, W., Schölkopf, B., Wichmann, F. A.,&Franz, M. O. (2007). How to find interesting locations in video: A spatiotemporal interest point detector learned from human eye movements. Proceedings of the 29th DAGM Conference on Pattern Recognition, LNCS 4713, 405-414.
47. Koch, C., & Ullman, S. (1985). Shifts in selective visual attention: Towards the underlying neural circuitry. Human Neurobiology, 4, 219-227.
48. Kraft, R. N. (1986). The role of cutting in the evaluation and retention of film. Journal of Experimental Psychology: Learning, Memory, and Cognition, 12, 155-163.
49. Kraft, R. N. (1987). Rules and strategies of visual narratives. Perceptual and Motor Skills, 64, 3-14.
50. Krieger, G., Rentschler, I., Hauske, G., Schill, K., & Zetzsche, C. (2000). Object and scene analysis by saccadic eye-movements: An investigation with higher-order statistics. Spatial Vision, 13, 201-214.
51. Land, M. F. (2009). Vision, eye movements, and natural behavior. Visual Neuroscience, 26, 51-62.
52. Land, M. F., & Hayhoe, M. (2001). In what ways do eye movements contribute to everyday activities? Vision Research, 41, 3559-3565.
53. Le Meur, O., Le Callet, P., & Barba, D. (2007). Predicting visual fixations on video based on low-level visual features. Vision Research, 47, 2483-2498.
54. Levin, D. T., & Simons, D. J. (1997). Failure to detect changes to attended objects in motion pictures. Psychonomic Bulletin & Review, 4, 501-506.
55. Levin, D. T., & Simons, D. J. (2000). Perceiving stability in a changing world: Combining shots and integrating views in motion pictures and the real world. Media Psychology, 2, 357-380.
56. Mack, A., & Rock, I. (1998). Inattentional blindness. Cambridge, MA: MIT Press.
57. MacLaclan, J., & Logan, M. (1993). Camera shot length in TV commercials and their memorability and persuasiveness. Journal of Advertising Research, 33, 57-61.
58. Mannan, S., Ruddock, K. H., & Wooding, D. S. (1995). Automatic control of saccadic eye movements made in visual inspection of briefly presented 2-D images. Spatial Vision, 9, 363-386.
59. May, J., Dean, M., & Barnard, P. (2003). Using film cutting techniques in interface design. Human-Computer Interaction, 18, 325-372.
60. McConkie, G. W., & Currie, C. B. (1996). Visual stability across saccades while viewing complex pictures. Journal of Experimental Psychology: Human Perception and Performance, 22, 563-581.
61. Melcher, D. (2006). Accumulation and persistence of memory for natural scenes. Journal of Vision, 6(1):2, 8-17, http://www.journalofvision.org/content/6/1/2, doi:10.1167/6.1.2.
62. Melcher, D., & Kowler, E. (2001). Visual scene memory and the guidance of saccadic eye movements. Vision Research, 41, 3597-3611.
63. Najemnik, J., & Geisler, W. S. (2005). Optimal eye movement strategies in visual search. Nature, 434, 387-391.
64. Navalpakkam, V., & Itti, L. (2005). Modeling the influence of task on attention. Vision Research, 45, 205-231.
65. Neider, M. B., & Zelinsky, G. J. (2006). Scene context guides eye movements during visual search. Vision Research, 46, 614-621.
66. Noton, D., & Stark, L. (1971). Scanpaths in saccadic eye movements while viewing and recognizing patterns. Vision Research, 11, 929-942.
67. Oliva, A., Torralba, A., Castelhano, M.,& Henderson, J. M. (2003). Top-down control of visual attention in object detection. Proceedings of the 2003 International Conference on Image Processing (ICIP), 1, I-253-I-256.
68. Orban, G. A. (2008). Higher order visual processing in macaque extrastriate cortex. Physiological Reviews, 88, 59-89.
69. O'Regan, J. K. (1992). Solving the " real" mysteries of visual perception: The world as an outside memory. Canadian Journal of Psychology, 46, 461-488.
70. Parkhurst, D., Law, K., & Niebur, E. (2002). Modeling the role of salience in the allocation of overt visual attention. Vision Research, 42, 107-123.
71. Parkhurst, D. J., & Niebur, E. (2003). Scene content selected by active vision. Spatial Vision, 16, 125-154.
72. Pelli, D. G. (1997). The VideoToolbox software for visual psychophysics: Transforming numbers into movies. Spatial Vision, 10, 437-442.
73. Peters, R. J., & Itti, L. (2007). Beyond bottom-up: Incorporating task-dependent influences into a computational model of spatial attention. Proceedings of the 2007 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 1-8.
74. Peters, R. J., Iyer, A., Itti, L., & Koch, C. (2005). Components of bottom-up gaze allocation in natural images. Vision Research, 45, 2397-2416.
75. Rajashekar, U., van der Linde, I., Bovik, A. C., & Cormack, L. K. (2007). Foveated analysis of image features at fixations. Vision Research, 47, 3160-3172.
76. Reinagel, P., & Zador, A. M. (1999). Natural scene statistics at the centre of gaze. Network, 10, 341-350.
77. Reisz, K., & Millar, G. (1953). Technique of film editing. London: Focal Press.
78. Rensink, R. A., O'Regan, J. K., & Clark, J. J. (1997). To see or not to see: The need for attention to perceive changes in scenes. Psychological Science, 8, 368-373.
79. Rothkopf, C. A., Ballard, D. H., & Hayhoe, M. M. (2007). Task and context determine where you look. Journal of Vision, 7(14):16, 1-20, http://www.journalofvision. org/content/7/14/16, doi:10.1167/7.14.16.
80. Salt, B. (1992). Film style and technology: History and analysis (2nd ed.). London: Starword.
81. Scholl, B. J. (2001). Objects and attention: The state of the art. Cognition, 80, 1-46.
82. Schwan, S., Garsoffky, B., & Hesse, F. W. (2000). Do film cuts facilitate the perceptual and cognitive organization of activity sequences? Memory & Cognition, 28, 214-223.
83. Shepherd, S. V., Steckenfinger, S. A., Hasson, U., & Ghazanfar, A. A. (2010). Human-monkey gaze correlations reveal convergent and divergent patterns of movie viewing. Current Biology, 20, 649-656.
84. Smith, T. J., & Henderson, J. M. (2008). The relationship between attention and global change blindness in dynamic scenes. Journal of Eye Movement Research, 2, 1-17.
85. Sun, Y., Fisher, R., Wang, F., & Gomes, H. M. (2008). A computer vision model for visual-object-based attention and eye movements. Computer Vision and Image Understanding, 112, 126-142.
86. Tatler, B., Gilchrist, I., & Land, M. (2005). Visual memory for objects in natural scenes: From fixations to object files. Quarterly Journal of Experimental Psychology, 58A, 931-960.
87. Tatler, B. W. (2007). The central fixation bias in scene viewing: Selecting an optimal viewing position independently of motor biases and image feature distributions. Journal of Vision, 7(14):4, 1-17, http://www.journalofvision.org/content/7/14/4, doi:10.1167/ 7.14.4.
88. Tatler, B. W., Baddeley, R. J., & Gilchrist, I. D. (2005). Visual correlates of fixation selection: Effects of scale and time. Vision Research, 45, 643-659. Tatler, B. W., Gilchrist, I. D., & Rusted, J. (2003). The time course of abstract visual representation. Perception, 32, 579-592.
89. 't Hart, B. M., Vockeroth, J., Schumann, F., Bartl, K., Schneider, E., König, P., et al. (2009). Gaze allocation in natural stimuli: Comparing free exploration to head-fixed viewing conditions. Visual Cognition, 17, 1132-1158.
90. Torralba,A., Oliva, A.,Castelhano,M. S.,&Henderson, J.M. (2006). Contextual guidance of eye movements and attention in real-world scenes: The role of global features in object search. Psychological Review, 113, 766-786.
91. Tosi, V., Mecacci, L., & Pasquali, E. (1997). Scanning eye movements made when viewing film: Preliminary observations. International Journal of Neuroscience, 92, 47-52.
92. Treisman, A. M., & Gelade, G. (1980). A featureintegration theory of attention. Cognitive psychology, 12, 97-136.
93. Tseng, P. H., Carmi, R., Cameron, I. G. M., Munoz, D. P., & Itti, L. (2009). Quantifying center bias of observers in free viewing of dynamic natural scenes. Journal of Vision, 9(7):4, 1-16, http://www.journalofvision.org/ content/9/7/4, doi:10.1167/9.7.4.
94. Turano, K. A., Geruschat, D. R., & Baker, F. H. (2003). Oculomotor strategies for the direction of gaze tested with a real-world activity. Vision Research, 43, 333-346.
95. Vig, E., Dorr, M., & Barth, E. (2009). Efficient visual coding and the predictability of eye movements on natural movies. Spatial Vision, 22, 397-408.
96. Wischnewski, M., Belardinelli, A., Schneider, W. X., & Steil, J. J. (2010). Where to look next? Combining static and dynamic proto-objects in a TVA-based model of visual attention. Cognitive Computation, 2, 326-343.
97. Yamada, K., & Cottrell, G. W. (1995). A model of scan paths applied to face recognition. Proceedings of the 17th Annual Conference of the Cognitive Science Society, 55-60.
98. Yantis, S., & Jonides, J. (1984). Abrupt visual onsets and selective attention: Evidence from visual search. Journal of Experimental Psychology: Human Perception and Performance, 10, 601-621.
99. Yarbus, A. (1967). Eye movements and vision. New York: Plenum Press.
100. Zhang, L., Tong, M. H., & Cottrell, G. W. (2009). SUNDAy: Saliency using natural statistics for dynamic analysis of scenes. Proceedings of the 31st Annual Conference of the Cognitive Science Society, 2944-2949.