Eye movement prediction and variability on natural video data sets

Visual Cognition

Volumn 20, Issue 4-5, 2012, Pages 495-514

  Dorr, Michael ^a,b   Vig, Eleonora ^a   Barth, Erhardt ^a  

^a UNIVERSITY OF LÜBECK   (Germany)

^b SCHEPENS EYE RESEARCH INSTITUTE   (United States)

Author keywords

Dynamic natural scenes; Eye movement variability; Intrinsic dimension; Saliency; Structure tensor

Indexed keywords

ALGORITHM; ARTICLE; AUDIOVISUAL EQUIPMENT; EYE MOVEMENT; GAZE; GEOMETRY; HUMAN; MACHINE LEARNING; PREDICTION; PRIORITY JOURNAL; STIMULUS RESPONSE; VIDEORECORDING;

EID: 84862199923     PISSN: 13506285     EISSN: 14640716     Source Type: Journal    
DOI: 10.1080/13506285.2012.667456     Document Type: Article

References (55)

1. Barth, E., Caelli, T. and Zetzsche, C. 1993. Image encoding, labeling, and reconstruction from differential geometry. CVGIP: Graphical Models and Image Processing, 55(6): 428-446.
2. Barth, E., Dorr, M., Böhme, M., Gegenfurtner, K. R., & Martinetz, T. 2006. Guiding the mind's eye: Improving communication and vision by external control of the scanpath. In B. E. Rogowitz, T. N. Pappas, & S. J. Daly, Human vision and electronic imaging XI 6057, 116-123. Bellingham, WA: Society of Photo-Optical Instrumentation Engineers
3. Barth, E., Dorr, M., Vig, E., Pomarjanschi, L. and Mota, C. 2010. Efficient coding and multiple motions. Vision Research, 50(22): 2190-2199.
4. Bruce, N. and Tsotsos, J. K. 2009. Saliency, attention, and visual search: An information theoretic approach. Journal of Vision, 9(3): 1-24.
5. Chang, C. C., & Lin, C. J. 2001. LIBSVM: A library for support vector machines [Software]. Retrieved fromhttp://www.csie.ntu.edu.tw/cjlin/libsvm
6. Dorr, M., Martinetz, T., Gegenfurtner, K. and Barth, E. 2010. Variability of eye movements when viewing dynamic natural scenes. Journal of Vision, 10(10): 1-17.
7. Drewes, J., Trommershäuser, J. and Gegenfurtner, K. R. 2011. Parallel visual search and rapid animal detection in natural scenes. Journal of Vision, 11(2): 1-21.
8. Einhäuser, W., Rutishauser, U. and Koch, C. 2008. Task-demands can immediately reverse the effects of sensory-driven saliency in complex visual stimuli. Journal of Vision, 8(2): 1-19.
9. Einhäuser, W., Spain, M. and Perona, P. 2008. Objects predict fixations better than early saliency. Journal of Vision, 8(14): 11-26.
10. Everingham, M., van Gool, L., Williams, C. K. I., Winn, J. and Zisserman, A. 2010. The Pascal Visual Object Classes (VOC) challenge. International Journal of Computer Vision, 88(2): 303-338.
11. Field, D. J. 1987. Relations between the statistics of natural images and the response profiles of cortical cells. Journal of the Optical Society of America, 4A: 2379-2394.
12. Foulsham, T., & Kingstone, A. forthcoming. Modeling the influence of central and peripheral information on saccade biases in gaze-contingent scene viewing. Visual Cognition.
13. Gao, D. and Vasconcelos, N. 2009. Decision-theoretic saliency: Computational principles, biological plausibility, and implications for neurophysiology and psychophysics. Neural Computation, 21(1): 239-271.
14. Guo, C. and Zhang, L. 2010. A novel multiresolution spatiotemporal saliency detection model and its applications in image and video compression. IEEE Transactions on Image Processing, 19(1): 185-198.
15. Hasson, U., Landesman, O., Knappmeyer, B., Vallines, I., Rubin, N. and Heeger, D. J. 2008. Neurocinematics: The neuroscience of film. Projections, 2(1): 1-26.
16. Itti, L. 2004. Automatic foveation for video compression using a neurobiological model of visual attention. IEEE Transactions on Image Processing, 13(10): 1304-1318.
17. Itti, L. 2005. Quantifying the contribution of low-level saliency to human eye movements in dynamic scenes. Visual Cognition, 12(6): 1093-1123.
18. Itti, L., & Baldi, P. 2006. Bayesian surprise attracts human attention. In Advances in neural information processing systems, 19 (NIPS 2005) 547-554. Cambridge, MA: MIT Press.
19. Itti, L., Koch, C. and Niebur, E. 1998. A model of saliency-based visual attention for rapid scene analysis. IEEE Transactions on Pattern Analysis and Machine Intelligence, 20(11): 1254-1259.
20. Jähne, B. 1999. Local structure. In B. Jähne & H. Haußecker, Handbook of computer vision and applications 2, 209-238. San Diego, CA: Academic Press.
21. Judd, T., Ehinger, K., Durand, F., & Torralba, A. 2009. Learning to predict where humans look. In Proceedings of IEEE international conference on Computer Vision (ICCV) 2106 2113. Los Alamitos, CA: IEEE Computer Society.
22. Kienzle, W., Franz, M. O., Schölkopf, B. and Wichmann, F. A. 2009. Center-surround patterns emerge as optimal predictors for human saccade targets. Journal of Vision, 9(5): 1-15.
23. Kienzle, W., Schölkopf, B., Wichmann, F. A., & Franz, M. O. 2007. How to find interesting locations in video: A spatiotemporal interest point detector learned from human eye movements. In Proceedings of the 29th annual symposium of the German Association for Pattern Recognition (DAGM 2007) 405-414. Berlin, Germany: Springer Verlag.
24. Kirchner, H. and Thorpe, S. 2005. Ultra-rapid object detection with saccadic eye movements: Visual processing speed revisited. Vision Research, 46: 1762-1776.
25. Koch, C. and Ullman, S. 1985. Shifts in selective visual attention: Towards the underlying neural circuitry. Human Neurobiology, 4(4): 219-227.
26. Land, M. and McLeod, P. 2000. From eye movements to actions: How batsmen hit the ball. Nature Neuroscience, 3(12): 1340-1345.
27. Land, M. F. and Hayhoe, M. 2001. In what ways do eye movements contribute to everyday activities?. Vision Research, 41: 3559-3565.
28. Land, M. F., Mennie, N. and Rusted, J. 1999. The roles of vision and eye movements in the control of activities of daily living. Perception, 28(11): 1311-1328.
29. Laptev, I., Marszalek, M., Schmid, C., & Rozenfeld, B. 2008. Learning realistic human actions from movies. In IEEE conference on Computer Vision and Pattern Recognition 1-8. Los Alamitos, CA: IEEE Computer Society
30. Le Meur, O., Le Callet, P., Barba, D. and Thoreau, D. 2006. A coherent computational approach to model bottom-up visual attention. IEEE Transactions on Pattern Analysis and Machine Intelligence, 28(5): 802-817.
31. Li, Z., Qin, S. and Itti, L. 2011. Visual attention guided bit allocation in video compression. Image and Vision Computing, 29: 1-14.
32. Mahadevan, V. and Vasconcelos, N. 2010. Spatiotemporal saliency in dynamic scenes. IEEE Transactions on Pattern Analysis and Machine Intelligence, 32: 171-177.
33. Mital, P. K., Smith, T. J., Hill, R. and Henderson, J. M. 2011. Clustering of gaze during dynamic scene viewing is predicted by motion. Cognitive Computation, 3(1): 5-24.
34. Mota, C., & Barth, E. 2000. On the uniqueness of curvature features. In G. Baratoff & H. Neumann, Dynamische Perzeption 9, 175-178. Köln, Germany: Infix Verlag.
35. Nuthmann, A., & Henderson, J. forthcoming. Using CRISP to model global characteristics of fixation durations in scene viewing and reading with a common mechanism. Visual Cognition.
36. Nyström, M. and Holmqvist, K. 2010. Effect of compressed off-line foveated video on viewing behavior and subjective quality. ACM Transactions on Multimedia Computing, Communications, and Applications, 6(1): 1-14.
37. Olshausen, B. A. and Field, D. J. 1996. Emergence of simple-cell receptive field properties by learning a sparse code for natural images. Nature, 381: 607-609.
38. Parkhurst, D. J. and Niebur, E. 2003. Scene content selected by active vision. Spatial Vision, 16(2): 125-154.
39. Peters, R. J., Iyer, A., Itti, L. and Koch, C. 2005. Components of bottom-up gaze allocation in natural images. Vision Research, 45(8): 2397-2416.
40. Poynton, C. 2003. Digital video and HDTV, San Francisco, CA: Morgan Kaufmann Publishers.
41. Reinagel, P. and Zador, A. M. 1999. Natural scene statistics at the centre of gaze. Network: Computation in Neural Systems, 10: 341-350.
42. Smith, T. J. and Henderson, J. M. 2008. Edit blindness: The relationship between attention and global change blindness in dynamic scenes. Journal of Eye Movement Research, 2(2): 1-17.
43. Tatler, B. W., Baddeley, R. J. and Gilchrist, I. D. 2005. Visual correlates of fixation selection: Effects of scale and time. Vision Research, 45: 643-659.
44. Tatler, B. W., Baddeley, R. J. and Vincent, B. T. 2006. The long and the short of it: Spatial statistics at fixation vary with saccade amplitude and task. Vision Research, 46: 1857-1862.
45. Tatler, B. W., Hayhoe, M. M., Land, M. F. and Ballard, D. H. 2011. Eye guidance in natural vision: Reinterpreting salience. Journal of Vision, 11(5): 1-23.
46. Tatler, B. W. and Vincent, B. T. 2009. The prominence of behavioural biases in eye guidance. Visual Cognition, 17(6-7): 1029-1054.
47. Vig, E., Dorr, M. and Barth, E. 2009. Efficient visual coding and the predictability of eye movements on natural movies. Spatial Vision, 22(5): 397-408.
48. Vig, E., Dorr, M., Martinetz, T. and Barth, E. 2011. Eye movements show optimal average anticipation with natural dynamic scenes. Cognitive Computation, 3(1): 79-88.
49. Vig, E., Dorr, M., Martinetz, T., & Barth, E. in press. Intrinsic dimensionality predicts the saliency of natural dynamic scenes. IEEE Transactions on Pattern Analysis and Machine Intelligence.
50. Wandell, B. A. 1995. Foundations of vision. Sunderland, MA: Sinauer Associates.
51. Wolfe, J. M. 1998. "Visual search". In Attention, Edited by: Pashler, H. 13-73. Hove, UK: Psychology Press.
52. Zelinsky, G. forthcoming. TAM: Explaining off-object fixations and central fixation tendencies as effects of population averaging during search. Visual Cognition.
53. Zetzsche, C. and Barth, E. 1990. Fundamental limits of linear filters in the visual processing of two-dimensional signals. Vision Research, 30: 1111-1117.
54. Zhang, L., Tong, M. H., & Cottrell, G. W. 2009. SUNDAy: Saliency using natural statistics for dynamic analysis of scenes. In Proceedings of the 31st annual Cognitive Science conference, Amsterdam, The Netherlands 2944 2949. Mahwah, NJ: Lawrence Erlbaum.
55. Zhang, L., Tong, M. H., Marks, T. K., Shan, H. and Cottrell, G. W. 2008. SUN: A Bayesian framework for saliency using natural statistics. Journal of Vision, 8(7): 1-20.