Modelling spatio-temporal saliency to predict gaze direction for short videos

International Journal of Computer Vision

Volumn 82, Issue 3, 2009, Pages 231-243

  Marat, Sophie ^a   Ho Phuoc, Tien ^a   Granjon, Lionel ^a   Guyader, Nathalie ^a   Pellerin, Denis ^a   Guérin Dugué, Anne ^a  

^a GIPSA LAB   (France)

Author keywords

Gaze prediction; Saliency; Spatio temporal model; Video viewing

Indexed keywords

IMAGE SEGMENTATION;

DYNAMIC PATHWAYS; HUMAN VISUAL SYSTEM; LARGE DATABASE; SALIENCY; SALIENCY MODELING; SPATIO TEMPORAL; SPATIO-TEMPORAL MODELS; VIDEO VIEWING;

EYE MOVEMENTS;

EID: 61349170126     PISSN: 09205691     EISSN: 15731405     Source Type: Journal    
DOI: 10.1007/s11263-009-0215-3     Document Type: Article

References (35)

1. Beaudot, W. H. (1994). The neural information in the vertebra retina: a melting pot of ideas for artificial vision. PHD thesis, Tirf laboratory, Grenoble, France.
2. Beaudot, W. H. A., Palagi, P., & Hérault, J. (1993). Realistic simulation tool for early visual processing including space, time and colour data. In Lecture notes in computer science: Vol. 686. IWANN (pp. 370-375). Barcelona, June 1993. Berlin: Springer.
3. Bruno, E., & Pellerin, D. (2002). Robust motion estimation using spatial Gabor-like filters. Signal Processing, 82, 297-309.
4. Carmi, R., & Itti, L. (2006). Visual causes versus correlates of attentional selection in dynamic scenes. Vision Research, 46, 4333-4345.
5. Daugman, J. G. (1980). Two-dimensional spectral analysis of cortical receptive field profiles. Vision Research, 20, 847-856.
6. DeValois, R. L. (1991). Orientation and spatial frequency selectivity: properties and modular organization. In A. Valberg & B. B. Lee (Eds.). From pigment to perception. New York: Plenum.
7. Egeth, H. E., & Yantis, S. (1997). Visual attention: control representation and time course. Annual Review of Psychology, 48, 269-297.
8. Guironnet, M., Pellerin, D., Guyader, N., & Ladret, P. (2007). Video summarization based on camera motion and a subjective evaluation method. EURASIP Journal on Image and Video Processing, 2007, Article ID 60245, 12 pages.
9. Hansen, T., Sepp, W., & Neumann, H. (2001). Recurrent long-range interactions in early vision. In Lecture notes in computer science/Lecture notes in artificial intelligence: Vol. 2036. Emergent neural computational architectures based on neuroscience (pp. 139-153). Berlin: Springer.
10. Henderson, J. M. (2003). Human gaze control during real-world scene perception. Trends in Cognitive Sciences, 7, 498-504.
11. Hubel, D. H., & Wiesel, T. N. (1977). Functional architecture of macaque visual cortex. Proceedings of the Royal Society of London, B, 198, 1-59.
12. Itti, L., Koch, C., & Niebur, E. (1998). A model of saliency-based visual attention for rapid scene analysis. IEEE Transactions on Pattern Analysis and Machine Intelligence, 20, 1254-1259.
13. Koch, C., & Ullman, S. (1985). Shifts in selective visual attention: towards the underlying neural circuitry. Human Neurobiology, 4, 219-227.
14. Le Meur, O., Le Callet, P., & Barba, D. (2006). A coherent computational approach to model bottom-up visual attention. IEEE Transactions on Pattern Analysis and Machine Intelligence, 28, 802-817.
15. Le Meur, O., Le Callet, P., & Barba, D. (2007). Predicting visual fixations on video based on low-level visual features. Vision Research, 47, 2483-2498.
16. Lisberger, S. G., Morris, E. J., & Tychsen, L. (1987). Visual motion processing and sensory-motor integration for smooth pursuit eye movements. Annual Review-Neuroscience, 10, 97-129.
17. Ma, Y.-F., Hua, X.-S., Lu, L., & Zhang, H.-J. (2005). A generic framework of user attention model and its application in video summarization. IEEE Transactions on Multimedia, 7.
18. Marat, S., Ho Phuoc, T., Granjon, L., Guyader, N., Pellerin, D., & Guérin-Dugué, A. (2008). Spatiotemporal saliency model to predict eye movements in video free viewing. In EUSlPCO'08 - 16th European signal processing conference, Lausanne, Switzerland, 2008.
19. Massot, C., & Hérault, J. (2008). Model of frequency analysis in the visual cortex and the shape from texture problem. International Journal of Computer Vision, 76, 165-182.
20. Milanese, R., Wechsler, H., Gil, S., Bost, J.-M., & Pun, T. (1994). Integration of bottom-up and top-down cues for visual attention using non-linear relaxation. In Proc. CVPR (pp. 781-785) 1994.
21. Odobez, J.-M., & Bouthemy, P. (1995). Robust multiresolution estimation of parametric motion models. Journal of Visual Communication and Image Representation, 6, 348-365.
22. Palmer, S. E. (1999). Vision science: photons to phenomenology (1st edn.). Cambridge: MIT Press.
23. Parkhurst, D., Law, K., & Niebur, E. (2002). Modeling the role of salience in the allocation of overt visual attention. Vision Research, 42, 107-123.
24. Peters, R. J., & Itti, L. (2008). Applying computational tools to predict gaze direction in interactive visual environments. ACM Transactions on Applied Perception, 5.
25. Peters, R. J., Iyer, A., Itti, L., & Koch, C. (2005). Components of bottom-up gaze allocation in natural images. Vision Research, 45, 2397-2416.
26. Rajashekar, U., Cormack, L. K., & Bovik, A. C. (2004). Point of gaze analysis reveals visual search strategies. In Proceedings of SPIE: Vol. 5292. Human vision and electronic imaging IX 2004 (pp. 296-306). Bellingham: SPIE Press.
27. Reinagel, P., & Zador, A. (1999). Natural scene statistics at the center of gaze. Network: Computation in Neural Systems, 10, 341-350.
28. Schwartz, S. H. (2004). Visual perception: a clinical orientation (3rd edn.). New-York: McGraw-Hill.
29. Tatler, B. W., Baddeley, R. J., & Gilchrist, I. D. (2005). Visual correlates of fixation selection: effects of scale and time. Vision Research, 45, 643-659.
30. Torralba, A., Oliva, A., Castelhano, M. S., & Henderson, J. M. (2006). Contextual guidance of eye movements and attention in real-world scenes: the role of global features in object search. Psychological Review, 113, 766-786.
31. Treisman, A. M., & Gelade, G. (1980). A. feature-integration theory of attention. Cognitive Psychology, 12, 97-136.
32. Tsotsos, J. K., Culhane, S. M., Winky, Y. K. W., Lai, Y., Davis, N., & Nuflo, F. (1995). Modeling visual attention via selective tuning. Artificial Intelligence, 78, 507-545.
33. Tsotsos, J. K., Rodríguez-Sánchez, A. J., Rothenstein, A. L., & Simine, E. (2008). The different stages of visual recognition need different attentional binding strategies. Brain Research, 1225, 119-132.
34. Wolfe, J. M., Alvarez, G. A., & Horowitz, T. S. (2000). Attention is fast but volition is slow. Nature, 406, 691.
35. Wolfe, J. M., Cave, K. R., & Franzel, S. L. (2006). Guided search: an alternative to the feature integration model for visual search. Journal of Experimental Psychology: Human Perception and Performance, 15, 419-433.