Exploring the role of gaze behavior and object detection in scene understanding

Frontiers in Psychology

Volumn 4, Issue DEC, 2013, Pages

  Yun, Kiwon ^a   Peng, Yifan ^a   Samaras, Dimitris ^a   Zelinsky, Gregory J ^a,b   Berg, Tamara L ^c  

^a Stony Brook University   (United States)

^b STONY BROOK UNIVERSITY   (United States)

^c University of North Carolina   (United States)

Author keywords

Computer vision; Description; Gaze; Image annotation; Object detection; Scene understanding

Indexed keywords

EID: 84891713152     PISSN: None     EISSN: 16641078     Source Type: Journal    
DOI: 10.3389/fpsyg.2013.00917     Document Type: Article

References (42)

1. Baldi, P. F., and Itti, L. (2010). Of bits and wows: a bayesian theory of surprise with applications to attention. Neural Netw. 23, 649-666. doi: 10.1016/j.neunet.2009.12.007.
2. Banks, M. S., and Ginsburg, A. P. (1985). "Infant visual preferences: a review and new theoretical treatment," in ADV in Child Development and Behavior, Vol. 19, ed H. W. Reese (New York, NY: Academic Press), 207-246.
3. Berg, A. C., Berg, T. L., Daume, H., Dodge, J., Goyal, A., Han, X., et al. (2012). "Understanding and predicting importance in images," in IEEE Conference on Computer Vision and Pattern Recognition (CVPR) (Providence, RI), 3562-3569.
4. Choi, M. J., Lim, J., Torralba, A., and Willsky, A. (2010). "Exploiting hierarchical context on a large database of object categories," in IEEE Conference on Computer VIsion and Pattern Recognition (CVPR) (San Francisco, CA), 129-136. 10.1109/CVPR.2010.5540221.
5. De Campos, T., Csurka, G., and Perronnin, F. (2012). Images as sets of locally weighted features. Comput. Vis. Image Und. 116, 68-85. doi: 10.1016/j.cviu.2011.07.011.
6. Deng, J., Berg, A., Satheesh, S., Su, H., Khosla, A., and Li, F.-F. (2012). Large Scale Visual Recognition Challenge. Available online at: http://www.image-net.org/challenges/LSVRC/2012/index.
7. Deng, J., Berg, A. C., Li, K., and Fei-Fei, L. (2010). "What does classifying more than 10,000 image categories tell us?" in Proceedings of the 11th European Conference on Computer Vision (ECCV): Part V (Crete, Greece), 71-84. doi: 10.1007/978-3-642-15555-0_6.
8. Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., and Fei-Fei, L. (2009). "ImageNet: a Large-Scale Hierarchical Image Database," in IEEE Conference on Computer VIsion and Pattern Recognition (CVPR) (Miami, FL), 248-255.
9. Einhäuser, W., Spain, M., and Perona, P. (2008). Objects predict fixations better than early saliency. J. Vis. 8, 1-26. doi: 10.1167/8.14.18.
10. Everingham, M., Van Gool, L., Williams, C. K. I., Winn, J., and Zisserman, A. (2008). The PASCAL Visual Object Classes Challenge 2008 (VOC2008) results. Available online at: http://www.pascal-network.org/challenges/VOC/voc2008/workshop/index.html.
11. Everingham, M., Van Gool, L., Williams, C. K. I., Winn, J., and Zisserman, A. (2012). The PASCAL Visual Object Classes Challenge 2012 (VOC2012) results. Available online at: http://www.pascal-network.org/challenges/VOC/voc2012/workshop/index.html.
12. Felzenszwalb, P., Girshick, R., McAllester, D., and Ramanan, D. (2010). "Object detection with discriminatively trained part-based models," IEEE Trans. Pattern Anal. Mach. Intell. 32, 1627-1645. doi: 10.1109/TPAMI.2009.167.
13. Griffin, Z. M., and Bock, K. (2000). What the eyes say about speaking. Psychol. Sci. 11, 24-279. doi: 10.1111/1467-9280.00255.
14. Henderson, J. M. (2003). Human gaze control during real-world scene perception. Trends Cogn. Sci. 7, 498-504. doi: 10.1016/j.tics.2003.09.006.
15. Itti, L. (2005). Quantifying the contribution of low-level saliency to human eye movements in dynamic scenes. Vis. Cogn. 12, 1093-1123. doi: 10.1080/13506280444000661.
16. Itti, L., and Baldi, P. F. (2009). Bayesian surprise attracts human attention. Vis. Res. 49, 1295-1306. doi: 10.1016/j.visres.2008.09.007.
17. Itti, L. and Koch, C. (2000). A saliency-based search mechanism for overt and covert shifts of visual attention. Vis. Res. 40, 1489-1506. doi: 10.1016/S0042-6989(99)00163-7.
18. Itti, L., and Koch, C. (2001). Computational modeling of visual attention. Nat. Rev. Neurosci. 2, 194-203. doi: 10.1038/35058500.
19. Judd, T., Durand, F., and Torralba, A. (2011). Fixations on low-resolution images. J. Vis. 11, 1-20. doi: 10.1167/11.4.14.
20. Judd, T., Ehinger, K. A., Durand, F., and Torralba, A. (2009). "Learning to predict where humans look," in International Conference on Computer Vision (ICCV) (Kyoto).
21. Krizhevsky, A., Sutskever, I., and Hinton, G. E. (2012). "Imagenet classification with deep convolutional neural networks," in Conference on Advances in Neural Information Processing Systems 25 (Lake Tahoe, NV), 1106-1114.
22. Lin, Y., Lv, F., Zhu, S., Yang, M., Cour, T., Yu, K., et al. (2011). "Large-scale image classification: fast feature extraction and svm training," in IEEE Conference on Computer VIsion and Pattern Recognition (CVPR) (Colorado Springs, CO), 1689-1696.
23. Mackworth, N., and Morandi, A. (1967). The gaze selects informative details within pictures. Percept. Psychophys. 2, 547-552. doi: 10.3758/BF03210264.
24. Meyer, A., van der Meulen, F., and Brooks, A. (2004). Eye movements during speech planning: talking about present and remembered objects. Visual Cogn. 11, 553-576. doi: 10.1080/13506280344000248.
25. Meyer, A. S. (2004). "The use of eye tracking in studies of sentence generation," in The Interface of Language, Vision, and Action: eye Movements and the Visual World, eds J. M. Henderson and F. Ferreira (Hove: Psychology Press), 191-212.
26. Neider, M. B., and Zelinsky, G. J. (2006a). Scene context guides eye movements during search. Vis. Res. 46, 614-621. doi: 10.1016/j.visres.2005.08.025.
27. Neider, M. B., and Zelinsky, G. J. (2006b). Searching for camouflaged targets: effects of target-background similarity on visual search. Vis. Res. 46, 2217-2235. doi: 10.1016/j.visres.2006.01.006.
28. Nuthmann, A., and Henderson, J. M. (2010). Object-based attentional selection in scene viewing. J. Vis. 10, 1-19. doi: 10.1167/10.8.20.
29. Pajak, M., and Nuthmann, A. (2013). Object-based saccadic selection during scene perception: evidence from viewing position effects. J. Vis. 13, 1-21. doi: 10.1167/13.5.2.
30. Parkhurst, D. J., Law, K., and Niebur, E. (2002). Modeling the role of salience in the allocation of overt visual selective attention. Vis. Res. 42, 107-123. doi: 10.1016/S0042-6989(01)00250-4.
31. Perronnin, F., Akata, Z., Harchaoui, Z., and Schmid, C. (2012). "Towards good practice in large-scale learning for image classification," in IEEE Conference on Computer VIsion and Pattern Recognition (CVPR) (Providence, RI), 3482-3489.
32. Phan, X.-H. (2006). CRFTagger: CRF English POS Tagger. Available online at: http://crftagger.sourceforge.net/.
33. Rashtchian, C., Young, P., Hodosh, M., and Hockenmaier, J. (2010). "Collecting image annotations using amazon's mechanical turk," in Proceedings of the NAACL HLT 2010 Workshop on Creating Speech and Language Data with Amazon's Mechanical Turk (Los Angeles, CA), 139-147.
34. Renninger, L. W., Vergheese, P., and Coughlan, J. (2007). Where to look next? eye movements reduce local uncertainty. J. Vis. 3, 1-17. doi: 10.1167/7.3.6.
35. Tatler, B. W. (2007). The central fixation bias in scene viewing: Selecting an optimal viewing position independently of motor biases and image feature distributions. J. Vis. 7, 1-17. doi: 10.1167/7.14.4.
36. Tatler, B. W., Baddeley, R. J., and Vincent, B. T. (2006). The long and the short of it: spatial statistics at fixation vary with saccade amplitude and task. Vis. Res. 46, 1857-1862. doi: 10.1016/j.visres.2005.12.005.
37. Theeuwes, J. (1994). Stimulus-driven capture and attentional set: selective search for color and visual abrupt onsets. J. Exp. Psychol. 20, 799-806.
38. Theeuwes, J., Kramer, A., Hahn, S., Irwin, D., and Zelinsky, G. (1999). Influence of attentional capture on oculomotor control. J. Exp. Psychol. 25, 1595-1608.
39. Wu, Z., and Palmer, M. (1994). "Verbs semantics and lexical selection," in Proceedings of the 32nd Annual Meeting on Association for Computational Linguistics (Las Cruces, NM), 133-138. doi: 10.3115/981732.981751.
40. Yarbus, A. (1967). Eye Movements and Vision. (New York, NY: Plenum Press), doi: 10.1007/978-1-4899-5379-7.
41. Zelinsky, G. J. (2008). A theory of eye movements during target acquisition. Psychol. Rev. 115, 787-835. doi: 10.1037/a0013118.
42. Zelinsky, G. J., and Schmidt, J. (2009). An effect of referential scene constraint on search implies scene segmentation. Vis. Cogn. 17, 1004-1028. doi: 10.1080/13506280902764315.