Look who's talking? Sound changes gaze behaviour in a dynamic social scene

Visual Cognition

Volumn 21, Issue 7, 2013, Pages 922-944

  Foulsham, Tom ^a   Sanderson, Lucy Anne ^a  

^a UNIVERSITY OF ESSEX   (United Kingdom)

Author keywords

Eye movements; Scene perception; Social attention; Speech perception

Indexed keywords

ADULT; ARTICLE; BINOCULAR CONVERGENCE; CONVERSATION; EYE FIXATION; EYE POSITION; EYE TRACKING; FACIES; FEMALE; GAZE; HUMAN; LIP READING; MALE; NORMAL HUMAN; PRIORITY JOURNAL; SACCADIC EYE MOVEMENT; SELECTIVE ATTENTION; SOUND; SPEECH; SPEECH PERCEPTION; VIDEORECORDING;

EID: 84889239104     PISSN: 13506285     EISSN: 14640716     Source Type: Journal    
DOI: 10.1080/13506285.2013.849785     Document Type: Article

References (43)

1. Birmingham, E., Bischof, W. F., & Kingstone, A. (2008). Gaze selection in complex social scenes. Visual Cognition, 16(2-3), 341-355. doi:10.1080/13506280701434532
2. Buchan, J. N., Pare, M., & Munhall, K. G. (2007). Spatial statistics of gaze fixations during dynamic face processing. Social Neuroscience, 2(1), 1-13. doi:10.1080/17470910601043644
3. Cherry, E. C. (1953). Some experiments on the recognition of speech, with one and with two ears. Journal of the Acoustical Society of America, 25, 975-979. doi:10.1121/1.1907229
4. Coutrot, A., Guyader, N., Ionescu, G., & Caplier, A. (2012). Influence of soundtrack on eye movements during video exploration. Journal of Eye Movement Research, 5(4), 1-10.
5. Emery, N. J. (2000). The eyes have it: The neuroethology, function and evolution of social gaze. Neuroscience and Biobehavioral Reviews, 24(6), 581-604. doi:10.1016/S0149-7634(00)00025-7
6. Emmorey, K., Thompson, R., & Colvin, R. (2009). Eye gaze during comprehension of American sign language by native and beginning signers. Journal of Deaf Studies and Deaf Education, 14(2), 237-243. doi:10.1093/deafed/enn037
7. Foulsham, T., Cheng, J. T., Tracy, J. L., Henrich, J., & Kingstone, A. (2010). Gaze allocation in a dynamic situation: Effects of social status and speaking. Cognition, 117(3), 319-331. doi:10.1016/j.cognition.2010.09.003
8. Freeth, M., Foulsham, T., & Kingstone, A. (2013). What affects social attention? Social presence, eye contact and autistic traits. PLoS ONE, 8(1), e53286. doi:10.1371/journal.pone.0053286
9. Friesen, C. K., & Kingstone, A. (1998). The eyes have it! Reflexive orienting is triggered by nonpredictive gaze. Psychonomic Bulletin and Review, 5(3), 490-495. doi:10.3758/BF03208827
10. Gallup, A. C., Chong, A., & Couzin, I. D. (2012). The directional flow of visual information transfer between pedestrians. Biology Letters, 8(4), 520-522. doi:10.1098/rsbl.2012.0160
11. Ghazanfar, A. A., Nielsen, K., & Logothetis, N. K. (2006). Eye movements of monkey observers viewing vocalizing conspecifics. Cognition, 101(3), 515-529. doi:10.1016/j.cognition.2005.12.007
12. Goldstein, R. B., Woods, R. L., & Peli, E. (2007). Where people look when watching movies: Do all viewers look at the same place? Computers in Biology and Medicine, 37(7), 957-964. doi:10.1016/j.compbiomed.2006.08.018
13. Gullberg, M. (2002). Eye movements and gestures in human interaction. In J. Hyona, R. Radach, & H. Deubel (Eds.), The mind's eye: Cognitive and applied aspects of eye movement research (pp. 685-703). Amsterdam: Elsevier.
14. Hanna, J. E., & Brennan, S. E. (2007). Speakers' eye gaze disambiguates referring expressions early during face-to-face conversation. Journal of Memory and Language, 57, 596-615. doi:10.1016/j.jml.2007.01.008
15. Itier, R. J., & Batty, M. (2009). Neural bases of eye and gaze processing: The core of social cognition. Neuroscience and Biobehavioral Reviews, 33(6), 843-863. doi:10.1016/j.neubiorev.2009.02.004
16. Itti, L. (2005). Quantifying the contribution of low-level saliency to human eye movements in dynamic scenes. Visual Cognition, 12(6), 1093-1123. doi:10.1080/13506280444000661
17. Kendon, A. (1967). Some functions of gaze-direction in social interaction. Acta Psychologica, 26(1), 22-63. doi:10.1016/0001-6918(67)90005-4
18. Klin, A., Jones, W., Schultz, R., Volkmar, F., & Cohen, D. (2002). Visual fixation patterns during viewing of naturalistic social situations as predictors of social competence in individuals with autism. Archives of General Psychiatry, 59(9), 809-816. doi:10.1001/archpsyc.59.9.809
19. Laidlaw, K. E., Foulsham, T., Kuhn, G., & Kingstone, A. (2011). Potential social interactions are important to social attention. Proceedings of the National Academy of Sciences, 108(14), 5548-5553. doi:10.1073/pnas.1017022108
20. Lansing, C. R., & McConkie, G. W. (1999). Attention to facial regions in segmental and prosodic visual speech perception tasks. Journal of Speech, Language and Hearing Research, 42(3), 526-539.
21. Lansing, C. R., & McConkie, G. W. (2003). Word identification and eye fixation locations in visual and visual-plus-auditory presentations of spoken sentences. Perception and Psychophysics, 65(4), 536-552. doi:10.3758/BF03194581
22. Levy, J., Foulsham, T., & Kingstone, A. (2013). Monsters are people too. Biology Letters, 9, 20120850.
23. Macdonald, R. G., & Tatler, B. W. (2013). Do as eye say: Gaze cueing and language in a real-world social interaction. Journal of Vision, 13(4). doi:10.1167/13.4.6
24. McGurk, H., & MacDonald, J. (1976). Hearing lips and seeing voices. Nature, 264, 746-748. doi:10.1038/264746a0
25. Mital, P. K., Smith, T. J., Hill, R. L., & Henderson, J. M. (2011). Clustering of gaze during dynamic scene viewing is predicted by motion. Cognitive Computation, 3(1), 5-24. doi:10.1007/s12559-010-9074-z
26. Nummenmaa, L., Hyona, J., & Hietanen, J. K. (2009). I'll walk this way: Eyes reveal the direction of locomotion and make passersby look and go the other way. Psychological Science, 20(12), 1454-1458. doi:10.1111/j.1467-9280.2009.02464.x
27. Pelphrey, K. A., Morris, J. P., & McCarthy, G. (2005). Neural basis of eye gaze processing deficits in autism. Brain, 128(5), 1038-1048. doi:10.1093/brain/awh404
28. Ricciardelli, P., Carcagno, S., Vallar, G., & Bricolo, E. (2013). Is gaze following purely reflexive or goal-directed instead? Revisiting the automaticity of orienting attention by gaze cues. Experimental Brain Research, 224(1), 93-106. doi:10.1007/s00221-012-3291-5
29. Risko, E. F., Laidlaw, K., Freeth, M., Foulsham, T., & Kingstone, A. (2012). Social attention with real versus reel stimuli: Toward an empirical approach to concerns about ecological validity. Frontiers in Human Neuroscience, 6. doi:10.3389/fnhum.2012.00143
30. Ross, N. M., & Kowler, E. (2013). Eye movements while viewing narrated, captioned, and silent videos. Journal of Vision, 13(4). doi:10.1167/13.4.1
31. Shen, J., & Itti, L. (2012). Top-down influences on visual attention during listening are modulated by observer sex. Vision Research, 65, 62-76. doi:10.1016/j.visres.2012.06.001
32. Shepherd, S. V., Steckenfinger, S. A., Hasson, U., & Ghazanfar, A. A. (2010). Human-monkey gaze correlations reveal convergent and divergent patterns of movie viewing. Current Biology, 20(7), 649-656. doi:10.1016/j.cub.2010.02.032
33. Smith, T. J., & Henderson, J. M. (2008). Edit blindness: The relationship between attention and global change blindness in dynamic scenes. Journal of Eye Movement Research, 2, 1-17.
34. Spezio, M. L., Huang, P. Y. S., Castelli, F., & Adolphs, R. (2007). Amygdala damage impairs eye contact during conversations with real people. Journal of Neuroscience, 27(15), 3994-3997. doi:10.1523/JNEUROSCI.3789-06.2007
35. Spivey, M. J., & Geng, J. J. (2001). Oculomotor mechanisms activated by imagery and memory: Eye movements to absent objects. Psychological Research, 65(4), 235-241. doi:10.1007/s004260100059
36. Steinman, R. M. (1965). Effect of target size, luminance, and color on monocular fixation. Journal of the Optical Society of America, 55, 1158-1165. doi:10.1364/JOSA.55.001158
37. Stivers, T., Enfield, N. J., Brown, P., Englert, C., Hayashi, M., Heinemann, T., & Levinson, S. C. (2009). Universals and cultural variation in turn-taking in conversation. Proceedings of the National Academy of Sciences, 106(26), 10587-10592. doi:10.1073/pnas.0903616106
38. Tice, M., & Henetz, T. (2011). The eye gaze of 3rd party observers reflects turn-end boundary projection. In Proceedings of the 15th workshop on the Semantics and Pragmatics of Dialogue (SEMDIAL 2011) (pp. 204-205). Los Angeles, CA.
39. Van der Burg, E., Olivers, C. N. L., Bronkhorst, A. W., & Theeuwes, J. (2008). Pip and pop: Nonspatial auditory signals improve spatial visual search. Journal of Experimental Psychology: Human Perception and Performance, 34(5), 1053-1065. doi:10.1037/0096-1523.34.5.1053
40. Vatikiotis-Bateson, E., Eigsti, I. M., Yano, S., & Munhall, K. G. (1998). Eye movement of perceivers during audiovisual speech perception. Perception and Psychophysics, 60(6), 926-940. doi:10.3758/BF03211929
41. Vertegaal, R., Slagter, R., van der Veer, G., & Nijholt, A. (2001). Eye gaze patterns in conversations: There is more to conversational agents than meets the eyes. In Proceedings of the SIGCHI conference on Human Factors in Computing Systems (pp. 301-308). ACM.
42. Võ, M. L. H., Smith, T. J., Mital, P. K., & Henderson, J. M. (2012). Do the eyes really have it? Dynamic allocation of attention when viewing moving faces. Journal of Vision, 12(13).
43. Yarbus, A. L. (1967). Eye movements and vision. New York, NY: Plenum.