Social eye gaze modulates processing of speech and co-speech gesture

Cognition

Volumn 133, Issue 3, 2014, Pages 692-697

  Holler, Judith ^a,b   Schubotz, Louise ^a,f   Kelly, Spencer ^c   Hagoort, Peter ^a,e   Schuetze, Manuela ^a   Özyürek, Asli ^a,d  

^a MAX PLANCK INSTITUTE FOR PSYCHOLINGUISTICS   (Netherlands)

^b UNIVERSITY OF MANCHESTER   (United Kingdom)

^c Assistant Professor in the University Libraries   (United States)

^d RADBOUD UNIVERSITY NIJMEGEN   (Netherlands)

^e RADBOUD UNIVERSITY   (Netherlands)

^f MAX PLANCK INSTITUTE FOR DEMOGRAPHIC RESEARCH   (Germany)

Author keywords

Co speech iconic gesture; Communicative intent; Eye gaze; Language processing; Multi party communication; Recipient status

Indexed keywords

ADULT; ARTICLE; FACTORIAL DESIGN; GAZE; GESTURE; HUMAN; INTERPERSONAL COMMUNICATION; LANGUAGE PROCESSING; NORMAL HUMAN; RESPONSE TIME; SOCIAL EYE GAZE; SPEECH DEVELOPMENT; VISUAL STIMULATION; COMPREHENSION; EYE MOVEMENT; FEMALE; MALE; PHYSIOLOGY; REACTION TIME; SPEECH; SPEECH PERCEPTION; YOUNG ADULT;

COMMUNICATION; COMPREHENSION; EYE MOVEMENTS; FEMALE; GESTURES; HUMANS; MALE; REACTION TIME; SPEECH; SPEECH PERCEPTION; YOUNG ADULT;

EID: 84907247006     PISSN: 00100277     EISSN: 18737838     Source Type: Journal    
DOI: 10.1016/j.cognition.2014.08.008     Document Type: Article

References (32)

1. Boersma, P., & Weenink, D. (2014). Praat: doing phonetics by computer [Computer program]. Retrieved from. http://www.praat.org/.
2. Clark H.H., Carlson T.B. Hearers and speech acts. Language 1982, 58:332-373.
3. Feyereisen, P. (1998). Le rôle des gestes dans la mémorisation d'énoncés oraux. In S. Santi, I. Guaïtella, C. Cavé, & G. Konopczynski (Eds.), Oralité et gestualité. Communication multimodale, interaction. Actes du colloque Orage 98 (pp. 355-360). Paris: L'Harmattan.
4. Goodwin C. Conversational organization: Interaction between speakers and hearers 1981, Academic Press, New York.
5. Hanna J.E., Brennan S.E. Speakers' eye gaze disambiguates referring expressions early during face-to-face conversation. Journal of Memory & Language 2007, 57:596-615.
6. Holle H., Gunter T.C. The role of iconic gestures in speech disambiguation: ERP evidence. Journal of Cognitive Neuroscience 2007, 19:1175-1192.
7. Holler J., Kelly S., Hagoort P., Özyürek A. When gestures catch the eye: The influence of gaze direction on co-speech gesture comprehension in triadic communication. Proceedings of the 34th Annual Meeting of the Cognitive Science Society 2012, 467-472. Cognitive Society, Austin, TX. N. Miyake, D. Peebles, R.P. Cooper (Eds.).
8. Kelly S.D., Barr D., Church R.B., Lynch K. Offering a hand to pragmatic understanding: The role of speech and gesture in comprehension and memory. Journal of Memory & Language 1999, 40:577-592.
9. Kelly S.D., Creigh P., Bartolotti J. Integrating speech and iconic gestures in a Stroop-like task: Evidence for automatic processing. Journal of Cognitive Neuroscience 2010, 22:683-694.
10. Kelly S.D., Kravitz C., Hopkins M. Neural correlates of bimodal speech and gesture comprehension. Brain & Language 2004, 89:253-260.
11. Kelly S.D., McDevitt T., Esch M. Brief training with co-speech gesture lends a hand to word learning in a foreign language. Language & Cognitive Processes 2009, 24:313-334.
12. Kelly S.D., Özyürek A., Maris E. Two sides of the same coin: Speech and gesture mutually interact to enhance comprehension. Psychological Science 2010, 21:260-267.
13. Kelly S.D., Ward S., Creigh P., Bartolotti J. An intentional stance modulates the integration of gesture and speech during comprehension. Brain & Language 2007, 101:222-233.
14. Kendon A. Some functions of gaze direction in social interaction. Acta Psychologica 1967, 26:22-63.
15. Knöferle P., Kreysa H. Can speaker gaze modulate syntactic structuring and thematic role assignment during spoken sentence comprehension?. Frontiers in Cognitive Science 2012, 3:538.
16. Lerner G.H. Selecting next speaker: The context-sensitive operation of a context-free organization. Language in Society 2003, 32:177-201.
17. Leys C., Schumann S. A nonparametric method to analyze interactions: The adjusted rank transform test. Journal of Experimental Social Psychology 2010, 46:684-688.
18. Obermeier C., Dolk T., Gunter T.C. The benefit of gestures during communication: Evidence from hearing and hearing-impaired individuals. Cortex 2012, 48:857-870.
19. Özyürek A. Do speakers design their cospeech gestures for their addressees? The effects of addressee location on representational gestures. Journal of Memory & Language 2002, 46:688-704.
20. Özyürek A., Willems R.M., Kita S., Hagoort P. On-line integration of semantic information from speech and gesture: Insights from event-related brain potentials. Journal of Cognitive Neuroscience 2007, 19:605-616.
21. Paivio A. Mental representations: A dual coding approach 1986, Oxford University Press, Oxford, UK.
22. Rossano, F. (2012). Gaze behavior in face-to-face interaction. Published PhD thesis, Radboud University, Nijmegen.
23. Schober M.F., Clark H.H. Understanding by addressees and overhearers. Cognitive Psychology 1989, 21:211-232.
24. Senju A., Johnson M.H. The eye contact effect: Mechanisms and development. Trends in Cognitive Sciences 2009, 13:127-134.
25. Staudte M., Crocker M. Investigating joint attention mechanisms through spoken human-robot interaction. Cognition 2011, 120:268-291.
26. Straube B., Green A., Jansen A., Chatterjee A., Kircher T. Social cues, mentalizing and the neural processing of speech accompanied by gestures. Neuropsychologia 2010, 48:382-393.
27. Willems R.M., Özyürek A., Hagoort P. When language meets action: The neural integration of speech and gesture. Cerebral Cortex 2007, 17:2322-2333.
28. Willems R.M., Özyürek A., Hagoort P. Differential roles for left inferior frontal and superior temporal cortex in multimodal integration of action and language. NeuroImage 2009, 47:1992-2004.
29. Wu Y.C., Coulson S. Meaningful gestures: Electrophysiological indices of iconic gesture comprehension. Psychophysiology 2005, 42:654-667.
30. Wu Y.C., Coulson S. How iconic gestures enhance communication: An ERP study. Brain & Language 2007, 101:234-245.
31. Wu Y.C., Coulson S. Iconic gestures prime related concepts: An ERP study. Psychonomic Bulletin & Review 2007, 14:57-63.
32. Yap D.F., So W.C., Melvin Yap J.M., Tan Y.Q., Teoh R.L.S. Iconic gestures prime words. Cognitive Science 2011, 35:171-183.