Two sides of the same coin: Speech and gesture mutually interact to enhance comprehension

Psychological Science

Volumn 21, Issue 2, 2010, Pages 260-267

  Kelly, Spencer D ^a   Özyürek, Asli ^b,c   Maris, Eric ^d,e  

^a Colgate University ^*  (United States)

^b RADBOUD UNIVERSITY NIJMEGEN   (Netherlands)

^c MAX PLANCK INSTITUTE FOR PSYCHOLINGUISTICS   (Netherlands)

^d RADBOUD UNIVERSITY   (Netherlands)

^e RADBOUD UNIVERSITY NIJMEGEN   (Netherlands)

Author keywords

Action; Communication; Comprehension; Iconic gestures; Integrated systems hypothesis; Language; Multimodal; Mutual; Obligatory; Semantic processing; Speech

Indexed keywords

ARTICLE; ATTENTION; COMPREHENSION; FEMALE; GESTURE; HUMAN; MALE; SEMANTICS; SPEECH;

ATTENTION; COMPREHENSION; FEMALE; GESTURES; HUMANS; MALE; SEMANTICS; SPEECH;

EID: 77951272213     PISSN: 09567976     EISSN: 14679280     Source Type: Journal    
DOI: 10.1177/0956797609357327     Document Type: Article

References (29)

1. Bernardis, P., & Gentilucci, M. (2006). Speech and gesture share the same communication system. Neuropsychologia, 44, 178-190.
2. Bernardis, P., Salillas, E., & Caramelli, N. (2008). Behavioural and neurophysiological evidence of semantic interaction between iconic gestures and words. Cognitive Neuropsychology, 25, 1114-1128.
3. Calvert, G.A. (2001). Cross-modal processing in the human brain: Insights from functional neuroimaging studies. Cerebral Cortex, 11, 1110-1123.
4. Calvert, G.A., Spence, C., & Stein, B.E. (2004). The handbook of multisensory processes. Cambridge, MA: MIT Press.
5. Clark, H.H. (1996). Using language. Cambridge, England: Cambridge University Press.
6. Cornejo, C., Simonetti, F., Ibáñez, A., Aldunate, N., Ceric, F., Lopez, V., et al. (2009). Gesture and metaphor comprehension: Electrophysiological evidence of cross-modal coordination by audiovisual stimulation. Brain and Cognition, 70, 42-52.
7. de Gelder, B., & Vroomen, J. (2000). Perceiving emotions by ear and by eye. Cognition & Emotion, 14, 289-311.
8. Fischer, M.H., & Zwaan, R.A. (2008). Embodied language: A review of the role of the motor system in language comprehension. The Quarterly Journal of Experimental Psychology, 61, 825-850.
9. Goldin-Meadow, S. (2003). Hearing gesture: How our hands help us think. Cambridge, MA: Harvard University Press.
10. Holle, H., & Gunter, T.C. (2007). The role of iconic gestures in speech disambiguation: ERP evidence. Journal of Cognitive Neuroscience, 19, 1175-1192.
11. Kelly, S.D., Barr, D.J., Church, R.B., & Lynch, K. (1999). Offering a hand to pragmatic understanding: The role of speech and gesture in comprehension and memory. Journal of Memory and Language, 40, 577-592.
12. Kelly, S.D., Kravitz, C., & Hopkins, M. (2004). Neural correlates of bimodal speech and gesture comprehension. Brain and Language, 89, 253-260.
13. Kelly, S.D., Manning, S., & Rodak, S. (2008). Gesture gives a hand to language and learning: Perspectives from cognitive neuroscience, developmental psychology and education. Language and Linguistics Compass, 2, 1-20.
14. Kelly, S.D., Ward, S., Creigh, P., & Bartolotti, J. (2007). An intentional stance modulates the integration of gesture and speech during comprehension. Brain and Language, 101, 222-233.
15. Kendon, A. (2004). Gesture: Visible action as utterance. Cambridge, England: Cambridge University Press.
16. Kita, S., & Özyürek, A. (2003). What does cross-linguistic variation in semantic coordination of speech and gesture reveal? Evidence for an interface representation of spatial thinking and speaking. Journal of Memory and Language, 48, 16-32.
17. Kutas, M., & Hillyard, S. (1984). Brain potentials during reading reflect word expectancy and semantic association. Nature, 307, 161-163.
18. Masson, M.E.J., Bub, D.N., & Warren, C.M. (2008). Kicking calculators: Contribution of embodied representations to sentence comprehension. Journal of Memory and Language, 59, 256-265.
19. McNeill, D. (1992). Hand and mind: What gestures reveal about thought. Chicago: University of Chicago Press.
20. McNeill, D. (2005). Gesture and thought. Chicago: University of Chicago Press.
21. McNeill, D. (in press). Gesture. In P.C. Hogan (Ed.), Cambridge encyclopedia of the language sciences. Cambridge, England: Cambridge University Press.
22. Özyürek, A., Willems, R.M., Kita, S., & Hagoort, P. (2007). On-line integration of semantic information from speech and gesture: Insights from event-related brain potentials. Journal of Cognitive Neuroscience, 19, 605-616.
23. Pulvermüller, F. (2005). Brain mechanisms linking language and action. Nature Reviews Neuroscience, 6, 576-582.
24. Ratcliff, R. (1993). Methods for dealing with reaction time outliers. Psychological Bulletin, 114, 510-532.
25. Singer, M.A., & Goldin-Meadow, S. (2005). Children learn when their teacher's gestures and speech differ. Psychological Science, 16, 85-89.
26. van Wassenhove, V., Grant, K.W., & Poeppel, D. (2005). Visual speech speeds up the neural processing of auditory speech. Proceedings of the National Academy of Sciences, USA, 102, 1181-1186.
27. Willems, R.M., & Hagoort, P. (2007). Neural evidence for the interplay between language, gesture, and action: A review. Brain and Language, 101, 278-289.
28. Willems, R.M., Özyürek, A., & Hagoort, P. (2007). When language meets action: The neural integration of gesture and speech. Cerebral Cortex, 17, 2322-2333.
29. Wu, Y.C., & Coulson, S. (2007). How iconic gestures enhance communication: An ERP study. Brain and Language, 101, 234-245.