Gesture facilitates the syntactic analysis of speech

Frontiers in Psychology

Volumn 3, Issue MAR, 2012, Pages

  Holle, Henning ^a   Obermeier, Christian ^b   Schmidt Kassow, Maren ^c   Friederici, Angela D ^b   Ward, Jamie ^d   Gunter, Thomas C ^b  

^a UNIVERSITY OF HULL   (United Kingdom)

^b MAX PLANCK INSTITUTE FOR HUMAN COGNITIVE AND BRAIN SCIENCES   (Germany)

^c GOETHE UNIVERSITY   (Germany)

^d UNIVERSITY OF SUSSEX   (United Kingdom)

Author keywords

Ambiguity; Audiovisual; Language; P600; Syntax

Indexed keywords

EID: 84867078083     PISSN: None     EISSN: 16641078     Source Type: Journal    
DOI: 10.3389/fpsyg.2012.00074     Document Type: Article

References (38)

1. Alibali, M. W., Heath, D. C., and Myers, H. J. (2001). Effects of visibility between speaker and listener on gesture production: some gestures are meant to be seen. J. Mem. Lang. 44, 169-188.
2. Bader, M., and Meng, M. (1999). Subject-object ambiguities in German embedded clauses: an across-the-board comparison. J. Psycholinguist. Res. 28, 121-143.
3. Bavelas, J., Gerwing, J., Sutton, C., and Prevost, D. (2008). Gesturing on the telephone: independent effects of dialogue and visibility. J. Mem. Lang. 58, 495-520.
4. Boersma, P., and Weenink, D. (2010). Praat: Doing Phonetics by Computer, Version 5.1.15 [Computer program]. Available at: http://www.praat.org/
5. Crocker, M. W., Knoeferle, P., and Mayberry, M. R. (2010). Situated sentence processing: the coordinated interplay account and a neurobehavioral model. Brain Lang. 112, 189-201.
6. Efron, D. (1941/1972). Gesture, Race and Culture. The Hague: Mouton.
7. Friederici, A. D. (2002). Towards a neural basis of auditory sentence processing. Trends Cogn. Sci. (Regul. Ed.) 6, 78-84.
8. Friederici, A. D., Pfeifer, E., and Hahne, A. (1993). Event-related brain potentials during natural speech processing: effects of semantic, morphological and syntactic violations. Brain Res. Cogn. Brain Res. 1, 183-192.
9. Gentilucci, M., Bernardis, P., Crisi, G., and Dalla Volta, R. (2006). Repetitive transcranial magnetic stimulation of Broca's area affects verbal responses to gesture observation. J. Cogn. Neurosci. 18, 1059-1074.
10. Grice, H. P. (1975). "Logic and conversation," in Speech Acts: Syntax and Semantics, eds P. Cole and J. L. Morgan (New York: Academic Press), 41-58.
11. Hagoort, P., Brown, C., and Groothusen, J. (1993). The syntactic positive shift (SPS) as an ERP measure of syntactic processing. Lang. Cogn. Process. 8, 439-483
12. Haupt, F. S., Schlesewsky, M., Roehm, D., Friederici, A. D., and Bornkessel-Schlesewsky, I. (2008). The status of subject-object reanalyses in the language comprehension architecture. J. Mem. Lang. 59, 54-96.
13. Holle, H., and Gunter, T. C. (2007). The role of iconic gestures in speech disambiguation: ERP evidence. J. Cogn. Neurosci. 19, 1175-1192.
14. Holle, H., Gunter, T. C., Ruschemeyer, S. A., Hennenlotter, A., and Iacoboni, M. (2008). Neural correlates of the processing of co-speech gestures. Neuroimage 39, 2010-2024.
15. Holle, H., Obleser, J., Rueschemeyer, S. A., and Gunter, T. C. (2010). Integration of iconic gestures and speechinleftsuperiortemporal areas boosts speech comprehension under adverse listening conditions. Neuroimage 49, 875-884.
16. Hubbard, A. L., Wilson, S. M., Callan, D. E., and Dapretto, M. (2009). Giving speech a hand: gesture modulates activity in auditory cortex during speech perception. Hum. Brain Mapp. 30, 1028-1037.
17. Kaiser, E., and Trueswell, J. C. (2004). The role of discourse context in the processing of a flexible word-order language. Cognition 94, 113-147.
18. Kelly, S. D., Creigh, P., and Bartolotti, J. (2010a). Integrating speech and iconic gestures in a Stroop-like task: evidence for automatic processing. J. Cogn. Neurosci. 22, 683-694.
19. Kelly, S. D., Özyürek, A., and Maris, E. (2010b). Two sides of the same coin: speech and gesture mutually interact to enhance comprehension. Psychol. Sci. 21, 260-267.
20. Kempen, G., and Harbusch, K. (2003). An artificial opposition between grammaticality and frequency: comment on Bornkessel, Schlesewsky, and Friederici. (2002). Cognition 90, 205-210; discussion 211-203, 215.
21. Kita, S., and Özyürek, A. (2003). What does cross-linguistic variation in semantic coordination of speech and gesture reveal? Evidence for an interface representation of spatial thinking and speaking. J. Mem. Lang. 48, 16-32.
22. Knoeferle, P., Habets, B., Crocker, M. W., and Munte, T. F. (2008). Visual scenes trigger immediate syntactic reanalysis: evidence from ERPs during situated spoken comprehension. Cereb. Cortex 18, 789-795.
23. Krahmer, E., and Swerts, M. (2007). The effects of visual beats on prosodic prominence: acoustic analyses, auditory perception and visual perception. J. Mem. Lang. 57, 396-414.
24. Krauss, R. M. (1998). Why do we gesture when we speak. Curr. Dir. Psychol. Sci. 7, 54-60.
25. Krauss, R. M., Dushay, R. A., Chen, Y. S., and Rauscher, F. (1995). The communicative value of conversational hand gestures. J. Exp. Soc. Psychol. 31, 533-552.
26. Krauss, R. M., Morrel-Samuels, P., and Colasante, C. (1991). Do conversational hand gestures communicate? J. Pers. Soc. Psychol. 61, 743-754.
27. Levelt, W. J. M., Richardson, G., and la Heij, W. (1985). Pointing and voicing in deictic expressions. J. Mem. Lang. 24, 133-164.
28. McNeill, D. (1992). Hand and Mind What Gestures Reveal about Thought. Chicago: The University of Chicago Press.
29. Obermeier, C., Holle, H., and Gunter, T. C. (2011). What iconic gesture fragments reveal about gesture-speech integration: when synchrony is lost, memory can help. J. Cogn. Neurosci. 23, 1648-1663.
30. Oldfield, R. C. (1971). The assessment and analysis of handedness: the Edinburgh inventory. Neuropsychologia 9, 97-113.
31. Osterhout, L., and Holcomb, P. J. (1992). Event-related brain potentials elicited by syntactic anomaly. J. Mem. Lang. 31, 785-806.
32. Özyürek, A., Willems, R. M., Kita, S., and Hagoort, P. (2007). Online integration of semantic information from speech and gesture: insights from event-related brain potentials. J. Cogn. Neurosci. 19, 605-616.
33. Schriefers, H., Friederici, A. D., and Kuhn, K. (1995). The processing of locally ambiguous relative clauses in German. J. Mem. Lang. 34, 499-520. van de Meerendonk, N., Kolk, H. H., Vissers, C. T., and Chwilla, D. J.
34. (2010). Monitoring in language perception: mild and strong conflicts elicit different ERP patterns. J. Cogn. Neurosci. 22, 67-82.
35. Willems, R. M., Özyürek, A., and Hagoort, P. (2007). When language meets action: the neural integration of gesture and speech. Cereb. Cortex 17, 2322-2333.
36. Willems, R. M., Özyürek, A., and Hagoort, P. (2009). Differential roles for left inferior frontal and superior temporal cortex in multimodal integration of action and language. Neuroimage 47, 1992-2004.
37. Wu, Y. C., and Coulson, S. (2010). Gestures modulate speech processing early in utterances. Neuroreport 21, 522-526.
38. Xu, J., Gannon, P. J., Emmorey, K., Smith, J. F., and Braun, A. R. (2009). Symbolic gestures and spoken language are processed by a common neural system. Proc. Natl. Acad. Sci. U.S.A. 106, 20664-20669.