Neural correlates of the processing of co-speech gestures

NeuroImage

Volumn 39, Issue 4, 2008, Pages 2010-2024

  Holle, Henning ^a   Gunter, Thomas C ^a   Rüschemeyer, Shirley Ann ^a   Hennenlotter, Andreas ^a   Iacoboni, Marco ^b  

^a MAX PLANCK INSTITUTE FOR HUMAN COGNITIVE AND BRAIN SCIENCES   (Germany)

^b UNIVERSITY OF CALIFORNIA   (United States)

Author keywords

[No Author keywords available]

Indexed keywords

ADULT; ARTICLE; AUDIOVISUAL EQUIPMENT; BRAIN FUNCTION; BRAIN REGION; FEMALE; FUNCTIONAL MAGNETIC RESONANCE IMAGING; GESTURE; GROOMING; HUMAN; HUMAN EXPERIMENT; MALE; MOVEMENT (PHYSIOLOGY); NORMAL HUMAN; PRIORITY JOURNAL; SPEECH; VIDEORECORDING; WORD LIST RECALL;

ADULT; COMMUNICATION; DECISION MAKING; FEMALE; FRONTAL LOBE; GESTURES; HAND; HUMANS; IMAGE PROCESSING, COMPUTER-ASSISTED; LANGUAGE; MAGNETIC RESONANCE IMAGING; MALE; MOTION PERCEPTION; MOVEMENT; PSYCHOMOTOR PERFORMANCE; REACTION TIME; SPEECH PERCEPTION; VIDEOTAPE RECORDING; VISUAL PERCEPTION;

EID: 38949110718     PISSN: 10538119     EISSN: None     Source Type: Journal    
DOI: 10.1016/j.neuroimage.2007.10.055     Document Type: Article

References (76)

1. Alibali M.W., Flevares L.M., and Goldin-Meadow S. Assessing knowledge conveyed in gesture - do teachers have the upper hand. J. Educ. Psychol. 89 (1997) 183-193
2. Allison T., Puce A., and McCarthy G. Social perception from visual cues: role of the STS region. Trends Cogn. Sci. 4 (2000) 267-278
3. Baddeley A.D. Is working memory still working?. Eur. Psychol. 7 (2002) 85-97
4. Beattie G., and Shovelton H. Do iconic hand gestures really contribute anything to the semantic information conveyed by speech? An experimental investigation. Semiotica 123 (1999) 1-30
5. Beattie G., and Shovelton H. An experimental investigation of some properties of individual iconic gestures that mediate their communicative power. Br. J. Psychol. 93 (2002) 179-192
6. Beauchamp M.S. See me, hear me, touch me: multisensory integration in lateral occipital-temporal cortex. Curr. Opin. Neurobiol. 15 (2005) 145-153
7. Beauchamp M.S., Argall B.D., Bodurka J., Duyn J.H., and Martin A. Unraveling multisensory integration: patchy organization within human STS multisensory cortex. Nat. Neurosci. 7 (2004) 1190-1192
8. Beauchamp M.S., Lee K.E., Argall B.D., and Martin A. Integration of auditory and visual information about objects in superior temporal sulcus. Neuron 41 (2004) 809-823
9. Binkofski F., and Buccino G. The role of ventral premotor cortex in action execution and action understanding. J. Physiol. (Paris) 99 (2006) 396-405
10. Calvert G.A., and Thesen T. Multisensory integration: methodological approaches and emerging principles in the human brain. J. Physiol. (Paris) 98 (2004) 191-205
11. Calvert G.A., Campbell R., and Brammer M.J. Evidence from functional magnetic resonance imaging of crossmodal binding in the human heteromodal cortex. Curr. Biol. 10 (2000) 649-657
12. Cassell J., McNeill D., and McCullough K.-E. Speech-gesture mismatches: evidence for one underlying representation of linguistic and nonlinguistic information. Pragmat. Cogn. 7 (1999) 1-33
13. Chomsky N. Aspects of the theory of syntax (1965), MIT Press, London
14. Culham J., He S., Dukelow S., and Verstraten F.A. Visual motion and the human brain: what has neuroimaging told us?. Acta Psychol. (Amst.) 107 (2001) 69-94
15. Dupont P., De Bruyn B., Vandenberghe R., Rosier A.M., Michiels J., Marchal G., Mortelmans L., and Orban G.A. The kinetic occipital region in human visual cortex. Cereb. Cortex 7 (1997) 283-292
16. Feyereisen P., Van de Wiele M., and Dubois F. The meaning of gestures: what can be understood without speech?. Cah. Psychol. Cogn./Curr. Psychol. Cogn. 8 (1988) 3-25
17. Fiebach C.J., Friederici A.D., Muller K., and von Cramon D.Y. fMRI Evidence for dual routes to the Mental Lexicon in visual word recognition. J. Cogn. Neurosci. 14 (2002) 11-23
18. Fiez J.A., Balota D.A., Raichle M.E., and Petersen S.E. Effects of lexicality, frequency, and spelling-to-sound consistency on the functional anatomy of reading. Neuron 24 (1999) 205-218
19. Fridman E.A., Immisch I., Hanakawa T., Bohlhalter S., Waldvogel D., Kansaku K., Wheaton L., Wu T., and Hallett M. The role of the dorsal stream for gesture production. NeuroImage 29 (2006) 417-428
20. Friston K.J., Fletcher P., Josephs O., Holmes A., Rugg M.D., and Turner R. Event-related fMRI: characterizing differential responses. NeuroImage 7 (1998) 30-40
21. Gallese V., Fadiga L., Fogassi L., and Rizzolatti G. Action recognition in the premotor cortex. Brain 119 (1996) 593-609
22. Gallese V., Fadiga L., Fogassi L., and Rizzolatti G. Action representation and the inferior parietal lobule. In: Prinz W., and Hommel B. (Eds). Common mechanisms in perception and action (2002), Oxford Univ. Press, New York
23. Gauthier I., and Bukach C. Should we reject the expertise hypothesis?. Cognition 103 (2007) 322-330
24. Gauthier I., Skudlarski P., Gore J.C., and Anderson A.W. Expertise for cars and birds recruits brain areas involved in face recognition. Nat. Neurosci. 3 (2000) 191-197
25. Goldin-Meadow S. Hearing Gesture - How Our Hands Help Us Think (2003), The Belknap Press of Harvard Univ. Press, Cambridge
26. Grill-Spector K., and Malach R. The human visual cortex. Annu. Rev. Neurosci. 27 (2004) 649-677
27. Grodzinsky Y., and Friederici A.D. Neuroimaging of syntax and syntactic processing. Curr. Opin. Neurobiol. 16 (2006) 240-246
28. Grosjean F. Gating. Lang. Cogn. Processes 11 (1996) 597-604
29. Gunter T.C., and Bach P. Communicating hands: ERPs elicited by meaningful symbolic hand postures. Neurosci. Lett. 372 (2004) 52-56
30. Gunter T.C., Wagner S., and Friederici A.D. Working memory and lexical ambiguity resolution as revealed by ERPs: a difficult case for activation theories. J. Cogn. Neurosci. 15 (2003) 643-657
31. Hadar U., and Pinchas-Zamir L. The semantic specificity of gesture - Implications for gesture classification and function. J. Lang. Soc. Psychol. 23 (2004) 204-214
32. Holle H., and Gunter T.C. The role of iconic gestures in speech disambiguation: ERP evidence. J. Cogn. Neurosci. 19 (2007) 1175-1192
33. Holler J., and Beattie G. Pragmatic aspects of representational gestures: do speakers use them to clarify verbal ambiguity for the listener?. Gesture 3 (2003) 127-154
34. Iacoboni M. Neural mechanisms of imitation. Curr. Opin. Neurobiol. 15 (2005) 632-637
35. Iacoboni M., and Dapretto M. The mirror neuron system and the consequences of its dysfunction. Nat. Rev., Neurosci. 7 (2006) 942-951
36. Iacoboni M., and Wilson S.M. Beyond a single area: motor control and language within a neural architecture encompassing Broca's area. Cortex 42 (2006) 503-506
37. Josephs O., Turner R., and Friston K. Event-related fMRI. Hum. Brain Mapp. 5 (1997) 243-248
38. Joubert S., Beauregard M., Walter N., Bourgouin P., Beaudoin G., Leroux J.M., Karama S., and Lecours A.R. Neural correlates of lexical and sublexical processes in reading. Brain Lang. 89 (2004) 9-20
39. Kanwisher N., McDermott J., and Chun M.M. The fusiform face area: a module in human extrastriate cortex specialized for face perception. J. Neurosci. 17 (1997) 4302-4311
40. Kelly S.D., Kravitz C., and Hopkins M. Neural correlates of bimodal speech and gesture comprehension. Brain Lang. 89 (2004) 253-260
41. Kita S., and Özyürek A. What does cross-linguistic variation in semantic coordination of speech and gesture reveal? Evidence for an interface representation of spatial thinking and speaking. J. Mem. Lang. 48 (2003) 16-32
42. Krauss R.M., Morrel-Samuels P., and Colasante C. Do conversational hand gestures communicate?. J. Pers. Soc. Psychol. 61 (1991) 743-754 (November)
43. Lee J.H., Garwood M., Menon R., Adriany G., Andersen P., Truwit C.L., and Ugurbil K. High contrast and fast three-dimensional magnetic resonance imaging at high fields. Magn. Reson. Med. 34 (1995) 308-312
44. Levelt W.J.M., Richardson G., and la Heij W. Pointing and voicing in deictic expressions. J. Mem. Lang. 24 (1985) 133-164
45. Lohmann G., Muller K., Bosch V., Mentzel H., Hessler S., Chen L., Zysset S., and von Cramon D.Y. LIPSIA - a new software system for the evaluation of functional magnetic resonance images of the human brain. Comput. Med. Imaging Graph. 25 (2001) 449-457
46. McNeill D. Hand and Mind - What Gestures Reveal about Thought (1992), The University of Chicago Press, Chicago
47. In: McNeill D. (Ed). Language and Gesture (2000), Cambridge Univ. Press, Cambridge
48. McNeill D. Gesture and Thought (2005), University of Chicago Press, Chicago and London
49. Molnar-Szakacs I., Kaplan J., Greenfield P.M., and Iacoboni M. Observing complex action sequences: the role of the fronto-parietal mirror neuron system. NeuroImage 33 (2006) 923-935
50. Morrel-Samuels P., and Krauss R.M. Word familiarity predicts temporal asynchrony of hand gestures and speech. Journal of Experimental Psychology: Learning. Mem. Cogn. 18 (1992) 615-622
51. Nixon P., Lazarova J., Hodinott-Hill I., Gough P., and Passingham R. The inferior frontal gyrus and phonological processing: an investigation using rTMS. J. Cogn. Neurosci. 16 (2004) 289-300
52. Nobe S. Where do most spontaneous representational gestures actually occur with respect to speech?. In: McNeill D. (Ed). Language and Gesture (2000), Cambridge Univ. Press, Cambridge 186-198
53. Norris D.G. Reduced power multislice MDEFT imaging. JMRI-J. Magn. Reson. Imaging 11 (2000) 445-451
54. Oldfield R.C. The assessment and analysis of handedness: The Edinburgh inventory. Neuropsychologia 9 (1971) 97-113
55. Özyürek A., Willems R.M., Kita S., and Hagoort P. On-line integration of semantic information from speech and gesture: insights from event-related brain potentials. J. Cogn. Neurosci. 19 (2007) 605-616
56. Paulesu E., Perani D., Blasi V., Silani G., Borghese N.A., De Giovanni U., Sensolo S., and Fazio F. A functional-anatomical model for lipreading. J. Neurophysiol. 90 (2003) 2005-2013
57. Pelphrey K.A., Mitchell T.V., McKeown M.J., Goldstein J., Allison T., and McCarthy G. Brain activity evoked by the perception of human walking: controlling for meaningful coherent motion. J. Neurosci. 23 (2003) 6819-6825
58. Rizzolatti G., and Craighero L. The mirror-neuron system. Annu. Rev. Neurosci. 27 (2004) 169-192
59. Rizzolatti G., Fogassi L., and Gallese V. Neurophysiological mechanisms underlying the understanding and imitation of action. Nat. Rev., Neurosci. 2 (2001) 661-670
60. Rizzolatti G., Fogassi L., and Gallese V. Motor and cognitive functions of the ventral premotor cortex. Curr. Opin. Neurobiol. 12 (2002) 149-154
61. Rose M.L. The utility of arm and hand gestures in the treatment of aphasia. Adv. Speech-Lang. Pathol. 8 (2006) 92-109
62. Saygin A.P., Dick F., Wilson S.M., Dronkers N.F., and Bates E. Neural resources for processing language and environmental sounds: evidence from aphasia. Brain 126 (2003) 928-945
63. Schubotz R.I. Human Premotor Cortex: Beyond Motor Performance (2004), Max Planck Institute for Human Cognitive and Brain Sciences, Leipzig
64. Sekiyama K., Kanno I., Miura S., and Sugita Y. Auditory-visual speech perception examined by fMRI and PET. Neurosci. Res. 47 (2003) 277-287
65. Skipper J.I., Nusbaum H.C., and Small S.L. Listening to talking faces: motor cortical activation during speech perception. NeuroImage 25 (2005) 76-89
66. Tanenhaus M.K., and Trueswell J.C. Sentence Comprehension. In: Miller J., and Eimas P. (Eds). Speech, language, and communication. Handbook of perception and cognition (1995), Academic Press, San Diego 217-262
67. Tarr M.J., and Gauthier I. FFA: a flexible fusiform area for subordinate-level visual processing automatized by expertise. Nat. Neurosci. 3 (2000) 764-769
68. Ugurbil K., Garwood M., Ellermann J., Hendrich K., Hinke R., Hu X., Kim S.G., Menon R., Merkle H., Ogawa S., et al. Imaging at high magnetic fields: initial experiences at 4 T. Magn. Reson. Q. 9 (1993) 259-277
69. Umiltà M.A., Kohler E., Gallese V., Fogassi L., Fadiga L., Keysers C., and Rizzolatti G. I know what you are doing. a neurophysiological study. Neuron 31 (2001) 155-165
70. van Atteveldt N., Formisano E., Goebel R., and Blomert L. Integration of letters and speech sounds in the human brain. Neuron 43 (2004) 271-282
71. Wagner, S., 2003. Verbales Arbeitsgedächtnis und die Verarbeitung lexikalisch ambiger Wörter in Wort- und Satzkontexten (PhD thesis). Max-Planck-Institute for Cognitive Neuroscience, Leipzig.
72. Ward B.D. AlphaSim - Simultaneous Inference for FMRI Data (2000), Biophysics Research Institute, Medical College of Wisconsin, Milwaukee, WI
73. Willems, R.M., Özyürek, A., Hagoort, P., 2006. When Language Meets Action: The Neural Integration of Gesture and Speech. Cerebral Cortex, bhl141.
74. Worsley K.J., Liao C.H., Aston J., Petre V., Duncan G.H., Morales F., and Evans A.C. A general statistical analysis for fMRI data. NeuroImage 15 (2002) 1-15
75. Wright T.M., Pelphrey K.A., Allison T., McKeown M.J., and McCarthy G. Polysensory interactions along lateral temporal regions evoked by audiovisual speech. Cereb. Cortex 13 (2003) 1034-1043
76. Wu Y.C., and Coulson S. How iconic gestures enhance communication: an ERP study. Brain Lang. 101 (2007) 234-245