Neural integration of iconic and unrelated coverbal gestures: A functional MRI study

Human Brain Mapping

Volumn 30, Issue 10, 2009, Pages 3309-3324

  Green, Antonia ^a,c   Straube, Benjamin ^a   Weis, Susanne ^b   Jansen, Andreas ^a,c   Willmes, Klaus ^a   Konrad, Kerstin ^a   Kircher, Tilo ^c  

^a RWTH AACHEN UNIVERSITY   (Germany)

^b UNIVERSITY HOSPITAL RWTH AACHEN   (Germany)

^c PHILIPPS UNIVERSITY MARBURG   (Germany)

Author keywords

fMRI; Language; Multimodal integration; Perception; Semantic processing

Indexed keywords

ADULT; ARTICLE; BRAIN REGION; ELECTROENCEPHALOGRAM; FUNCTIONAL MAGNETIC RESONANCE IMAGING; GESTURE; HUMAN; HUMAN EXPERIMENT; INFORMATION; LANGUAGE; LINGUISTICS; MALE; NORMAL HUMAN; OCCIPITAL CORTEX; PARIETAL LOBE; PRIORITY JOURNAL; SPATIAL DISCRIMINATION; SPEECH; TEMPORAL CORTEX; VERBAL COMMUNICATION; VIDEORECORDING;

ADULT; ANALYSIS OF VARIANCE; BRAIN; BRAIN MAPPING; COMPREHENSION; GESTURES; HUMANS; IMAGE PROCESSING, COMPUTER-ASSISTED; MAGNETIC RESONANCE IMAGING; MALE; MIDDLE AGED; OXYGEN; PHOTIC STIMULATION; RECOGNITION (PSYCHOLOGY); SPEECH; YOUNG ADULT;

EID: 70349316482     PISSN: 10659471     EISSN: None     Source Type: Journal    
DOI: 10.1002/hbm.20753     Document Type: Article

References (81)

1. Barsalou LW, Kyle Simmons W, Barbey AK, Wilson CD (2003): Grounding conceptual knowledge in modality-specific systems. Trends Cogn Sci 7:84-91. (Pubitemid 36173638)
2. Bartolo A, Weisbecker A, Coello Y (2007): Linguistic and spatial information for action. Behav Brain Res 184:19-30.
3. Baumgärtner A, Weiller C, Büchel C (2002): Event-related fMRI reveals cortical sites involved in contextual sentence integration. Neuroimage 16:736-745.
4. Beauchamp MS (2005): Statistical criteria in fMRI studies of multisensory integration. Neuroinformatics 3:93-113.
5. Beauchamp MS, Argall BD, Bodurka J, Duyn JH, Martin A (2004a): Unraveling multisensory integration: Patchy organization within human STS multisensory cortex. Nat Neurosci 7:1190-1192. (Pubitemid 39426237)
6. Beauchamp MS, Lee KE, Argall BD, Martin A (2004b): Integration of auditory and visual information about objects in superior temporal sulcus. Neuron 41:809-823.
7. Bookheimer S (2002): Functional MRI of language: New approaches to understanding the cortical organization of semantic processing. Ann Rev Neurosci 25:151-188.
8. Bruce C, Desimone R, Gross CG (1981): Visual properties of neurons in a polysensory area in superior temporal sulcus of the macaque. J Neurophysiol 46:369-384. (Pubitemid 11087648)
9. Calvert GA (2001): Crossmodal processing in the human brain: Insights from functional neuroimaging studies. Cereb Cortex 11:1110-1123. (Pubitemid 33108238)
10. Calvert GA, Campbell R, Brammer MJ (2000): Evidence from functional magnetic resonance imaging of crossmodal binding in the human heteromodal cortex. Curr Biol 10:649-657. (Pubitemid 30354700)
11. Chao LL, Haxby JV, Martin A (1999): Attribute-based neural substrates in temporal cortex for perceiving and knowing about objects. Nat Neurosci 2:913-919.
12. Chao LL, Martin A (2000): Representation of manipulable manmade objects in the dorsal stream. Neuroimage 12:478-484.
13. Corbetta M, Shulman GL (2002): Control of goal-directed and stimulus-driven attention in the brain. Nat Rev Neurosci 3:201-215. (Pubitemid 135706590)
14. Craighero L, Metta G, Sandini G, Fadiga L (2007): The mirror-neurons system: Data and models. In: von Hofsten C, Rosander K, editors. From Action to Cognition. Amsterdam: Elsevier. pp 39-59.
15. Culham JC, Kanwisher NG (2001): Neuroimaging of cognitive functions in human parietal cortex. Curr Opin Neurobiol 11:157-163. (Pubitemid 32289401)
16. De Renzi E, Lucchelli F (1988): Ideational apraxia. Brain 111:1173-1185.
17. Démonet JF, Thierry G, Cardebat D (2005): Renewal of the neurophysiology of language: Functional neuroimaging. Physiol Rev 85:49-95. (Pubitemid 40171258)
18. Ebisch SJH, Babiloni C, Del Gratta C, Ferretti A, Perrucci MG, Caulo M, Sitskoorn MM, Luca Romani G (2007): Human neural systems for conceptual knowledge of proper object use: A functional magnetic resonance imaging study. Cereb Cortex 17:2744-2751. (Pubitemid 47537835)
19. Eickhoff SB, Stephan KE, Mohlberg H, Grefkes C, Fink GR, Amunts K, Zilles K (2005): A new SPM toolbox for combining probabilistic cytoarchitectonic maps and functional imaging data. Neuroimage 25:1325-1335. (Pubitemid 40591672)
20. Farah MJ (1995): Current issues in the neuropsychology of image generation. Neuropsychologia 33:1455-1471.
21. Friederici AD, Ruschemeyer SA, Hahne A, Fiebach CJ (2003): The role of left inferior frontal and superior temporal cortex in sentence comprehension: Localizing syntactic and semantic processes. Cereb Cortex 13:170-177. (Pubitemid 36084046)
22. Gentilucci M, Bernardis P, Crisi G, Volta RD (2006): Repetitive transcranial magnetic stimulation of Broca's area affects verbal responses to gesture observation. J Cogn Neurosci 18:1059-1074. (Pubitemid 44066111)
23. Goebel R, Khorram-Sefat D, Muckli L, Hacker H, Singer W (1998): The constructive nature of vision: Direct evidence from functional magnetic resonance imaging studies of apparent motion and motion imagery. Eur J Neurosci 10:1563-1573. (Pubitemid 28327304)
24. Grafton ST, Hazeltine E, Ivry RB (1998): Abstract and effector-specific representations of motor sequences identified with PET. J Neurosci 18:9420-9428. (Pubitemid 28512472)
25. Grèzes J, Decety J (2001): Functional anatomy of execution, mental simulation, observation, and verb generation of actions: A meta analysis. Hum Brain Mapp 12:1-19. (Pubitemid 32048353)
26. Hadar U, Krauss RK (1999): Iconic gestures: The grammatical categories of lexical affiliates. J Neurolinguistics 12:1-12.
27. Hadar U, Pinchas-Zamir L (2004): The semantic specificity of gesture: Implications for gesture classification and function. J Lang Soc Psychol 23:204-214. (Pubitemid 38614892)
28. Hein G, Doehrmann O, Müller NG, Kaiser J, Muckli L, Naumer MJ (2007): Object familiarity and semantic congruency modulate responses in cortical audiovisual integration areas. J Neurosci 27:7881-7887. (Pubitemid 47173312)
29. Hickok G, Poeppel D (2000): Towards a functional neuroanatomy of speech perception. Trends Cogn Sci 4:131-138. (Pubitemid 30182529)
30. Hickok G, Poeppel D (2007): The cortical organization of speech processing. Nat Rev Neurosci 8:393-402.
31. Hocking J, Price CJ (2008): The role of the posterior superior temporal sulcus in audiovisual processing. Cereb Cortex 18:2439-2449.
32. Holle H, Gunter TC (2007): The role of iconic gestures in speech disambiguation: ERP evidence. J Cogn Neurosci 19:1175-1192. (Pubitemid 47026506)
33. Holle H, Gunter TC, Rüschemeyer SA, Hennenlotter A, Iacoboni M (2008): Neural correlates of the processing of co-speech gestures. Neuroimage 39:2010-2024.
34. Husain M, Nachev P (2007): Space and the parietal cortex. Trends Cogn Sci 11:30-36.
35. Ishai A, Ungerleider LG, Haxby JV (2000a): Distributed neural systems for the generation of visual images. Neuron 28:979-990. (Pubitemid 32038086)
36. Ishai A, Ungerleider LG, Martin A, Haxby JV (2000b): The representation of objects in the human occipital and temporal cortex. J Cogn Neurosci 12:35-51.
37. Ishai A, Ungerleider LG, Martin A, Schouten JL, Haxby JV (1999): Distributed representation of objects in the human ventral visual pathway. Proc Natl Acad Sci USA 96:9379-9384. (Pubitemid 29374763)
38. Jahn K, Deutschländer A, Stephan T, Strupp M, Wiesmann M, Brandt T (2004): Brain activation patterns during imagined stance and locomotion in functional magnetic resonance imaging. Neuroimage 22:1722-1731. (Pubitemid 38962660)
39. Jenkins IH, Brooks DJ, Nixon PD, Frackowiak RS, Passingham RE (1994): Motor sequence learning: A study with positron emission tomography. J Neurosci 14:3775-3790. (Pubitemid 24174123)
40. Johnson MR, Mitchell KJ, Raye CL, D'Esposito M, Johnson MK (2007): A brief thought can modulate activity in extrastriate visual areas: Top-down effects of refreshing just-seen visual stimuli. Neuroimage 37:290-299. (Pubitemid 47068992)
41. Kable JW, Kan IP, Wilson A, Thompson-Schill SL, Chatterjee A (2005): Conceptual representations of action in the lateral temporal cortex. J Cogn Neurosci 17:1855-1870.
42. Katanoda K, Matsuda Y, Sugishita M (2002): A spatio-temporal regression model for the analysis of functional MRI data. Neuroimage 17:1415-1428. (Pubitemid 35333861)
43. Kircher T, Straube B, Leube D, Weis S, Sachs O, Willmes K, Konrad K, Green A (2009): Neural interaction of speech and gesture: Differential activations of metaphoric co-verbal gestures. Neuropsychologia 47:169-179.
44. Kita S, Ozyürek A (2003): What does cross-linguistic variation in semantic coordination of speech and gesture reveal? Evidence for an interface representation of spatial thinking and speaking. J Mem Lang 48:16-32. (Pubitemid 36177475)
45. Kourtzi Z, Kanwisher N (2000): Activation in human MT/MST by static images with implied motion. J Cogn Neurosci 12:48-55. (Pubitemid 30142783)
46. Kuperberg GR, Holcomb PJ, Sitnikova T, Greve D, Dale AM, Caplan D (2003): Distinct patterns of neural modulation during the processing of conceptual and syntactic anomalies. J Cogn Neurosci 15:272-293. (Pubitemid 36293136)
47. Kuperberg GR, McGuire PK, Bullmore ET, Brammer MJ, Rabe- Hesketh S, Wright IC, Lythgoe DJ, Williams SCR, David AS (2000): Common and distinct neural substrates for pragmatic, semantic, and syntactic processing of spoken sentences: An fMRI study. J Cogn Neurosci 12:321-341.
48. Kuperberg GR, Sitnikova T, Lakshmanan BM (2008): Neuroanatomical distinctions within the semantic system during sentence comprehension: Evidence from functional magnetic resonance imaging. Neuroimage 40:367-388.
49. Kutas M, Hillyard SA (1980): Reading senseless sentences: Brain potentials reflect semantic incongruity. Science 207:203-205. (Pubitemid 10141432)
50. Lewis JW (2006): Cortical networks related to human use of tools. Neuroscientist 12:211-231.
51. Malikovic A, Amunts K, Schleicher A, Mohlberg H, Eickhoff SB, Wilms M, Palomero-Gallagher N, Armstrong E, Zilles K (2007): Cytoarchitectonic analysis of the human extrastriate cortex in the region of V5/MT: A probabilistic, stereotaxic map of area hOc5. Cereb Cortex 17:562-574. (Pubitemid 46254695)
52. Martin A, Chao LL (2001): Semantic memory and the brain: Structure and processes. Curr Opin Neurobiol 11:194-201. (Pubitemid 32289406)
53. Martin A, Wiggs CL, Ungerleider LG, Haxby JV (1996): Neural correlates of category-specific knowledge. Nature 379:649-652.
54. McNeill D (1992): Hand and Mind-What Gestures Reveal About Thought. Chicago, Illinois, and London, England: The University of Chicago Press.
55. Michelon P, Snyder AZ, Buckner RL, McAvoy M, Zacks JM (2003): Neural correlates of incongruous visual information: An event-related fMRI study. Neuroimage 19:1612-1626. (Pubitemid 37025858)
56. Mummery CJ, Patterson K, Hodges JR, Price CJ (1998): Functional neuroanatomy of the semantic system: Divisible by what? J Cogn Neurosci 10:766-777.
57. Ni W, Constable RT, Mencl WE, Pugh KR, Fulbright RK, Shaywitz SE, Shaywitz BA, Gore JC, Shankweiler D (2000): An event-related neuroimaging study distinguishing form and content in sentence processing. J Cogn Neurosci 12:120-133. (Pubitemid 30142788)
58. Nichols T, Brett M, Andersson J, Wager T, Poline JB (2005): Valid conjunction inference with the minimum statistic. Neuroimage 25:653-660. (Pubitemid 40461704)
59. Nishitani N, Schurmann M, Amunts K, Hari R (2005): Broca's region: From action to language. Physiology 20:60-69.
60. Novick JM, Trueswell JC, Thompson-Schill SL (2005): Cognitive control and parsing: Reexamining the role of Broca's area in sentence comprehension. Cogn Affect Behav Neurosci 5:263-281.
61. Oldfield RC (1971): The assessment and analysis of handedness: The Edinburgh inventory. Neuropsychologia 9:97-113.
62. Olivetti Belardinelli M, Sestieri C, Matteo R, Delogu F, Gratta C, Ferretti A, Caulo M, Tartaro A, Romani G (2004): Audio-visual crossmodal interactions in environmental perception: An fMRI investigation. Cogn Process 5:167-174.
63. Ozyürek A, Willems RM, Kita S, Hagoort P (2007): On-line integration of semantic information from speech and gesture: Insights from event-related brain potentials. J Cogn Neurosci 19:605-616. (Pubitemid 46495292)
64. Padberg J, Seltzer B, Cusick CG (2003): Architectonics and cortical connections of the upper bank of the superior temporal sulcus in the rhesus monkey: An analysis in the tangential plane. J Comp Neurol 467:418-434. (Pubitemid 37421084)
65. Perani D, Schnur T, Tettamanti M, Italy, Cappa SF, Fazio F (1999): Word and picture matching: A PET study of semantic category effects. Neuropsychologia 37:293-306. (Pubitemid 29113255)
66. Rizzolatti G, Arbib MA (1998): Language within our grasp. Trends Neurosci 21:188-194.
67. Rizzolatti G, Craighero L (2004): The mirror neuron system. Ann Rev Neurosci 27:169-192.
68. Ross RS, Slotnick SD (2008): The hippocampus is preferentially associated with memory for spatial context. J Cogn Neurosci 20:432-446. (Pubitemid 351317602)
69. Seltzer B, Pandya DN (1978): Afferent cortical connections and architectonics of the superior temporal sulcus and surrounding cortex in the rhesus monkey. Brain Res 149:1-24. (Pubitemid 8350020)
70. Slotnick SD, Schacter DL (2004): A sensory signature that distinguishes true from false memories. Nat Neurosci 7:664-672.
71. Straube B, Green A, Weis S, Chatterjee A, Kircher T (2008): Memory effects of speech and gesture binding: Cortical and hippocampal activation in relation to subsequent memory performance. J Cogn Neurosci [Epub ahead of print].
72. Taylor KI, Moss HE, Stamatakis EA, Tyler LK (2006): Binding crossmodal object features in perirhinal cortex. Proc Natl Acad Sci USA 103:8239-8244.
73. Teder-Sälejärvi WA, McDonald JJ, Di Russo F, Hillyard SA (2002): An analysis of audio-visual crossmodal integration by means of event-related potential (ERP) recordings. Brain Res Cogn Brain Res 14:106-114.
74. Tranel D, Damasio H, Damasio AR (1997): A neural basis for the retrieval of conceptual knowledge. Neuropsychologia 35:1319-1327. (Pubitemid 27455559)
75. Tranel D, Feinstein J, Manzel K (2008): Further lesion evidence for the neural basis of conceptual knowledge for persons and other concrete entities. J Neuropsychol 2:301-320.
76. Van Essen DC (2004): Surface-based approaches to spatial localization and registration in primate cerebral cortex. Neuroimage 23:S97-S107.
77. Vigneau M, Beaucousin V, Hervé PY, Duffau H, Crivello F, Houdé O, Mazoyer B, Tzourio-Mazoyer N (2006): Meta-analyzing left hemisphere language areas: Phonology, semantics, and sentence processing. Neuroimage 30:1414-1432.
78. Willems RM, Hagoort P (2007): Neural evidence for the interplay between language, gesture, and action: A review. Brain Lang 101:278-289. (Pubitemid 46802720)
79. Willems RM, Ozyürek A, Hagoort P (2007): When language meets action: The neural integration of gesture and speech. Cereb Cortex 17:2322-2333. (Pubitemid 47517468)
80. Wu YC, Coulson S (2007): How iconic gestures enhance communication: An ERP study. Brain Lang 101:234-245.
81. Zarahn E, Aguirre GK, D'Esposito M (1997): Empirical analyses of BOLD fMRI statistics. Neuroimage 5:179-197.