The impact of iconic gestures on foreign language word learning and its neural substrate

Human Brain Mapping

Volumn 32, Issue 6, 2011, Pages 982-998

  Macedonia, Manuela ^a   Müller, Karsten ^a   Friederici, Angela D ^a  

^a MAX PLANCK INSTITUTE FOR HUMAN COGNITIVE AND BRAIN SCIENCES   (Germany)

Author keywords

Cognitive control; Foreign language learning; Gestures; Memory; Premotor cortex; Vocabulary acquisition

Indexed keywords

ADULT; ARTICLE; BRAIN FUNCTION; CONTROLLED STUDY; EXECUTIVE FUNCTION; FEMALE; FUNCTIONAL MAGNETIC RESONANCE IMAGING; GESTURE; HUMAN; LANGUAGE; LEARNING; LINGUISTICS; MALE; PREMOTOR CORTEX; PRIORITY JOURNAL; SEMANTICS; SPEECH; VERBAL MEMORY; WORD RECOGNITION;

BRAIN; BRAIN MAPPING; FEMALE; GESTURES; HUMANS; IMAGE INTERPRETATION, COMPUTER-ASSISTED; LANGUAGE; LEARNING; MAGNETIC RESONANCE IMAGING; MALE; YOUNG ADULT;

EID: 79955705551     PISSN: 10659471     EISSN: 10970193     Source Type: Journal    
DOI: 10.1002/hbm.21084     Document Type: Article

References (123)

1. Anderson JR (1983): A spreading activation theory of memory. J Verb Learn Verb Behav 22: 261-295.
2. Arbib MA (2006): Action to Language via the Mirror Neuron System, Vol. 13. Cambridge: Cambridge University Press. 552p.
3. Bäckman L, Nilsson LG (1984): Aging effects in free recall: An exception to the rule. Hum Learn: J Pract Res Appl 3: 53-69.
4. Bäckman L, Nilsson LG (1985): Prerequisites for lack of age differences in memory performance. Exp Aging Res 11: 67-73.
5. Barbieri F, Buonocore A, Dalla Volta R, Gentilucci M (2009): How symbolic gestures and words interact with each other. Brain Lang 110: 1-11.
6. Beattie G, Coughlan J (1999): An experimental investigation of the role of iconic gestures in lexical access using the tip-of-the-tongue phenomenon. Br J Psychol 90: 35-56.
7. Beattie G, Shovelton H (1999): Do iconic hand gestures really contribute anything to the semantic information conveyed by speech? An experimental investigation. Semiotica 123: 1-30.
8. Bernardis P, Gentilucci M (2006): Speech and gesture share the same communication system. Neuropsychologia 44: 178-190.
9. Bernardis P, Salillas E, Caramelli N (2008): Behavioural and neurophysiological evidence of semantic interaction between iconic gestures and words. Cogn Neuropsychol 25: 1114-1128.
10. Blakemore S-J, Frith C (2005): The role of motor contagion in the prediction of action. Neuropsychologia 43: 260-267.
11. Blasi G, Goldberg TE, Weickert T, Das S, Kohn P, Zoltick B, Bertolino A, Callicott JH, Weinberger DR, Mattay VS (2006): Brain regions underlying response inhibition and interference monitoring and suppression. Eur J Neurosci 23: 1658-1664.
12. Botvinick MM (2007): Conflict monitoring and decision making: Reconciling two perspectives on anterior cingulate function. Cogn Affect Behav Neurosci 7: 356-366.
13. Botvinick MM, Braver TS, Barch DM, Carter CS, Cohen JD (2001): Conflict monitoring and cognitive control. Psychol Rev 108: 624-652.
14. Brass M, Derrfuss J, Forstmann B, Cramon DYv (2005a): The role of the inferior frontal junction area in cognitive control. Trends Cogn Sci 9: 314-316.
15. Brass M, Ullsperger M, Knoesche TR, von Cramon DY, Phillips NA (2005b): Who comes first? The role of the prefrontal and parietal cortex in cognitive control. J Cogn Neurosci 17: 1367-1375.
16. Calvert GA, Campbell R, Brammer MJ (2000): Evidence from functional magnetic resonance imaging of crossmodal binding in the human heteromodal cortex. Curr Biol 10: 649-657.
17. Calvo-Merino B, Grezes J, Glaser DE, Passingham RE, Haggard P (2006): Seeing or doing? Influence of visual and motor familiarity in action observation. Curr Biol 16: 1905-1910.
18. Carter CS, Braver T, Barch DM, Botvinick M, Noll D, Cohen JD (1998): The role of the anterior cingulate cortex in error detection and the on-line monitoring of performance: An event related fMRI study. Biol Psychiatry 43: 13s-13s.
19. Chen Q, Wei P, Zhou X (2006): Distinct neural correlates for resolving stroop conflict at inhibited and noninhibited locations in inhibition of return. J Cogn Neurosci 18: 1937-1946.
20. Chieffi S, Secchi C, Gentilucci M (2009): Deictic word and gesture production: Their interaction. Behav Brain Res 203: 200-206.
21. Cohen RL (1981): On the generality of some memory laws. Scand J Psychol 22: 267-281.
22. Cohen RL, Stewart M (1982): How to avoid developmental effects in free-recall. Scand J Psychol 23: 9-15.
23. Cole MW, Schneider W (2007): The cognitive control network: Integrated cortical regions with dissociable functions. Neuroimage 37: 343-360.
24. Denis M, Engelkamp J, Mohr G (1991): Memory of imagined actions-Imagining oneself or another person. Psychol Res 53: 246-250.
25. Depue BE, Curran T, Banich MT (2007): Memory suppression in PTSD treatment? Science 318: 1722.
26. Derrfuss J, Brass M, von Cramon DY (2004): Cognitive control in the posterior frontolateral cortex: Evidence from common activations in task coordination, interference control, and working memory. Neuroimage 23: 604-612.
27. Dick AS, Goldin-Meadow S, Hasson U, Skipper JI, Small SL (2009): Co-speech gestures influence neural activity in brain regions associated with processing semantic information. Hum Brain Mapp 30: 3509-3526.
28. Emmorey K (2006): The signer as an embodied mirror neuron system: Neural mechanisms underlying sing language and action. In: Arbib MA, editor. Action to Language via the Mirror Neuron System. Cambridge: Cambridge University Press. pp 110-135.
29. Engelkamp J, Krumnacker H (1980a): Imaginale und motorische Prozesse beim Behalten verbalen Materials. Zeitschrift für Experimentelle und Angewandte Psychologie 27: 511-533.
30. Engelkamp J, Zimmer HD (1984): Motor program information as a separable memory unit. Psychol Res 46: 283-299.
31. Engelkamp J, Zimmer HD (1985): Motor programs and their relation to semantic memory. Ger J Psychol 9: 239-254.
32. Feyereisen P (2006): Further investigation on the mnemonic effect of gestures: Their meaning matters. Eur J Cogn Psychol 18: 185-205.
33. Feyereisen P (2009): Enactment effects and integration processes in younger and older adults' memory for actions. Memory 17: 374-385.
34. Friston KJ, Penny W (2003): Posterior probability maps and SPMs. Neuroimage 19: 1240-1249.
35. Friston K, Chu C, Mourão-Miranda J, Hulme O, Rees G, Penny W, Ashburner J (2008): Bayesian decoding of brain images. Neuroimage 39: 181-205.
36. Gallese V, Fadiga L, Fogassi L, Rizzolatti G (1996): Action recognition in the premotor cortex. Brain 119: 593-609.
37. Gentilucci M, Corballis MC (2006): From manual gesture to speech: A gradual transition. Neurosci Biobehav Rev 30: 949-960.
38. Gentilucci M, Dalla Volta R (2008): Spoken language and arm gestures are controlled by the same motor control system. Q J Exp Psychol (Colchester) 61: 944-957.
39. Gentilucci M, Dalla Volta R, Gianelli C (2008): When the hands speak. J Physiol Paris 102: 21-30.
40. Gliga T, Csibra G (2009): One-year-old infants appreciate the referential nature of deictic gestures and words. Psychol Sci 20: 347-353.
41. Goldin-Meadow S (1999): The role of gesture in communication and thinking. Trends Cogn Sci 3: 419-429.
42. Goldin-Meadow S (2003): Hearing Gesture: How Our Hands Help Us Think, Vol. 14. Cambridge: Belknap Press of Harvard University Press. 280 p.
43. Goldin-Meadow S (2005): Watching language grow. Proc Natl Acad Sci USA 102: 2271-2272.
44. Grezes J, Decety J (2001): Functional anatomy of execution, mental simulation, observation, and verb generation of actions: A meta-analysis. Hum Brain Mapp 12: 1-19.
45. Guillot A, Collet C, Nguyen VA, Malouin F, Richards C, Doyon J (2008): Functional neuroanatomical networks associated with expertise in motor imagery. Neuroimage 41: 1471-1483.
46. Hauk O, Johnsrude I, Pulvermuller F (2004): Somatotopic representation of action words in human motor and premotor cortex. Neuron 41: 301-307.
47. Hazlett EA, Byne W, Brickman AM, Mitsis EM, Newmark R, Haznedar MM, Knatz DT, Chen AD, Buchsbaum MS (2010): Effects of sex and normal aging on regional brain activation during verbal memory performance. Neurobiol Aging 31: 826-838.
48. Helstrup T (1984): Serial position phenomena-Memory for acts, contents and spatial position patterns. Scand J Psychol 25: 131-146.
49. Helstrup T (1987): One, two, or three memories-A problem-solving approach to memory for performed acts. Acta Psychol 66: 37-68.
50. Helstrup T (1993): Actions, context, memory-What is the relation? Scand J Psychol 34: 19-26.
51. Hermsdörfer J, Goldenberg G, Wachsmuth C, Conrad B, Ceballos-Baumann AO, Bartenstein P, Schwaiger M, Boecker H (2001): Cortical correlates of gesture processing: Clues to the cerebral mechanisms underlying apraxia during the imitation of meaningless gestures. Neuroimage 14: 149-161.
52. Hlustík P, Solodkin A, Gullapalli RP, Noll DC, Small SL (2002): Functional lateralization of the human premotor cortex during sequential movements. Brain Cogn 49: 54-62.
53. Hockley WE (2008): The picture superiority effect in associative recognition. Mem Cogn 36: 1351-1359.
54. Holle H, Gunter TC (2007): The role of iconic gestures in speech disambiguation: ERP evidence. J Cogn Neurosci 19: 1175-1192.
55. Joubert S, Beauregard M, Walter N, Bourgouin P, Beaudoin G, Leroux JM, Karama S, Lecours AR (2004): Neural correlates of lexical and sublexical processes in reading. Brain Lang 89: 9-20.
56. Kaplan JT, Iacoboni M (2007): Multimodal action representation in human left ventral premotor cortex. Cogn Process 8: 103-113.
57. Karnath HO, Dieterich M (2006): Spatial neglect-A vestibular disorder? Brain 129 (Part 2): 293-305.
58. Kausler DH, Lichty W, Hakami MK, Freund JS (1986): Activity duration and adult age differences in memory for activity performance. Psychol Aging 1: 80-81.
59. Kelly SD, Kravitz C, Hopkins M (2004): Neural correlates of bimodal speech and gesture comprehension. Brain Lang 89: 253-260.
60. Kelly SD, McDevitt T, Esch M (2009): Brief training with co-speech gesture lends a hand to word learning in a foreign language. Lang Cogn Process 24: 313-334.
61. Kendon A (1981): Gestures, Their Origins and Distribution. Semiotica 37: 129-163.
62. Keysers C, Kohler E, Umilta MA, Nanetti L, Fogassi L, Gallese V (2003): Audiovisual mirror neurons and action recognition. Exp Brain Res 153: 628-636.
63. Kim JH, Park KY, Seo SW, Na DL, Chung CS, Lee KH, Kim GM (2007): Reversible verbal and visual memory deficits after left retrosplenial infarction. J Clin Neurol 3: 62-66.
64. Kircher T, Straube B, Leube D, Weis S, Sachs O, Willmes K, Konrad K, Green A (2009): Neural interaction of speech and gesture: Differential activations of metaphoric co-verbal gestures. Neuropsychologia 47: 169-179.
65. Knopf M (1992): Gedächtnis für Handlungen. Funktionsweise und Entwicklung. Heidelberg: University of Heidelberg.
66. Knopf M, Mack W, Lenel A, Ferrante S (2005): Memory for action events: Findings in neurological patients. Scand J Psychol 46: 11-19.
67. Kormi-Nouri R (1995): The nature of memory for action events: An episodic integration view. Eur J Cogn Psychol 7: 337-363.
68. Kormi-Nouri R (2000): The role of movement and object in action memory. Int J Psychol 35: 132.
69. Leynes PA, Bink ML (2002): Did I do that? An ERP study of memory for performed and planned actions. Int J Psychophysiol 45: 197-210.
70. Leynes PA, Grey JA, Crawford JT (2006): Event-related potential (ERP) evidence for sensory-based action memories. Int J Psychophysiol 62: 193-202.
71. Lohmann G, Muller K, Bosch V, Mentzel H, Hessler S, Chen L, Zysset S, von Cramon DY (2001): LIPSIA-A new software system for the evaluation of functional magnetic resonance images of the human brain. Comput Med Imaging Graph 25: 449-457.
72. Longcamp M, Tanskanen T, Hari R (2006): The imprint of action: Motor cortex involvement in visual perception of handwritten letters. Neuroimage 33: 681-688.
73. Macedonia M (2003): Sensorimotor enhancing of verbal memory through "Voice Movement Icons" during encoding of foreign language (German: Voice Movement Icons. Sensomotorische Encodierungsstrategie zur Steigerung der quantitativen und qualitativen Lerneffizienz bei Fremdsprachen) PhD Thesis. Salzburg: University of Salzburg.
74. Maguire EA (2001): The retrosplenial contribution to human navigation: A review of lesion and neuroimaging findings. Scand J Psychol 42: 225-238.
75. Masumoto K, Yamaguchi M, Sutani K, Tsuneto S, Fujita A, Tonoike M (2006): Reactivation of physical motor information in the memory of action events. Brain Res 1101: 102-109.
76. McDonald CR, Crosson B, Valenstein E, Bowers D (2001): Verbal encoding deficits in a patient with a left retrosplenial lesion. Neurocase: Neural Basis Cogn 7: 407-417.
77. McNeill D (1992): Hand and mind: What gestures reveal about thought, Vol. 11. Chicago: University of Chicago Press. 416p.
78. Mensebach C, Beblo T, Driessen M, Wingenfeld K, Mertens M, Rullkoetter N, Lange W, Markowitsch HJ, Ollech I, Saveedra AS, et al. (2009): Neural correlates of episodic and semantic memory retrieval in borderline personality disorder: An fMRI study. Psychiatry Res: Neuroimaging 171: 94-105.
79. Morris D, Collett P, Marsh P, Oshaughnessy M (1979): Gestures, Their Origins and Distribution. Stein and Day, NewYork.
80. Neumann J, Lohmann G (2003): Bayesian second-level analysis of functional magnetic resonance images. Neuroimage 20: 1346-1355.
81. Nilsson L-G, Nyberg L, Klingberg T, Åberg C, Persson J, Roland PE (2000): Activity in motor areas while remembering action events. Neuroreport 11: 2199-2201.
82. Norris DG (2000): Reduced power multislice MDEFT imaging. J Magn Reson Imaging 11: 445-451.
83. Nyberg L, Petersson KM, Nilsson LG, Sandblom J, Aberg C, Ingvar M (2001): Reactivation of motor brain areas during explicit memory for actions. Neuroimage 14: 521-528.
84. Paivio A (1971): Imagery and Verbal Processes, Vol. 11. New York: Holt. 596p.
85. Paivio A (1986): Mental Representations: A Dual Coding Approach, Vol 10. Oxford: Oxford University Press. 322p.
86. Paivio A, Csapo K (1969): Concrete image and verbal memory codes. J Exp Psychol 80: 279.
87. Peigneux P, Salmon E, van der Linden M, Garraux G, Aerts J, Delfiore G, Degueldre C, Luxen A, Orban G, Franck G (2000): The role of lateral occipitotemporal junction and area MT/V5 in the visual analysis of upper-limb postures. Neuroimage 11: 644-655.
88. Penny WD, Trujillo-Barreto NJ, Friston KJ (2005): Bayesian fMRI time series analysis with spatial priors. Neuroimage 24: 350-362.
89. Pine KJ, Bird H, Kirk E (2007): The effects of prohibiting gestures on children's lexical retrieval ability. Dev Sci 10: 747-754.
90. Porro CA, Francescato MP, Cettolo V, Diamond ME, Baraldi P, Zuiani C, Bazzocchi M, di Prampero PE (1996): Primary motor and sensory cortex activation during motor performance and motor imagery: A functional magnetic resonance imaging study. J Neurosci 16: 7688-7698.
91. Ramnani N, Owen AM (2004): Anterior prefrontal cortex: Insights into function from anatomy and neuroimaging. Nat Rev Neurosci 5: 184-194.
92. Ravizza S (2003): Movement and lexical access: Do noniconic gestures aid in retrieval? Psychon Bull Rev 10: 610-615.
93. Reynolds JR, Donaldson DI, Wagner AD, Braver TS (2004): Item- and task-level processes in the left inferior prefrontal cortex: Positive and negative correlates of encoding. Neuroimage 21: 1472-1483.
94. Rizzolatti G, Arbib MA (1998): Language within our grasp. Trends Neurosci 21: 188-194.
95. Rizzolatti G, Craighero L (2004): The mirror-neuron system. Annu Rev Neurosci 27: 169-192.
96. Rizzolatti G, Fogassi L, Gallese V (2000): Cortical mechanisms subserving object grasping and action recognition: A new view on the cortical motor functions. In: Gazzaniga MS, editor. The New Cognitive Neuroscience. Cambridge, MA: MIT Press. pp. 539-552.
97. Roberts KL, Hall DA (2008): Examining a supramodal network for conflict processing: A systematic review and novel functional magnetic resonance imaging data for related visual and auditory stroop tasks. J Cogn Neurosci 20: 1063-1078.
98. Ruscheweyh R, Willemer C, Kruger K, Duning T, Warnecke T, Sommer J, Volker K, Ho HV, Mooren F, Knecht S, et al.: Physical activity and memory functions: An interventional study. Neurobiol Aging (in press). doi: 10.1016/j.neurobiolaging.2009.08.001
99. Russ MO, Mack W, Grama CR, Lanfermann H, Knopf M (2003): Enactment effect in memory: Evidence concerning the function of the supramarginal gyrus. Exp Brain Res 149: 497-504.
100. Saltz E, Donnenwerthnolan S (1981): Does motoric imagery facilitate memory for sentences-A selective interference test. J Verb Learn Verb Behav 20: 322-332.
101. Schubotz RI, von Cramon DY (2001): Functional organization of the lateral premotor cortex: fMRI reveals different regions activated by anticipation of object properties, location and speed. Brain Res Cogn Brain Res 11: 97-112.
102. Schubotz RI, von Cramon DY, Lohmann G (2003): Auditory what, where, and when: A sensory somatotopy in lateral premotor cortex. Neuroimage 20: 173-185.
103. Senkfor AJ (2008): Memory for pantomimed actions versus actions with real objects. Cortex 44: 820-833.
104. Senkfor AJ, Van Petten C, Kutas M (2002): Episodic action memory for real objects: An ERP investigation with perform, watch, and imagine action encoding tasks versus a non-action encoding task. J Cogn Neurosci 14: 402-419.
105. Shams L, Seitz AR (2008): Benefits of multisensory learning. Trends Cogn Sci 12: 411-417.
106. Spaniol J, Davidson PSR, Kim ASN, Han H, Moscovitch M, Grady CL (2009): Event-related fMRI studies of episodic encoding and retrieval: Meta-analyses using activation likelihood estimation. Neuropsychologia 47: 1765-1779.
107. Sueyoshi Ayano HDM (2005): The role of gestures and facial cues in second language listening comprehension. Lang Learn 55: 661-699.
108. Talairach J, Tournoux P (1988): Co-planar Stereotaxic Atlas of the Human Brain: 3-Dimensional Proportional System: An Approach to Cerebral Imaging. Stuttgart: Thieme.
109. Tellier M (2008): The effect of gestures on second language memorisation by young children. Gesture 8: 219-235.
110. Tettamanti M, Buccino G, Saccuman MC, Gallese V, Danna M, Scifo P, Fazio F, Rizzolatti G, Cappa SF, Perani D (2005): Listening to action-related sentences activates fronto-parietal motor circuits. J Cogn Neurosci 17: 273-281.
111. Tomasello M (2005): Constructing a Language: A Usage-Based Theory of Language Acquisition, Vol. 8. Cambridge, MA: Harvard University Press. 388p.
112. Tomasello M (2008): Origins of Human Communication, Vol. 13. Cambridge, MA: MIT. 393p.
113. Ullsperger M, von Cramon DY (2004): Neuroimaging of performance monitoring: Error detection and beyond. Cortex 40: 593-604.
114. Van Mier H (2000): Human learning. In: Toga AW, Mazziotta JC, editors. Brain Mapping: The Systems. London: Academic Press. pp 605-620.
115. van Praag H (2009): Exercise and the brain: Something to chew on. Trends Neurosci 32: 283-290.
116. Vann SD, Aggleton JP, Maguire EA (2009): What does the retrosplenial cortex do? Nat Rev Neurosci 10: 792-802.
117. von Zerssen GC, Mecklinger A, Opitz B, von Cramon DY (2001): Conscious recollection and illusory recognition: An event-related fMRI study. Eur J Neurosci 13: 2148-2156.
118. Wagner AD, Shannon BJ, Kahn I, Buckner RL (2005): Parietal lobe contributions to episodic memory retrieval. Trends Cogn Sci 9: 445-453.
119. Wittfoth M, Kustermann E, Fahle M, Herrmann M (2008): The influence of response conflict on error processing: Evidence from event-related fMRI. Brain Res 1194: 118-129.
120. Worsley KJ, Liao CH, Aston J, Petre V, Duncan GH, Morales F, Evans AC (2002): A general statistical analysis for fMRI data. Neuroimage 15: 1-15.
121. Wu YC, Coulson S (2007a): How iconic gestures enhance communication: An ERP study. Brain Lang 101: 234-245.
122. Wu YC, Coulson S (2007b): Iconic gestures prime related concepts: An ERP study. Psychon Bull Rev 14: 57-63.
123. Ziemus B, Baumann O, Luerding R, Schlosser R, Schuierer G, Bogdahn U, Greenlee MW (2007): Impaired working-memory after cerebellar infarcts paralleled by changes in BOLD signal of a cortico-cerebellar circuit. Neuropsychologia 45: 2016-2024.