Gesture and metaphor comprehension: Electrophysiological evidence of cross-modal coordination by audiovisual stimulation

Brain and Cognition

Volumn 70, Issue 1, 2009, Pages 42-52

  Cornejo, Carlos ^a   Simonetti, Franco ^a   Ibáñez, Agustín ^b   Aldunate, Nerea ^a   Ceric, Francisco ^a   López, Vladimir ^a   Núñez, Rafael E ^c  

^a PONTIFICIA UNIVERSIDAD CATÓLICA DE CHILE   (Chile)

^b UNIVERSIDAD DIEGO PORTALES   (Chile)

^c UNIVERSITY OF CALIFORNIA   (United States)

Author keywords

Gesture; LPC; Metaphor; Multimodal integration; N400; Stroke; Video

Indexed keywords

ADULT; AMPLITUDE MODULATION; ARTICLE; AUDITORY STIMULATION; BRAIN ELECTROPHYSIOLOGY; COMPREHENSION; EVENT RELATED POTENTIAL; FEMALE; GESTURE; HUMAN; HUMAN EXPERIMENT; LANGUAGE ABILITY; LINGUISTICS; LITERATURE; MALE; NORMAL HUMAN; PRIORITY JOURNAL; SPEECH ARTICULATION; VISUAL STIMULATION;

ANALYSIS OF VARIANCE; BRAIN; COMPREHENSION; ELECTROENCEPHALOGRAPHY; ELECTROOCULOGRAPHY; EVOKED POTENTIALS; FEMALE; GESTURES; HUMANS; MALE; METAPHOR; PSYCHOLINGUISTICS; REACTION TIME; TASK PERFORMANCE AND ANALYSIS; YOUNG ADULT;

EID: 64449083492     PISSN: 02782626     EISSN: 10902147     Source Type: Journal    
DOI: 10.1016/j.bandc.2008.12.005     Document Type: Article

References (75)

1. Barrett S.E., and Rugg M.D. Event-related potentials and the semantic matching of pictures. Brain and Cognition 14 (1990) 201-212
2. Bates E., and Dick F. Language, gesture, and the developing brain. Developmental Psychobiology 40 (2002) 293-310
3. Bernardis P., Salillas E., and Caramelli N. Behavioural and neurophysiological evidence of semantic interaction between iconic gestures and words. Cognitive Neuropsychology 25 7 (2008) 1114-1128
4. Cienki A. STRAIGHT: An image schema and its metaphorical extensions. Cognitive Linguistics 9 (1998) 107-149
5. Cienki A., and Müller C. Gesture and metaphor (2008), John Benjamins, Amsterdam
6. Clark H.H. Using language (1996), Cambridge University Press, Cambridge
7. Cornejo C. Who says what the words say?: The problem of linguistic meaning in psychology. Theory and Psychology 14 (2004) 5-28
8. Cornejo C. Review essay: Conceptualizing metaphors versus embodying the language. Culture & Psychology 13 4 (2007) 474-487
9. Cornejo C. Intersubjectivity as co-phenomenology: From the holism of meaning to the being-in-the-world-with-others. Integrative Psychological and Behavioral Science 42 2 (2008) 171-178
10. Cornejo C., Simonetti F., Aldunate N., Ibáñez A., López V., and Melloni L. Electrophysiological evidence of different interpretative strategies in irony comprehension. Journal of Psycholinguistic Research 36 (2007) 411-430
11. Cosmelli D., and Ibáñez A. Human cognition in context: On the biologic, cognitive and social reconsideration of meaning. Integrative Psychological and Behavioral Sciences 42 2 (2008) 233-244
12. Coulson S. Electrophysiology and pragmatic language comprehension. In: Noveck I., and Sperber D. (Eds). Experimental pragmatics (2004), Palgrave Macmillan, San Diego 187-206
13. Coulson S. Constructing meaning. Metaphor and Symbol 21 4 (2006) 245-266
14. Coulson S., and Van Petten C. Conceptual integration and metaphor: An event-related potential study. Memory and Cognition 30 (2002) 958-968
15. Coulson S., and Van Petten C. A special role for the right hemisphere in metaphor comprehension? ERP evidence from hemifield presentation. Brain Research 1146 (2007) 128-145
16. Davis H., Mast T., Yoshie N., and Zerlin S. The slow response of the human cortex to auditory stimuli: Recovery process. Electroencephalography and Clinical Neurophysiology 21 (1966) 105-113
17. Delorme A., and Makeig S. EEGLAB: An open source toolbox for analysis of single-trial EEG dynamics including independent component analysis. Journal of Neuroscience Methods 134 1 (2004) 9-21
18. Donchin E., and Coles M.G. Is the P300 component a manifestation of context updating?. Behavioral and Brain Sciences 11 3 (1988) 357-427
19. Enfield N.J. Areal linguistics and mainland southeast Asia. Annual Review of Anthropology 34 (2005) 181-206
20. Federmeir K., and Kutas M. Right words and left words. Cognitive Brain Research 8 (1999) 373-392
21. Gibbs R.W.J., Leggitt J.S., and Turner E.A. What's special about figurative language in emotional communication?. In: Fussell S.R. (Ed). The verbal communication of emotions. Interdisciplinary perspectives (2002), Lawrence Erlbaum Associates, Mahwah, NJ 125-149
22. Goldin-Meadow S., Nusbaum H., Kelly S., and Wagner S. Explaining math: Gesturing lightens the load. Psychological Science 12 (2001) 516-522
23. Goodwin C. Action and embodiment within situated human interaction. Journal of Pragmatics 32 (2000) 1489-1522
24. Hagoort P. The fractionation of spoken language understanding by measuring electrical and magnetic brain signals. Philosophical Transactions of the Royal Society B Biological Sciences 363 1493 (2008) 1055-1069
25. Hadar U., Wenkert-Olenik D., Krauss R., and Soroker N. Gesture and the processing of speech: Neuropsychological evidence. Brain and Language 62 (1998) 107-126
26. Hahne A., and Friederici A.D. Differential task effects on semantic and syntactic processes as revealed by ERPs. Cognitive Brain Research 13 (2002) 339-356
27. Holcomb P.J., and Neville H.J. Auditory and visual semantic priming in lexical decision: A comparison using event-related brain potentials. Language and Cognitive Processes 5 (1990) 281-312
28. Holle H., and Gunter T. The role of iconic gestures in speech disambiguation: ERP evidence. Journal of Cognitive Neuroscience 19 (2007) 1175-1792
29. Ibáñez A., and Cosmelli D. Moving beyond computational cognitivism: Understanding intentionality, intersubjectivity and ecology of mind. Integrative Psychological and Behavioral Sciences 42 2 (2008) 129-136
30. Ibáñez A., López V., and Cornejo C. ERPs and contextual semantic discrimination: Evidence of degrees of congruency in wakefulness and sleep. Brain and Language 98 (2006) 264-275
31. Ibáñez, A., Haye, A., González, R., Hurtado, E., Henríquez, R. (in press). Multi-level analysis of cultural phenomena: The role of ERP approach to prejudice. Journal for Theory in Social Behavior. JTSB 39:1 0021-8308.
32. Iverson J., and Goldin-Meadow S. The nature and functions of gesture in children's communication (1998), Jossey-Bass, San Francisco
33. Kazmerski V., Blasko D., and Dessalegn B. ERP and behavioral evidence of individual differences in metaphor comprehension. Memory & Cognition 31 (2003) 673-689
34. Kazmerski V., Blasko D., and Marroquin A. ERP evidence for differences in automaticity of metaphor processing in younger and older adults. Psychophysiology 40 (2003) 52
35. Kelly S., Barr D., Church R., and Lynch K. Offering a hand to pragmatic understanding: The role of speech and gesture in comprehension and memory. Journal of Memory and Language 40 (1999) 577-592
36. Kelly S., Kravitz C., and Hopkins M. Neural correlates of bimodal speech and gesture comprehension. Brain and Language 89 (2004) 243-260
37. Kelly S., Ward S., Creigh P., and Bartolotti J. An intentional stance modulates the integration of gesture and speech during comprehension. Brain and Language 101 (2007) 222-233
38. Kendon A. Language and gesture: Unity or duality?. In: McNeill D. (Ed). Language and gesture (2000), Cambridge University Press, Cambridge 47-63
39. Kendon A. Gesture: Visible action as utterance (2004), Cambridge University Press, Cambridge
40. Kita S., Danziger E., and Stolz C. Cultural specificity of spatial schemas, as manifested in spontaneous gestures. In: Gattis M. (Ed). Spatial schemas in abstract thought (2001), MIT Press, Cambridge, MA 15-146
41. Kita S., and Özyürek A. What does cross-linguistic variation in semantic coordination of speech and gesture reveal?: Evidence for an interface representation of spatial thinking and speaking. Journal of Memory and Language 48 1 (2003) 16-32
42. Krauss R.M. Why do we gesture when we speak?. Current Directions in Psychological Science 7 (1998) 54-60
43. Krauss R.M., Morrel-Samuels P., and Colasante C. Do conversational hand gestures communicate?. Journal of Personality and Social Psychology 61 (1991) 743-754
44. Kutas M., and Federmeier K. Electrophysiology reveals semantic memory use in language comprehension. Trends in Cognitive Sciences 4 (2000) 463-470
45. Kutas M., and Hillyard S. Reading senseless sentences: Brain potentials reflect semantic incongruity. Science 207 (1980) 203-205
46. Kutas M., and Van Petten C. Psycholinguistics electrified: Event-related brain potentials investigations. Handbook of Psycholinguistics 4 (1994) 83-143
47. Kutas M., Van Petten C., and Kluender R. Psycholinguistics electrified II: 1994-2005. In: Gernsbacher M.T.M.A. (Ed). Handbook of psycholinguistics (2006), Elsevier, New York 659-724
48. Leezenberg M. Contexts of metaphor (2001), Elsevier, Amsterdam
49. Levin D.T., and Simons D.J. Perceiving stability in a changing world: Combining shots and intergrading views in motion pictures and the real world. Media Psychology 2 4 (2000) 357-380
50. McNeill D. Hand and mind (1992), University of Chicago Press, Chicago
51. McNeill D. Language and gesture (2000), Cambridge University Press, Cambridge
52. Morrel-Samuels P., and Krauss R.M. Word familiarity predicts temporal asynchrony of hand gestures and speech. Journal of Experimental Psychology: Learning, Memory and Cognition 18 (1992) 615-623
53. Münte T.F., Heinze H.-J., Matzke M., Wieringa B.M., and Johannes S. Brain potentials and syntactic violations revisited: No evidence for specificity of the syntactic positive shift. Neuropsychologia 36 3 (1998) 217-226
54. Nieuwland M.S., and Van Berkum J.J. When peanuts fall in love: N400 evidence for the power of discourse. Journal of Cognitive Neuroscience 18 7 (2006) 1098-1111
55. Núñez R. Do real numbers really move? The embodied cognitive foundations of mathematics. In: Lida F., Pfeifer R., Steels L., and Kuniyoshi Y. (Eds). Embodied artificial intelligence (2004), Springer, New York 54-73
56. Núñez R., and Sweetser E. With the future behind them: Convergent evidence from Aymara language and gesture in the crosslinguistic comparison of spatial construal's of time. Cognitive Science 30 (2006) 401-450
57. Oken B.S., and Chiappa K.H. Statistical issues concerning computerized analysis of brain wave topography. Annals of Neurology 19 (1986) 493-494
58. Oldfield R.C. The assessment and analysis of handedness: The edinburgh inventory. Neuropsychologia 9 (1971) 97-113
59. Özyüreck A., Willems R.M., Kita S., and Hagoort P. On-line integration of semantic information from speech and gesture: Insights from event-related brain potentials. Journal of Cognitive Neuroscience 19 (2007) 605-616
60. Pynte J.L., Besson M., Robichon F.H., and Poli J. The time-course of metaphor comprehension: An event-related potential study. Brain and Language 55 (1996) 293-316
61. Rauscher F., Krauss R.M., and Chen Y. Gesture, speech and lexical access: The role of lexical movements in speech production. Psychological Science 7 (1996) 226-231
62. Rugg M.D., and Yonelinas A. Human recognition memory: A cognitive neuroscience perspective. Trends in Cognitive Sciences 7 7 (2003) 313-319
63. Sadowsky, S., & Martínez, R. (2003) Lifcach. Retrieved December, 2003. Available from http://www2.udec.cl/~ssadowsky/lifcach.html.
64. Shanon B. The representational and the presentational: An essay on cognition and the study of the mind (1993), Harvester Wheatsheaf, New York
65. Sitnikova T., Kuperberg G., and Holcomb P.J. Semantic integration in videos of real-world events: An electrophysiological investigation. Psychophysiology 40 (2003) 160-164
66. Tartter V.C., Gomes H., Dubrovsky B., Molholm S., and Stewart R.V. Novel metaphors appear anomalous at least momentarily: Evidence from N400. Brain and Language 80 (2002) 488-509
67. Van den Brink D., Brown C., and Hagoort P. Electrophysiological evidence for early contextual influences during spoken-word recognition: N200 versus N400 effects. Journal of Cognitive Neuroscience 13 (2001) 967-985
68. Van Petten C., Kutas M., Kluender R., Mitchiner M., and McIsaac H. Fractionating the word repetition effect with event-related potentials. Journal of Cognitive Neuroscience 3 (1991) 131-150
69. Van Petten C., and Luka B. Neural localization of semantics contexts effects in electromagnetic and hemodynamic studies. Brain and Language 97 (2006) 279-293
70. West W.C., and Holcomb P.J. Event-related potentials during discourse-level semantic integration of complex pictures. Cognitive Brain Research 13 (2002) 363-375
71. Willems R., Özyürek A., and Hagoort P. When language meets action: The neural integration of gesture and speech. Cerebral Cortex 17 10 (2007) 2322-2333
72. Wu Y., and Coulson S. Is that a meaningful gesture?: Electrophysiological indices of gesture comprehension. Psychophysiology 42 (2005) 654-667
73. Wu Y., and Coulson S. How iconic gestures enhance communication: An ERP study. Brain and Language 101 (2007) 234-245
74. Wu Y., and Coulson S. Iconic gestures prime related concepts: An ERP study. Psychological Bulletin & Review 14 (2007) 57-63
75. Zhou S., Zhou W., and Chen X. Spatiotemporal analysis of ERP during Chinese idiom comprehension. Brain Topography 17 (2004) 27-37