How iconic gestures enhance communication: An ERP study

Brain and Language

Volumn 101, Issue 3, 2007, Pages 234-245

  Wu, Ying Choon ^a   Coulson, Seana ^a  

^a UNIVERSITY OF CALIFORNIA   (United States)

Author keywords

Conceptual integration; Embodiment; ERP; Gesture; Language comprehension; Meaning; N300; N400; Object recognition; Semantic integration; Simulation

Indexed keywords

AMPLITUDE MODULATION; ARTICLE; BEHAVIOR; COMMUNICATION SKILL; COMPREHENSION; CONTROLLED STUDY; DEPTH PERCEPTION; ELECTRODE; ELECTROENCEPHALOGRAPHY; EVENT RELATED POTENTIAL; FEMALE; GESTURE; HUMAN; HUMAN EXPERIMENT; MALE; NORMAL HUMAN; SPEECH ANALYSIS; STIMULUS RESPONSE;

ADULT; ANALYSIS OF VARIANCE; BRAIN; BRAIN MAPPING; COMMUNICATION; CONCEPT FORMATION; EVOKED POTENTIALS; FEMALE; GESTURES; HUMANS; MALE; PSYCHOLINGUISTICS; SEMANTICS; SPACE PERCEPTION; SPEECH PERCEPTION;

EID: 34249050851     PISSN: 0093934X     EISSN: 10902155     Source Type: Journal    
DOI: 10.1016/j.bandl.2006.12.003     Document Type: Article

References (58)

1. Alibali M.W., Flevares L., and Goldin-Meadow S. Assessing knowledge conveyed in gesture: do teachers have the upper hand?. Journal of Educational Psychology 89 (1997) 183-193
2. Bar M. A cortical mechanism for triggering top-down facilitation in visual object recognition. Journal of Cognitive Neuroscience 15 4 (2003) 600-609
3. Barrett S.E., and Rugg M.D. Event-related potentials and the semantic matching of pictures. Brain and Cognition 14 (1990) 201-212
4. Beattie G., and Shovelton H. Mapping the range of information contained in the iconic hand gestures that accompany spontaneous speech. Journal of Language and Social Psychology 18 (1999) 438-462
5. Beattie G., and Shovelton H. An experimental investigation of some properties of individual iconic gestures that mediate their communicative power. British Journal of Psychology 93 2 (2002) 179-192
6. Cassell J., McNeill D., and McCullough K. Speech-gesture mismatches: evidence for one underlying representation of linguistic and nonlinguistic information. Pragmatics and Cognition 7 1 (1999) 1-33
7. Coulson S., and Van Petten C.K. Conceptual integration and metaphor: an event-related potential study. Memory & Cognition 30 (2002) 958-968
8. DeLong K.A., Urbach T.P., and Kutas M. Probabilitstic word pre-activation during language comprehension inferred from electrical brain activity. Nature Neuroscience 8 8 (2005) 117-1121
9. Emmorey K., and Casey S. Gesture, thought, and spatial language. Gesture 1 1 (2001) 35-50
10. Fauconnier G., and Turner M. Conceptual integration networks. Cognitive Science 22 (1998) 133-187
11. Fauconnier G., and Turner M. The way we think (2002), Basic Books, New York
12. Federmeier K.D., and Kutas M. Meaning and modality: influences of context, semantic memory organization, and perceptual predictability on picture processing. Journal of Experimental Psychology: Learning, Memory, and Cognition 27 (2001) 202-224
13. Federmeier K.D., and Kutas M. Picture the difference: eletrophysiological investigations of picture processing in the two cerebral hemispheres. Neuropsychologia 40 (2002) 730-747
14. Ganis G., Kutas M., and Sereno M.I. The search for "common sense": an electrophysiological study of the comprehension of words and pictures in reading. Journal of Cognitive Neuroscience 8 2 (1996) 89-106
15. Geisser S., and Greenhouse S. On methods in the analysis of profile data. Psychometrika 24 (1959) 95-112
16. Glenberg A.M. What memory is for. Behavioral and Brain Sciences 20 (1997) 1-55
17. Glenberg A.M., and Robertson D.A. Indexical understanding of instructions. Discourse Processes 28 1 (1999) 1-26
18. Goldin-Meadow S. Hearing gesture: How our hands help us think (2003), Belknap Press, Cambridge, Massachusetts
19. Goldin-Meadow S., and Sandhofer C.M. Gesture conveys substantive information about a child's thoughts to ordinary listeners. Developmental Science 2 (1999) 67-74
20. Goldin-Meadow S., Wein D., and Chang C. Assessing knowledge through gestures: using children's hands to read their minds. Cognition and Instruction 9 3 (1992) 201-219
21. Graham J.A., and Argyle M. A cross-cultural study of the communication of extra-verbal meaning by gestures. International Journal of Psychology 10 (1975) 57-67
22. Gunter T.C., and Bach P. Communicating hands: ERPs elicited by meaningful symbolic hand postures. Neuroscience Letters 372 (2004) 52-56
23. Hamm J.P., Johnson B.W., and Kirk I.J. Comparison of the N300 and N400 ERPs to pictue stimuli in congruent and incongruent contexts. Clinical Neurophysiology 113 (2002) 1339-1450
24. Holcomb P.J. Semantic priming and stimulus degradation: implications for the role of the N400 in language processing. Psychophysiology 30 (1993) 47-61
25. Holcomb P.J., and McPherson W.B. Event-related brain potentials reflect semantic priming in an object decision task. Brain and Cognition 24 (1994) 259-276
26. Kelly S.D. Broadening the units of analysis in communication: speech and nonverbal behaviours in pragmatic comprehension. Journal of Child Language 28 (2001) 325-349
27. Kelly S.D., Barr D.J., Church R.B., and Lynch K. Offering a hand to pragmatic understanding: the role of speech and gesture in comprehension and memory. Journal of Memory and Language 40 (1999) 577-592
28. Kelly S.D., and Church R.B. A comparison between children's and adults' ability to detect conceptual information conveyed through representational gestures. Child Development 69 1 (1998) 85-93
29. Kelly S.D., Kravitz C., and Hopkins M. Neural correlates of bimodal speech and gesture comprehension. Brain and Language 89 1 (2004) 243-260
30. Krauss R.M., Dushay R.A., Chen Y., and Rauscher F. The communicative value of conversational hand gestures. Journal of Experimental Social Psychology 31 (1995) 533-552
31. Kutas M., Federmeier K., Coulson S., King J.W., and Muente T.F. Language. In: Berntson G.G. (Ed). Handbook of psychophysiology. 2nd ed. (2000), Cambridge University Press, Cambridge, UK 576-601
32. Kutas M., and Hillyard S.A. Reading senseless sentences: brain potentials reflect semantic incongruity. Science 207 (1980) 203-205
33. Landauer T.K., and Dumais S.T. A solution to Plato's problem: the latent semantic analysis theory of acquisition, induction, and representation of knowledge. Psychological Review 104 2 (1997) 211-240
34. McNeill D. Hand and mind (1992), Chicago University Press, Chicago
35. McNeill D. Speech and gesture integration. In: Iverson J., and Goldin-Meadow S. (Eds). The nature and functions of gesture in children's communication. New directions for child development Vol. 79 (1998), Jossey-Bass, San Francisco 11-27
36. McPherson W.B., and Holcomb P.J. An electrophysiological investigation of semantic priming with pictures of real objects. Psychophysiology 36 (1999) 53-65
37. Morford M., and Goldin-Meadow S. Comprehension and production of gesture in combination with speech in one-word speakers. Journal of Child Language 19 3 (1992) 559-580
38. Neville H.J., Coffey S.A., Lawson D.S., Fischer A., Emmorey K., and Bellugi U. Neural systems mediating American Sign Language: effects of sensory experience and age of acquisition. Brain and Language 57 3 (1997) 285-308
39. Nuwer M.R., Comi G., Emerson R., Fuglsang-Frederiksen A., Guerit J.M., Hinrichs H., et al. IFCN standards for digital recording of clinical EEG. The international federation of clinical neurophysiology. Electroencephalography and Clinical Neurophysiology Supplement 52 (1999) 11-14
40. Oldfield R.C. The assessment and analysis of handedness: the Edinburgh Inventory. Neuropsychologia 9 (1971) 97-113
41. Praterelli M.E. Semantic processing of pictures and spoken words: evidence from event-related brain potentials. Brain and Cognition 24 (1994) 137-157
42. Rizzolatti G., and Arbib M.A. Language within our grasp. Trends in Neuroscience 21 (1998) 188-194
43. Rogers W.T. The contribution of kinesic illustrators toward the comprehension of verbal behavior within utterances. Human Communication Research 5 (1978) 54-62
44. Schendan H.E., and Kutas M. Neurophysiological evidence for two processing times for visual object identification. Neuropsychologia 40 (2002) 931-945
45. Schendan H.E., and Kutas M. Time course of processes and representations supporting visual object identification and memory. Journal of Cognitive Neuroscience 15 1 (2003) 111-135
46. Schmid, A., Eddy, M., & Holcomb, P. J. (2005). Integration of bottom-up and top-down processes in visual object recognition. Paper presented at the Cognitive Neuroscience Society: Annual Meeting Program 2005, New York, NY.
47. Singer M.A., and Goldin-Meadow S. Children learn when their teacher's gestures and speech differ. Psychological Science 16 2 (2005) 85-89
48. Sitnikova T., Kuperberg G., and Holcomb P.J. Semantic integration in videos of real-world events: an electrophysiological investigation. Psychophysiology 40 1 (2003) 160-164
49. Sitnikova T., West W.C., Kuperberg G., and Holcomb P.J. The neural organization of semantic memory: electrophysiological activity suggests feature-based segregation. Biological Psychology 71 (2006) 326-340
50. Stanfield R.A., and Zwaan R.A. The effect of implied orientation derived from verbal context on picture recognition. Psychological Science 12 2 (2001) 153-156
51. Thompson L.A., and Massaro D.W. Evaluation and integration of speech and pointing gestures during referential understanding. Journal of Experimental Child Psychology 42 1 (1986) 144-168
52. Thompson L.A., and Massaro D.W. Children's integration of speech and pointing gestures in comprehension. Journal of Experimental Child Psychology 57 (1994) 327-354
53. Valenzeno L., Alibali M.W., and Klatzky R. Teacher's gestures facilitate students' learning: a lesson in symmetry. Contemporary Educational Psychology 29 (2003) 187-204
54. West W.C., and Holcomb P.J. Event-related potentials during discourse-level semantic integration of complex pictures. Cognitive Brain Research 13 3 (2002) 363-375
55. Wicha N.Y.Y., Bates E., Moreno E.M., and Kutas M. Potato not Pope: human brain potentials to gender expectation and agreement in Spanish spoken sentences. Neuroscience Letters 346 (2003) 165-168
56. Wu, Y.C. (2005). Meaning in gestures: what event-related potentials reveal about processes underlying the comprehension of iconic gestures. Retrieved August, 2005, from .
57. Wu Y.C., and Coulson S. Is that a meaningful gesture: electrophysiological indices of gesture comprehension. Psychophysiology 42 6 (2005) 654-667
58. Zwaan R.A., Stanfield R.A., and Yaxley R.H. Language comprehenders mentally represent the shapes of objects. Psychological Science 13 2 (2002) 168-171