The benefit of gestures during communication: Evidence from hearing and hearing-impaired individuals

Cortex

Volumn 48, Issue 7, 2012, Pages 857-870

  Obermeier, Christian ^a   Dolk, Thomas ^a   Gunter, Thomas C ^a  

^a MAX PLANCK INSTITUTE FOR HUMAN COGNITIVE AND BRAIN SCIENCES   (Germany)

Author keywords

Asynchrony; Babble noise; Communication; Gesture speech integration; Hearing impaired; N400

Indexed keywords

ADULT; ARTICLE; CLINICAL ARTICLE; COMPREHENSION; CONTROLLED STUDY; EVENT RELATED POTENTIAL; FEMALE; GESTURE; HEARING LOSS; HUMAN; HUMAN EXPERIMENT; INTERPERSONAL COMMUNICATION; MALE; NOISE; SPEECH;

ADULT; BRAIN; COMMUNICATION; COMPREHENSION; EVOKED POTENTIALS; FEMALE; GESTURES; HEARING IMPAIRED PERSONS; HEARING LOSS; HEARING TESTS; HUMANS; LANGUAGE; MALE; SPEECH; SPEECH PERCEPTION;

EID: 84861841946     PISSN: 00109452     EISSN: 19738102     Source Type: Journal    
DOI: 10.1016/j.cortex.2011.02.007     Document Type: Article

References (56)

1. Agrafiotis D., Canagarajah N., Bull D.R. Perceptually optimised sign language coding based on eye-tracking analysis. Electronic Letters 2003, 39:1703-1705.
2. Agrafiotis D., Canagarajah N., Bull D.R., Dye M., Twyford H., Kyle J., et al. Optimised sign language coding based on eye-tracking analysis. Proceedings of the Society of Photo-optical Instrumentation Engineers (SPIE) 2003, 5150:1244-1252.
3. Bavelier D., Brozinsky C., Tomann A., Mitchell T., Neville H., Liu G. Impact of early deafness and early exposure to sign language on the cerebral organization for motion processing. Journal of Neuroscience 2001, 21(22):8931-8942.
4. Bavelier D., Dye M.W.G., Hauser P.C. Do deaf individuals see better?. Trends in Cognitive Science 2006, 10(11):512-518.
5. Bavelier D., Tomann A., Hutton C., Mitchell T., Corina D., Liu G., et al. Visual attention to the periphery is enhanced in congenitally deaf individuals. Journal of Neuroscience 2000, 20(17):1-6. RC93.
6. Beattie G., Shovelton H. Mapping the range of information contained in the iconic gestures that accompany spontaneous speech. Journal of Language and Social Psychology 1999, 18(4):438-462.
7. Beattie G., Shovelton H. An experimental investigation of the role of the different types of iconic gesture in communication: A semantic feature approach. Gesture 2001, 1(2):129-149.
8. Bernstein L.E., Demorest M.E., Tucker P.E. Speech perception without hearing. Perception and Psychophysics 2000, 62:233-252.
9. Bies D.A., Hansen C.H. Engineering Noise Control: Theory and Practice 2003, Chapman and Hall, New York, p. 132.
10. Boersma P., Weenink D. Praat: Doing Phonetics by Computer (Version 4.5.01) [Computer Program] 2005, Institute of Phonetic Sciences, Retrieved from, Amsterdam. http://www.praat.org/.
11. Clark J.G. Uses and abuses of hearing loss classification. Asha 1981, 23:493-500.
12. Culbertson D.N.S., Kricos P.B. Language and speech of the deaf and hard of hearing. Introduction to Audiologic Rehabilitation 2002, 183-224. Allyn & Bacon, London. R.L. Schow, M.A. Nerbonne (Eds.).
13. Chui K. Temporal patterning of speech and iconic gestures in conversational discourse. Journal of Pragmatics 2005, 37(6):871-887.
14. Driver J., Spence C. Multisensory perception: Beyond modularity and convergence. Current Biology 2000, 10:R731-R735.
15. Dye M.W.G., Hauser P.C., Bavelier D. Is visual selective attention in deaf individuals enhanced or deficient? The case of the useful field of view. PLoS ONE 2009, 4(5):e5640.
16. Emmorey K., Thompson R., Colvin R. Eye gaze during comprehension of American sign language by native and beginning signers. Journal of Deaf Studies and Deaf Education 2009, 14(2):237-243.
17. Feyereisen P. Further investigation on the mnemonic effect of gestures: Their meaning matters. European Journal of Cognitive Psychology 2006, 18(2):185-205.
18. Grant K.W., Tufts J.B., Greenberg S. Integration efficiency for speech perception within and across sensory modalities by normal-hearing and hearing-impaired individuals. Journal of the Acoustic Society America 2007, 121(2):1164-1176.
19. Greenhouse S.W., Geisser S. On methods in the analysis of profile data. Psychometrika 1959, 24(2):95-112.
20. Grosjean F. Gating. Language and Cognitive Processes 1996, 11(6):597-604.
21. Gunter T.C., Wagner S., Friederici A.D. Working memory and lexical ambiguity resolution as revealed by ERPs: A difficult case for activation theories. Journal of Cognitive Neuroscience 2003, 15(5):643-657.
22. Gullberg M., Holmqvist K. What speakers do and what addressees look at: Visual attention to gestures in human interaction live and on video. Pragmatics and Cognition 2006, 14(1):53-82.
23. Habets B., Kita S., Shao Z., özyürek A., Hagoort P. The role of synchrony and ambiguity in speech-gesture integration during comprehension. Journal of Cognitive Neuroscience 2011, 23(8):1845-1854.
24. Hadar U., Pinchas-Zamir L. The semantic specificity of gesture: Implications for gesture classification and function. Journal of Language and Social Psychology 2004, 23:204-214.
25. Hinojosa J.A., Martin-Loeches M., Rubia F.J. Event-related potentials and semantics: An overview and an integrative proposal. Brain and Language 2001, 78(1):128-139.
26. Holcomb P.J. Semantic priming and stimulus degradation: Implications for the role of the N400 in language processing. Psychophysiology 1993, 30:47-61.
27. Holle H., Gunter T.C. The role of iconic gestures in speech disambiguation: ERP evidence. Journal of Cognitive Neuroscience 2007, 19(7):1175-1192.
28. Holle H., Obleser J., Rueschemeyer S.A., Gunter T.C. Integration of iconic gestures and speech in left superior temporal areas boosts speech comprehension under adverse listening conditions. NeuroImage 2010, 49(1):875-884.
29. Hoskin J., Herman R. The communication, speech and gesture of a group of hearing-impaired children. International Journal of Language and Communication Disorders 2001, 36(s1):206-209.
30. Johnson R.E. An extension of Oregon sawmill sign language. Current Anthropology 1977, 18(2):353-354.
31. Kelly S.D., Barr D.J., Church R.B., Lynch K. Offering a hand to pragmatic understanding: The role of speech and gesture in comprehension and memory. Journal of Memory and Language 1999, 40(4):577-592.
32. Kelly S.D., özyürek A., Maris E. Two sides of the same coin: Speech and gesture mutually interact to enhance comprehension. Psychological Science 2010, 21(2):260-267.
33. Kelly S.D., Ward S., Creigh P., Bartolotti J. An intentional stance modulates the integration of gesture and speech during comprehension. Brain and Language 2007, 101(3):222-233.
34. Kendon A. Gesture. Annual Review of Anthropology 1997, 26:109-128.
35. Kendon A. Gesture. Visible Action as Utterance 2004, Cambridge University Press, Cambridge.
36. Krauss R.M., Morrel-Samuels P., Colasante C. Do conversational hand gestures communicate?. Journal of Personality and Social Psychology November 1991, 61(5):743-754.
37. Ma W.J., Zhou X., Ross L.A., Foxe J.J., Parra L.C. Lip-reading aids word recognition most in moderate noise: A Bayesian explanation using high-dimensional feature space. PLoS ONE 2009, 4(3):e4638.
38. Martin C., Vu H., Kellas G., Metcalf K. Strength of discourse context as a determinant of the subordinate bias effect. Quarterly Journal of Experimental Psychology Section A-Human Experimental Psychology 1999, 52(4):813-839.
39. McNeill D. Hand and Mind - What Gestures Reveal about Thought 1992, The University of Chicago Press, Chicago, Illinois, and London, England.
40. McNeill D. Gesture and Thought 2005, University of Chicago Press, Chicago, Illinois, and London, England.
41. McNeill D., Cassell J., McCullough K.E. Communicative effects of speech-mismatched gestures. Research on Language and Social Interaction 1994, 27(3):223-237.
42. Meissner M., Philpott S.B. The sign language of sawmill workers in British Columbia. Sign Language Studies 1975, 9:291-308.
43. Moore B.C.J. An Introduction to the Psychology of Hearing 2003, Academic Press Elsevier Science, NY. 5th ed.
44. Muir L.J., Richardson I.E.G. Percepetion of sign language and its application to visual communications for deaf people. Journal of Deaf Studies and Deaf Education 2005, 10(4):390-401.
45. Natalizia A., Casale M., Guglielmelli E., Rinaldi V., Bressi F., Salvinelli F. An overview of hearing impairment in older adults: Perspectives for rehabilitation with hearing aids. European Review for Medical and Pharmacologial Science 2010, 14(3):223-229.
46. Obermeier C., Holle H., Gunter T.C. What iconic gesture fragments reveal about gesture-speech integration: When synchrony is lost, memory can help. Journal of Cognitive Neuroscience 2011, 23(7):1648-1663.
47. Oldfield R.C. The assessment and analysis of handedness: The Edinburgh inventory. Neuropsychologia 1971, 9(1):97-113.
48. Paul P.V., Gustafson G. Comprehension of high-frequency multimeaning words by students with hearing impairment. Remedial and Special Education (RASE) 1991, 12(4):52-62.
49. Paul P.V., O'Rourke J.P. Multimeaning words and reading comprehension: Implications for special education students. Remedial and Special Education (RASE) 1988, 9(3):42-52.
50. Proksch J., Bavelier D. Changes in the spatial distribution of visual attention after early deafness. Journal of Cognitive Neuroscience 2002, 14(5):687-701.
51. Rogers W.T. The contribution of kinesic illustrators toward the comprehension of verbal behavior within utterances. Human Communication Research 1978, 5(1):54-62.
52. Ross L.A., Saint-Amour D., Leavitt V.M., Javitt D.C., Foxe J.J. Do you see what I am saying? Exploring visual enhancement of speech comprehension in noisy environments. Cerebral Cortex 2007, 17(5):1147-1153.
53. Schegloff E.A. On some gestures' relation to talk. Structures of Social Action 1984, 266-298. Cambridge University Press, Cambridge. J.M. Atkinson, J. Heritage (Eds.).
54. Simpson G.B. Meaning dominance and semantic context in the processing of lexical ambiguity. Journal of Verbal Learning and Verbal Behavior 1981, 20(1):120-136.
55. Schwartz J.L., Berthommier F., Savariaux C. Seeing to hear better: Evidence for early audio-visual interactions in speech identification. Cognition 2004, 93:69-78.
56. Van Atteveldt N.M., Formisano E., Goebel R., Blomert L. Top-down task effects overrule automatic multisensory responses to letter-sound pairs in auditory association cortex. NeuroImage 2007, 36(4):1345-1360.