McGurk effects in cochlear-implanted deaf subjects

Brain Research

Volumn 1188, Issue 1, 2008, Pages 87-99

  Rouger, Julien ^a   Fraysse, Bernard ^b   Deguine, Olivier ^a,b   Barone, Pascal ^a  

^a Brain and Cognition Research Center in Toulouse   (France)

^b CHU PURPAN   (France)

Author keywords

Articulatory feature; Audiovisual speech; Cochlear implantation; Cross modal reorganization; McGurk effect

Indexed keywords

ADULT; AGED; ARTICLE; AUDITORY CORTEX; CLINICAL ARTICLE; COMPARATIVE STUDY; COMPREHENSION; CONSONANT; CONTROLLED STUDY; FEMALE; HEARING; HEARING IMPAIRMENT; HUMAN; IMPLANTATION; LIP READING; MALE; PRIORITY JOURNAL; SENSORY ANALYSIS; SPEECH; STIMULUS RESPONSE; VISION; VISUAL CORTEX; VOICE; VOWEL;

ACOUSTIC STIMULATION; ADULT; CEREBRAL CORTEX; COCHLEAR IMPLANTS; CUES; DEAFNESS; FEMALE; HUMANS; ILLUSIONS; LANGUAGE TESTS; MALE; MIDDLE AGED; NEURAL PATHWAYS; NEURONAL PLASTICITY; PHOTIC STIMULATION; READING; RECOVERY OF FUNCTION; SPEECH PERCEPTION; VISUAL PERCEPTION;

EID: 37349057822     PISSN: 00068993     EISSN: None     Source Type: Journal    
DOI: 10.1016/j.brainres.2007.10.049     Document Type: Article

References (95)

1. Arnold P., and Kopsel A. Lipreading, reading and memory of hearing and hearing-impaired children. Scand. Audiol. 25 (1996) 13-20
2. Aronson E., and Rosenbloom S. Space perception in early infancy: perception within a common auditory-visual space. Science 172 (1971) 1161-1163
3. Bavelier D., and Neville H.J. Cross-modal plasticity: where and how?. Nat. Rev., Neurosci. 3 (2002) 443-452
4. Bavelier D., Dye M.W., and Hauser P.C. Do deaf individuals see better?. Trends Cogn. Sci. 10 (2006) 512-518
5. Benoit C., Mohamadi T., and Kandel S. Effects of phonetic context on audio-visual intelligibility of French. J. Speech Hear. Res. 37 (1994) 1195-1203
6. Besle J., Fort A., Delpuech C., and Giard M.H. Bimodal speech: early suppressive visual effects in human auditory cortex. Eur. J. Neurosci. 20 (2004) 2225-2234
7. Binnie C.A., Montgomery A.A., and Jackson P.L. Auditory and visual contributions to the perception of consonants. J. Speech Hear. Res. 17 (1974) 619-630
8. Boothroyd A., Mulhearn B., Gong J., and Ostroff J. Effects of spectral smearing on phoneme and word recognition. J. Acoust. Soc. Am. 100 (1996) 1807-1818
9. Brancazio L. Lexical influences in audiovisual speech perception. J. Exp. Psychol. Hum. Percept. Perform. 30 (2004) 445-463
10. Brozinsky C.J., and Bavelier D. Motion velocity thresholds in deaf signers: changes in lateralization but not in overall sensitivity. Brain Res. Cogn. Brain Res. 21 (2004) 1-10
11. Calmels M.N., Saliba I., Wanna G., Cochard N., Fillaux J., Deguine O., and Fraysse B. Speech perception and speech intelligibility in children after cochlear implantation. Int. J. Pediatr. Otorhinolaryngol. 68 (2003) 347-351
12. Calvert G.A., Campbell R., and Brammer M.J. Evidence from functional magnetic resonance imaging of crossmodal binding in the human heteromodal cortex. Curr. Biol. 10 (2000) 649-657
13. Cappe C., and Barone P. Heteromodal connections supporting multisensory integration at low levels of cortical processing in the monkey. Eur. J. Neurosci. 22 (2005) 2886-2902
14. Champoux F., Tremblay C., Mercier C., Lassonde M., Lepore F., Gagne J.P., and Theoret H. A role for the inferior colliculus in multisensory speech integration. NeuroReport 17 (2006) 1607-1610
15. Chouard C.H., Fugain C., Meyer B., and Lacombe H. Long-term results of the multichannel cochlear implant. Ann. N.Y. Acad. Sci. 405 (1983) 387-411
16. Danhauer J.L., Erratt J.D., and Edgerton B.J. Auditory/Visual speech perception by cochlear implant users and hearing aid wearers. Am. J. Otol. 7 (1986) 354-360
17. Dodd B. Lipreading in infants: attention to speech presented in and out of synchrony. Cogn. Psychol. 11 (1979) 478-484
18. Dorman M.F., Soli S., Dankowski K., Smith L.M., McCandless G., and Parkin J. Acoustic cues for consonant identification by patients who use the Ineraid cochlear implant. J. Acoust. Soc. Am. 88 (1990) 2074-2079
19. Dowell R.C., Martin L.F., Tong Y.C., Clark G.M., Seligman P.M., and Patrick J.F. A 12-consonant study on a multiple-channel cochlear implant patient. J. Speech Hear. Res. 25 (1982) 509-516
20. Erber N.P. Interaction of audition and vision in the recognition of oral speech stimuli. J. Speech Hear. Res. 12 (1969) 423-425
21. Erber N.P. Auditory, visual and auditory-visual recognition of consonants by children with normal and impaired hearing. J. Speech Hear. Res. 15 (1972) 413-422
22. Erber N.P. Auditory-visual perception of speech. J. Speech Hear. Disord. 33 (1975) 771-778
23. Fishman K.E., Shannon R.V., and Slattery W.H. Speech recognition as a function of the number of electrodes used in the SPEAK cochlear implant speech processor. J. Speech Lang. Hear. Res. 40 (1997) 1201-1215
24. Fu Q.J., Shannon R.V., and Wang X. Effects of noise and spectral resolution on vowel and consonant recognition: acoustic and electric hearing. J. Acoust. Soc. Am. 104 (1998) 3586-3596
25. Geers A., and Brenner C. Speech perception results: audition and lipreading enhancement. Volta Rev. 96 (1994) 97-108
26. Geers A., Brenner C., and Davidson L. Factors associated with development of speech perception skills in children implanted by age five. Ear Hear. 24 (2003) 24S-35S
27. Giraud A.L., Truy E., Frackowiak R.S., Gregoire M.C., Pujol J.F., and Collet L. Differential recruitment of the speech processing system in healthy subjects and rehabilitated cochlear implant patients. Brain 123 (2000) 1391-1402
28. Giraud A.L., Price C.J., Graham J.M., Truy E., and Frackowiak R.S. Cross-modal plasticity underpins language recovery after cochlear implantation. Neuron 30 (2001) 657-663
29. Grant K.W., and Braida L.D. Evaluating the articulation index for audiovisual input. J. Acoust. Soc. Am. 89 (1991) 2952-2960
30. Grant K.W., and Seitz P. The use of visible speech cues for improving auditory detection of spoken sentences. J. Acoust. Soc. Am. 108 (2000) 1197-1208
31. Green K.P., and Kuhl P.K. Integral processing of visual place and auditory voicing information during phonetic perception. J. Exp. Psychol. Hum. Percept. Perform. 17 (1991) 278-288
32. Halle M., Hughes G.W., and Radley J.P.A. Acoustic properties of stop consonants. J. Acoust. Soc. Am. 20 (1957) 107-116
33. Hamilton R.H., Shenton J.T., and Coslett H.B. An acquired deficit of audiovisual speech processing. Brain Lang. 98 (2006) 66-73
34. Jones J.A., and Callan D.E. Brain activity during audiovisual speech perception: an fMRI study of the McGurk effect. NeuroReport 14 (2003) 1129-1133
35. Kaiser A.R., Kirk K.I., Lachs L., and Pisoni D.B. Talker and lexical effects on audiovisual word recognition by adults with cochlear implants. J. Speech Lang. Hear. Res. 46 (2003) 390-404
36. Kang E., Lee D.S., Kang H., Hwang C.H., Oh S.H., Kim C.S., Chung J.K., and Lee M.C. The neural correlates of cross-modal interaction in speech perception during a semantic decision task on sentences: a PET study. NeuroImage 32 (2006) 423-431
37. Kim J., and Davis C. Hearing foreign voices: does knowing what is said affect masked visual speech detection?. Perception 32 (2003) 111-120
38. Kuhl P.K., and Meltzoff A.N. The bimodal development of speech in infancy. Science 218 (1982) 1138-1141
39. Kuhl P.K., and Meltzoff A.N. The intermodal representation of speech in infants. Infant Behav. Dev. 7 (1984) 361-381
40. Kuhl P.K., and Meltzoff A.N. Speech as an intermodal object of perception. In: Yonas A. (Ed). Perceptual development in infancy: The Minnesota Symposia on child psychology vol. 20 (1988), Erlbaum, Hillsdale, NJ
41. Lachs L., Pisoni D.B., and Kirk K.I. Use of audiovisual information in speech perception by prelingually deaf children with cochlear implants: a first report. Ear Hear. 22 (2001) 236-251
42. Lagleyre S., Rouger J., Laborde M.L., Demonet J.F., Fraysse B., Deguine O., and Barone P. Role of visuo-auditory integration in speech comprehension in deaf subjects with cochlear implants. 2nd Meeting of the European Societies of Neuropsychology (2006) 162
43. Legerstee M. Infants use multimodal information to imitate speech sounds. Infant Behav. Dev. 17 (1990) 829-840
44. Macaluso E., George N., Dolan R., Spence C., and Driver J. Spatial and temporal factors during processing of audiovisual speech: a PET study. NeuroImage 21 (2004) 725-732
45. MacDonald J., and McGurk H. Visual influences on speech perception processes. Percept. Psychophys. 24 (1978) 253-257
46. MacKain K., Studdert-Kennedy M., Spieker S., and Stern D. Infant intermodal speech perception is a left hemisphere function. Science 219 (1983) 1347-1349
47. MacLeod A., and Summerfield Q. Quantifying the contribution of vision to speech perception in noise. Br. J. Audiol. 21 (1987) 131-141
48. McGurk H., and MacDonald J. Hearing lips and seeing voices. Nature 264 (1976) 746-748
49. McKay C.M., and McDermott H.J. Perceptual performance of subjects with cochlear implants using the Spectral Maxima Sound Processor (SMSP) and the Mini Speech Processor (MSP). Ear Hear. 14 (1993) 350-367
50. Miller G.A., and Nicely P. An analysis of perceptual confusions among some English consonants. J. Acoust. Soc. Am. 27 (1955) 338-352
51. Moody-Antonio S., Takayanagi S., Masuda A., Auer Jr. E.T., Fisher L., and Bernstein L.E. Improved speech perception in adult congenitally deafened cochlear implant recipients. Otol. Neurotol. 26 (2005) 649-654
52. Möttönen R., Schürmann M., and Sams M. Time course of multisensory interactions during audiovisual speech perception in humans: a magnetoencephalographic study. Neurosci. Lett. 363 (2004) 112-115
53. Munson B., and Nelson P.B. Phonetic identification in quiet and in noise by listeners with cochlear implants. J. Acoust. Soc. Am. 118 (2005) 2607-2617
54. Musacchia G., Sams M., Nicol T., and Kraus N. Seeing speech affects acoustic information processing in the human brainstem. Exp. Brain Res. 168 (2006) 1-10
55. Naito Y., Okazawa H., Honjo I., Hirano S., Takahashi H., Shiomi Y., Hoji W., Kawano M., Ishizu K., and Yonekura Y. Cortical activation with sound stimulation in cochlear implant users demonstrated by positron emission tomography. Cogn. Brain Res. 2 (1995) 207-214
56. Neville H.J. Intermodal competition and compensation in development. Evidence from studies of the visual system in congenitally deaf adults. Ann. N.Y. Acad. Sci. 608 (1990) 71-91
57. Neville H., and Bavelier D. Human brain plasticity: evidence from sensory deprivation and altered language experience. Prog. Brain Res. 138 (2002) 177-188
58. Neville H.J., and Lawson D. Attention to central and peripheral visual space in a movement detection task: an event-related potential and behavioral study: II. Congenitally deaf adults. Brain Res. 405 (1987) 268-283
59. Neely K.K. Effect of visual factors on the intelligibility of speech. J. Acoust. Soc. Am. 28 (1956) 1275-1277
60. 2(15)O and PET. Brain 119 (1996) 1297-1306
61. Pelizzone M., Cosendai G., and Tinembart J. Within-patient longitudinal speech reception measures with continuous interleaved sampling processors for Ineraid implanted subjects. Ear Hear. 20 (1999) 228-237
62. Pickens J., Field T., Nawrocki T., Martinez A., Soutullo D., and Gonzalez J. Full-term and preterm infants' perception of face-voice synchrony. Infant Behav. Dev. 17 (1994) 447-455
63. Proksch J., and Bavelier D. Changes in the spatial distribution of visual attention after early deafness. J. Cogn. Neurosci. 14 (2002) 687-701
64. Rabinowitz W.M., Eddington D.K., Delhorne L.A., and Cuneo P.A. Relations among different measures of speech reception in subjects using a cochlear implant. J. Acoust. Soc. Am. 92 (1992) 1869-1881
65. Reisberg D., McLean J., and Goldfield A. Easy to hear but hard to understand. In: Dodd B., and Campbell R. (Eds). Hearing by Eye: The Psychology of Lipreading (1987), Erlbaum, Hillsdale, NJ 97-114
66. Roder B., Teder-Salejarvi W., Sterr A., Rosler F., Hillyard S.A., and Neville H.J. Improved auditory spatial tuning in blind humans. Nature 400 (1999) 162-166
67. Rosen S., Walliker J., Brimacombe J.A., and Edgerton B.J. Prosodic and segmental aspects of speech perception with the House/3M single-channel implant. J. Speech Hear. Res. 32 (1989) 93-111
68. Ross L.A., Saint-Amour D., Leavitt V.M., Javitt D.C., and Foxe J.J. Do you see what I am saying? Exploring visual enhancement of speech comprehension in noisy environments. Cereb. Cortex 17 (2007) 1147-1153
69. Rouger J., Lagleyre S., Fraysse B., Deneve S., Deguine O., and Barone P. Evidence that cochlear implanted deaf patients are better multisensory integrators. Proc. Natl. Acad. Sci. U. S. A. 104 (2007) 7295-7300
70. Saito D.N., Yoshimura K., Kochiyama T., Okada T., Honda M., and Sadato N. Cross-modal binding and activated attentional networks during audio-visual speech integration: a functional MRI study. Cereb. Cortex 15 (2005) 1750-1760
71. Sams M., Manninen P., Surakka V., Helin P., and Kättö R. McGurk effect in Finnish syllables, isolated words, and words in sentences: effects of word meaning and sentence context. Speech Commun. 26 (1998) 75-87
72. Schorr E.A., Fox N.A., van Wassenhove V., and Knudsen E.L. Auditory-visual fusion in speech perception in children with cochlear implants. Proc. Natl. Acad. Sci. U. S. A. 102 (2005) 18748-18750
73. Schwartz J.L., Berthommier F., and Savariaux C. Seeing to hear better: evidence for early audio-visual interactions in speech identification. Cognition 93 (2004) B69-B78
74. Sekiyama K., Kanno I., Miura S., and Sugita Y. Audio-visual speech perception examined by fMRI and PET. Neurosci. Res. 47 (2003) 277-287
75. Shannon R.V. Understanding hearing through deafness. Proc. Natl. Acad. Sci. U. S. A. 104 (2007) 6883-6884
76. Shannon R.V., Zeng F.G., Kamath V., Wygonsky J., and Ekelid M. Speech recognition with primarily temporal cues. Science 270 (1995) 303-304
77. Skinner M.W., Fourakis M.S., Holden T.A., Holden L.K., and Demorest M.E. Identification of speech by cochlear implant recipients with the multipeak (MPEAK) and spectral peak (SPEAK) speech coding strategies: II. Consonants. Ear Hear. 20 (1999) 443-460
78. Skipper J.I., Nusbaum H.C., and Small S.L. Listening to talking faces: motor cortical activation during speech perception. NeuroImage 25 (2005) 76-89
79. Spelke E.S., and Cortelyou A. Perceptual aspects of social knowing: looking and listening in Infancy. In: Lamb E., and Sherrod L. (Eds). Infant social cognition (1980), Erlbaum, Hillsdale, NJ
80. Spelke E.S., and Owsley C. Intermodal exploration and knowledge in infancy. Infant Behav. Dev. 2 (1979) 13-27
81. Staller S.J., Dowell R.C., Beiter A.L., and Brimacombe J.A. Perceptual abilities of children with the nucleus 22-channel cochlear implant. Ear Hear. 12 (1991) 34S-47S
82. Stein B.E., and Meredith M.A. The Merging of the Senses (1993), MIT Press, Cambridge, MA
83. Sumby W.H., and Pollack I. Visual contribution to speech intelligibility in noise. J. Acoust. Soc. Am. 26 (1954) 212-215
84. Summerfield Q. Use of visual information for phonetic perception. Phonetica 36 (1979) 314-331
85. Tyler R.S., Preece J.P., Lansing C.R., Otto S.R., and Gantz B.J. Previous experience as a confounding factor in comparing cochlear-implant processing schemes. J. Speech Hear. Res. 29 (1986) 282-287
86. Tyler R.S., Fryauf-Bertschy H., Kelsay D.M.R., Gantz B.J., Woodworth G.P., and Parkinson A. Speech perception by prelingually deaf children using cochlear implants. Otolaryngol. Head Neck Surg. 117 (1997) 180-187
87. United Kingdom Cochlear Implant Study Group. Criteria of candidacy for unilateral cochlear implantation in postlingually deafened adults I: theory and measures of effectiveness. Ear Hear. 25 (2004) 310-335
88. Valimaa T.T., Maatta T.K., Lopponen H.J., and Sorri M.J. Phoneme recognition and confusions with multichannel cochlear implants: consonants. J. Speech Lang. Hear. Res. 45 (2002) 1055-1069
89. van Wieringen A., and Wouters J. Natural vowel and consonant recognition by Laura cochlear implantees. Ear Hear. 20 (1999) 89-103
90. Walden B.E., Prosek R.A., Montgomery A.A., Scherr C.K., and Jones C.J. Effects of training on the visual recognition of consonants. J. Speech Hear. Res. 20 (1977) 130-145
91. Walker A.S. Intermodal perception of expression behaviors by human infants. J. Exp. Child Psychol. 33 (1982) 514-535
92. Walton G.E., and Bower T.G.R. Amodal representation of speech in infants. Infant Behav. Dev. 16 (1993) 233-243
93. Wilson B.S., Finley C.C., Lawson D.T., Wolford R.D., Eddington D.K., and Rabinowitz W.M. Better speech recognition with cochlear implants. Nature 352 (1991) 236-238
94. Wong D., Miyamoto R.T., Pisoni D.B., Sehgal M., and Hutchins G.D. PET imaging of cochlear-implant and normal-hearing subjects listening to speech and nonspeech. Hear. Res. 132 (1999) 34-42
95. Wright T.M., Pelphrey K.A., Allison T., McKeown M.J., and McCarthy G. Polysensory interactions along lateral regions evoked by audiovisual speech. Cereb. Cortex 13 (2003) 1034-1043