Role of speechreading in audiovisual interactions during the recovery of speech comprehension in deaf adults with cochlear implants

Scandinavian Journal of Psychology

Volumn 50, Issue 5, 2009, Pages 437-444

  Strelnikov, K ^a,b   Rouger, J ^a,b   Barone, P ^a,b   Deguine, O ^a,b,c  

^a Brain and Cognition Research Center in Toulouse   (France)

^b CNRS   (France)

^c CHU PURPAN   (France)

Author keywords

Brain plasticity; Cochlear implantation; Deafness; Lip reading; Speechreading

Indexed keywords

EID: 70349337886     PISSN: 00365564     EISSN: 14679450     Source Type: Journal    
DOI: 10.1111/j.1467-9450.2009.00741.x     Document Type: Article

References (79)

1. Arnold, P. Hill, F. (2001). Bisensory augmentation: A speechreading advantage when speech is clearly audible and intact. British Journal of Psychology, 92 (2 339 355.
2. Auer, E. T., Jr (2009). Spoken word recognition by eye. Scandinavian Journal of Psychology, 50, 419 425.
3. Bavelier, D., Dye, M. W. Hauser, P. C. (2006). Do deaf individuals see better? Trends in Cognitive Sciences, 10 (11 512 518.
4. Bavelier, D. Neville, H. J. (2002). Cross-modal plasticity: Where and how? Nature Reviews, 3 (6 443 452.
5. Benoit, C., Mohamadi, T. Kandel, S. (1994). Effects of phonetic context on audio-visual intelligibility of French. Journal of Speech and Hearing Research, 37 (5 1195 1203.
6. Bergeson, T. R., Pisoni, D. B. Davis, R. A. (2005). Development of audiovisual comprehension skills in prelingually deaf children with cochlear implants. Ear and Hearing, 26 (2 149 164.
7. Bernstein, L. E., Auer, E. T., Jr. Takayanagi, S. (2004). Auditory speech detection in noise enhanced by lipreading. Speech Communication, 44 (1-4 5 18.
8. Bernstein, L. E., Auer, E. T., Jr. Tucker, P. E. (2001). Enhanced speechreading in deaf adults: Can short-term training/practice close the gap for hearing adults? Journal of Speech, Language, and Hearing Research, 44 (1 5 18.
9. Bernstein, L. E., Demorest, M. E. Tucker, P. E. (2000). Speech perception without hearing. Perception & Psychophysics, 62 (2 233 252.
10. Brancazio, L. Brancazio, L. (2004). Lexical influences in audiovisual speech perception. Journal of Experimental Psychology, 30 (3 445 463.
11. Calmels, M. N., Saliba, I., Wanna, G., Cochard, N., Fillaux, J., Deguine, O., et al. (2004). Speech perception and speech intelligibility in children after cochlear implantation. International Journal of Pediatric Otorhinolaryngology, 68 (3 347 351.
12. Campbell, R. (2008). The processing of audio-visual speech: Empirical and neural bases. Philosophical Transactions of the Royal Society of London, 363 (1493 1001 1010.
13. Capek, C. M., MacSweeney, M., Woll, B., Waters, D., McGuire, P. K., David, A. S., et al. (2008). Cortical circuits for silent speechreading in deaf and hearing people. Neuropsychologia, 46 (5 1233 1241.
14. Cappe, C. Barone, P. (2005). Heteromodal connections supporting multisensory integration at low levels of cortical processing in the monkey. The European Journal of Neuroscience, 22 (11 2886 2902.
15. Copeland, B. J. Pillsbury, H. C. (2004). Cochlear implantation for the treatment of deafness. Annual Review of Medicine, 55, 157 167.
16. Criteria of candidacy for unilateral cochlear implantation in postlingually deafened adults. I: Theory and measures of effectiveness 2004). Ear and Hearing, 25 (4 310 335.
17. Deggouj, N., Gersdorff, M., Garin, P., Castelein, S. Gerard, J. M. (2007). Today's indications for cochlear implantation. Acta oto-rhino- laryngologica Belgica, 3 (1 9 14.
18. Desai, S., Stickney, G. Zeng, F. G. (2008). Auditory-visual speech perception in normal-hearing and cochlear-implant listeners. The Journal of the Acoustical Society of America, 123 (1 428 440.
19. Di Nardo, W., Cantore, I., Cianfrone, F., Melillo, P., Rigante, M. Paludetti, G. (2007). Residual hearing thresholds in cochlear implantation and reimplantation. Audiology & Neuro-otology, 12 (3 165 169.
20. Doucet, M. E., Bergeron, F., Lassonde, M., Ferron, P. Lepore, F. (2006). Cross-modal reorganization and speech perception in cochlear implant users. Brain, 129 (Pt 12 3376 3383.
21. Drennan, W. R. Rubinstein, J. T. (2008). Music perception in cochlear implant users and its relationship with psychophysical capabilities. Journal of Rehabilitation Research and Development, 45 (5 779 789.
22. Falchier, A., Clavagnier, S., Barone, P. Kennedy, H. (2002). Anatomical evidence of multimodal integration in primate striate cortex. Journal of Neuroscience, 22 (13 5749 5759.
23. Finney, E. M., Clementz, B. A., Hickok, G. Dobkins, K. R. (2003). Visual stimuli activate auditory cortex in deaf subjects: Evidence from MEG. Neuroreport, 14 (11 1425 1427.
24. Finney, E. M., Fine, I. Dobkins, K. R. (2001). Visual stimuli activate auditory cortex in the deaf. Nature Neuroscience, 4 (12 1171 1173.
25. Frassinetti, F., Bolognini, N., Bottari, D., Bonora, A. Ladavas, E. (2005). Audiovisual integration in patients with visual deficit. Journal of Cognitive Neuroscience, 17 (9 1442 1452.
26. Friesen, L. M., Shannon, R. V., Baskent, D. Wang, X. (2001). Speech recognition in noise as a function of the number of spectral channels: comparison of acoustic hearing and cochlear implants. The Journal of the Acoustical Society of America, 110 (2 1150 1163.
27. Fu, Q. J., Shannon, R. V. Wang, X. (1998). Effects of noise and spectral resolution on vowel and consonant recognition: acoustic and electric hearing. The Journal of the Acoustical Society of America, 104 (6 3586 3596.
28. Garstecki, D. C. O'Neill, J. J. (1980). Situational cue and strategy influence on speechreading. Scandinavian Audiology, 9 (3 147 151.
29. Giraud, A. L., Price, C. J., Graham, J. M. Frackowiak, R. S. (2001a). Functional plasticity of language-related brain areas after cochlear implantation. Brain, 124 (Pt 7 1307 1316.
30. Giraud, A. L., Price, C. J., Graham, J. M., Truy, E. Frackowiak, R. S. (2001b). Cross-modal plasticity underpins language recovery after cochlear implantation. Neuron, 30 (3 657 663.
31. Giraud, A. L., Truy, E. Frackowiak, R. (2001c). Imaging plasticity in cochlear implant patients. Audiology & Neuro-otology, 6 (6 381 393.
32. Giraud, A. L., Truy, E., Frackowiak, R. S., Gregoire, M. C., Pujol, J. F. Collet, L. (2000). Differential recruitment of the speech processing system in healthy subjects and rehabilitated cochlear implant patients. Brain, 123 (Pt 7 1391 1402.
33. Grant, K. W. Braida, L. D. (1991). Evaluating the articulation index for auditory-visual input. The Journal of the Acoustical Society of America, 89 (6 2952 2960.
34. Grant, K. W. Seitz, P. F. (2000). The use of visible speech cues for improving auditory detection of spoken sentences. The Journal of the Acoustical Society of America, 108 (3 Pt 1 1197 1208.
35. Grant, K. W., Walden, B. E. Seitz, P. F. (1998). Auditory-visual speech recognition by hearing-impaired subjects: consonant recognition, sentence recognition, and auditory-visual integration. The Journal of the Acoustical Society of America, 103 (5 Pt 1 2677 2690.
36. Gray, R. F., Quinn, S. J., Court, I., Vanat, Z. Baguley, D. M. (1995). Patient performance over eighteen months with the Ineraid intracochlear implant. The Annals of Otology, Rhinology & Laryngology, 166, 275 277.
37. Green, K. M., Julyan, P. J., Hastings, D. L. Ramsden, R. T. (2005). Auditory cortical activation and speech perception in cochlear implant users: effects of implant experience and duration of deafness. Hearing Research, 205 (1-2 184 192.
38. Ito, K., Momose, T., Oku, S., Ishimoto, S., Yamasoba, T., Sugasawa, M., et al. (2004). Cortical activation shortly after cochlear implantation. Audiology & Neuro-otology, 9 (5 282 293.
39. Kaiser, A. R., Kirk, K. I., Lachs, L. Pisoni, D. B. (2003). Talker and lexical effects on audiovisual word recognition by adults with cochlear implants. Journal of Speech, Language and Hearing Research, 46 (2 390 404.
40. Kim, J. Davis, C. (2003). Hearing foreign voices: Does knowing what is said affect visual-masked-speech detection? Perception, 32 (1 111 120.
41. Lagleyre, S., Rouger, J., Laborde, M. L., Demonet, J. F., Fraysse, B., Deguine, O., et al. (2006). Role of visuo-auditory integration in speech comprehension in deaf subjects with cochlear implants. Paper presented at the 2nd Meeting of the European Societies of Neuropsychology.
42. Lee, H. J., Giraud, A. L., Kang, E., Oh, S. H., Kang, H., Kim, C. S., et al. (2007). Cortical activity at rest predicts cochlear implantation outcome. Cerebral Cortex, 17 (4 909 917.
43. Lind, C., Erber, N. P. Doyle, J. (1999). Effects of related and unrelated questions on the speechreading of sentences. Journal of the Academy of Rehabilitative Audiology, 32, 61 83.
44. Lippert, M., Logothetis, N. K. Kayser, C. (2007). Improvement of visual contrast detection by a simultaneous sound. Brain Research, 1173, 102 109.
45. Lorenzi, C., Gilbert, G., Carn, H., Garnier, S. Moore, B. C. (2006). Speech perception problems of the hearing impaired reflect inability to use temporal fine structure. Proceedings of the National Academy of Sciences of the United States of America, 103 (49 18866 18869.
46. MacLeod, A. Summerfield, Q. (1987). Quantifying the contribution of vision to speech perception in noise. British Journal of Audiology, 21 (2 131 141.
47. McGurk, H. MacDonald, J. (1976). Hearing lips and seeing voices. Nature, 264 (5588 746 748.
48. Moller, A. R. (2006). History of cochlear implants and auditory brainstem implants. Adv Otorhinolaryngol, 64, 1 10.
49. Mortensen, M. V., Mirz, F. Gjedde, A. (2006). Restored speech comprehension linked to activity in left inferior prefrontal and right temporal cortices in postlingual deafness. Neuroimage, 31 (2 842 852.
50. Munson, B. Nelson, P. B. (2005). Phonetic identification in quiet and in noise by listeners with cochlear implants. The Journal of the Acoustical Society of America, 118 (4 2607 2617.
51. Nishimura, H., Doi, K., Iwaki, T., Hashikawa, K., Oku, N., Teratani, T., et al. (2000). Neural plasticity detected in short- and long-term cochlear implant users using PET. Neuroreport, 11 (4 811 815.
52. Nishimura, H., Hashikawa, K., Doi, K., Iwaki, T., Watanabe, Y., Kusuoka, H., et al. (1999). Sign language "heard" in the auditory cortex. Nature, 397 (6715 116.
53. Petitto, L. A., Zatorre, R. J., Gauna, K., Nikelski, E. J., Dostie, D. Evans, A. C. (2000). Speech-like cerebral activity in profoundly deaf people processing signed languages: Implications for the neural basis of human language. Proceedings of the National Academy of Sciences of the United States of America, 97 (25 13961 13966.
54. Pressnitzer, D., Bestel, J. Fraysse, B. (2005). Music to electric ears: Pitch and timbre perception by cochlear implant patients. Annals of the New York Academy of Sciences, 1060, 343 345.
55. Putzar, L., Goerendt, I., Lange, K., Rosler, F. Roder, B. (2007). Early visual deprivation impairs multisensory interactions in humans. Nature Neuroscience, 10 (10 1243 1245.
56. Rauschecker, J. P. (1991). Mechanisms of visual plasticity: Hebb synapses, NMDA receptors, and beyond. Physiological Reviews, 71 (2 587 615.
57. Reisberg, D., McLean, J. Goldfield, A. (1987). Easy to hear but hard to understand. In B. Dodd R. Campbell (Eds Hearing by eye: The psychology of lipreading (pp. 97 114). Hillsdale, NJ : Erlbaum.
58. Rockland, K. S. Ojima, H. (2003). Multisensory convergence in calcarine visual areas in macaque monkey. International Journal of Psychophysiology, 50 (1-2 19 26.
59. Roder, B., Teder-Salejarvi, W., Sterr, A., Rosler, F., Hillyard, S. A. Neville, H. J. (1999). Improved auditory spatial tuning in blind humans. Nature, 400 (6740 162 166.
60. Ronnberg, J., Samuelsson, S., Lyxell, B. Arlinger, S. (1996). Lipreading with auditory low-frequency information. Contextual constraints. Scandinavian Audiology, 25 (2 127 132.
61. Ross, L. A., Saint-Amour, D., Leavitt, V. M., Javitt, D. C. Foxe, J. J. (2007). Do you see what I am saying? Exploring visual enhancement of speech comprehension in noisy environments. Cerebral Cortex, 17 (5 1147 1153.
62. Rouger, J., Fraysse, B., Deguine, O. Barone, P. (2008). McGurk effects in cochlear-implanted deaf subjects. Brain Research, 1188, 87 99.
63. Rouger, J., Lagleyre, S., Fraysse, B., Deneve, S., Deguine, O. Barone, P. (2007). Evidence that cochlear-implanted deaf patients are better multisensory integrators. Proceedings of the National Academy of Sciences of the United States of America, 104 (17 7295 7300.
64. Rubinstein, A., Cherry, R., Hecht, P. Idler, C. (2000). Anticipatory strategy training: Implications for the postlingually hearing-impaired adult. Journal of the American Academy of Audiology, 11 (1 52 55.
65. Ruytjens, L., Albers, F., van Dijk, P., Wit, H. Willemsen, A. (2006). Neural responses to silent lipreading in normal hearing male and female subjects. The European Journal of Neuroscience, 24 (6 1835 1844.
66. Sams, M., Manninen, P., Surakka, V., Helin, P. Kättö, R. (1998). McGurk effect in Finnish syllables, isolated words, and words in sentences: Effects of word meaning and sentence context. Speech Communication, 26 (1-2 75 87.
67. Schwartz, J. L., Berthommier, F. Savariaux, C. (2004). Seeing to hear better: Evidence for early audio-visual interactions in speech identification. Cognition, 93 (2 B69 78.
68. Seitz, A. R., Kim, R. Shams, L. (2006). Sound facilitates visual learning. Current Biology, 16 (14 1422 1427.
69. Shannon, R. V., Zeng, F. G., Kamath, V., Wygonski, J. Ekelid, M. (1995). Speech recognition with primarily temporal cues. Science, 270 (5234 303 304.
70. Shannon, R. V., Zeng, F. G. Wygonski, J. (1998). Speech recognition with altered spectral distribution of envelope cues. The Journal of the Acoustical Society of America, 104 (4 2467 2476.
71. Stein, B. E. Meredith, M. A. (1993). The Merging of the Senses. Cambridge, MA : MIT Press.
72. Stenfelt, S. Rönnberg, J. (2009). The Signal-Cognition interface: Interactions between degraded auditory signals and cognitive processes. Scandinavian Journal of Psychology, 50, 385 393.
73. Strelnikov, K. (2008). Activation-verification in continuous speech processing. Interaction of cognitive strategies as a possible theoretical approach. Journal of Neurolinguistics, 21, 1 17.
74. Strelnikov, K., Rouger, J., Lagleyre, S., Fraysse, B., Deguine, O. Barone, P. (2009). Improvement in speech-reading ability by auditory training: Evidence from gender differences in normally hearing, deaf and cochlear implanted subjects. Neuropsychologia, 47 (4 972 979.
75. Sumby, W. H. Pollack, I. (1954). Visual contribution to speech intelligibility in noise. The Journal of the Acoustical Society of America, 26, 212 215.
76. Summerfield, Q. (1979). Use of visual information for phonetic perception. Phonetica, 36 (4-5 314 331.
77. Summerfield, Q. (1992). Lipreading and audio-visual speech perception. Philosophical Transactions of the Royal Society of London, 335 (1273 71 78.
78. Tyler, R. S., Parkinson, A. J., Woodworth, G. G., Lowder, M. W. Gantz, B. J. (1997). Performance over time of adult patients using the Ineraid or Nucleus cochlear implant. Journal of Acoustical Society of America, 102 (1 508 522.
79. Wong, D., Miyamoto, R. T., Pisoni, D. B., Sehgal, M. Hutchins, G. D. (1999). PET imaging of cochlear-implant and normal-hearing subjects listening to speech and nonspeech. Hearing Research, 132 (1-2 34 42.