Neural restoration of degraded audiovisual speech

NeuroImage

Volumn 60, Issue 1, 2012, Pages 530-538

  Shahin, Antoine J ^a   Kerlin, Jess R ^b   Bhat, Jyoti ^a   Miller, Lee M ^b  

^a OHIO STATE UNIVERSITY   (United States)

^b UNIVERSITY OF CALIFORNIA   (United States)

Author keywords

Audiovisual integration; Auditory evoked potentials; EEG; Illusory filling in; Phase locking; Theta band

Indexed keywords

ADULT; ARTICLE; AUDITORY CORTEX; AUDITORY DISCRIMINATION; AUDITORY STIMULATION; BEHAVIORAL SCIENCE; BRAIN FUNCTION; CONTROLLED STUDY; EVOKED AUDITORY RESPONSE; FEMALE; HUMAN; HUMAN EXPERIMENT; ILLUSION; LIP READING; MALE; NERVE CELL NETWORK; NOISE REDUCTION; NORMAL HUMAN; OSCILLATION; PRIORITY JOURNAL; SPEECH DISCRIMINATION; SPEECH INTELLIGIBILITY; TASK PERFORMANCE; THETA RHYTHM; VISUAL DISCRIMINATION; VISUAL STIMULATION; WHITE NOISE;

ADULT; AUDITORY CORTEX; ELECTROENCEPHALOGRAPHY; FEMALE; HUMANS; ILLUSIONS; MALE; SPEECH PERCEPTION; VISUAL PERCEPTION;

EID: 84857132150     PISSN: 10538119     EISSN: 10959572     Source Type: Journal    
DOI: 10.1016/j.neuroimage.2011.11.097     Document Type: Article

References (31)

1. Bashford J.A., Meyers M.D., Brubaker B.S., Warren R.M. Illusory continuity of interrupted speech: speech rate determines durational limits. J. Acoust. Soc. Am. 1988, 84:1635-1638.
2. Bashford J.A., Warren R.M., Brown C.A. Use of speech-modulated noise adds strong "bottom-up" cues for phonemic restoration. Percept Psychophys. 1996, 58:342-350.
3. Bhat J.V. Signal Processing Techniques For Artifact Removal In Electroencephalogram (EEG) (Master's Thesis, University of Texas at Arlington) 2009, Retrieved from http://dspace.uta.edu/handle/10106/2047.
4. Chau W., McIntosh A.R., Robinson S.E., Schulz M., Pantev C. Improving permutation test power for group analysis of spatially filtered MEG data. Neuroimage 2004, 23:983-996.
5. Delorme A., Makeig S. EEGLAB: an open source toolbox for analysis of single-trial EEG dynamics including independent component analysis. J. Neurosci. Methods 2004, 134:9-21.
6. Fraser S., Gagne J.P., Alepins M., Dubois P. Evaluating the effort expended to understand speech in noise using a dual-task paradigm: the effects of providing visual speech cues. J. Speech Lang. Hear. Res. 2010, 53:18-33.
7. Grant K.W., Walden B.E., Seitz P.F. Auditory-visual speech recognition by hearing-impaired subjects: consonant recognition, sentence recognition, and auditory-visual integration. J. Acoust. Soc. Am. 1998, 103:2677-2690.
8. Groppe D.M., Choi M., Huang T., Schilz J., Topkins B., Urbach T.P., Kutas M. The phonemic restoration effect reveals pre-N400 effect of supportive sentence context in speech perception. Brain Res. 2010, 1361:54-66.
9. Heinrich A., Carlyon R.P., Davis M.H., Johnsrude I.S. Illusory vowels resulting from perceptual continuity: a functional magnetic resonance imaging study. J. Cogn. Neurosci. 2008, 20:1737-1752.
10. Heinrich A., Carlyon R.P., Davis M.H., Johnsrude I.S. The continuity illusion does not depend on attentional state: FMRI evidence from illusory vowels. J. Cogn. Neurosci. 2011, 23:2675-2689.
11. Hillyard S.A., Picton T.W. On and off components in the auditory evoked potential. Percept. Psychophys. 1978, 24:391-398.
12. Kaiser A.R., Kirk K.I., Lachs L., Pisoni D.B. Talker and lexical effects on audiovisual word recognition by adults with cochlear implants. J. Speech Lang. Hear. Res. 2003, 46:390-404.
13. Keil J., Muller N., Ihssen N., Weisz N. On the variability of the McGurk effect: audiovisual integration depends on prestimulus brain states. Cereb Cortex 2012, 22:221-231.
14. Luo H., Poeppel D. Phase patterns of neuronal responses reliably discriminate speech in human auditory cortex. Neuron 2007, 54:1001-1010.
15. Luo H., Liu Z., Poeppel D. Auditory cortex tracks both auditory and visual stimulus dynamics using low-frequency neuronal phase modulation. PLoS Biol. 2010, 8:e1000445.
16. Lyzenga J., Carlyon R.P., Moore B.C. Dynamic aspects of the continuity illusion: perception of level and of the depth, rate, and phase of modulation. Hear. Res. 2005, 210:30-41.
17. O'Connell M.N., Falchier A., McGinnis T., Schroeder C.E., Lakatos P. Dual mechanism of neuronal ensemble inhibition in primary auditory cortex. Neuron 2011, 69:805-817.
18. Petkov C.I., O'Connor K.N., Sutter M.L. Encoding of illusory continuity in primary auditory cortex. Neuron 2007, 54:153-165.
19. Riecke L., Esposito F., Bonte M., Formisano E. Hearing illusory sounds in noise: the timing of sensory-perceptual transformations in auditory cortex. Neuron 2009, 64:550-561.
20. Samuel A.G. Phonemic restoration: insights from a new methodology. J. Exp. Psychol. Gen. 1981, 110:474-494.
21. Samuel A.G. Lexical activation produces potent phonemic percepts. Cogn. Psychol. 1997, 32:97-127.
22. Schroeder C.E., Lakatos P., Kajikawa Y., Partan S., Puce A. Neuronal oscillations and visual amplification of speech. Trends Cogn. Sci. 2008, 12:106-113.
23. Shahin A.J., Miller L.M. Multisensory integration enhances phonemic restoration. J. Acoust. Soc. Am. 2009, 125:1744-1750.
24. Shahin A.J., Bishop C.W., Miller L.M. Neural mechanisms for illusory filling-in of degraded speech. Neuroimage 2009, 44:1133-1143.
25. Shahin A.J., Picton T.W., Miller L.M. Brain oscillations during semantic evaluation of speech. Brain Cogn. 2009, 70(3):259-266. (Aug).
26. Shannon R.V., Zeng F.G., Kamath V., Wygonski J., Ekelid M. Speech recognition with primarily temporal cues. Science 1995, 270:303-304.
27. Sivonen P., Maess B., Friederici A.D. Semantic retrieval of spoken words with an obliterated initial phoneme in a sentence context. Neurosci. Lett. 2006, 408:220-225.
28. Sivonen P., Maess B., Lattner S., Friederici A.D. Phonemic restoration in a sentence context: evidence from early and late ERP effects. Brain Res. 2006, 1121:177-189.
29. Tallon-Baudry C., Bertrand O., Delpuech C., Pernier J. Stimulus specificity of phase-locked and non-phase-locked 40Hz visual responses in human. J. Neurosci. 1996, 16:4240-4249.
30. Wager T.D., Nichols T.E. Optimization of experimental design in fMRI: a general framework using a genetic algorithm. Neuroimage 2003, 18:293-309.
31. Warren R.M. Perceptual restoration of missing speech sounds. Science 1970, 167:392-393.