Multisensory integration enhances phonemic restoration

Journal of the Acoustical Society of America

Volumn 125, Issue 3, 2009, Pages 1744-1750

  Shahin, Antoine J ^a   Miller, Lee M ^a  

^a UNIVERSITY OF CALIFORNIA   (United States)

Author keywords

[No Author keywords available]

Indexed keywords

ADVERSE ENVIRONMENTS; AUDIO VISUALS; AUDITORY MODALITIES; CONTEXTUAL KNOWLEDGE; FILLING-IN; MULTISENSORY INTEGRATIONS; PHONEMIC RESTORATIONS; SPEECH SIGNALS; SPOKEN WORDS; TEMPORAL GAPS; VISUAL SPEECH;

RESTORATION; WHITE NOISE;

SPEECH ANALYSIS;

ADULT; COGNITION; HUMANS; NOISE; PHONETICS; SEMANTICS; SOUND SPECTROGRAPHY; SPEECH PERCEPTION; TIME FACTORS; VISUAL PERCEPTION;

EID: 62249162266     PISSN: 00014966     EISSN: None     Source Type: Journal    
DOI: 10.1121/1.3075576     Document Type: Article

References (38)

1. Bashford, J. A., Jr., Meyers, M. D., Brubaker, B. S., and Warren, R. M. (1988). " Illusory continuity of interrupted speech: Speech rate determines durational limits.," J. Acoust. Soc. Am. 0001-4966 10.1121/1.397178 84, 1635-1638.
2. Bashford, Jr., J. A., Riener, K. R., and Warren, R. M. (1992). " Increasing the intelligibility of speech through multiple phonemic restorations.," Percept. Psychophys. 51, 211-217. 0031-5117
3. Bashford, Jr., J. A., Warren, R. M., and Brown, C. A. (1996). " Use of speech-modulated noise adds strong "bottom-up" cues for phonemic restoration.," Percept. Psychophys. 58, 342-350. 0031-5117
4. Binder, J. R., McKiernan, K. A., Parsons, M. E., Westbury, C. F., Possing, E. T., Kaufman, J. N., and Buchanan, L. (2003). " Neural correlates of lexical access during visual word recognition.," J. Cogn. Neurosci. 15, 372-393. 0898-929X
5. Bishop, C. W., and Miller, L. M. (2009). " A multisensory cortical network for understanding speech in noise.," J. Cogn Neurosci. (in press).
6. Brancazio, L., Miller, J. L., and Pare, M. A. (2003). " Visual influences on the internal structure of phonetic categories.," Percept. Psychophys. 65, 591-601. 0031-5117
7. Calvert, G. A., Bullmore, E. T., Brammer, M. J., Campbell, R., Williams, S. C., McGuire, P. K., Woodruff, P. W., Iversen, S. D., and David, A. S. (1997). " Activation of auditory cortex during silent lipreading.," Science 0036-8075 10.1126/science.276.5312.593 276, 593-596.
8. Friederici, A. D. (2002). " Towards a neural basis of auditory sentence processing.," Trends Cogn. Sci. 6, 78-84. 1364-6613
9. Grant, K. W. (2001). " The effect of speechreading on masked detection thresholds for filtered speech.," J. Acoust. Soc. Am. 0001-4966 10.1121/1.1362687 109, 2272-2275.
10. Grant, K. W., and Seitz, P. F. (2000). " The use of visible speech cues for improving auditory detection of spoken sentences.," J. Acoust. Soc. Am. 0001-4966 10.1121/1.1288668 108, 1197-1208.
11. Grant, K. W., Walden, B. E., and Seitz, P. F. (1998). " Auditory-visual speech recognition by hearing-impaired subjects: Consonant recognition, sentence recognition, and auditory-visual integration.," J. Acoust. Soc. Am. 0001-4966 10.1121/1.422788 103, 2677-2690.
12. Helfer, K. S., and Freyman, R. L. (2005). " The role of visual speech cues in reducing energetic and informational masking.," J. Acoust. Soc. Am. 0001-4966 10.1121/1.1836832 117, 842-849.
13. Jancke, L., and Shah, N. J. (2004). " Hearing syllables by seeing visual stimuli.," Eur. J. Neurosci. 19, 2603-2608. 0953-816X
14. Kobayashi, M., Osada, Y., and Kashino, M. (2007). " The effect of a flashing visual stimulus on the auditory continuity illusion.," Percept. Psychophys. 69, 393-399. 0031-5117
15. Martuzzi, R., Murray, M. M., Michel, C. M., Thiran, J. P., Maeder, P. P., Clarke, S., and Meuli, R. A. (2007). " Multisensory interactions within human primary cortices revealed by BOLD dynamics.," Cereb. Cortex 17, 1672-1679. 1047-3211
16. McGurk, H., and MacDonald, J. (1976). " Hearing lips and seeing voices.," Nature (London) 0028-0836 10.1038/264746a0 264, 746-748.
17. Petkov, C. I., O'Connor, K. N., and Sutter, M. L. (2007). " Encoding of illusory continuity in primary auditory cortex.," Neuron 0896-6273 10.1016/j.neuron.2007.02.031 54, 153-165.
18. Repp, B. H., Frost, R., and Zsiga, E. (1992). " Lexical mediation between sight and sound in speechreading.," Q. J. Exp. Psychol. 45, 1-20. 1747-0218
19. Romei, V., Murray, M. M., Merabet, L. B., and Thut, G. (2007). " Occipital transcranial magnetic stimulation has opposing effects on visual and auditory stimulus detection: Implications for multisensory interactions.," J. Neurosci. 27, 11465-11472. 0270-6474
20. Saldana, H. M., and Rosenblum, L. D. (1994). " Selective adaptation in speech perception using a compelling audiovisual adaptor.," J. Acoust. Soc. Am. 0001-4966 10.1121/1.409935 95, 3658-3661.
21. Sams, M., Aulanko, R., Hamalainen, M., Hari, R., Lounasmaa, O. V., Lu, S. T., and Simola, J. (1991). " Seeing speech: Visual information from lip movements modifies activity in the human auditory cortex.," Neurosci. Lett. 0304-3940 10.1016/0304-3940(91)90914-F 127, 141-145.
22. Samuel, A. G. (1981a). " Phonemic restoration: Insights from a new methodology.," J. Exp. Psychol. 110, 474-494. 0022-1015
23. Samuel, A. G. (1981b). " The role of bottom-up confirmation in the phonemic restoration illusion.," J. Exp. Psychol. Hum. Percept. Perform. 0096-1523 10.1037//0096-1523.7.5.1124 7, 1124-1131.
24. Samuel, A. G. (1991). " A further examination of attentional effects in the phonemic restoration illusion.," Q. J. Exp. Psychol. 43, 679-699. 1747-0218
25. Samuel, A. G. (1997). " Lexical activation produces potent phonemic percepts.," Cogn. Psychol. 32, 97-127. 0010-0285
26. Samuel, A. G., and Ressler, W. H. (1986). " Attention within auditory word perception: insights from the phonemic restoration illusion.," J. Exp. Psychol. Hum. Percept. Perform. 12, 70-79. 0096-1523
27. Shahin, A. J., Bishop, C. W., and Miller, L. M. (2009). " Neural mechanisms for illusory filling-in of degraded speech.," Neuroimage 44, 1133-1143.
28. Shinn-Cunningham, B. G., and Wang, D. (2008). " Influences of auditory object formation on phonemic restoration.," J. Acoust. Soc. Am. 0001-4966 10.1121/1.2804701 123, 295-301.
29. Sivonen, P., Maess, B., and Friederici, A. D. (2006a). " Semantic retrieval of spoken words with an obliterated initial phoneme in a sentence context.," Neurosci. Lett. 0304-3940 10.1016/j.neulet.2006.09.001 408, 220-225.
30. Sivonen, P., Maess, B., Lattner, S., and Friederici, A. D. (2006b). " Phonemic restoration in a sentence context: Evidence from early and late ERP effects.," Brain Res. 1121, 177-189. 0006-8993
31. Sumby, W. H., and Pollack, I. (1954). " Visual contribution to speech intelligibility in noise.," J. Acoust. Soc. Am. 0001-4966 10.1121/1.1907309 26, 212-215.
32. Trout, J. D., and Poser, W. J. (1990). " Auditory and visual influences on phonemic restoration.," Lang Speech 33, 121-135. 0023-8309
33. van Wassenhove, V., Grant, K. W., and Poeppel, D. (2005). " Visual speech speeds up the neural processing of auditory speech.," Proc. Natl. Acad. Sci. U.S.A. 102, 1181-1186. 0027-8424
34. Vroomen, J., and Baart, M. (2009). " Recalibration of phonetic categories by lipread speech: Measuring aftereffects after a twenty-four hours delay.," Lang. Speech (in press). 0023-8309
35. Wager, T. D., and Nichols, T. E. (2003). " Optimization of experimental design in fMRI: A general framework using a genetic algorithm.," Neuroimage 18, 293-309. 1053-8119
36. Warren, R. M. (1970). " Perceptual restoration of missing speech sounds.," Science 0036-8075 10.1126/science.167.3917.392 167, 392-393.
37. Warren, R. M., Hainsworth, K. R., Brubaker, B. S., Bashford, Jr., J. A., and Healy, E. W. (1997). " Spectral restoration of speech: Intelligibility is increased by inserting noise in spectral gaps.," Percept. Psychophys. 59, 275-283. 0031-5117
38. Yehia, H. C., Rubin, P. E., and Vatikiotis-Bateson, E. (2001). " Quantitative association of vocal-tract and facial behavior.," Speech Commun. 0167-6393 10.1016/S0167-6393(98)00048-X 26, 23-43.