Auditory speech detection in noise enhanced by lipreading

Speech Communication

Volumn 44, Issue 1-4 SPEC. ISS., 2004, Pages 5-18

  Bernstein, Lynne E ^a,b   Auer Jr , Edward T ^a   Takayanagi, Sumiko ^a  

^a HOUSE EAR INSTITUTE   (United States)

^b NATIONAL SCIENCE FOUNDATION   (United States)

Author keywords

Audiovisual speech perception; Audiovisual speech processing; Lipreading; Speech detection in noise; Speech in noise; Speech processing; Speechreading

Indexed keywords

AUDIOVISUAL SPEECH PROCESSING; LIPREADING; SPEECH DETECTION; SPEECHREADING;

ACOUSTIC NOISE; CORRELATION METHODS; SENSORY PERCEPTION; SIGNAL TO NOISE RATIO; VIDEO RECORDING; WHITE NOISE;

SPEECH RECOGNITION;

EID: 10444276578     PISSN: 01676393     EISSN: None     Source Type: Journal    
DOI: 10.1016/j.specom.2004.10.011     Document Type: Article

References (32)

1. American National Standards Institute, 1989. Specification for audiometers (ANSI S3.6-1989), New York: Author.
2. Arnold, P., Hill, F., 2001. Bisensory augmentation: a speech-reading advantage when speech is clearly audible and intact. Br. J. Psychol. 92, 339-355.
3. Bernstein, L.E., Auer Jr., E.T., Moore, J.K., 2004. Audiovisual speech binding: convergence or association?. In: Calvert, C., Spence, C., Stein, B.E. (Eds.), Handbook of Multisensory Processing. MIT, Cambridge, MA.
4. Bernstein, L.E., Demorest, M.E., Tucker, P.E., 2000. Speech perception without hearing. Percept. Psychophys. 62, 233-252.
5. Colin, C., Radeau, M., Soquet, A., Dachy, B., Deltenre, P., 2002, Electrophysiology of spatial scene analysis: the mismatch negativity (MMN) is sensitive to the ventriloquism illusion. Clin. Neurophysiol. 113, 507-518.
6. De Gelder, B., Bertelson, P., 2003. Multisensory integration, perception and ecological validity. Trends Cogn. Sci. 7, 460-467.
7. Foxe, J.J., Simpson, G.V., 2002. Flow of activation from V1 to frontal cortex in humans. A framework for defining "early" visual processing. Exp. Brain Res. 142, 139-150.
8. Grant, K.W., 2001. The effect of speechreading on masked detection thresholds for filtered speech. J. Acoust. Soc. Amer. 109, 2272-2275.
9. Grant, K.W., Seitz, P.F., 2000. The use of visible speech cues for improving auditory detection of spoken sentences. J. Acoust. Soc. Amer. 108, 1197-1208.
10. Krolak-Salmon, P., Henaff, M.A., Tallon-Baudry, C., Yvert, B., Fischer, C., Vighetto, A., Betrand, O., Mauguiere, F., 2001. How fast can the human lateral geniculate nucleus and visual striate cortex see?. Soc. Neurosci. Abstracts 27, 913.
11. Levitt, H., 1971. Transformed up-down method in psychoacoustics. J. Acoust. Soc. Amer. 49, 467-477.
12. McGurk, H., MacDonald, J., 1976. Hearing lips and seeing voices. Nature 264, 746-748.
13. Meredith, M.A., 2002. On the neuronal basis for multisensory convergence: a brief overview. Cognit. Brain Res. 14, 31-40.
14. Meredith, M.A., Nemitz, J.W., Stein, B.E., 1987. Determinants of multisensory integration in superior colliculus neurons. I. Temporal factors. J. Neurosci. 7, 3215-3229.
15. Mesulam, M.M., 1998. From sensation to cognition. Brain 121 (Pt 6), 1013-1052.
16. Moutoussis, K., Zeki, S., 1997. A direct demonstration of perceptual asynchrony in vision. Proc. Roy. Soc. Lond. B, Biol. Sci. 264, 393-399.
17. Naatanen, R., 2001. The perception of speech sounds by the human brain as reflected by the mismatch negativity (MMN) and its magnetic equivalent (MMNm). Psychophysiology 38, 1-21.
18. Nilsson, M., Soli, S.D., Sullivan, J.A., 1994. Development of the hearing in noise test for the measurement of speech reception thresholds in quiet and in noise. J. Acoust. Soc. Amer. 95.
19. Puce, A., Perrett, D., 2003. Electrophysiology and brain imaging of biological motion. Philos. Trans. Roy. Soc. Lond. B 358, 435-445.
20. Ratcliff, R., 1993. Methods for dealing with reaction time outliers. Psychol. Bull. 114, 510-532.
21. Reisberg, D., McLean, J., Goldfield, A., 1987. Easy to hear but hard to understand: a lip-reading advantage with intact auditory stimuli. In: Dodd, B., Campbell, R. (Eds.), Hearing by Eye: The Psychology of Lip-Reading. Lawrence Erlbaum, London, pp. 97-113.
22. Schroeder, C.E., Foxe, J.J., 2002. The timing and laminar profile of converging inputs to multisensory areas of the macaque neocortex. Cognit. Brain Res. 14, 187-198.
23. Schwartz, J.-L., Berthommier, F., Savariaux, C., 2002. Audiovisual scene analysis: evidence for a "very-early" integration process in audio-visual speech perception. In: Proc. 7th Internat. Conf. on Spoken Language Processing, Denver, CO.
24. Schwartz, J.-L., Berthommier, F., Savariaux, C., 2003. Auditory syllabic identification enhanced by non-informative visible speech. In: Proc. 7th AVSP (Audiovisual Speech Processing) 2003, St. Jorioz, France.
25. Stein, B.E., Meredith, M.A., 1993. The Merging of the Senses. MIT, Cambridge, MA.
26. Steinschneider, M., Volkov, I.O., Noh, M.D., Garell, P.C., Howard III, M.A., 1999. Temporal encoding of the voice onset time phonetic parameter by field potentials recorded directly from human auditory cortex. J. Neurophysiol. 82, 2346-2357.
27. Stevens, K.N., 1998. Acoustic Phonetics. MIT Press, Cambridge, MA.
28. Sumby, W.H., Pollack, I., 1954. Visual contribution to speech intelligibility in noise. J. Acoust. Soc. Amer. 26, 212-215.
29. Treisman, A., 1996. The binding problem. Curr. Opin. Neurobiol. 6, 171-178.
30. von der Malsburg, C., 1995. Binding in models of perception and brain function. Curr. Opin. Neurobiol. 5, 520-526.
31. Yvert, B., Crouzeix, A., Bertrand, O., Seither-Preisler, A., Pantev, C., 2001. Multiple supratemporal sources of magnetic and electric auditory evoked middle latency components in humans. Cereb. Cortex 11, 411-423.
32. Zeki, S., 1998. Parallel processing, asynchronous perception, and a distributed system of consciousness in vision. The Neuroscientist 4, 365-372.