Detection of auditory (cross-spectral) and auditory-visual (cross-modal) synchrony

Speech Communication

Volumn 44, Issue 1-4 SPEC. ISS., 2004, Pages 43-53

  Grant, Ken W ^a   Van Wassenhove, Virginie ^b   Poeppel, David ^b  

^a WALTER REED ARMY MEDICAL CENTER   (United States)

^b UNIVERSITY OF MARYLAND   (United States)

Author keywords

Auditory visual speech processing; Cross modal asynchrony; Spectro temporal asynchrony

Indexed keywords

ACOUSTIC SPEECH SIGNALS; CROSS-MODAL ASYNCHRONY; SPECTRO-TEMPORAL ASYNCHRONY;

ACOUSTICS; BANDWIDTH; REVERBERATION; SIGNAL PROCESSING; SPEECH PROCESSING; SPEECH RECOGNITION;

SPEECH COMMUNICATION;

EID: 10444249633     PISSN: 01676393     EISSN: None     Source Type: Journal    
DOI: 10.1016/j.specom.2004.06.004     Document Type: Article

References (29)

1. Abry, C., Lallouache, M.-T., Caithiard, M.-A., 1996. How can coarticulation models account for speech sensitivity to audio-visual desynchronization? In: D. Stork and M. Hennecke (Eds.), Speechreading by Humans ands Machines, NATO ASI Series F: Computer and Systems Sciences, vol. 150, pp. 247-255.
2. ANSI, 1969. ANSI S3.5-1969, American National Standard Methods for the Calculation of the Articulation Index, American National Standards Institute, New York.
3. ANSI, 1989. ANSI S3.6-1989, American national standard specification for audiometers, American National Standards Institute, New York.
4. Dixon, N., Spitz, L., 1980. The detection of audiovisual desynchrony. Perception 9, 719-721.
5. Doherty, K.A., Turner, C.W., 1996. Use of a correlational method to estimate a listener's weighting function for speech. J. Acoust. Soc. Am. 100, 3769-3773.
6. Grant, K.W., 2001. The effect of speechreading on masked detection thresholds for filtered speech. J. Acoust. Soc. Amer. 109, 2272-2275.
7. Grant, K.W., Braida, L.D., 1991. Evaluating the articulation index for audiovisual input. J. Acoust. Soc. Amer. 89, 2952-2960.
8. Grant, K.W., Greenberg, S., 2001. Speech intelligibility derived from asynchronous processing of auditory-visual information. In: Proc. Auditory-Visual Speech Processing (AVSP 2001), Scheelsminde, Denmark, September 7-9.
9. Grant, K.W., Seitz, P.P., Measures of auditory-visual integration in nonsense syllables and sentences, J., 1998. Acoust. Soc. Amer. 104, 2438-2450.
10. Grant, K.W., Seitz, P.F., 2000. The use of visible speech cues for improving auditory detection of spoken sentences. J. Acoust. Soc. Amer. 108, 1197-1208.
11. Grant, K.W., Walden, B.E., 1996. The spectral distribution of prosodic information. J. Speech Hear. Res. 39, 228-238.
12. Grant, K.W., Walden, B.E., Seitz, P.P., 1998. Auditory-visual speech recognition by hearing-impaired subjects: Consonant recognition, sentence recognition, and auditory-visual integration. J. Acoust. Soc. Amer. 103, 2677-2690.
13. Greenberg, S., 2003. Pronunciation variation is key to understanding spoken language. In: Proceedings of the International Phonetics Congress, Barcelona, Spain, August, pp. 219-222.
14. Greenberg, S., Arai, T. 2001. The relation between speech intelligibility and the complex modulation spectrum. In: Proc. 7th European Conference on Speech Communication and Technology (Eurospeech-2001), Aalborg, Denmark, September, pp. 473-476.
15. IEEE - Institute of Electrical and Electronic Engineers, 1969. IEEE recommended practice for speech quality measures. IEEE, New York.
16. Massaro, D.W., Cohen, M.M., Smeele, P.M., 1996. Perception of asynchronous and conflicting visual and auditory speech. J. Acoust, Soc. Amer. 100, 1777-1786.
17. McGrath, M., Summerfield, Q., 1985. Intermodal timing relations and audio-visual speech recognition by normal-hearing adults. J. Acoust. Soc. Amer. 77, 678-685.
18. McGurk, H., McDonald, J., 1976. Hearing lips and seeing voices. Nature 264, 746-747.
19. Munhall, K.G., Gribble, P., Sacco, L., Ward, M., 1996. Temporal constraints on the McGurk effect. Perception Psychophys. 58, 351-362.
20. Munhall, K.G., Tohkura, Y., 1998. Audiovisual gating and the time course of speech perception. J. Acoust. Soc. Amer. 104, 530-539.
21. Pandey, P.C., Kunov, H., Abel, S.M., 1986. Disruptive effects of auditory signal delay on speech perception with lipreading. J. Aud. Res. 26, 27-41.
22. Seitz, P.P., Grant, K.W., 1999. Modality, perceptual encoding speed, and time-course of phonetic information. In: AVSF99 Proceedings, Aug, 7-9, 1999, Santa Cruz, CA.
23. Silipo, R., Greenberg, S., Arai, T., 1999. Temporal constraints on speech intelligibility as deduced from exceedingly sparse spectral representations. In Proceedings of Eurospech 1999 Budepest, pp. 2687-2690.
24. Stevens, K.N., Blumstein, S.E., 1978. Invariant cues for place of articulation in stop consonants. J. Acoust. Soc. Amer. 64, 1358-1368.
25. Stone, M.A., Moore, B.C.J., 2003. Tolerable hearing aid delays. III. Effects of speech production and perception of acrossfrequency variation in delay. Ear Hear. 24, 175-183.
26. Sumby, W.H., Pollack, I., 1954. Visual contribution to speech intelligibility in noise. J. Acoust. Soc. Am. 26, 212-215.
27. Turner, C.W., Kwon, B.J., Tanaka, C., Knapp, J., Hubbartt, K.A., Doherty, K.A., 1998. Frequency-weighting functions for broadband speech as estimated by a correlational method. J. Acoust. Soc. Amer. 104, 1580-1586.
28. van Wassenhove, V., Grant, K.W., Poeppel, D., 2001. Timing of auditory-visual integration in the McGurk effect. Presented at the Society of Neuroscience Annual Meeting, San Diego, CA, November, 488.
29. Zue, V., Seneff, S., Glass, J., 1990. Speech database development at MIT: TIMIT and beyond. Speech Comm. 9, 351-356.