Auditory-visual speech perception and synchrony detection for speech and nonspeech signals

Journal of the Acoustical Society of America

Volumn 119, Issue 6, 2006, Pages 4065-4073

  Conrey, Brianna ^a   Pisoni, David B ^a  

^a INDIANA UNIVERSITY   (United States)

Author keywords

[No Author keywords available]

Indexed keywords

SIGNAL PROCESSING; SPEECH PROCESSING; SPEECH SYNTHESIS; SYNCHRONIZATION;

AUDITORY-VISUAL (AV) ASYNCHRONIES; MULTISENSORY SPEECH PERCEPTION; SPEECH SIGNALS; TEMPORAL PROCESSING;

SENSORY PERCEPTION;

ADULT; ARTICLE; AUDITORY STIMULATION; AUDITORY SYSTEM; FEMALE; HEARING; HUMAN; HUMAN EXPERIMENT; MALE; MEDICAL RESEARCH; NORMAL HUMAN; PRIORITY JOURNAL; SIGNAL TRANSDUCTION; SPEECH; SPEECH PERCEPTION; VISION; VISUAL STIMULATION;

ACOUSTIC STIMULATION; ADOLESCENT; ADULT; AUDITORY CORTEX; FEMALE; HUMANS; MALE; PHOTIC STIMULATION; PSYCHOACOUSTICS; SPEECH PERCEPTION; TIME FACTORS; VISUAL PERCEPTION;

EID: 33745102745     PISSN: 00014966     EISSN: None     Source Type: Journal    
DOI: 10.1121/1.2195091     Document Type: Article

References (27)

1. Boothroyd, A., Hanin, L., and Hnath, T. (1985). " A sentence test of speech perception: Reliability, set equivalence, and short term learning., " Speech and Hearing Science Report No. RC110, City University of New York.
2. Bushara, K. O., Grafman, J., and Hallett, M. (2001). " Neural correlates of auditory-visual stimulus onset asynchrony detection., " J. Neurosci. 21 (1), 300-304.
3. Cohen, J. D., MacWhinney, B., Flatt, M., and Provost, J. (1993). " PsyScope: A new graphic interactive environment for designing psychology experiments., " Behav. Res. Methods Instrum. Comput. 25 (2), 257-271.
4. Conrey, B., and Pisoni, D. B. (2003). " Audiovisual asynchrony detection for speech and nonspeech signals., " paper presented at the ISCA 2003 International Conference on Auditory-Visual Speech Processing.
5. Dixon, N., and Spitz, L. (1980). " The detection of audiovisual desynchrony., " Perception 9, 719-721.
6. Fujisaki, W., Shinsuke, S., Kashino, M., and Nishida, S. (2004). " Recalibration of audiovisual simultaneity., " Nat. Neurosci. 7, 773-778.
7. Grant, K. W., and Greenberg, S. (2001). " Speech intelligibility derived from asynchronous processing of auditory-visual information., " paper presented at the ISCA International Conference on Auditory-Visual Speech Processing.
8. Grant, K. W., and Seitz, P. F. (1998). " Measures of auditory-visual integration in nonsense syllables and sentences., " J. Acoust. Soc. Am. 104, 2438-2450.
9. Grant, K. W., van Wassenhove, V., and Poeppel, D. (2003). " Discrimination of auditory-visual synchrony., " paper presented at the ISCA 2003 International Conference on Auditory-Visual Speech Processing.
10. King, A. J., and Palmer, A. R. (1985). " Integration of visual and auditory information in bimodal neurones in the guinea-pig superior colliculus., " Exp. Brain Res. 60, 492-500.
11. Lachs, L. (1999). " Use of partial stimulus information in spoken word recognition without auditory stimulation., " in Research on Spoken Language Processing Progress Report No. 23, Speech Research Laboratory, Indiana University, Bloomington, IN, pp. 82-114.
12. Lachs, L., and Hernandez, L. R. (1998). " Update: The Hoosier Audiovisual Multitalker Database., " in Research on Spoken Language Processing Progress Report No. 22, Speech Research Laboratory, Indiana University, Bloomington, IN, pp. 377-388.
13. Lachs, L., and Pisoni, D. B. (2004a). " Crossmodal source identification in speech perception., " Ecological Psychol. 16, 159-187.
14. Lachs, L., and Pisoni, D. B. (2004b). " Crossmodal source information and spoken word recognition., " J. Exp. Psychol. Hum. Percept. Perform. 30 (2), 378-396.
15. Lewkowicz, D. J. (1996). " Perception of auditory-visual temporal synchrony in human infants., " J. Exp. Psychol. Hum. Percept. Perform. 22, 1094-1106.
16. Massaro, D., and Cohen, M. (1993). " Perceiving asynchronous bimodal speech in consonant-vowel and vowel syllables., " Speech Commun. 13, 127-134.
17. Massaro, D., Cohen, M. M., and Smeele, P. M. T. (1996). " Perception of asynchronous and conflicting visual and auditory speech., " J. Acoust. Soc. Am. 100 (3), 1777-1786.
18. McGrath, M., and Summerfield, Q. (1985). " Intermodal timing relations and audio-visual speech recognition by normal-hearing adults., " J. Acoust. Soc. Am. 77 (2), 678-684.
19. Meredith, M. A. (2002). " On the neuronal basis for multisensory convergence: A brief overview., " Cognit. Brain Res. 14, 31-40.
20. Meredith, M. A., Nemitz, J. W., and Stein, B. E. (1987). " Determinants of multisensory integration in superior colliculus neurons. I. Temporal factors., " J. Neurosci. 7 (10), 3215-3229.
21. Munhall, K. G., Gribble, P., Sacco, L., and Ward, M. (1996). " Temporal constraints on the McGurk effect., " Percept. Psychophys. 58 (3), 351-362.
22. Pandey, C. P., Kunov, H., and Abel, M. S. (1986). " Disruptive effects of auditory signal delay on speech perception with lip-reading., " J. Aud. Res. 26, 27-41.
23. Saberi, K., and Perrott, D. R. (1999). " Cognitive restoration of reversed speech., " Nature (London) 398, 760.
24. Sheffert, S. M., Lachs, L., and Hernandez, L. R. (1996). " The Hoosier Audiovisual Multitalker Database., " in Research on Spoken Language Processing Progress Report No. 21, Speech Research Laboratory, Indiana University, Bloomington, IN, pp. 578-583.
25. Stein, B., and Meredith, M. A. (1993). The Merging of the Senses (MIT, Cambridge, MA).
26. Stone, J. V., Hunkin, N. M., Porrill, J., Wood, R., Keeler, V., Beanland, M., (2001). " When is now? Perception of simultaneity., " Proc. R. Soc. London, Ser. B 268, 31-38.
27. van Wassenhove, V., Grant, K. W., and Poeppel, D. (2002). " Temporal integration in the McGurk effect., " poster presented at the annual meeting of the Society for Cognitive Neuroscience, San Francisco.