Seeing to hear better: Evidence for early audio-visual interactions in speech identification

Cognition

Volumn 93, Issue 2, 2004, Pages

  Schwartz, Jean Luc ^a   Berthommier, Frédéric ^a   Savariaux, Christophe ^a  

^a UNIV GRENOBLE ALPES   (France)

Author keywords

Audio visual interactions; Lipreading; Speech detection; Speech identification; Temporal cues

Indexed keywords

ACOUSTICS; ARTICLE; AUDITORY STIMULATION; CONTROLLED STUDY; HUMAN; HUMAN EXPERIMENT; LIP READING; NEUROPHYSIOLOGY; NORMAL HUMAN; PRIORITY JOURNAL; SPEECH; VISUAL STIMULATION; ASSOCIATION; SPEECH PERCEPTION; TIME; VISION;

CUES; HUMANS; LIPREADING; SPEECH PERCEPTION; TIME FACTORS; VISUAL PERCEPTION;

EID: 4544333803     PISSN: 00100277     EISSN: None     Source Type: Journal    
DOI: 10.1016/j.cognition.2004.01.006     Document Type: Article

References (24)

1. Barker J., Berthommier F., Schwartz J.L. Is primitive coherence an aid to segment the scene? Proceedings of AVSP'98, Terrigal, Australia. 1998;103-108
2. Bernstein L.E., Takayanagi S., Auer E.T. Jr. Enhanced auditory detection with AV speech: Perceptual evidence for speech and non-speech mechanisms. Proceedings of AVSP'2003, St Jorioz, France. 2003;13-18
3. Berthommier F. A phonetically neutral model of the low-level audiovisual interaction. Proceedings of AVSP'2003, St Jorioz, France. 2003;89-94
4. Bregman A.S. Auditory scene analysis: The perceptual organization of sound. 1990;MIT Press, Cambridge, MA
5. Calvert G.A., Campbell R., Brammer M.J. Evidence from functional magnetic resonance imaging of crossmodal binding in the human heteromodal cortex. Current Biology. 10:2000;649-657
6. Cherry E.C. Some experiments on the recognition of speech, with one and two ears. Journal of the Acoustical Society of America. 25:1953;975-979
7. Colin C., Radeau M., Soquet A., Demolin D., Colin F., Deltenre P. Mismatch negativity evoked by the McGurk-MacDonald effect: A phonetic representation within short-term memory. Clinical Neurophysiology. 113:2002;495-506
8. Driver J. Enhancement of selective listening by illusory mislocation of speech sounds due to lip-reading. Nature. 381:1996;66-68
9. Egan J.P., Greenberg G.Z., Shulman A. Interval of time uncertainty in auditory detection. Journal of the Acoustical Society of America. 33:1961;771-778
10. Erber N.P. Auditory-visual perception of speech. Journal of Speech and Hearing Disorders. 40:1975;481-492
11. Giard M.H., Peronnet F. Auditory-visual integration during multimodal object recognition in humans: A behavioral and electrophysiological study. Journal of Cognitive Neuroscience. 11:1999;473-490
12. Gordon P.C. Coherence masking protection in speech sounds: The role of formant synchrony. Perception and Psychophysics. 59:1997;232-242
13. Grant K.W. The effect of speechreading on masked detection thresholds for filtered speech. Journal of the Acoustical Society of America. 109:2001;2272-2275
14. Grant K.W., Seitz P. The use of visible speech cues for improving auditory detection of spoken sentences. Journal of the Acoustical Society of America. 108:2000;1197-1208
15. Kim J., Davis C. Hearing foreign voices: Does knowing what is said affect masked visual speech detection? Perception. 32:2003;111-120
16. Kim J., Davis C. Testing the cuing hypothesis for the AV speech detection. Proceedings of AVSP'2003, St Jorioz, France. 2003;9-12
17. Lallouache M.T. Un poste 'visage-parole'. Acquisition et traitement de contours labiaux. Proceedings of the XVIII Journées d'Études sur la Parole, Montréal, Canada. 1990;282-286
18. Lebib R., Papo D., de Bode S., Baudonniere P.-M. Evidence of a visual-to-auditory cross-modal sensory gating phenomenon as reflected by the P50 event-related brain potential modulation. Neuroscience Letters. 341:2003;185-188
19. Lisker L., Abramson A.S. A cross-language study of voicing in initial stops: Acoustical measurements. Word. 20:1964;384-422
20. Robert-Ribes J., Schwartz J.L., Lallouache T., Escudier P. Complementarity and synergy in bimodal speech: Auditory, visual and audiovisual identification of French oral vowels in noise. Journal of the Acoustical Society of America. 103:1998;3677-3689
21. Sodoyer D., Schwartz J.-L., Girin L., Klinkisch J., Jutten C. Separation of audio-visual speech sources: A new approach exploiting the audiovisual coherence of speech stimuli. Eurasip Journal on Applied Signal. 11:2002;1165-1173
22. Sumby W.H., Pollack I. Visual contribution to speech intelligibility in noise. Journal of the Acoustical Society of America. 26:1954;212-215
23. Summerfield Q. Use of visual information for phonetic perception. Phonetica. 36:1979;314-331
24. van Wassenhove V., Grant K.W., Poeppel D. Electrophysiology of auditory-visual speech integration. Proceedings of AVSP'2003, St Jorioz, France. 2003;37-42