Effects of audio-visual integration on the detection of masked speech and non-speech sounds

Brain and Cognition

Volumn 75, Issue 1, 2011, Pages 60-66

  Eramudugolla, Ranmalee ^a   Henderson, Rachel ^a   Mattingley, Jason B ^a  

^a UNIVERSITY OF QUEENSLAND   (Australia)

Author keywords

Audio visual; Auditory perception; Lip reading; Multisensory integration; Speech; Visual perception

Indexed keywords

ADULT; ARTICLE; AUDITORY DISCRIMINATION; AUDITORY MASKING; AUDITORY STIMULATION; AUDITORY THRESHOLD; CONTROLLED STUDY; FEMALE; HUMAN; HUMAN EXPERIMENT; LIP READING; MALE; MENTAL TASK; NORMAL HUMAN; PRIORITY JOURNAL; SIGNAL DETECTION; SIGNAL NOISE RATIO; SOUND DETECTION; SPEECH ANALYSIS; SPEECH AND LANGUAGE ASSESSMENT; SPEECH PERCEPTION; TASK PERFORMANCE; VISUAL DISCRIMINATION; VISUAL INFORMATION; VISUAL MASKING; VISUAL STIMULATION;

ACOUSTIC STIMULATION; ADULT; FEMALE; HUMANS; LIPREADING; LOUDNESS PERCEPTION; MALE; MIDDLE AGED; NEUROPSYCHOLOGICAL TESTS; NOISE; PATTERN RECOGNITION, VISUAL; PERCEPTUAL MASKING; SPEECH; SPEECH PERCEPTION; VERBAL BEHAVIOR;

EID: 78649449584     PISSN: 02782626     EISSN: 10902147     Source Type: Journal    
DOI: 10.1016/j.bandc.2010.09.005     Document Type: Article

References (34)

1. Bernstein L.E., Auer E.T., Moore J.K., Ponton C.W., Don M., Singh M. Visual speech perception without primary auditory cortex activation. Neuroreport 2002, 13:311-315.
2. Bernstein L.E., Auer E.T., Takayanagi S. Auditory speech detection in noise enhanced by lipreading. Speech and Communication 2004, 44:5-18.
3. Besle, J., Bertrand, O., Giard, M. -H. (2009). Electrophysiological (EEG, sEEG, MEG) evidence for multiple audiovisual interactions in the human auditory cortex. Hearing Research doi:10.1016/j.heares.2009.06.016.
4. Besle J., Fort A., Delpuech C., Giard M.-H. Bimodal speech: Early suppressive visual effects in human auditory cortex. European Journal of Neuroscience 2004, 20:2225-2234.
5. Calvert G.A., Bullmore E.T., Brammer M.J., Campbell R., Williams S.C., McGuire P.K. Activation of auditory cortex during silent lipreading. Science 1997, 276:593-596.
6. Calvert G.A., Campbell R. Reading speech from still and moving faces: The neural substrates of visible speech. Journal of Cognitive Neuroscience 2003, 15:57-70.
7. Campanella S., Belin P. Integrating face and voice in person perception. Trends in Cognitive Sciences 2007, 11:535-543.
8. Fagel, S., 2005. Auditory speech illusion evoked by moving lips. In: Proceedings of the audio-visual speech processing conference. British Columbia, Canada.
9. Ghazanfar A.A., Maier J.X., Hoffman K.L., Logothetis N.K. Multisensory integration of dynamic faces and voices in rhesus monkey auditory cortex. The Journal of Neuroscience 2005, 25:5004-5012.
10. Giard M.H., Peronnet F. Auditory-visual integration during multimodal object recognition in humans: a behavioural and electrophysiological study. Journal of Cognitive Neuroscience 1999, 11:473-490.
11. Grant K.W., Seitz P.-F. The use of visible cues for improving auditory detection of spoken sentences. Journal of the Acoustical Society of America 2000, 108:1197-1208.
12. Jones J.A., Jarick M. Multisensory integration of speech signals: The relationship between space and time. Experimental Brain Research 2006, 174:588-594.
13. Kim J., Davis C. Hearing foreign voices: Does knowing what is said affect audiovisual masked-speech detection?. Speech Communication 2003, 44:19-30.
14. Kim J., Davis C. Investigating the audio-visual speech detection advantage. Speech Communication 2004, 44:19-30.
15. Kim J., Davis C. Restoration effects in auditory and visual speech. Proceedings of the audio-visual speech processing conference 2007, Hilvarenbeek, The Netherlands. J. Vroomen, M. Swerts, E. Krahmer (Eds.).
16. Klucharev V., Mottonen R., Sams M. Electrophysiological indicators of phonetic and non-phonetic multisensory interactions during audiovisual speech perception. Brain Research: Cognitive Brain Research 2008, 18:65-75.
17. Levitt H. Transformed up-down methods in psychoacoustics. Journal of the Acoustical Society of America 1971, 49:467-477.
18. Macaluso E., Driver J. Mutlisensory spatial interactions: A window onto functional integration in the human brain. Trends in Neurosciences 2005, 28:264-271.
19. Miller J. Channel interaction and the redundant-targets effect in bimodal divided attention. Journal of Experimental Psychology: Human Perception and Performance 1991, 17:160-169.
20. Molholm S., Ritter W., Murray M.M., Javitt D.C., Schroeder C.E., Foxe J.J. Multisensory auditory-visual interactions during early sensory processing in humans: A high-density electrial mapping study. Brain Research Cognition, Brain Research 2002, 13:417-425.
21. Pekkola J., Ojanen V., Autti T., Jaaskelainen I.P., Mottonen R., Tarkainen A., et al. Primary auditory cortex activation by visual speech: An fMRI study at 3T. Neuroreport 2005, 16:125-128.
22. Repp B.H., Frost R., Zsiga E. Lexical mediation between sight and sound in speech reading. The Quarterly Journal of Experimental Psychology 1992, 45A.
23. Ross L.A., Saint-Amour D., Leavitt V.M., Javitt D.C., Foxe J.J. Do you see what I am saying? Exploring visual enhancement of speech comprehension in noisy environments. Cerebral Cortex 2007, 17:1147-1153.
24. Saldana H.M., Rosenblum L.D. Visual influences on auditory pluck and bow judgments. Perception and Psychophysics 1993, 54:406-416.
25. Stein B.E., Stanford T.R. Multisensory integration: Current issues from the perspective of the single neuron. Nature Reviews Neuroscience 2008, 9:255-266.
26. Stein B.E., Stanford T.R. Multisensory integration: Current issues from the perspective of the single neuron. Nature Reviews, Neuroscience 2008, 9:255-267.
27. Stekelenberg J.J., Vroomen J. Neural correlates of multisensory integration of ecologically valid audiovisual events. Journal of Cognitive Neuroscience 2007, 19:1964-1973.
28. Sumby W.H., Pollack I. Visual contribution to speech intelligibility in noise. Journal of the Acoustical Society of America. 1954, 26:212-215.
29. Summerfield Q. Lipreading and audiovisual speech perception. Philosophical Transactions of the royal Society of London, B. 1992, 335.
30. Tuomainen J., Andersen T.S., Tiippana K., Sams M. Audio-visual speech perception is special. Cognition 2005, 96:B13-22.
31. van Wassenhove V., Grant K.W., Poeppel D. Visual speech speeds up the neural processing of auditory speech. Proceedings of the National Academy of Sciences USA 2005, 102:1181-1186.
32. Vatakis A., Spence C. Evaluating the unity 'assumption' using audiovisual speech stimuli. Perception and Psychophysics 2007, 69:744-756.
33. Vatakis A., Spence C. Evaluating the influence of the 'unity assumption' on the temporal perception of realistic audiovisual stimuli. Acta Psychologica 2008, 127:12-23.
34. Zampini M., Shore D.I., Spence C. Audiovisual temporal order judgment. Experimental Brain Research 2003, 152:198-210.