A visual or tactile signal makes auditory speech detection more efficient by reducing uncertainty

European Journal of Neuroscience

Volumn 39, Issue 8, 2014, Pages 1323-1331

  Tjan, Bosco S ^a   Chao, Ewen ^b   Bernstein, Lynne E ^a,c  

^a UNIVERSITY OF SOUTHERN CALIFORNIA   (United States)

^b ALBANY MEDICAL COLLEGE   (United States)

^c GEORGE WASHINGTON UNIVERSITY   (United States)

Author keywords

Ideal observer analysis; Multisensory enhancement; Speech detection

Indexed keywords

ADULT; ARTICLE; AUDITORY MASKING; AUDITORY STIMULATION; AUDITORY THRESHOLD; CONTROLLED STUDY; FEMALE; HEARING ACUITY; HUMAN; HUMAN EXPERIMENT; MALE; PRIORITY JOURNAL; SENSORY STIMULATION; SOUND DETECTION; SPEECH DISCRIMINATION; TACTILE STIMULATION; TASK PERFORMANCE; VISUAL STIMULATION; BIOLOGICAL MODEL; PHYSIOLOGY; SPEECH PERCEPTION; TOUCH; UNCERTAINTY; VISION;

ADULT; FEMALE; HUMANS; MALE; MODELS, NEUROLOGICAL; SPEECH PERCEPTION; TOUCH; TOUCH PERCEPTION; UNCERTAINTY; VISION, OCULAR; VISUAL PERCEPTION;

EID: 84899071277     PISSN: 0953816X     EISSN: 14609568     Source Type: Journal    
DOI: 10.1111/ejn.12471     Document Type: Article

References (48)

1. ANSI. S3.6- (2004) Specification for audiometers.
2. Auer, E.T. Jr. & Bernstein, L.E. (2007) Enhanced visual speech perception in individuals with early-onset hearing impairment. J. Speech Lang. Hear. R., 50, 1157-1165.
3. Bernstein, L.E., Auer, E.T. Jr. & Takayanagi, S. (2004) Auditory speech detection in noise enhanced by lipreading. Speech Commun., 44, 5-18.
4. Bernstein, L.E., Lu, Z.L. & Jiang, J. (2008) Quantified acoustic-optical speech signal incongruity identifies cortical sites of audiovisual speech processing. Brain Res., 1242, 172-184.
5. Bernstein, L.E., Auer, E.T. Jr., Jiang, J. & Eberhardt, S.P. (2013) Auditory perceptual learning with degraded speech can be enhanced by audiovisual training. Front. Neurosci., 7, 1-16.
6. Chandrasekaran, C., Lemus, L. & Ghazanfar, A.A. (2013) Dynamic faces speed up the onset of auditory cortical spiking responses during vocal detection. Proc. Natl. Acad. Sci. USA, 110, E4668-E4677.
7. Eskelund, K., Tuomainen, J. & Andersen, T.S. (2011) Multistage audiovisual integration of speech: dissociating identification and detection. Exp. Brain Res., 208, 447-457.
8. Ghazanfar, A.A., Maier, J.X., Hoffman, K.L. & Logothetis, N.K. (2005) Multisensory integration of dynamic faces and voices in rhesus monkey auditory cortex. J. Neurosci., 25, 5004-5012.
9. Gillmeister, H. & Eimer, M. (2007) Tactile enhancement of auditory detection and perceived loudness. Brain Res., 1160, 58-68.
10. Gold, J., Bennett, P.J. & Sekuler, A.B. (1999) Signal but not noise changes with perceptual learning. Nature, 402, 176-178.
11. Graham, N.V.S. (1989) Visual Pattern Analyzers, vol 16. Oxford University, Oxford, UK.
12. Grant, K.W. (2001) The effect of speechreading on masked detection thresholds for filtered speech. J. Acoust. Soc. Am., 109, 2272-2275.
13. Grant, K.W. & Seitz, P.F. (2000) The use of visible speech cues for improving auditory detection of spoken sentences. J. Acoust. Soc. Am., 108, 1197-1208.
14. Green, D.M. & Swets, J.A. (1966) Signal Detection Theory and Psychophysics. Wiley & Sons, New York.
15. Harper, D.W. (1979) Signal detection analysis of effect of white noise intensity on sensitivity to visual flicker. Percept. Motor Skill., 48, 791-798.
16. Hochstein, S. & Ahissar, M. (2002) View from the top: hierarchies and reverse hierarchies in the visual system. Neuron, 36, 791-804.
17. Jiang, J. & Bernstein, L.E. (2011) Psychophysics of the McGurk and other audiovisual speech integration effects. J. Exp. Psychol. Human, 37, 1193-1209.
18. Kayser, C. & Logothetis, N.K. (2007) Do early sensory cortices integrate cross-modal information? Brain Struct. Funct., 212, 121-132.
19. Kayser, C., Petkov, C.I. & Logothetis, N.K. (2008) Visual modulation of neurons in auditory cortex. Cereb. Cortex, 18, 1560-1574.
20. Kim, J. & Davis, C. (2004) Investigating the audio-visual speech detection advantage. Speech Commun., 44, 19-30.
21. Lakatos, P., Chen, C.M., O'connell, M.N., Mills, A. & Schroeder, C.E. (2007) Neuronal oscillations and multisensory interaction in primary auditory cortex. Neuron, 53, 279-292.
22. Legge, G.E., Kersten, D. & Burgess, A.E. (1987) Contrast discrimination in noise. J. Opt. Soc. Am. A, 4, 391-404.
23. Levitt, H. (1971) Transformed up-down methods in psychoacoustics. J. Acoust. Soc. Am., 49, 467-477.
24. Lu, Z.-L. & Dosher, B.A. (1998) External noise distinguishes attention mechanisms. Vision Res., 38, 1183-1198.
25. Lu, Z.-L. & Dosher, B.A. (2008) Characterizing observers using external noise and observer models: assessing internal representations with external noise. Psychol. Rev., 115, 44-82.
26. Lugo, J.E., Doti, R., Wittich, W. & Faubert, J. (2008) Multisensory integration: central processing modifies peripheral systems. Psychol. Sci., 19, 989-997.
27. Lugo, J.E., Doti, R. & Faubert, J. (2012) Effective tactile noise facilitates visual perception. Seeing and Perceiving, 25, 29-44.
28. Megevand, P., Molholm, S., Nayak, A. & Foxe, J.J. (2013) Recalibration of the multisensory temporal window of integration results from changing task demands. PLoS ONE, 8, e71608.
29. Meyer, G.F., Mulligan, J. & Wuerger, S.M. (2004) Continuous audio-visual digit recognition using N-best decision fusion. Inform. Fusion, 5, 91-101.
30. Nahum, M., Nelken, I. & Ahissar, M. (2008) Low-level information and high-level perception: the case of speech in noise. PLoS Biol., 6, 978-991.
31. Nilsson, M., Soli, S.D. & Sullivan, J.A. (1994) Development of the Hearing in Noise Test for the measurement of speech reception thresholds in quiet and in noise. J. Acoust. Soc. Am., 95, 1085-1099.
32. Odgaard, E.C., Arieh, Y. & Marks, L.E. (2004) Brighter noise: sensory enhancement of perceived loudness by concurrent visual stimulation. Cogn. Affect. Behav. Ne., 4, 127-132.
33. Pelli, D.G. (1985) Uncertainty explains many aspects of visual contrast detection and discrimination. J. Opt. Soc. Am. A, 2, 1508-1532.
34. Pelli, D.G. & Farell, B. (1999) Why use noise? J. Opt. Soc. Am. A, 16, 647-653.
35. Power, A.J., Lalor, E.C. & Reilly, R.B. (2011) Endogenous auditory spatial attention modulates obligatory sensory activity in auditory cortex. Cereb. Cortex, 21, 1223-1230.
36. Raij, T., Ahveninen, J., Lin, F.-H., Witzel, T., Jaaskelainen, I.P., Letham, B., Israeli, E., Sahyoun, C., Vasios, C., Stufflebeam, S., Hamalainen, M. & Belliveau, J.W. (2010) Onset timing of cross-sensory activations and multisensory interactions in auditory and visual sensory cortices. Eur. J. Neurosci., 31, 1772-1782.
37. Ro, T., Hsu, J., Yasar, N.E., Elmore, L.C. & Beauchamp, M.S. (2009) Sound enhances touch perception. Exp. Brain Res., 195, 135-143.
38. Schroeder, C.E. & Foxe, J.J. (2002) The timing and laminar profile of converging inputs to multisensory areas of the macaque neocortex. Brain Res. Cogn. Brain Res., 14, 187-198.
39. Schroeder, C.E., Lakatos, P., Kajikawa, Y., Partan, S. & Puce, A. (2008) Neuronal oscillations and visual amplification of speech. Trends Cogn. Sci., 12, 106-113.
40. Schurmann, M., Caetano, G., Jousmaki, V. & Hari, R. (2004) Hands help hearing: facilitatory audiotactile interaction at low sound-intensity levels. J. Acoust. Soc. Am., 115, 830-832.
41. Schwartz, J.L., Berthommier, F. & Savariaux, C. (2004) Seeing to hear better: evidence for early audio-visual interactions in speech identification. Cognition, 93, B69-B78.
42. Stein, B.E., London, N., Wilkinson, L.K. & Price, D.D. (1996) Enhancement of perceived visual intensity by auditory stimuli: a psychophysical analysis. J. Cognitive Neurosci., 8, 497-506.
43. Stein, B.E., Burr, D., Constantinidis, C., Laurienti, P.J., Meredith, A.M., Perrault, T.J. Jr., Ramachandran, R., Roder, B., Rowland, B.A., Sathian, K., Schroeder, C.E., Shams, L., Stanford, T.R., Wallace, M.T., Yu, L. & Lewkowicz, D.J. (2010) Semantic confusion regarding the development of multisensory integration: a practical solution. Eur. J. Neurosci., 31, 1713-1720.
44. Sun, G.J., Chung, S.T. & Tjan, B.S. (2010) Ideal observer analysis of crowding and the reduction of crowding through learning. J. Vision, 10, 16.
45. Tanner, W.P. Jr. (1961) Physiological implication of psychophysical data. Ann. NY Acad. Sci., 89, 752-765.
46. Tjan, B.S., Braje, W.L., Legge, G.E. & Kersten, D. (1995) Human efficiency for recognizing 3-D objects in luminance noise. Vision Res., 35, 3053-3069.
47. Tjan, B.S., Lestou, V. & Kourtzi, Z. (2006) Uncertainty and invariance in the human visual cortex. J. Neurophysiol., 96, 1556-1568.
48. Zeng, F.G., Fu, Q.J. & Morse, R. (2000) Human hearing enhanced by noise. Brain Res., 869, 251-255.