Synchronous sounds enhance visual sensitivity without reducing target uncertainty

Seeing and Perceiving

Volumn 24, Issue 6, 2011, Pages 623-638

  Chen, Yi Chuan ^a   Huang, Pi Chun ^b   Yeh, Su Ling ^c   Spence, Charles ^a  

^a UNIVERSITY OF OXFORD   (United Kingdom)

^b MCGILL UNIVERSITY   (Canada)

^c NATIONAL TAIWAN UNIVERSITY   (Taiwan)

Author keywords

AUDIOVISUAL; MULTISENSORY FACILITATION; SIGNAL ENHANCEMENT; TEMPORAL UNCERTAINTY; TRANSIENT

Indexed keywords

AUDIOVISUAL; CONTRAST THRESHOLD; CROSS-MODAL; INTERNAL NOISE; MULTISENSORY; NOISE LEVELS; ONSET-TIME; PSYCHOMETRIC FUNCTION; SIGNAL ENHANCEMENT; TEMPORAL UNCERTAINTY; VISUAL DETECTION; VISUAL SENSITIVITY; VISUAL SYSTEMS; VISUAL TARGETS;

TRANSIENTS; UNCERTAINTY ANALYSIS; WHITE NOISE;

ACOUSTIC NOISE;

ARTICLE; AUDITORY STIMULATION; BIOLOGICAL MODEL; HEARING; HUMAN; METHODOLOGY; PERCEPTION; PHOTOSTIMULATION; PHYSIOLOGY; PSYCHOMETRY; UNCERTAINTY; VISION;

ACOUSTIC STIMULATION; AUDITORY PERCEPTION; HUMANS; MODELS, NEUROLOGICAL; PERCEPTUAL MASKING; PHOTIC STIMULATION; PSYCHOMETRICS; UNCERTAINTY; VISUAL PERCEPTION;

EID: 84858196559     PISSN: 18784755     EISSN: 18784763     Source Type: Journal    
DOI: 10.1163/187847611X603765     Document Type: Article

References (47)

1. Anderson, T. S. and Mamassian, P. (2008). Audiovisual integration of stimulus transients, Vision Res. 48, 2537-2544.
2. Arieh, Y. and Marks, L. E. (2008). Cross-modal interaction between vision and hearing: a speedaccuracy analysis, Percept. Psychophys. 70, 412-421.
3. Bernstein, I. H., Rose, R. and Ashe, V. M. (1970). Energy integration in intersensory facilitation, J. Exper. Psychol. 86, 196-203.
4. Bolognini, N., Frassinetti, F., Serino, A. and Ladavas, E. (2005). 'Acoustical vision' of below threshold stimuli: interaction among spatially converging audiovisual input, Exper. Brain Res. 160, 273-282. (Pubitemid 40169419)
5. Chen, Y.-C. and Yeh, S.-L. (2008). Visual events modulated by sound in repetition blindness, Psychon. B Rev. 15, 404-408.
6. Chen, Y.-C. and Yeh, S.-L. (2009). Catch the moment: multisensory enhancement of rapid visual events by sound, Exper. Brain Res. 198, 209-219.
7. de Gelder, B. and Bertelson, P. (2003). Multisensory integration, perception and ecological validity, Trends Cognit. Sci. 7, 460-467. (Pubitemid 37188790)
8. Dosher, B. A. and Lu, Z.-L. (1998). Perceptual learning reflects external noise filtering and internal noise reduction through channel reweighting, Proc. Nat. Acad. Sci. USA 95, 13988-13993. (Pubitemid 28523085)
9. Downar, J., Crawley, A. P., Mikulis, D. J. and Davis, K. D. (2000). A multimodal cortical network for the detection of changes in the sensory environment, Nat. Neurosci. 3, 277-283. (Pubitemid 30140791)
10. Driver, J. and Noesselt, T. (2008). Multisensory interplay reveals crossmodal influences on 'sensory specific' brain regions, neural responses, and judgments, Neuron 57, 11-23.
11. Earle, D. C. and Lowe, G. (1971). Channel, temporal, and composite uncertainty in the detection and recognition of auditory and visual signals, Percept. Psychophys. 9, 177-181.
12. Felleman, D. J. and van Essen, D. C. (1991). Distributed hierarchical processing in the primate cerebral cortex, Cereb. Cortex 1, 1-47.
13. Fodor, J. A. (1983). The Modularity of Mind. MIT Press, Cambridge, MA, USA.
14. Frassinetti, F., Pavani, F. and Lavadas, E. (2002). Enhancement of visual perception by crossmodal visuo-auditory interaction, Exper. Brain Res. 147, 332-343. (Pubitemid 35341007)
15. Ghazanfar, A. A. and Schroeder, C. E. (2006). Is neocortex essentially multisensory? Trends Cognit. Sci. 10, 278-285. (Pubitemid 43870163)
16. Hairol,M. I. andWaugh, S. J. (2010). Lateral interactions across space reveal links between processing streams for luminance-modulated and contrast-modulated stimuli, Vision Res. 50, 889-903.
17. Hershenson, M. (1962). Reaction time as a measure of intersensory facilitation, J. Exper. Psychol. 63, 289-293.
18. Kelly, D. H. (1974). Spatio-temporal frequency characteristics of color-vision mechanisms, J. Opt. Soc. Am. 64, 983-990.
19. Kingdom, F. A. A. and Prins, N. (2010). Psychophysics: A Practical Introduction. Academic Press, London, UK.
20. Lippert, M., Logothetis, N. K. and Kayser, C. (2007). Improvement of visual contrast detection by a simultaneous sound, Brain Res. 1173, 102-109. (Pubitemid 47445452)
21. Lovelace, C. T., Stein, B. E. andWallace, M. T. (2003). An irrelevant light enhances auditory detection in humans: a psychophysical analysis of multisensory integration in stimulus detection, Cognit. Brain Res. 17, 447-453. (Pubitemid 36860069)
22. Lu, Z.-L. and Dosher, B. A. (1998). External noise distinguishes attention mechanisms, Vision Res. 38, 1183-1198. (Pubitemid 28204060)
23. Lu, Z.-L. and Dosher, B. A. (1999). Characterizing human perceptual inefficiencies with equivalent internal noise, J. Optic. Soc. Amer. A 16, 764-778. (Pubitemid 129589252)
24. Ma,W. J., Zhou, X., Ross, L. A., Foxe, J. J. and Parra, L. C. (2009). Lip-reading aids word recognition most in moderate noise: a Bayesian explanation using high-dimensional feature space, PLoS ONE 4, e4638. DOI: 10.1371/journal.pone.0004638.
25. Marks, L. E., Ben-Artzi, E. and Lakatos, S. (2003). Cross-modal interactions in auditory and visual discrimination, Intl J. Psychophysiol. 50, 125-145. (Pubitemid 37162876)
26. Ngo, M. K. and Spence, C. (2010a). Crossmodal facilitation of masked visual target identification, Atten. Percept. Psychol. 72, 1938-1947.
27. Ngo, M. K. and Spence, C. (2010b). Auditory, tactile, and multisensory cues facilitate search for dynamic visual stimuli, Atten. Percept. Psychol. 72, 1654-1665.
28. Nickerson, R. S. (1973). Intersensory facilitation of reaction time: energy summation or preparation enhancement? Psychol. Rev. 80, 489-509.
29. Noesselt, T., Bergmann, D., Hake, M., Heinze, H.-J. and Fendrich, R. (2008). Sound increases the saliency of visual events, Brain Res. 1220, 157-163.
30. Noesselt, T., Tyll, S., Boehler, C. N., Budinger, E., Heinze, H.-J. and Driver, J. (2010). Sound-induced enhancement of low-intensity vision: multisensory influences on human sensory-specific cortices and thalamic bodies relate to perceptual enhancement of visual detection sensitivity, J. Neurosci. 30, 13609-13623.
31. Odgaard, E. C., Arieh, Y. and Marks, L. E. (2003). Cross-modal enhancement of perceived brightness: sensory interaction versus response bias, Percept. Psychophys. 65, 123-132. (Pubitemid 37544903)
32. Odgaard, E. C., Arieh, Y. and Marks, L. E. (2004). Brighter noise: sensory enhancement of perceived loudness by concurrent visual stimulation, Cognit. Affect. Behav. Neurosci. 4, 127-132. (Pubitemid 39167155)
33. Olivers, C. N. L. and van der Burg, E. (2008). Bleeping you out of the blink: sound saves vision from oblivion, Brain Res. 1242, 191-199.
34. Pelli, D. G. (1985). Uncertainty explains many aspects of visual contrast detection and discrimination, J. Optic. Soc. Amer. A 2, 1508-1532.
35. Petrov, Y., Verghese, P. and McKee, S. P. (2006). Collinear facilitation is largely uncertainty reduction, J. Vision 6, 170-178. (Pubitemid 43338065)
36. Prins, N. and Kingdom, F. A. A. (2009). Palamedes: Matlab Routines for Analyzing Psychophysical Data. www.palamedestoolbox.org.
37. Pylyshyn, Z. W. (1999). Is vision continuous with cognition? The case for cognitive impenetrability of visual perception, Behav. Brain Sci. 22, 341-423.
38. Robson, J. G. (1966). Spatial and temporal contrast-sensitivity functions of the visual system, J. Optic. Soc. Amer. 56, 1141-1142.
39. Ross, L. A., Saint-Amour, D., Leavitt, V. M., Javitt, D. C. and Foxe, J. J. (2007). Do you see what I am saying? Exploring visual enhancement of speech comprehension in noisy environments, Cereb. Cortex 17, 1147-1153. (Pubitemid 46598855)
40. Spence, C. (2010). Crossmodal attention, Scholarpedia 5, 6309. DOI: 10.4249/scholarpedia.6309.
41. Spence, C. and Ngo, M.-C. (in press). Does attention or multisensory integration explain the crossmodal facilitation of masked visual target identification? in: The New Handbook of Multisensory Processing, B. E. Stein (Ed.). MIT Press, Cambridge, MA, USA.
42. Stein, B. E., London, N., Wilkinson, L. K. and Price, D. D. (1996). Enhancement of perceived visual intensity by auditory stimuli: a psychophysical analysis, J. Cognit. Neurosci. 8, 497-506. (Pubitemid 27075432)
43. van der Burg, E., Olivers, C. N. L., Bronkhorst, A. W. and Theeuwes, J. (2008). Pip and pop: nonspatial auditory signals improve spatial visual search, J. Exper. Psychol. Human 34, 1053-1065.
44. Vroomen, J. and de Gelder, B. (2000). Sound enhances visual perception: crossmodal effects of auditory organization on vision, J. Exper. Psychol. Human 26, 1583-1590.
45. Watkins, S., Shams, L., Josephs, O. and Rees, G. (2007). Activity in human V1 follows multisensory perception, Neuroimage 37, 572-578. (Pubitemid 47454512)
46. Werner, S. and Noppeney, U. (2010). Perceptual decisions formed by accumulation of audiovisual evidence in prefrontal cortex, J. Neurosci. 30, 7434-7446.
47. Yu, C., Klein, S. A. and Levi, D. M. (2002). Facilitation of contrast detection by cross-oriented surround stimuli and its psychophysical mechanisms, J. Vision 2, 243-255. (Pubitemid 39047317)