Sound-driven enhancement of vision: Disentangling detection-level from decision-level contributions

Journal of Neurophysiology

Volumn 109, Issue 4, 2013, Pages 1065-1077

  Pérez Bellido, Alexis ^a   Soto Faraco, Salvador ^b,c   López Moliner, Joan ^a,d  

^a UNIVERSITY OF BARCELONA   (Spain)

^b UNIVERSITAT POMPEU FABRA   (Spain)

^c INSTITUCIÓ CATALANA DE RECERCA I ESTUDIS AVANÇATS ICREA   (Spain)

^d Institute for Brain Cognition and Behaviour (IR3C)   (Spain)

Author keywords

Audio visual stimuli; Magnocellular pathway; Psychophysics; Race models; Reaction time; Sensory integration

Indexed keywords

ADULT; ARTICLE; AUDITORY STIMULATION; CLINICAL ARTICLE; CONTRAST SENSITIVITY; FACILITATION; FEMALE; HUMAN; MALE; PRIORITY JOURNAL; PSYCHOPHYSICS; REACTION TIME; SPATIAL FREQUENCY DISCRIMINATION; STIMULUS RESPONSE; VISION; VISUAL STIMULATION;

ACOUSTIC STIMULATION; ADULT; AUDITORY PERCEPTION; DECISION MAKING; FEMALE; HUMANS; MALE; PHOTIC STIMULATION; REACTION TIME; SIGNAL DETECTION, PSYCHOLOGICAL; SOUND; VISION, OCULAR; VISUAL PERCEPTION;

EID: 84874084850     PISSN: 00223077     EISSN: 15221598     Source Type: Journal    
DOI: 10.1152/jn.00226.2012     Document Type: Article

References (83)

1. Andersen TS, Mamassian P. Audiovisual integration of stimulus transients. Vision Res 48: 2537-2544, 2008.
2. Arnold DH, Tear M, Schindel R, Roseboom W. Audio-visual speech cue combination. PloS One 5: e10217, 2010.
3. Benjamini Y, Hochberg Y. Controlling the false discovery rate: a practical and powerful approach to multiple testing. J R Stat Soc B 57: 289-300, 1995.
4. Berson DM. Retinal and cortical inputs to cat superior colliculus: composition, convergence and laminar specificity. Prog Brain Res 75: 17-26, 1988.
5. Bolognini N, Frassinetti F, Serino A, Làdavas E. "Acoustical vision" of below threshold stimuli: interaction among spatially converging audiovisual inputs. Exp Brain Res 160: 273-282, 2005.
6. Breitmeyer BG. Simple reaction time as a measure of the temporal response properties of transient and sustained channels. Vision Res 15: 1411-1412, 1975.
7. Calvert GA, Hansen PC, Iversen SD, Brammer MJ. Detection of audiovisual integration sites in humans by application of electrophysiological criteria to the BOLD effect. Neuroimage 14: 427-438, 2001.
8. Cappe C, Thut G, Romei V, Murray MM. Selective integration of auditoryvisual looming cues by humans. Neuropsychologia 47: 1045-1052, 2009.
9. Carpenter RH. Contrast, probability, and saccadic latency?: evidence for independence of detection and decision. Curr Biol 14: 1576-1580, 2004.
10. Carpenter RH, Williams ML. Neural computation of log likelihood in control of saccadic eye movements. Nature 377: 59-62, 1995.
11. Colonius H, Arndt P. A two-stage model for visual-auditory interaction in saccadic latencies. Percept Psychophys 63: 126-147, 2001.
12. Corneil BD, Van Wanrooij M, Munoz DP, Van Opstal AJ. Auditory-visual interactions subserving goal-directed saccades in a complex scene. J Neurophysiol 88: 438-454, 2002.
13. Diederich A, Colonius H. Bimodal and trimodal multisensory enhancement: effects of stimulus onset and intensity on reaction time. Percept Psychophys 66: 1388-1404, 2004.
14. Doyle MC, Snowden RJ. Identification of visual stimuli is improved by accompanying auditory stimuli: the role of eye movements and sound location. Perception 30: 795-810, 2001.
15. Driver J, Spence C. Cross-modal links in spatial attention. Philos Trans R Soc Lond B Biol Sci 353: 1319-1331, 1998.
16. Driver J, Noesselt T. Multisensory interplay reveals crossmodal influences on "sensory-specific" brain regions, neural responses, and judgments. Neuron 57: 11-23, 2008.
17. Ernst MO, Bülthoff HH. Merging the senses into a robust percept. Trends Cogn Sci 8: 162-169, 2004.
18. Falchier A, Clavagnier S, Barone P, Kennedy H. Anatomical evidence of multimodal integration in primate striate cortex. J Neurosci 22: 5749-5759, 2002.
19. Frassinetti F, Bolognini N, Ladavas E. Enhancement of visual perception by crossmodal visuo-auditory interaction. Exp Brain Res 147: 332-343, 2002.
20. Giard MH, Peronnet F. Auditory-visual integration during multimodal object recognition in humans: a behavioral and electrophysiological study. J Cogn Neurosci 11: 473-490, 1999.
21. Green DM. Detection of multiple component signals in noise. J Acoust Soc Am 30: 904-911, 1958.
22. Harwerth RS, Levi DM. Reaction time as a measure of suprathreshold grating detection. Vision Res 18: 1579-1586, 1978.
23. Jaekl PM, Harris LR. Sounds can affect visual perception mediated primarily by the parvocellular pathway. Vis Neurosci 26: 477-486, 2009.
24. Jaekl PM, Soto-Faraco S. Audiovisual contrast enhancement is articulated primarily via the M-pathway. Brain Res 1366: 85-92, 2010.
25. Kinchla RA. Detecting target elements in multielement arrays?: a confusability model. Perception 15: 149-158, 1974.
26. Lakatos P, Chen CM, O'Connell MN, Mills A, Schroeder CE. Neuronal oscillations and multisensory interaction in primary auditory cortex. Neuron 53: 279-292, 2007.
27. Laurienti PJ, Kraft RA, Maldjian JA, Burdette JH, Wallace MT. Semantic congruence is a critical factor in multisensory behavioral performance. Exp Brain Res 158:, 405-414, 2004.
28. Legge GE. Sustained and transient mechanisms in human vision: temporal and spatial properties. Vision Res 18: 69-81, 1978.
29. Leo F, Bertini C, di Pellegrino G, Ladavas E. Multisensory integration for orienting responses in humans requires the activation of the superior colliculus. Exp Brain Res 186: 67-77, 2008.
30. Levi DM, Harwerth RS, Manny RE. Suprathreshold spatial frequency detection and binocular interaction in strabismic and anisometropic amblyopia. Invest Ophthalmol Vis Sci 18: 714-725, 1979.
31. Lippert M, Logothetis NK, Kayser C. Improvement of visual contrast detection by a simultaneous sound. Brain Res 1173: 102-109, 2007.
32. Livingstone MS, Hubel DH. Do the relative mapping densities of the magnoand parvocellular systems vary with eccentricity? J Neurosci 8: 4334-4339, 1988.
33. Logan GD, Cowan WB, Davis KA. On the ability to inhibit simple and choice reaction time responses: a model and a method. J Exp Psychol Hum Percept Perform 10: 276-291, 1984.
34. López-Moliner J, Soto-Faraco S. Vision affects how fast we hear sounds move. J Vis 7: 6.1-6.7, 2007.
35. Manjarrez E, Mendez I, Martinez L, Flores A, Mirasso CR. Effects of auditory noise on the psychophysical detection of visual signals: crossmodal stochastic resonance. Neurosci Lett 415: 231-236, 2007.
36. McDonald JJ, Teder-Sälejärvi WA, Hillyard SA. Involuntary orienting to sound improves visual perception. Nature 407: 906-908, 2000.
37. McDonald JJ, Teder-Sälejärvi WA, Ward LM. Multisensory integration and crossmodal attention effects in the human brain. Science 292: 1791, 2001.
38. McDonald JJ, Teder-Sälejärvi WA, Di Russo F, Hillyard SA. Neural basis of auditory-induced shifts in visual time-order perception. Nat Neurosci 8: 1197-1202, 2005.
39. Meredith MA, Stein BE. Visual, auditory, and somatosensory convergence on cells in superior colliculus results in multisensory integration. J Neurophysiol 56: 640-662, 1986.
40. Meredith MA, Stein BE. Spatial determinants of multisensory integration in cat superior colliculus neurons. J Neurophysiol 75: 1843-1857, 1996.
41. Meredith M, Stein B. Interactions among converging sensory inputs in the superior colliculus. Science 221: 389-391, 1983.
42. Merigan WH, Maunsell JH. How parallel are the primate visual pathways? Annu Rev Neurosci 16: 369-402, 1993.
43. Meyer GF, Wuerger SM, Röhrbein F, Zetzsche C. Low-level integration of auditory and visual motion signals requires spatial co-localisation. Exp Brain Res 166: 538-547, 2005.
44. Miller J. Divided attention: evidence for coactivation with redundant signals. Cogn Psychol 14: 247-279, 1982.
45. Mitov D, Totev T. How many pathways determine the speed of grating detection? Vision Res 45: 821-825, 2005.
46. Molholm S, Ritter W, Murray MM, Javitt DC, Schroeder CE, Foxe JJ. Multisensory auditory-visual interactions during early sensory processing in humans?: a high-density electrical mapping study. Cogn Brain Res 14: 115-128, 2002.
47. Morrell F. Visual system's view of acoustic space. Nature 238: 44-46, 1972.
48. Morrone MC, Burr DC, Vaina LM. Two stages of visual processing for radial and circular motion. Nature 376: 507-509, 1995.
49. Murray IJ, Plainis S. Contrast coding and magno/parvo segregation revealed in reaction time studies. Vision Res 43: 2707-2719, 2003.
50. Nickerson RS. Intersensory facilitation of reaction time: energy summation or preparation enhancement? Psychol Rev 80: 489-509, 1973.
51. Nieuwenhuis S, Ridderinkhof KR, Blom J, Band GP, Kok A. Error-related brain potentials are differentially related to awareness of response errors: evidence from an antisaccade task. Psychophysiology 38: 752-760, 2001.
52. Noesselt T, Tyll S, Boehler CN, Budinger E, Heinze HJ, Driver J. Sound-induced enhancement of low-intensity vision: multisensory influences on human sensory-specific cortices and thalamic bodies relate to perceptual enhancement of visual detection sensitivity. J Neurosci 30: 13609-13623, 2010.
53. Noesselt T, Bergmann D, Hake M, Heinze HJ, Fendrich R. Sound increases the saliency of visual events. Brain Res 1220: 157-163, 2008.
54. Nozawa G, Reuter-Lorenz PA, Hughes HC. Parallel and serial processes in the human oculomotor system: bimodal integration and express saccades. Biol Cybern 72: 19-34, 1994.
55. Odgaard EC, Arieh Y, Marks LE. Cross-modal enhancement of perceived brightness: sensory interaction versus response bias. Percept Psychophys 65: 123-132, 2003.
56. Otto TU, Mamassian P. Noise and correlations in parallel perceptual decision making. Curr Biol 22: 1391-1396, 2012.
57. Pannunzi M, Pérez-Bellido A, Baños AP, López-Moliner J, Deco G, Soto-Faraco S. Scrutinizing integrative effects in a multi-stimuli detection task. Seeing Perceiving 25: 100, 2012.
58. Piéron H. Recherches sur les lois de variation des temps de latence sensorielle en fonction des intensités excitatrices. L'Année Psychol 20: 17-96, 1914.
59. Pins D, Bonnet C. The Piéron function in the threshold region. Percept Psychophys 62: 127-136, 2000.
60. Pins D, Bonnet C. On the relation between stimulus intensity and processing time?: Pieron's law and choice reaction time. Perception 58: 390-400, 1996.
61. Plainis S, Murray IJ. Neurophysiological interpretation of human visual reaction times: effect of contrast, spatial frequency and luminance. Neuropsychologia 38: 1555-1564, 2000.
62. Quick RF. A vector-magnitude model of contrast detection. Kybernetik 16: 65-67, 1974.
63. Raab DH. Statistical facilitation of simple reaction times. Trans NY Acad Sci 24: 574-590, 1962.
64. Romei V, Murray MM, Cappe C, Thut G. Preperceptual and stimulusselective enhancement of low-level human visual cortex excitability by sounds. Curr Biol? 19: 1799-1805, 2009.
65. Rowland BA, Quessy S, Stanford TR, Stein BE. Multisensory integration shortens physiological response latencies. J Neurosci 27: 5879-5884, 2007.
66. Schroeder CE, Foxe J. Multisensory contributions to low-level, "unisensory" processing. Curr Opin Neurobiol 15: 454-458, 2005.
67. Silveira LC, Perry VH. The topography of magnocellular projecting ganglion cells (M-ganglion cells) in the primate retina. Neuroscience 40: 217-237, 1991.
68. Smiley JF, Falchier A. Multisensory connections of monkey auditory cerebral cortex. Hear Res 258: 37-46, 2009.
69. Spence C, Driver J. Covert orienting in audition: endogenous and exogenous mechanisms. J Exp Psychol Hum Percept Perform 20: 555-574, 1994.
70. Stein BE. Neural mechanisms for synthesizing sensory information and producing adaptive behaviors. Exp Brain Res 123: 124-135, 1998.
71. Stein BE, Stanford TR, Ramachandran R, Perrault TJ, Rowland BA. Challenges in quantifying multisensory integration: alternative criteria, models, and inverse effectiveness. Exp Brain Res 198: 113-126, 2009.
72. Stein BE, London N, Wilkinson LK, Price DD. Enhancement of perceived visual intensity by auditory stimuli: a psychophysical analysis. J Cogn Neurosci 8: 497-506, 1996.
73. Stein BE, Meredith MA, Huneycutt WS, McDade L. Behavioral indices of multisensory integration: orientation to visual cues is affected by auditory stimuli. J Cogn Neurosci 1: 12-24, 1989.
74. Thomas JP, Fagerholm P, Bonnet C. One spatial filter limits speed of detecting low and middle frequency gratings. Vision Res 39: 1683-1693, 1999.
75. Treisman M. Combining information: probability summation and probability averaging in detection and discrimination. Psychol Methods 3: 252-265, 1998.
76. Ulrich R, Miller J, Schröter H. Testing the race model inequality: an algorithm and computer programs. Behav Res Methods 39: 291-302, 2007.
77. Valls-Solé J, Solé A, Valldeoriola F, Muñoz E, Gonzalez LE, Tolosa ES. Reaction time and acoustic startle in normal human subjects. Neurosci Lett 195: 97-100, 1995.
78. Van der Burg E, Cass J, Olivers CN, Theeuwes J, Alais D. Efficient visual search from synchronized auditory signals requires transient audiovisual events. PloS One 5: e10664, 2010.
79. Wallace MT, Meredith MA, Stein BE. Converging influences from visual, auditory, and somatosensory cortices onto output neurons of the superior colliculus. J Neurophysiol 69: 1797-1809, 1993.
80. Wallace MT, Meredith MA, Stein BE. Multisensory integration in the superior colliculus of the alert cat. J Neurophysiol 80: 1006-1010, 1998.
81. Watkins S, Shams L, Tanaka S, Haynes JD, Rees G. Sound alters activity in human V1 in association with illusory visual perception. Neuroimage 31: 1247-1256, 2006.
82. Werner S, Noppeney U. The contributions of transient and sustained response codes to audiovisual integration. Cereb Cortex 21: 920-931, 2011.
83. Wuerger SM, Hofbauer M, Meyer GF. The integration of auditory and visual motion signals at threshold. Percept Psychophys 65: 1188-1196, 2003.