A neurocomputational analysis of the sound-induced flash illusion

NeuroImage

Volumn 92, Issue , 2014, Pages 248-266

  Cuppini, Cristiano ^a   Magosso, Elisa ^a   Bolognini, Nadia ^b,c   Vallar, Giuseppe ^b,c   Ursino, Mauro ^a  

^a UNIVERSITY OF BOLOGNA   (Italy)

^b UNIVERSITY OF MILANO BICOCCA   (Italy)

^c ISTITUTO AUXOLOGICO ITALIANO   (Italy)

Author keywords

Multisensory integration; Neural network modeling; Sound induced flash illusion; Spatial processing; Temporal processing; Visual auditory illusions

Indexed keywords

ACOUSTIC NERVE FIBER; ARTICLE; AUDIOVISUAL ILLUSION; AUDITORY NERVOUS SYSTEM; AUDITORY STIMULATION; BAYES THEOREM; COMPUTER ANALYSIS; CONCEPTUAL FRAMEWORK; EXCITATORY JUNCTION POTENTIAL; NERVE CELL NETWORK; NEUROCOMPUTATIONAL ANALYSIS; PRIORITY JOURNAL; RECEPTIVE FIELD; SOUND INDUCED FLASH ILLUSION; SPATIOTEMPORAL ANALYSIS; VENTRILOQUISM ILLUSION; VISUAL ILLUSION; VISUAL NEURON; VISUAL STIMULATION; ASSOCIATION; AUDITORY CORTEX; BIOLOGICAL MODEL; COMPUTER SIMULATION; HEARING; HUMAN; ILLUSION; NERVE TRACT; PHYSIOLOGY; PROCEDURES; VISION; VISUAL CORTEX;

ACOUSTIC STIMULATION; AUDITORY CORTEX; AUDITORY PERCEPTION; COMPUTER SIMULATION; CUES; HUMANS; ILLUSIONS; MODELS, NEUROLOGICAL; NERVE NET; NEURAL PATHWAYS; VISUAL CORTEX; VISUAL PERCEPTION;

EID: 84895935730     PISSN: 10538119     EISSN: 10959572     Source Type: Journal    
DOI: 10.1016/j.neuroimage.2014.02.001     Document Type: Article

References (124)

1. Alais D., Burr D. The ventriloquist effect results from near-optimal bimodal integration. Curr. Biol. 2004, 14(3):257-262.
2. Alais D., Newell F.N., Mamassian P. Multisensory processing in review: from physiology to behaviour. Seeing Perceiving 2010, 23(1).
3. Anastasio T.J., Patton P.E., Belkacem-Boussaid K. Using Bayes rule to model multisensory enhancement in the superior colliculus. Neural Comput. 2000, 12:1165-1187.
4. Andersen T.S., Tiippana K., Sams M. Factors influencing audiovisual fission and fusion illusions. Cogn. Brain Res. 2004, 21(3):301-308.
5. Bastos A.M., Usrey W.M., Adams R.A., Mangun G.R., Fries P., Friston K.J. Canonical microcircuits for predictive coding. Neuron 2012, 76(4):695-711.
6. Battaglia P.W., Jacobs R.A., Aslin R.N. Bayesian integration of visual and auditory signals for spatial localization. J. Opt. Soc. Am. A 2003, 20(7):1391-1397.
7. Beer A.L., Plank T., Greenlee M.W. Diffusion tensor imaging shows white matter tracts between human auditory and visual cortex. Exp. Brain Res. 2011, 213(2-3):299-308.
8. Ben-Yishai R., Bar O., Sompolinsky H. Theory of orientation tuning in visual cortex. Proc. Natl. Acad. Sci. U. S. A. 1995, 92:3844-3848.
9. Bertelson P., Aschersleben G. Automatic visual bias of perceived auditory location. Psychon. Bull. Rev. 1998, 5(3):482-489.
10. Bertelson P., Radeau M. Cross-modal bias and perceptual fusion with auditory-visual spatial discordance. Atten. Percept. Psychophys. 1981, 29(6):578-584.
11. Bolognini N., Maravita A. Proprioceptive alignment of visual and somatosensory maps in the posterior parietal cortex. Curr. Biol. 2007, 17(21):1890-1895.
12. Bolognini N., Maravita A. Uncovering multisensory processing through non-invasive brain stimulation. Front. Psychol. 2011, 2:46.
13. Bolognini N., Frassinetti F., Serino A., Ládavas E. "Acoustical vision" of below threshold stimuli: interaction among spatially converging audiovisual inputs. Exp. Brain Res. 2005, 160(3):273-282.
14. Bolognini N., Leo F., Passamonti C., Stein B.E., Làdavas E. Multisensory-mediated auditory localization. Perception 2007, 36(10):1477-1485.
15. Bolognini N., Senna I., Maravita A., Pascual-Leone A., Merabet L.B. Auditory enhancement of visual phosphene perception: the effect of temporal and spatial factors and of stimulus intensity. Neurosci. Lett. 2010, 477(3):109-114.
16. Bolognini N., Rossetti A., Casati C., Mancini F., Vallar G. Neuromodulation of multisensory perception: a tDCS study of the sound-induced flash illusion. Neuropsychologia 2011, 49(2):231-237.
17. Bresciani J.P., Dammeier F., Ernst M.O. Vision and touch are automatically integrated for the perception of sequences of events. J. Vis. 2006, 6(5).
18. Buchel C., Price C., Friston K. A multimodal language region in the ventral visual pathway. Nature 1998, 394:274-277.
19. Cappe C., Barone P. Heteromodal connections supporting multisensory integration at low levels of cortical processing in the monkey. Eur. J. Neurosci. 2005, 22(11):2886-2902.
20. Carandini M., Heeger D.J., Senn W. A synaptic explanation of suppression in visual cortex. J. Neurosci. 2002, 22(22):10053-10065.
21. Chen C.C., Henson R.N., Stephan K.E., Kilner J.M., Friston K.J. Forward and backward connections in the brain: a DCM study of functional asymmetries. NeuroImage 2009, 45(2):453-462.
22. Convento S., Vallar G., Galantini C., Bolognini N. Neuromodulation of early multisensory interactions in the visual cortex. J. Cogn. Neurosci. 2013, 25(5):685-696.
23. Corneil B.D., Munoz D.P. The influence of auditory and visual distractors on human orienting gaze shifts. J. Neurosci. 1996, 16(24):8193-8207.
24. Craik K.J.W. The Nature of Explanation 1945, CUP, Cambridge, UK.
25. Driver J., Noesselt T. Multisensory interplay reveals crossmodal influences on 'sensory specific' brain regions, neural responses, and judgments. Neuron 2008, 57:11-23.
26. Eckert M.A., Kamdar N.V., Chang C.E., Beckmann C.F., Greicius M.D., Menon V. A cross-modal system linking primary auditory and visual cortices: evidence from intrinsic fMRI connectivity analysis. Hum. Brain Mapp. 2008, 29(7):848-857.
27. Ernst M.O., Banks M.S. Humans integrate visual and haptic information in a statistically optimal fashion. Nature 2002, 415(6870):429-433.
28. Ernst M.O., Banks M.S., Bülthoff H.H. Touch can change visual slant perception. Nat. Neurosci. 2000, 3(1):69-73.
29. Falchier A., Clavagnier S., Barone P., Kennedy H. Anatomical evidence of multimodal integration in primate striate cortex. J. Neurosci. 2002, 22(13):5749-5759.
30. Foxe J.J., Schroeder C.E. The case for feedforward multisensory convergence during early cortical processing. NeuroReport 2005, 16(5).
31. Foxe J.J., Morocz I.A., Murray M.M., Higgins B.A., Javitt D.C., Schroeder C.E. Multisensory auditory-somatosensory interactions in early cortical processing revealed by high-density electrical mapping. Cogn. Brain Res. 2000, 10(1ΓÇÔ2):77-83.
32. Frassinetti F., Bolognini N., Ládavas E. Enhancement of visual perception by crossmodal visuo-auditory interaction. Exp. Brain Res. 2002, 147(3):332-343.
33. Frassinetti F., Bolognini N., Bottari D., Bonora A., Ládavas E. Audiovisual integration in patients with visual deficit. J. Cogn. Neurosci. 2005, 17(9):1442-1452.
34. Freeman E., Driver J. Direction of visual apparent motion driven solely by timing of a static sound. Curr. Biol. 2008, 18(16):1262-1266.
35. Friston K. A theory of cortical responses. Philos. Trans. R. Soc. B Biol. Sci. 2005, 360(1456):815-836.
36. Friston K. The free-energy principle: a unified brain theory?. Nat. Rev. Neurosci. 2010, 11(2):127-138.
37. Friston K., Kiebel S. Cortical circuits for perceptual inference. Neural Netw. 2009, 22(8):1093-1104.
38. Garrido M.I., Kilner J.M., Kiebel S.J., Friston K.J. Evoked brain responses are generated by feedback loops. Proc. Natl. Acad. Sci. 2007, 104(52):20961-20966.
39. Gebhard J.W., Mowbray G.H. On discriminating the rate of visual flicker and auditory flutter. Am. J. Psychol. 1959, 72:521-529.
40. Getzmann S. The effect of brief auditory stimuli on visual apparent motion. Perception 2007, 36(7):1089-1103.
41. Ghazanfar A.A., Schroeder C.E. Is neocortex essentially multisensory?. Trends Cogn. Sci. 2006, 10(6):278-285.
42. Giard M.H., Peronnet F. Auditory-visual integration during multimodal object recognition in humans: a behavioral and electrophysiological study. J. Cogn. Neurosci. 1999, 11(5):473-490.
43. Hairston W.D., Laurienti P.J., Mishra G., Burdette J.H., Wallace M.T. Multisensory enhancement of localization under conditions of induced myopia. Exp. Brain Res. 2003, 152:404-408.
44. Harrison L.M., Stephan K.E., Rees G., Friston K.J. Extra-classical receptive field effects measured in striate cortex with fMRI. NeuroImage 2007, 34(3):1199-1208.
45. Howard I.P., Templeton W.B. Human Spatial Orientation 1966, John Wiley & Sons, Oxford, England.
46. Huang Y., Rao R.P.N. Predictive coding. Wiley Interdiscip. Rev. Cogn. Sci. 2011, 2(5):580-593.
47. Hubel D.H., Wiesel T.N. Receptive fields, binocular interaction and functional architecture in the cat's visual cortex. J. Physiol. 1962, 160(1):106-154.
48. Hughes H.C., Reuter-Lorenz P.A., Nozawa G., Fendrich R. Visual-auditory interactions in sensorimotor processing: saccades versus manual responses. J. Exp. Psychol. Hum. Percept. Perform. 1994, 20(1):131-153.
49. Innes-Brown H., Crewther D. The impact of spatial incongruence on an auditory-visual illusion. PLoS ONE 2009, 4(7):e6450.
50. Jansen B., Rit V. Electroencephalogram and visual evoked potential generation in a mathematical model of coupled cortical columns. Biol. Cybern. 1995, 73(4):357-366.
51. Kamke M.R., Vieth H.E., Cottrell D., Mattingley J.B. Parietal disruption alters audiovisual binding in the sound-induced flash illusion. NeuroImage 2012, 62(3):1334-1341.
52. Knill D.C., Pouget A. The Bayesian brain: the role of uncertainty in neural coding and computation. Trends Neurosci. 2004, 27(12):712-719.
53. Knox G.W. Investigations of flicker and fusion: IV. The effect of auditory flicker on the pronouncedness of visual flicker. J. Gen. Psychol. 1945, 33:145-154.
54. Komssi S., Kähkönen S. The novelty value of the combined use of electroencephalography and transcranial magnetic stimulation for neuroscience research. Brain Res. Rev. 2006, 52(1):183-192.
55. Körding K.P., Beierholm U., Ma W.J., Quartz S., Tenenbaum J.B., Shams L. Causal inference in multisensory perception. PLoS ONE 2007, 2(9):e943.
56. Leo F., Bolognini N., Passamonti C., Stein B.E., Ladavas E. Cross-modal localization in hemianopia: new insights on multisensory integration. Brain 2008, 131(3):855-865.
57. Ma W.J., Beck J.M., Latham P.E., Pouget A. Bayesian inference with probabilistic population codes. Nat. Neurosci. 2006, 9(11):1432-1438.
58. Macaluso E. Multisensory processing in sensory-specific cortical areas. Neuroscientist 2006, 12(4):327-338.
59. Macaluso E., Frith C.D., Driver J. Modulation of human visual cortex by crossmodal spatial attention. Science 2000, 289(5482):1206-1208.
60. Magosso E., Cuppini C., Ursino M. A neural network model of ventriloquism effect and aftereffect. PLoS ONE 2012, 7(8):e42503.
61. Maravita A., Bolognini N., Bricolo E., Marzi C.A., Savazzi S. Is audiovisual integration subserved by the superior colliculus in humans?. NeuroReport 2008, 19(3).
62. Maunsell J.H., Gibson J.R. Visual response latencies in striate cortex of the macaque monkey. J. Neurophysiol. 1992, 68(4):1332-1344.
63. McDonald J.J., Teder-Salejarvi W.A., Di Russo F., Hillyard S.A. Neural substrates of perceptual enhancement by cross-modal spatial attention. J. Cogn. Neurosci. 2003, 15:10-19.
64. McDonald J.J., Teder-Salejarvi W.A., Di Russo F., Hillyard S.A. Neural basis of auditory-induced shifts in visual time-order perception. Nat. Neurosci. 2005, 8:1197-1202.
65. Meredith M.A. On the neuronal basis for multisensory convergence: a brief overview. Cogn. Brain Res. 2002, 14(1):31-40.
66. Meredith M.A., Stein B.E. Visual, auditory, and somatosensory convergence on cells in superior colliculus results in multisensory integration. J. Neurophysiol. 1986, 56(3):640-662.
67. Middlebrooks J.C., Knudsen E.I. A neural code for auditory space in the cat's superior colliculus. J. Neurosci. 1984, 4(10):2621-2634.
68. Mishra J., Martinez A., Hillyard S.A. Cortical processes underlying sound-induced flash fusion. Brain Res. 2008, 1242:102-115.
69. Mishra J., Martinez A., Hillyard S.A. Effect of attention on early cortical processes associated with the sound-induced extra flash illusion. J. Cogn. Neurosci. 2009, 22(8):1714-1729.
70. Molholm S., Ritter W., Murray M.M., Javitt D.C., Schroeder C.E., Foxe J.J. Multisensory auditory-visual interactions during early sensory processing in humans: a high-density electrical mapping study. Cogn. Brain Res. 2002, 14(1):115-128.
71. Morein-Zamir S., Soto-Faraco S., Kingstone A. Auditory capture of vision: examining temporal ventriloquism. Cogn. Brain Res. 2003, 17(1):154-163.
72. Musacchia G., Schroeder C.E. Neuronal mechanisms, response dynamics and perceptual functions of multisensory interactions in auditory cortex. Hear. Res. 2009, 258(1-2):72-79.
73. Nitsche M.A., Liebetanz D., Antal A., Lang N., Tergau F., Paulus W. Modulation of cortical excitability by weak direct current stimulation-technical, safety and functional aspects. Suppl. Clin. Neurophysiol. 2003, 56:255-276.
74. Noesselt T., Rieger J.W., Schoenfeld M.A., Kanowski M., Hinrichs H., Heinze H.J., Driver J. Audiovisual temporal correspondence modulates human multisensory superior temporal sulcus plus primary sensory cortices. J. Neurosci. 2007, 27(42):11431-11441.
75. Ogilvie J.C. Effect of auditory flutter on the visual critical flicker frequency. Can. J. Psychol./Rev. Can. Psychol. 1956, 10(2):61-68.
76. Parise C., Spence C. Synesthetic congruency modulates the temporal ventriloquism effect. Neurosci. Lett. 2008, 442(3):257-261.
77. Paulus W. Transcranial electrical stimulation (tES-tDCS; tRNS, tACS) methods. Neuropsychol. Rehabil. 2011, 21(5):602-617.
78. Raij T., Ahveninen J., Lin F.H., Witzel T., Jääskeläinen I.P., Letham B., Israeli E., Sahyoun C., Vasios C., Stufflebeam S., Hämäläinen M., Belliveau J.W. Onset timing of cross-sensory activations and multisensory interactions in auditory and visual sensory cortices. Eur. J. Neurosci. 2010, 31(10):1772-1782.
79. Rao R.P.N., Ballard D.H. Predictive coding in the visual cortex: a functional interpretation of some extra-classical receptive-field effects. Nat. Neurosci. 1999, 2(1):79-87.
80. Recanzone G.H. Auditory influences on visual temporal rate perception. J. Neurophysiol. 2003, 89(2):1078-1093.
81. Recanzone G.H. Interactions of auditory and visual stimuli in space and time. Hear. Res. 2009, 258(1ΓÇÔ2):89-99.
82. Recanzone G.H., Guard D.C., Phan M.L. Frequency and intensity response properties of single neurons in the auditory cortex of the behaving macaque monkey. J. Neurophysiol. 2000, 83(4):2315-2331.
83. Roach N.W., Heron J., McGraw P.V. Resolving multisensory conflict: a strategy for balancing the costs and benefits of audio-visual integration. Proc. R. Soc. B Biol. Sci. 2006, 273(1598):2159-2168.
84. Rockland K.S., Ojima H. Multisensory convergence in calcarine visual areas in macaque monkey. Int. J. Psychophysiol. 2003, 50(1ΓÇÔ2):19-26.
85. Romei V., Murray M.M., Merabet L.B., Thut G. Occipital transcranial magnetic stimulation has opposing effects on visual and auditory stimulus detection: implications for multisensory interactions. J. Neurosci. 2007, 27(43):11465-11472.
86. Romei V., Murray M.M., Cappe C., Thut G. Preperceptual and stimulus-selective enhancement of low-level human visual cortex excitability by sounds. Curr. Biol. 2009, 19(21):1799-1805.
87. Rosenthal O., Shimojo S., Shams L. Sound-induced flash illusion is resistant to feedback training. Brain Topogr. 2009, 21(3):185-192.
88. Rowland B., Stanford T., Stein B. A Bayesian model unifies multisensory spatial localization with the physiological properties of the superior colliculus. Exp. Brain Res. 2007, 180(1):153-161.
89. Salminen N.H., Tiitinen H., May P.J. Auditory spatial processing in the human cortex. Neuroscientist 2012, 18(6):602-612.
90. Scheier C.R., Nijhawan R., Shimojo S. Sound alters visual temporal resolution. Invest. Ophthalmol. Vis. Sci. 1999, 40(Suppl. 4):4169.
91. Schroeder C.E., Foxe J. Multisensory contributions to low-level, 'unisensory' processing. Curr. Opin. Neurobiol. 2005, 15(4):454-458.
92. Schröger E., Widmann A. Speeded responses to audiovisual signal changes result from bimodal integration. Psychophysiology 1998, 35(6):755-759.
93. Sekuler R., Sekuler A.B., Lau R. Sound alters visual motion perception. Nature 1997, 385(6614):308.
94. Sereno M.I., Dale A.M., Reppas J.B., Kwong K.K., Belliveau J.W., Brady T.J., Rosen B.R., Tootell R.B. Borders of multiple visual areas in humans revealed by functional magnetic resonance imaging. Science 1995, 268(5212):889-893.
95. Shams L., Beierholm U.R. Causal inference in perception. Trends Cogn. Sci. 2010, 14(9):425-432.
96. Shams L., Kim R. Crossmodal influences on visual perception. Phys. Life Rev. 2010, 7(3):269-284.
97. Shams L., Kamitani Y., Shimojo S. Illusions: what you see is what you hear. Nature 2000, 408(6814):788.
98. Shams L., Kamitani Y., Thompson S., Shimojo S., Shams L., Kamitani Y., Thompson S., Shimojo S. Sound alters visual evoked potentials in humans. NeuroReport 2001, 12(17).
99. Shams L., Kamitani Y., Shimojo S. Visual illusion induced by sound. Cogn. Brain Res. 2002, 14(1):147-152.
100. Shams L., Ma W.J., Beierholm U. Sound-induced flash illusion as an optimal percept. NeuroReport 2005, 16(17).
101. Shams L., Iwaki S., Chawla A., Bhattacharya J. Early modulation of visual cortex by sound: an MEG study. Neurosci. Lett. 2005, 378(2):76-81.
102. Shipley T. Auditory flutter-driving of visual flicker. Science 1964, 145(3638):1328-1330.
103. Spierer L., Manuel A.L., Bueti D., Murray M.M. Contributions of pitch and bandwidth to sound-induced enhancement of visual cortex excitability in humans. Cortex 2013, 10.1016/j.cortex.2013.01.001.
104. Srinivasan M.V., Laughlin S.B., Dubs A. Predictive coding: a fresh view of inhibition in the retina. Proc. R. Soc. Lond. B 1982, 216(1205):427-459.
105. Stein B.E. Neural mechanisms for synthesizing sensory information and producing adaptive behaviors. Exp. Brain Res. 1998, 123(1):124-135.
106. Stein B.E., Meredith M.A. The Merging of the Senses 1993, MIT Press, Cambridge, MA.
107. Stein B.E., Stanford T.R. Multisensory integration: current issues from the perspective of the single neuron. Nat. Rev. Neurosci. 2008, 9(4):255-266.
108. Stein B.E., London N., Wilkinson L.K., Price D.D. Enhancement of perceived visual intensity by auditory stimuli: a psychophysical analysis. J. Cogn. Neurosci. 1996, 8(6):497-506.
109. Thurlow W.R., Jack C.E. Certain determinants of the "ventriloquism effect". Percept. Mot. Skills 1973, 36(3c):1171-1184.
110. Treves A. Mean-field analysis of neuronal spike dynamics. Network 1993, 4:259-284.
111. Ursino M., Cona F., Zavaglia M. The generation of rhythms within a cortical region: analysis of a neural mass model. NeuroImage 2010, 52(3):1080-1094.
112. Vallar G., Bolognini N. Behavioural facilitation following brain stimulation: implications for neurorehabilitation. Neuropsychol. Rehabil. 2011, 21(5):618-649.
113. Vroomen J., de Gelder B. Perceptual effects of cross-modal stimulation: ventriloquism and the freezing phenomenon. The Handbook of Multisensory Processes 2004, 141-150. MIT Press, Cambridge, MA, US. G.A. Calvert, C. Spence, B.E. Stein (Eds.).
114. Vroomen J., de Gelder B. Temporal ventriloquism: sound modulates the flash-lag effect. J. Exp. Psychol. Hum. Percept. Perform. 2004, 30(3):513-518.
115. Walker J.T., Scott K.J. Auditory-visual conflicts in the perceived duration of lights, tones, and gaps. J. Exp. Psychol. Hum. Percept. Perform. 1981, 7(6):1327-1339.
116. Wallace M.T., Stein B.E. Early experience determines how the senses will interact. J. Neurophysiol. 2007, 97(1):921-926.
117. Wallace M.T., Ramachandran R., Stein B.E. A revised view of sensory cortical parcellation. Proc. Natl. Acad. Sci. U. S. A. 2004, 101:2167-2172.
118. Wallace M.T., Roberson G.E., Hairston W.D., Stein B.E., Vaughan J.W., Schirillo J.A. Unifying multisensory signals across time and space. Exp. Brain Res. 2004, 158(2):252-258.
119. Watkins S., Shams L., Tanaka S., Haynes J.D., Rees G. Sound alters activity in human V1 in association with illusory visual perception. NeuroImage 2006, 31(3):1247-1256.
120. Watkins S., Shams L., Josephs O., Rees G. Activity in human V1 follows multisensory perception. NeuroImage 2007, 37(2):572-578.
121. Welch R.B. Meaning, attention, and the "unity assumption" in the intersensory bias of spatial and temporal perceptions. Cognitive Contributions to the Perception of Spatial and Temporal Events 1999, 371-387. Elsevier, Amsterdam. G. Aschersleben, T. Bachmann, J. Müsseler (Eds.).
122. Welch R.B., Warren D.H. Immediate perceptual response to intersensory discrepancy. Psychol. Bull. 1980, 88:638-667.
123. Welch R., Duttenhurt L., Warren D. Contributions of audition and vision to temporal rate perception. Atten. Percept. Psychophys. 1986, 39(4):294-300.
124. Wendling F., Bartolomei F., Bellanger J.J., Chauvel P. Epileptic fast activity can be explained by a model of impaired GABAergic dendritic inhibition. Eur. J. Neurosci. 2002, 15(9):1499-1508.