Early cross-modal interactions in auditory and visual cortex underlie a sound-induced visual illusion

Journal of Neuroscience

Volumn 27, Issue 15, 2007, Pages 4120-4131

  Mishra, Jyoti ^a   Martinez, Antigona ^b,c   Sejnowski, Terrence J ^a,d   Hillyard, Steven A ^b,e  

^a UNIVERSITY OF CALIFORNIA SAN DIEGO   (United States)

^b UNIVERSITY OF CALIFORNIA   (United States)

^c NATHAN S KLINE INSTITUTE FOR PSYCHIATRIC RESEARCH   (United States)

^d HOWARD HUGHES MEDICAL INSTITUTE   (United States)

^e UNIVERSITY OF CALIFORNIA   (United States)

Author keywords

Auditory cortex; Cross modal interaction; ERPs; Illusory flash; Source analysis; Visual cortex

Indexed keywords

ADULT; ARTICLE; AUDITORY CORTEX; CONTROLLED STUDY; EVENT RELATED POTENTIAL; FEMALE; HUMAN; HUMAN EXPERIMENT; MALE; PRIORITY JOURNAL; SOUND TRANSMISSION; VISUAL CORTEX; VISUAL ILLUSION; VISUAL STIMULATION;

ACOUSTIC STIMULATION; AUDITORY CORTEX; AUDITORY PERCEPTION; EVOKED POTENTIALS; FEMALE; HUMANS; ILLUSIONS; MALE; PHOTIC STIMULATION; PSYCHOMOTOR PERFORMANCE; REACTION TIME; VISUAL CORTEX;

EID: 34247134672     PISSN: 02706474     EISSN: 02706474     Source Type: Journal    
DOI: 10.1523/JNEUROSCI.4912-06.2007     Document Type: Article

References (52)

1. Andersen TS, Tiippana K, Sams M (2004) Factors influencing audiovisual fission and fusion illusions. Brain Res Cogn Brain Res 21:301-308.
2. Arden GB, Wolf JE, Messiter C (2003) Electrical activity in visual cortex associated with combined auditory and visual stimulation in temporal sequences known to be associated with a visual illusion. Vision Res 43:2469-2478.
3. Baier B, Kleinschmidt A, Muller NG (2006) Cross-modal processing in early visual and auditory cortices depends on expected statistical relationship of multisensory information. J Neurosci 26:12260-12265.
4. Bavelier D, Neville HJ (2002) Cross-modal plasticity: where and how? Nat Rev Neurosci 3:443-452.
5. Bhattacharya J, Shams L, Shimojo S (2002) Sound-induced illusory flash perception: role of gamma band responses. NeuroReport 13:1727-1730.
6. Busse L, Roberts KC, Crist RE, Weissman DH, Woldorff MG (2005) The spread of attention across modalities and space in a multisensory object. Proc Natl Acad Sci USA 102:18751-18756.
7. Calvert GA (2001) Crossmodal processing in the human brain: insights from functional neuroimaging studies. Cereb Cortex 11:1110-1123.
8. Calvert GA, Thesen T (2004) Multisensory integration: methodological approaches and emerging principles in the human brain. J Physiol (Paris) 98:191-205.
9. Clark VP, Hillyard SA (1996) Spatial selective attention affects early extrastriate but not striate components of the visual evoked potential. J Cogn Neurosci 8:387-402.
10. Clavagnier S, Falchier A, Kennedy H (2004) Long-distance feedback projections to area V1: implications for multisensory integration, spatial awareness, and visual consciousness. Cogn Affect Behav Neurosci 4:117-126.
11. Cox RW (1996) AFNI: software for analysis and visualization of functional magnetic resonance neuroimages. Comput Biomed Res 29:162-173.
12. Di Russo F, Martinez A, Sereno MI, Pitzalis S, Hillyard SA (2002) Cortical sources of the early components of the visual evoked potential. Hum Brain Mapp 15:95-111.
13. Di Russo F, Martinez A, Hillyard SA (2003) Source analysis of event-related cortical activity during visuo-spatial attention. Cereb Cortex 13:486-499.
14. Falchier A, Clavagnier S, Barone P, KennedyH (2002) Anatomical evidence of multimodal integration in primate striate cortex. J Neurosci 22:5749-5759.
15. Fendrich R, Corballis PM (2001) The temporal cross-capture of audition and vision. Percept Psychophys 63:719-725.
16. Ghazanfar AA, Schroeder CE (2006) Is neocortex essentially multisensory? Trends Cogn Sci 10:278-285.
17. Giard MH, Peronnet F (1999) Auditory-visual integration during multimodal object recognition in humans: a behavioral and electrophysiological study. J Cogn Neurosci 11:473-490.
18. Gondan M, Röder B (2006) A new method for detecting interactions between the senses in event-related potentials. Brain Res 1073-1074:389-397.
19. Grave de Peralta R, Gonzalez Andino S, Lantz G, Michel CM, Landis T (2001) Noninvasive localization of electromagnetic epileptic activity. I. Method descriptions and simulations. Brain Topogr 14:131-137.
20. Johnson JA, Zatorre RJ (2006) Neural substrates for dividing and focusing attention between simultaneous auditory and visual events. NeuroImage 31:1673-1681.
21. Kiebel SJ, Tallon-Baudry C, Friston KJ (2005) Parametric analysis of oscillatory activity as measured with EEG/MEG. Hum Brain Mapp 26:170-177.
22. Lakatos P, Shah AS, Knuth KH, Ulbert I, Karmos G, Schroeder CE (2005) An oscillatory hierarchy controlling neuronal excitability and stimulus processing in the auditory cortex. J Neurophysiol 94:1904-1911.
23. Lütkenhöner B, Steinsträter O (1998) High-precision neuromagnetic study of the functional organization of the human auditory cortex. Audiol Neurootol 3:191-213.
24. Macaluso E (2006) Multisensory processing in sensory-specific cortical areas. The Neuroscientist 12:327-338.
25. Martuzzi R, Murray MM, Michel CM, Thiran JP, Maeder PP, Clarke S, Meuli RA (2006) Multisensory interactions within human primary cortices revealed by BOLD dynamics. Cereb Cortex, in press.
26. McCarthy G, Wood CC (1985) Scalp distributions of event-related potentials: an ambiguity associated with analysis of variance models. Electroencephalogr Clin Neurophysiol 62:203-208.
27. McCormick D, Mamassian P (2006) What does the illusory flash look like? J Vision 6:189a.
28. McDonald JJ, Teder-Sälejärvi WA, Di Russo F, Hillyard SA (2003) Neural substrates of perceptual enhancement by cross-modal spatial attention. J Cogn Neurosci 15:10-19.
29. McDonald JJ, Teder-Sälejärvi WA, Di Russo F, Hillyard SA (2005) Neural basis of auditory-induced shifts in visual time-order perception. Nat Neurosci 8:1197-1202.
30. Meylan RV, Murray MM (2007) Auditory-visual multisensory interactions attenuate subsequent visual responses in humans. NeuroImage 35:244-254.
31. Michel CM, Thut G, Morand S, Khateb A, Pegna AJ, Grave de Peralta R, Gonzalez S, Seeck M, Landis T (2001) Electric source imaging of human brain functions. Brain Res Brain Res Rev 36:108-118.
32. Molholm S, Ritter W, Murray MM, Javitt DC, Schroeder CE, Foxe JJ (2002) Multisensory auditory-visual interactions during early sensory processing in humans: a high-density electrical mapping study. Brain Res Cogn Brain Res 14:115-128.
33. Murray MM, Molholm S, Michel CM, Heslenfeld DJ, Ritter W, Javitt DC, Schroeder CE, Foxe JJ (2005) Grabbing your ear: rapid auditory-somatosensory multisensory interactions in low-level sensory cortices are not constrained by stimulus alignment. Cereb Cortex 15:963-974.
34. Recanzone GH (2003) Auditory influences on visual temporal rate perception. J Neurophysiol 89:1078-1093.
35. Rockland KS, Ojima H (2003) Multisensory convergence in calcarine visual areas in macaque monkey. Int J Psychophysiol 50:19-26.
36. Rorden C, Brett M (2000) Stereotaxic display of brain lesions. Behav Neurol 12:191-200.
37. Schroeder CE, Foxe J (2005) Multisensory contributions to low-level, "unisensory" processing. Curr Opin Neurobiol 15:454-458.
38. Sekuler R, Sekuler AB, Lau R (1997) Sound alters visual motion perception. Nature 385:308.
39. Shams L, Kamitani Y, Shimojo S (2000) Illusions. What you see is what you hear. Nature 408:788.
40. Shams L, Kamitani Y, Thompson S, Shimojo S (2001) Sound alters visual evoked potentials in humans. NeuroReport 12:3849-3852.
41. Shams L, Kamitani Y, Shimojo S (2002) Visual illusion induced by sound. Brain Res Cogn Brain Res 14:147-152.
42. Shams L, Iwaki S, Chawla A, Bhattacharya J (2005a) Early modulation of visual cortex by sound: an MEG study. Neurosci Lett 378:76-81.
43. Shams L, Ma WJ, Beierholm U (2005b) Sound-induced flash illusion as an optimal percept. NeuroReport 16:1923-1927.
44. Stein BE, London R, Wilkinson LK, Price DD (1996) Enhancement of perceived visual intensity by auditory stimuli: a psychophysical analysis. J Cogn Neurosci 8:497-506.
45. Talairach J, Tournoux P (1988) Co-planar stereotaxic atlas of the human brain. New York: Thieme.
46. Tallon-Baudry C, Bertrand O, Peronnet F, Pernier J (1998) Induced gamma-band activity during the delay of a visual short-term memory task in humans. J Neurosci 18:4244-4254.
47. Talsma D, Woldorff MG (2005) Selective attention and multisensory integration: multiple phases of effects on the evoked brain activity. J Cogn Neurosci 17:1098-1114.
48. Talsma D, Doty TJ, Woldorff MG (2007) Selective attention and audiovisual integration: is attending to both modalities a prerequisite for early integration? Cereb Cortex 17:679-690.
49. Teder-Sälejärvi WA, McDonald JJ, Di Russo F, Hillyard SA (2002) An analysis of audio-visual crossmodal integration by means of event-related potential (ERP) recordings. Brain Res Cogn Brain Res 14:106-114.
50. Teder-Sälejärvi WA, Di Russo F, McDonald JJ, Hillyard SA (2005) Effects of spatial congruity on audio-visual multimodal integration. J Cogn Neurosci 17:1396-1409.
51. Vroomen J, de Gelder B (2004) Temporal ventriloquism: sound modulates the flash-lag effect. J Exp Psychol Hum Percept Perform 30:513-518.
52. Watkins S, Shams L, Tanaka S, Haynes JD, Rees G (2006) Sound alters activity in human V1 in association with illusory visual perception. NeuroImage 31:1247-1256.