Audio-visual synchrony modulates the ventriloquist illusion and its neural/spatial representation in the auditory cortex

NeuroImage

Volumn 98, Issue , 2014, Pages 425-434

  Bonath, Björn ^a   Noesselt, Toemme ^a,c   Krauel, Kerstin ^a   Tyll, Sascha ^a   Tempelmann, Claus ^a   Hillyard, Steven A ^b  

^a OTTO VON GUERICKE UNIVERSITY   (Germany)

^b UNIVERSITY OF CALIFORNIA   (United States)

^c Center for Behavioral and Brain Sciences   (Germany)

Author keywords

[No Author keywords available]

Indexed keywords

ACCURACY; ADULT; ARTICLE; AUDITORY CORTEX; AUDITORY STIMULATION; BOLD SIGNAL; CHRONOLOGY; FEMALE; FUNCTIONAL MAGNETIC RESONANCE IMAGING; HEARING; HUMAN; HUMAN EXPERIMENT; MALE; PLANUM TEMPORALE; PRIORITY JOURNAL; SOUND DETECTION; SOUND RELATED PHENOMENA; VENTRILOQUIST ILLUSION; VISUAL STIMULATION; BRAIN MAPPING; ILLUSION; NUCLEAR MAGNETIC RESONANCE IMAGING; PHOTOSTIMULATION; PHYSIOLOGY; VISION; YOUNG ADULT;

ACOUSTIC STIMULATION; ADULT; AUDITORY CORTEX; BRAIN MAPPING; FEMALE; HUMANS; ILLUSIONS; MAGNETIC RESONANCE IMAGING; MALE; PHOTIC STIMULATION; SOUND LOCALIZATION; VISUAL PERCEPTION; YOUNG ADULT;

EID: 84903575981     PISSN: 10538119     EISSN: 10959572     Source Type: Journal    
DOI: 10.1016/j.neuroimage.2014.04.077     Document Type: Article

References (55)

1. Ahissar M., Ahissar E., Bergman H., Vaadia E. Encoding of sound-source location and movement: activity of single neurons and interactions between adjacent neurons in the monkey auditory cortex. J. Neurophysiol. 1992, 67:203-215.
2. Altmann C.F., Bledowski C., Wibral M., Kaiser J. Processing of location and pattern changes of natural sounds in the human auditory cortex. Neuroimage 2007, 35:1192-1200.
3. Barrett D.J., Hall D.A. Response preferences for "what" and "where" in human non-primary auditory cortex. Neuroimage 2006, 32:968-977.
4. Bertelson P. Ventriloquism: A case of cross-modal perceptual grouping. Cognitive contributions to the perception of spatial and temporal events 1999, 347-362. Elsevier, Amsterdam. G. Aschersleben, T. Bachmann, J. Müsseler (Eds.).
5. Bischoff M., Walter B., Blecker C.R., Morgen K., Vaitl D., Sammer G. Utilizing the ventriloquist-effect to investigate audio-visual binding. Neuropsychologia 2007, 45:578-586.
6. Bonath B., Noesselt T., Martinez A., Mishra J., Schwiecker K., Heinze H.J., Hillyard S.A. Neural basis of the ventriloquist illusion. Curr. Biol. 2007, 17:1697-1703.
7. Bruns P., Röder B. Tactile capture of auditory localization: an event-related potential study. Eur. J. Neurosci. 2010, 31:1844-1857.
8. Budinger E., Scheich H. Anatomical connections suitable for the direct processing of neuronal information of different modalities via the rodent primary auditory cortex. Hear. Res. 2009, 258:16-27.
9. Busse L., Roberts K.C., Crist R.E., Weissman D.H., Woldorff M.G. The spread of attention across modalities and space in a multisensory object. Proc. Natl. Acad. Sci. U. S. A. 2005, 102:18751-18756.
10. Callan A., Callan D.E., Ando H. Neural correlates of sound externalization. Neuroimage 2013, 66(1):22-27.
11. Camalier C.R., D'Angelo W.R., Sterbing-D'Angelo S.J., de la Mothe L.A., Hackett T.A. Neural latencies across auditory cortex of macaque support a dorsal stream supramodal timing advantage in primates. Proc. Natl. Acad. Sci. U. S. A. 2012, 109:18168-18173.
12. Colonius H., Diederich A. Models of the Time Window of Integration. The New Handbook of Multisensory Processes 2012, 545-555. The MIT Press, Cambridge, MA. B.E. Stein (Ed.).
13. Deouell L.Y., Heller A.S., Malach R., D'Esposito M., Knight R.T. Cerebral responses to change in spatial location of unattended sounds. Neuron 2007, 55:985-996.
14. Driver J., Noesselt T. Multisensory interplay reveals crossmodal influences on 'sensory-specific' brain regions, neural responses, and judgments. Neuron 2008, 57:11-23.
15. Falchier A., Clavagnier S., Barone P., Kennedy H. Anatomical evidence of multimodal integration in primate striate cortex. J. Neurosci. 2002, 22:5749-5759.
16. Ghazanfar A.A., Lemus L. Multisensory integration: vision boosts information through suppression in auditory cortex. Curr. Biol. 2010, 20:R22-R23.
17. Glascher J. Visualization of group inference data in functional neuroimaging. Neuroinformatics 2009, 7:73-82.
18. Gondan M., Niederhaus B., Rosler F., Roder B. Multisensory processing in the redundant-target effect: a behavioral and event-related potential study. Percept. Psychophys. 2005, 67:713-726.
19. Harrington I.A., Stecker G.C., Macpherson E.A., Middlebrooks J.C. Spatial sensitivity of neurons in the anterior, posterior, and primary fields of cat auditory cortex. Hear. Res. 2008, 240:22-41.
20. Hinrichs H., Scholz M., Tempelmann C., Woldorff M.G., Dale A.M., Heinze H.J. Deconvolution of event-related fMRI responses in fast-rate experimental designs: tracking amplitude variations. J. Cogn. Neurosci. 2000, 12(Suppl. 2):76-89.
21. Hoefer M., Tyll S., Kanowski M., Brosch M., Schoenfeld M.A., Heinze H.-J., Noesselt T. Tactile stimulation and hemispheric asymmetries modulate auditory perception and neural responses in primary auditory cortex. Neuroimage 2013, 79:371-382.
22. Inui K., Okamoto H., Miki K., Gunji A., Kakigi R. Serial and parallel processing in the human auditory cortex: a magnetoencephalographic study. Cereb. Cortex 2005, 16:18-30.
23. Johnson B.W., Hautus M.J. Processing of binaural spatial information in human auditory cortex: neuromagnetic responses to interaural timing and level differences. Neuropsychologia 2010, 48:2610-2619.
24. Kanowski M., Rieger J.W., Noesselt T., Tempelmann C., Hinrichs H. Endoscopic eye tracking system for fMRI. J. Neurosci. Methods 2007, 160:10-15.
25. Kayser C., Petkov C.I., Augath M., Logothetis N.K. Functional imaging reveals visual modulation of specific fields in auditory cortex. J. Neurosci. 2007, 27:1824-1835.
26. Kayser C., Petkov C.I., Logothetis N.K. Visual modulation of neurons in auditory cortex. Cereb. Cortex 2008, 18:1560-1574.
27. Kayser C., Logothetis N.K., Panzeri S. Visual enhancement of the information representation in auditory cortex. Curr. Biol. 2009, 2009:19-24.
28. Kriegeskorte N., Simmons W.K., Bellgowan P.S., Baker C.I. Circular analysis in systems neuroscience: the dangers of double dipping. Nat. Neurosci. 2009, 12:535-540.
29. Krumbholz K., Schonwiesner M., von Cramon D.Y., Rubsamen R., Shah N.J., Zilles K., Fink G.R. Representation of interaural temporal information from left and right auditory space in the human planum temporale and inferior parietal lobe. Cereb. Cortex 2005, 15:317-324.
30. Lakatos P., Chen C.M., O'Connell M.N., Mills A., Schroeder C.E. Neuronal oscillations and multisensory interaction in primary auditory cortex. Neuron 2007, 53:279-292.
31. Laurienti P.J., Burdette J.H., Wallace M.T., Yen Y.F., Field A.S., Stein B.E. Deactivation of sensory specific cortex by cross-modal stimuli. J. Cogn. Neurosci. 2002, 14:420-429.
32. Lehmann C., Herdener M., Esposito F., Hubl D., di Salle F., Scheffler K., Bach D.R., Federspiel A., Kretz R., Mai K.J., Paxinos G., Voss T. Atlas of the Human Brain 2008, Elsevier Academic Press. 3rd ed.
33. Malhotra S., Lomber S.G. Sound localization during homotopic and heterotopic bilateral cooling deactivation of primary and nonprimary auditory cortical areas in the cat. J. Neurophysiol. 2007, 97:26-43.
34. McEvoy L., Hari R., Imada T., Sams M. Human auditory cortical mechanisms of sound lateralization: II. Interaural time differences at sound onset. Hear. Res. 1993, 67:98-109.
35. Murray M.M., Molholm S., Michel C.M., Heslenfeld D.J., Ritter W., Javitt D.C., Schroeder C.E., Foxe J.J. Grabbing your ear: rapid auditory-somatosensory multisensory interactions in low-level sensory cortices are not constrained by stimulus alignment. Cereb. Cortex 2005, 15:963-974.
36. Noesselt T., Rieger J.W., Schoenfeld M.A., Kanowski M., Hinrichs H., Heinze H.J., Driver J. Audiovisual temporal correspondence modulates human multisensory superior temporal sulcus plus primary sensory cortices. J. Neurosci. 2007, 27:11431-11441.
37. Noesselt T., Tyll S., Böhler N.C., Budinger E., Heinze H.J., Driver J. Sound-induced enhancement of low-intensity vision: multisensory influences on human sensory-specific cortices and thalamic bodies relate to perceptual enhancement of visual detection sensitivity. J. Neurosci. 2010, 30(41):13609-13623.
38. Noesselt T., Bergmann D., Heinze H.-J., Münte T., Spence C. Codingofmultisensorytemporal patternsinhumansuperiortempo- ralsulcus. Front. Integr. Neurosci. 2012, 6:64. 10.3389/fnint.2012.00064.
39. Noesselt T., Bergmann D., Heinze H.-J., Münte T., Spence C. Coding of multisensory temporal patterns in human superior temporal sulcus. Front. Integr. Neurosci. 2013, 6:64.
40. Occelli V., Bruns P., Zampini M., Röder B. Audiotactile integration is reduced in congenital blindness in a spatial ventriloquism task. Neuropsychologia 2012, 50:36-43.
41. Palomaki K.J., Tiitinen H., Makinen V., May P.J., Alku P. Spatial processing in human auditory cortex: the effects of 3D, ITD, and ILD stimulation techniques. Brain Res. Cogn. Brain Res. 2005, 24:364-379.
42. Sams M., Hamalainen M., Hari R., McEvoy L. Human auditory cortical mechanisms of sound lateralization: I. Interaural time differences within sound. Hear. Res. 1993, 67:89-97.
43. Schroeder C.E., Smiley J., Fu K.G., McGinnis T., O'Connell M.N., Hackett T.A. Anatomical mechanisms and functional implications of multisensory convergence in early cortical processing. Int. J. Psychophysiol. 2003, 50:5-17.
44. Senkowski D., Talsma D., Grigutsch M., Herrmann C.S., Woldorff M.G. Good times for multisensory integration: effects of the precision of temporal synchrony as revealed by gamma-band oscillations. Neuropsychologia 2007, 45:561-571.
45. Shulman G.L., Corbetta M., Buckner R.L., Raichle M.E., Fiez J.A., Miezin F.M., Petersen S.E. Top-down modulation of early sensory cortex. Cereb. Cortex 1997, 7:193-206.
46. Slutsky D.A., Recanzone G.H. Temporal and spatial dependency of the ventriloquism effect. Neuroreport 2001, 12:7-10.
47. Stecker G.C., Harrington I.A., Middlebrooks J.C. Location coding by opponent neural populations in the auditory cortex. PLoS Biol. 2005, 3:e78.
48. Teder-Salejarvi W.A., Di Russo F., McDonald J.J., Hillyard S.A. Effects of spatial congruity on audiovisual multimodal integration. J. Cogn. Neurosci. 2005, 17:1396-1409.
49. Tiitinen H., Salminen N.H., Palomaki K.J., Makinen V.T., Alku P., May P.J. Neuromagnetic recordings reveal the temporal dynamics of auditory spatial processing in the human cortex. Neurosci. Lett. 2006, 396:17-22.
50. van Atteveldt N.M., Formisano E., Blomert L., Goebel R. The effect of temporal asynchrony on the integration of letters and speech sounds. Cereb. Cortex 2007, 17:962-974.
51. van Eijk R.L., Kohlrausch A., Juola J.F., van de Par S. Audiovisual synchrony and temporal order judgments: effects of experimental method and stimulus type. Percept. Psychophys. 2008, 70:955-968.
52. Vroomen J., de Gelder B. Perceptual effects of cross-modal stimulation: ventriloquism and the freezing phenomenon. The Handbook of Multisensory Processes 2004, 35-48. The MIT Press, Cambridge, MA. G.A. Calvert, C. Spence, B.E. Stein (Eds.).
53. Warren J.D., Griffiths T.D. Distinct mechanisms for processing spatial sequences and pitch sequences in the human auditory brain. J. Neurosci. 2003, 23:5799-5804.
54. Werner-Reiss U., Groh J.M. A rate code for sound azimuth in monkey auditory cortex: implications for human neuroimaging studies. J. Neurosci. 2008, 28:3747-3758.
55. Zatorre R.J., Bouffard M., Ahad P., Belin P. Where is 'where' in the human auditory cortex?. Nat. Neurosci. 2002, 5:905-909.