Auditory modulation of visual stimulus encoding in human retinotopic cortex

NeuroImage

Volumn 70, Issue , 2013, Pages 258-267

  De Haas, Benjamin ^a,b   Schwarzkopf, D Samuel ^a,b   Urner, Maren ^a,b   Rees, Geraint ^a,b  

^a UNIVERSITY COLLEGE LONDON   (United Kingdom)

^b UNIVERSITY COLLEGE LONDON   (United Kingdom)

Author keywords

Audio visual; Decoding; FMRI; Multisensory; MVPA; V2

Indexed keywords

ADULT; AMPLITUDE MODULATION; ARTICLE; AUDITORY CORTEX; AUDITORY FEEDBACK; AUDITORY STIMULATION; BOLD SIGNAL; BRAIN RADIOGRAPHY; EVOKED AUDITORY RESPONSE; EVOKED VISUAL RESPONSE; FEMALE; FUNCTIONAL MAGNETIC RESONANCE IMAGING; FUNCTIONAL NEUROIMAGING; HUMAN; HUMAN EXPERIMENT; MALE; NEUROMODULATION; NOISE; NORMAL HUMAN; PRIORITY JOURNAL; RETINOTOPIC CORTEX; SECONDARY VISUAL CORTEX; SOUND DETECTION; SUPERIOR TEMPORAL GYRUS; VISUAL AREA V3; VISUAL CORTEX; VISUAL STIMULATION;

ACOUSTIC STIMULATION; ADULT; AUDITORY PERCEPTION; FEMALE; HUMANS; MAGNETIC RESONANCE IMAGING; MALE; MULTIVARIATE ANALYSIS; PHOTIC STIMULATION; VISUAL CORTEX;

EID: 84872723012     PISSN: 10538119     EISSN: 10959572     Source Type: Journal    
DOI: 10.1016/j.neuroimage.2012.12.061     Document Type: Article

References (55)

1. Alink A., Euler F., Kriegeskorte N., Singer W., Kohler A. Auditory motion direction encoding in auditory cortex and high-level visual cortex. Hum. Brain Mapp. 2012, 33:969-978.
2. Allman B.L., Meredith M.A. Multisensory processing in "unimodal" neurons: cross-modal subthreshold auditory effects in cat extrastriate visual cortex. J. Neurophysiol. 2007, 98:545-549.
3. Allman B.L., Bittencourt-Navarrete R.E., Keniston L.P., Medina A.E., Wang M.Y., Meredith M.A. Do cross-modal projections always result in multisensory integration?. Cereb. Cortex 2008, 18:2066-2076.
4. Allman B.L., Keniston A.L.P., Meredith M.A. Not just for bimodal neurons anymore: the contribution of unimodal neurons to cortical multisensory processing. Brain Topogr. 2009, 157-167.
5. Beauchamp M.S. Statistical criteria in FMRI studies of multisensory integration. Neuroinformatics 2005, 3:93-113.
6. Brainard D.H. The psychophysics toolbox. Spat. Vis. 1997, 10:433-436.
7. Budinger E., Heil P., Hess A., Scheich H. Multisensory processing via early cortical stages: connections of the primary auditory cortical field with other sensory systems. Neuroscience 2006, 143:1065-1083.
8. Cappe C., Thut G., Romei V., Murray M.M. Auditory-visual multisensory interactions in humans: timing, topography, directionality, and sources. J. Neurosci. 2010, 30:12572-12580.
9. Cardoso M.M.B., Sirotin Y.B., Lima B., Glushenkova E., Das A. The neuroimaging signal is a linear sum of neurally distinct stimulus- and task-related components. Nat. Neurosci. 2012, 15:1298-1306.
10. Clavagnier S., Falchier A., Kennedy H. Long-distance feedback projections to area V1: implications for multisensory integration, spatial awareness, and visual consciousness. Cogn. Affect. Behav. Neurosci. 2004, 4:117-126.
11. Dahl C.D., Logothetis N.K., Kayser C. Modulation of visual responses in the superior temporal sulcus by audio-visual congruency. Front. Integr. Neurosci. 2010, 4:10.
12. Deichmann R., Schwarzbauer C., Turner R. Optimisation of the 3D MDEFT sequence for anatomical brain imaging: technical implications at 1.5 and 3T. Neuroimage 2004, 21:757-767.
13. Destrieux C., Fischl B., Dale A., Halgren E. Automatic parcellation of human cortical gyri and sulci using standard anatomical nomenclature. Neuroimage 2010, 53:1-15.
14. Driver J., Noesselt T. Multisensory interplay reveals crossmodal influences on "sensory-specific" brain regions, neural responses, and judgments. Neuron 2008, 57:11-23.
15. Ernst M.O., Banks M.S. Humans integrate visual and haptic information in a statistically optimal fashion. Nature 2002, 415:429-433.
16. Falchier A., Clavagnier S., Barone P., Kennedy H. Anatomical evidence of multimodal integration in primate striate cortex. J. Neurosci. 2002, 22:5749-5759.
17. Fetsch C.R., Pouget A., DeAngelis G.C., Angelaki D.E. Neural correlates of reliability-based cue weighting during multisensory integration. Nat. Neurosci. 2012, 15:146-154.
18. Fishman M.C., Michael P. Integration of auditory information in the cat's visual cortex. Vision Res. 1973, 13:1415-1419.
19. Formisano E., De Martino F., Bonte M., Goebel R. "Who" is saying "what"? Brain-based decoding of human voice and speech. Science 2008, 970-973.
20. Friston K. The free-energy principle: a rough guide to the brain?. Trends Cogn. Sci. 2009, 13:293-301.
21. Friston K.J. What does functional MRI measure? Two complementary perspectives. Trends Cogn. Sci. 2012, 16:491-492.
22. Friston K.J., Buechel C., Fink G.R., Morris J., Rolls E., Dolan R.J. Psychophysiological and modulatory interactions in neuroimaging. Neuroimage 1997, 6:218-229.
23. Giani A.S., Ortiz E., Belardinelli P., Kleiner M., Preissl H., Noppeney U. Steady-state responses in MEG demonstrate information integration within but not across the auditory and visual senses. Neuroimage 2012, 60:1478-1489.
24. Hsieh P.-J., Colas J.T., Kanwisher N. Spatial pattern of BOLD fMRI activation reveals cross-modal information in auditory cortex. J. Neurophysiol. 2012, 107:3428-3432.
25. Hsu C.-W., Lin C.-J. A comparison of methods for multiclass support vector machines. EEE Trans. Neural Netw. 2002, 13:415-425.
26. Iurilli G., Ghezzi D., Olcese U., Lassi G., Nazzaro C., Tonini R., Tucci V., Benfenati F., Medini P. Sound-driven synaptic inhibition in primary visual cortex. Neuron 2012, 73:814-828.
27. Jones M.R. Time, our lost dimension: toward a new theory of perception, attention, and memory. Psychol. Rev. 1976, 83(5):323-355. (Retrieved from http://www.ncbi. Psychological review 83, 323-55).
28. Kayser C., Logothetis N.K., Panzeri S. Visual enhancement of the information representation in auditory cortex. Curr. Biol. 2010, 20:19-24.
29. Klemen J., Chambers C.D. Current perspectives and methods in studying neural mechanisms of multisensory interactions. Neurosci. Biobehav. Rev. 2012, 36:111-133.
30. Kriegeskorte N., Goebel R., Bandettini P. Information-based functional brain mapping. Proc. Natl. Acad. Sci. U. S. A. 2006, 103:3863-3868.
31. Lakatos P., Chen C.-M., O'Connell M.N., Mills A., Schroeder C.E. Neuronal oscillations and multisensory interaction in primary auditory cortex. Neuron 2007, 53:279-292.
32. Lakatos P., O'Connell M.N., Barczak A., Mills A., Javitt D.C., Schroeder C.E. The leading sense: supramodal control of neurophysiological context by attention. Neuron 2009, 64:419-430.
33. Large E., Jones M. The dynamics of attending: how people track time-varying events. Psychol. Rev. 1999, 106:119-159.
34. Larsson J., Heeger D.J. Two retinotopic visual areas in human lateral occipital cortex. J. Neurosci. 2006, 26:13128-13142.
35. Luo H., Liu Z., Poeppel D. Auditory cortex tracks both auditory and visual stimulus dynamics using low-frequency neuronal phase modulation. PLoS Biol. 2010, 8:e1000445.
36. Macaluso E., Frith C.D., Driver J. Modulation of human visual cortex by crossmodal spatial attention. Science 2000, 289:1206-1208.
37. Martuzzi R., Murray M.M., Michel C.M., Thiran J.-P., Maeder P.P., Clarke S., Meuli R.A. Multisensory interactions within human primary cortices revealed by BOLD dynamics. Cereb. Cortex 2007, 17:1672-1679.
38. Meienbrock A., Naumer M.J., Doehrmann O., Singer W., Muckli L. Retinotopic effects during spatial audio-visual integration. Neuropsychologia 2007, 45:531-539.
39. Mesulam M.M. From sensation to cognition. Brain 1998, 121(Pt 6):1013-1052.
40. Meyer K., Kaplan J.T., Essex R., Webber C., Damasio H., Damasio A. Predicting visual stimuli on the basis of activity in auditory cortices. Nat. Neurosci. 2010, 13:667-668.
41. Molholm S., Ritter W., Murray M.M., Javitt D.C., Schroeder C.E., Foxe J.J. Multisensory auditory-visual interactions during early sensory processing in humans: a high-density electrical mapping study. Brain Res. Cogn. Brain Res. 2002, 14:115-128.
42. Morey R.D. Confidence intervals from normalized data: a correction to Cousineau (2005). Tutor. Quant. Methods Psychol. 2008, 4:61-64.
43. Naue N., Rach S., Strüber D., Huster R.J., Zaehle T., Körner U., Herrmann C.S. Auditory event-related response in visual cortex modulates subsequent visual responses in humans. J. Neurosci. 2011, 31:7729-7736.
44. Nishimoto S., Vu A.T., Naselaris T., Benjamini Y., Yu B., Gallant J.L. Reconstructing visual experiences from brain activity evoked by natural movies. Curr. Biol. 2011, 21:1641-1646.
45. Noesselt T., Rieger J.W., Schoenfeld M.A., Kanowski M., Hinrichs H., Heinze H.-J., Driver J. Audiovisual temporal correspondence modulates human multisensory superior temporal sulcus plus primary sensory cortices. J. Neurosci. 2007, 27:11431-11441.
46. Pelli D.G. The VideoToolbox software for visual psychophysics: transforming numbers into movies. Spat. Vis. 1997, 10:437-442.
47. Rockland K.S., Ojima H. Multisensory convergence in calcarine visual areas in macaque monkey. Int. J. Psychophysiol. 2003, 50:19-26.
48. Romei V., Murray M.M., Cappe C., Thut G. Preperceptual and stimulus-selective enhancement of low-level human visual cortex excitability by sounds. Curr. Biol. 2009, 19:1799-1805.
49. Romei V., Gross J., Thut G. Sounds reset rhythms of visual cortex and corresponding human visual perception. Curr. Biol. 2012, 22:807-813.
50. Schroeder C.E., Lakatos P., Kajikawa Y., Partan S., Puce A. Neuronal oscillations and visual amplification of speech. Trends Cogn. Sci. 2008, 12:106-113.
51. Sereno M.I., Dale A.M., Reppas J.B., Kwong K.K., Belliveau J.W., Brady T.J., Rosen B.R., Tootell R.B. Borders of multiple visual areas in humans revealed by functional magnetic resonance imaging. Science 1995, 268:889-893.
52. Staeren N., Renvall H., De Martino F., Goebel R., Formisano E. Sound categories are represented as distributed patterns in the human auditory cortex. Curr. Biol. 2009, 19:498-502.
53. Thorne J.D., De Vos M., Viola F.C., Debener S. Cross-modal phase reset predicts auditory task performance in humans. J. Neurosci. 2011, 31:3853-3861.
54. Watkins S., Shams L., Tanaka S., Haynes J.D., Rees G. Sound alters activity in human V1 in association with illusory visual perception. Neuroimage 2006, 31:1247-1256.
55. Werner S., Noppeney U. Distinct functional contributions of primary sensory and association areas to audiovisual integration in object categorization. J. Neurosci. 2010, 30:2662-2675.