Contextual control of audiovisual integration in low-level sensory cortices

Human Brain Mapping

Volumn 35, Issue 5, 2014, Pages 2394-2411

  van Atteveldt, Nienke M ^a,b,c   Peterson, Bradley S ^c   Schroeder, Charles E ^c,d  

^a MAASTRICHT UNIVERSITY   (Netherlands)

^b NETHERLANDS INSTITUTE FOR NEUROSCIENCE   (Netherlands)

^c NEW YORK STATE PSYCHIATRIC INSTITUTE   (United States)

^d NATHAN S KLINE INSTITUTE FOR PSYCHIATRIC RESEARCH   (United States)

Author keywords

Auditory cortex; Eccentricity; fMRI; Multisensory; Task demand; Visual cortex

Indexed keywords

ADULT; ARTICLE; AUDITORY CORTEX; CLINICAL ARTICLE; CONTROLLED STUDY; EYE POSITION; FEEDBACK SYSTEM; FEMALE; FUNCTIONAL MAGNETIC RESONANCE IMAGING; HUMAN; HYPOTHESIS; MALE; PRIORITY JOURNAL; SENSORIMOTOR INTEGRATION; VELOCITY; VISION; VISUAL CORTEX; VISUAL FIELD; VISUAL STIMULATION; AUDITORY STIMULATION; BLOOD; BRAIN CORTEX; BRAIN MAPPING; EYE TUMOR; HEARING; IMAGE PROCESSING; NUCLEAR MAGNETIC RESONANCE IMAGING; PHOTOSTIMULATION; PHYSIOLOGY; SENSORY NERVE; VASCULARIZATION; YOUNG ADULT;

OXYGEN;

ACOUSTIC STIMULATION; ADULT; AFFERENT PATHWAYS; AUDITORY PERCEPTION; BRAIN MAPPING; CEREBRAL CORTEX; EYE NEOPLASMS; FEMALE; HUMANS; IMAGE PROCESSING, COMPUTER-ASSISTED; MAGNETIC RESONANCE IMAGING; MALE; OXYGEN; PHOTIC STIMULATION; VISUAL PERCEPTION; YOUNG ADULT;

EID: 84898039544     PISSN: 10659471     EISSN: 10970193     Source Type: Journal    
DOI: 10.1002/hbm.22336     Document Type: Article

References (79)

1. Amedi A, von Kriegstein K, Van Atteveldt NM, Beauchamp MS, Naumer MJ (2005): Functional imaging of human crossmodal identification and object recognition. Exp Brain Res 166:559-571.
2. Barnett-Cowan M, Harris L (2009): Perceived timing of vestibular stimulation relative to touch, light and sound. Exp Brain Res 198:221-231.
3. Bavelier D, Dye MWG, Hauser PC (2006): Do deaf individuals see better? Trends Cogn Sci 10:512-518.
4. Beauchamp M (2005): Statistical criteria in fMRI studies of multisensory integration. Neuroinformatics 3:93-113.
5. Beauchamp M, Lee K, Argall B, Martin A (2004a): Integration of auditory and visual information about objects in superior temporal sulcus. Neuron 41:809-823.
6. Beauchamp MS, Argall BD, Bodurka J, Duyn J, Martin A (2004b): Unraveling multisensory integration: Patchy organization within human STS multisensory cortex. Nat Neurosci 7:1190-1192.
7. Bizley JK, Nodal FR, Bajo VM, Nelken I, King AJ (2007): Physiological and anatomical evidence for multisensory interactions in auditory cortex. Cereb Cortex 17:2172-2189.
8. Bolognini N, Senna I, Maravita A, Pascual-Leone A, Merabet LB (2010): Auditory enhancement of visual phosphene perception: The effect of temporal and spatial factors and of stimulus intensity. Neurosci Lett 477:109-114.
9. Budinger E, Heil P, Hess A, Scheich H (2006): Multisensory processing via early cortical stages: Connections of the primary auditory cortical field with other sensory systems. Neuroscience 143:1065-1083.
10. Calvert GA (2001): Crossmodal processing in the human brain: Insights from functional neuroimaging studies. Cereb Cortex 11:1110-1123.
11. Cappe C, Barone P (2005): Heteromodal connections supporting multisensory integration at low levels of cortical processing. Eur J Neurosci 22:2886-2901.
12. Cappe C, Morel A, Barone P, Rouiller EM (2009a): The thalamocortical projection systems in primate: An anatomical support for multisensory and sensorimotor interplay. Cereb Cortex 19:2025-2037.
13. Cappe C, Morel A, Rouiller EM (2007): Thalamocortical and the dual pattern of corticothalamic projections of the posterior parietal cortex in macaque monkeys. Neuroscience 146:1371-1387.
14. Cappe C, Thelen A, Romei V, Thut G, Murray MM (2012): Looming signals reveal synergistic principles of multisensory integration. J Neurosci 32:1171-1182.
15. Cappe C, Thut G, Romei V, Murray MM (2009b): Selective integration of auditory-visual looming cues by humans. Neuropsychologia 47:1045-1052.
16. Cappe C, Thut G, Romei V, Murray MM (2010): Auditory-visual multisensory interactions in humans: Timing, topography, directionality, and sources. J Neurosci 30:12572-12580.
17. Dahl CD, Logothetis NK, Kayser C (2009): Spatial organization of multisensory responses in temporal association cortex. J Neurosci 29:11924-11932.
18. Doehrmann O, Naumer MJ (2008): Semantics and the multisensory brain: How meaning modulates processes of audio-visual integration. Brain Res 1242:136-150.
19. Dougherty R, Koch V, Brewer A, Fischer B, Modersitzki J, Wandell B (2003): Visual field representations and locations of visual areas V1/2/3 in human visual cortex. J Vis 3:586-598.
20. Driver J, Noesselt T (2008): Multisensory interplay reveals crossmodal influences on 'sensory-specific' brain regions, neural responses, and judgments. Neuron 57:11-23.
21. Falchier A, Clavagnier S, Barone P, Kennedy H (2002): Anatomical evidence of multimodal integration in primate striate cortex. J Neurosci 22:5749-5759.
22. Falchier A, Schroeder CE, Hackett TA, Lakatos P, Nascimento-Silva S, Ulbert I, Karmos G, Smiley JF (2010): Projection from visual areas V2 and prostriata to caudal auditory cortex in the monkey. Cereb Cortex 20:1529-1538.
23. Forman SD, Cohen JD, Fitzgerald M, Eddy W, Mintun M, Noll DC (1995): Improved assessment of significant activation in functional magnetic resonance imaging (fMRI): Use of a cluster-size threshold. Magn Reson Med 33:636-647.
24. Fort A, Delpuech C, Pernier J, Giard MH (2002): Early auditory-visual interactions in human cortex during nonredundant target identification. Brain Res Cogn Brain Res 14:20-30.
25. Fuhrmann Alpert G, Hein G, Tsai N, Naumer M, Knight R (2008): Temporal characteristics of audiovisual information processing. J Neurosci 28:5344-5349.
26. Ghazanfar AA, Schroeder C (2006): Is neocortex essentially multisensory? Trends Cogn Sci 10:278-285.
27. Giard MH, Peronnet F (1999): Auditory-visual integration during multimodal object recognition in humans: A behavioral and electrophysiological study. J Cogn Neurosci 11:473-490.
28. Goebel R, Esposito F, Formisano E (2006): Analysis of Functional Image Analysis Contest (FIAC) data with BrainVoyager QX: From single-subject to cortically aligned group general linear model analysis and self-organizing group independent component analysis. Hum Brain Mapp 27:392-402.
29. Goebel R, van Atteveldt N (2009): Multisensory functional magnetic resonance imaging: A future perspective. Exp Brain Res 198:153-164.
30. Hackett T, Preuss T, Kaas J (2001): Architectonic identification of the core region in auditory cortex of macaques, chimpanzees, and humans. J Comp Neurol 441:197-222.
31. Hackett TA, De La Mothe LA, Ulbert I, Karmos G, Smiley J, Schroeder CE (2007): Multisensory convergence in auditory cortex. II. Thalamocortical connections of the caudal superior temporal plane. J Comp Neurol 502:924-952.
32. Hein G, Doehrmann O, Müller NG, Kaiser J, Muckli L, Naumer MJ (2007): Object familiarity and semantic congruency modulate responses in cortical audiovisual integration areas. J Neurosci 27:7881-7887.
33. Hein G, Knight R (2008): Superior temporal sulcus--It's my area: Or is it? J Cogn Neurosci 20:2125-2136.
34. Hocking J, Price CJ (2008): The role of the posterior superior temporal sulcus in audiovisual processing. Cereb Cortex 18:2439-2449.
35. Kayser C, Logothetis NK (2007): Do early sensory cortices integrate cross-modal information? Brain Struct Funct 212:121-132.
36. Kreifelts B, Ethofer T, Shiozawa T, Grodd W, Wildgruber D (2009): Cerebral representation of non-verbal emotional perception: fMRI reveals audiovisual integration area between voice- and face-sensitive regions in the superior temporal sulcus. Neuropsychologia 47:3059-3066.
37. Lakatos P, Chen CM, O'Connell MN, Mills A, Schroeder CE (2007): Neuronal oscillations and multisensory interaction in primary auditory cortex. Neuron 53:279-292.
38. Laurienti PJ, Kraft RA, Maldjian JA, Burdette JH, Wallace MT (2004): Semantic congruence is a critical factor in multisensory behavioral performance. Exp Brain Res 158:405-414.
39. Laurienti PJ, Perrault TJ, Stanford TR, Wallace MT, Stein BE (2005): On the use of superadditivity as a metric for characterizing multisensory integration in functional neuroimaging studies. Exp Brain Res 166:289-297.
40. Lee H, Truy E, Mamou G, Sappey-Marinier D, Giraud A (2007): Visual speech circuits in profound acquired deafness: A possible role for latent multimodal connectivity. Brain 130:2929-2941.
41. Martuzzi R, Murray M, Michel C, Thiran J, Maeder P, Clarke S, Meuli R (2007): Multisensory interactions within human primary cortices revealed by BOLD dynamics. Cereb Cortex 17:1672-1679.
42. Meienbrock A, Naumer M, Doehrmann O, Singer W, Muckli L (2007): Retinotopic effects during spatial audio-visual integration. Neuropsychologia 45:531-539.
43. Miller J (1982): Divided attention: Evidence for coactivation with redundant signals. Cogn Psychol 14:247-279.
44. Molholm S, Ritter W, Murray MM, Javitt DC, Schroeder CE, Foxe JJ (2002): Multisensory auditory-visual interactions during early sensory processing in humans: A high-density electrical mapping study. Cogn Brain Res 14:115-128.
45. Murray MM, Foxe JJ, Higgins BA, Javitt DC, Schroeder CE (2001): Visuo-spatial neural response interactions in early cortical processing during a simple reaction time task: A high-density electrical mapping study. Neuropsychologia 39:828-844.
46. Musacchia G, Schroeder C (2009): Neuronal mechanisms, response dynamics and perceptual functions of multisensory interactions in auditory cortex. Hear Res 258:72-79.
47. Naumer MJ, Doehrmann O, Mueller NG, Muckli L, Kaiser J, Hein G (2008): Cortical plasticity of audio-visual object representations. Cereb Cortex 19:1641-1653.
48. Neville H, Lawson D (1987): Attention to central and peripheral visual space in a movement detection task: An event-related potential and behavioral study. II. Congenitally deaf adults. Brain Res 405:268-283.
49. Ochiai T, Grimault S, Scavarda D, Roch G, Hori T, Rivière D, Mangin J, Régis J (2004): Sulcal pattern and morphology of the superior temporal sulcus. Neuroimage 22:706-719.
50. Perrault TJJ, Vaughan JW, Stein BE, Wallace MT (2005): Superior colliculus neurons use distinct operational modes in the integration of multisensory stimuli. J Neurophysiol 93:2575-2586.
51. Raij T, Ahveninen J, Lin F, Witzel T, Jääskeläinen I, Letham B, Israeli E, Sahyoun C, Vasios C, Stufflebeam S, Hämäläinen M, Belliveau J (2010): Onset timing of cross-sensory activations and multisensory interactions in auditory and visual sensory cortices. Eur J Neurosci 31:1772-1782.
52. Rockland KS, Ojima H (2003): Multisensory convergence in calcarine visual areas in macaque monkey. Int J Psychophysiol 50:19-26.
53. Röder B, Teder-Sälejärvi W, Sterr A, Rösler F, Hillyard S, Neville H (1999): Improved auditory spatial tuning in blind humans. Nature 400:162-166.
54. Romei V, Murray M, Cappe C, Thut G (2009): Preperceptual and stimulus-selective enhancement of low-level human visual cortex excitability by sounds. Curr Biol 19:1799-1805.
55. Romei V, Murray M, Cappe C, Thut G: The contributions of sensory dominance and attentional bias to cross-modal enhancement of visual cortex excitability. J Cogn Neurosci (in press).
56. Romei V, Murray MM, Merabet LB, Thut G (2007): Occipital transcranial magnetic stimulation has opposing effects on visual and auditory stimulus detection: Implications for multisensory interactions. J Neurosci 27:11465-11472.
57. Schneider T, Engel A, Debener S (2008): Multisensory identification of natural objects in a two-way crossmodal priming paradigm. Exp Psychol 55:121-132.
58. Schroeder CE, Molholm S, Lakatos P, Ritter W, Foxe J (2004): Human-simian correspondence in the early cortical processing of multisensory cues. Cogn Process 5:140-151.
59. Schroeder CE, Smiley JF, Fu KG, McGinnis T, O'Connel MN, Hackett TA (2003): Anatomical mechanisms and functional implications of multisensory convergence in early cortical processing. Int J Psychophysiol 50:5-17.
60. Shams L, Kamitani Y, Thompson S, Shimojo S (2001): Sound alters visual evoked potentials in humans. Exp Brain Res 12:3849-3852.
61. Smiley JF, Hackett TA, Ulbert I, Karmas G, Lakatos P, Javitt DC, Schroeder CE (2007): Multisensory convergence in auditory cortex I. Cortical connections of the caudal superior temporal plane in macaque monkeys. J Comp Neurol 502:894-923.
62. Sperdin HF, Cappe C, Foxe JJ, Murray MM (2009): Early, low-level auditory-somatosensory multisensory interactions impact reaction time speed. Front Integr Neurosci 3:1-10.
63. Stein BE, Meredith MA (1993): The Merging of the Senses. Cambridge, MA: MIT Press.
64. Stekelenburg J, Vroomen J (2007): Neural correlates of multisensory integration of ecologically valid audiovisual events. J Cogn Neurosci 19:1964-1973.
65. Stevenson R, Kim S, James T (2009): An additive-factors design to disambiguate neuronal and areal convergence: Measuring multisensory interactions between audio, visual, and haptic sensory streams using fMRI. Exp Brain Res 198:183-194.
66. Takeshima Y, Gyoba J (2013): High-intensity sound increases the size of visually perceived objects. Atten Percept Psychophys 75:501-507.
67. Taylor KI, Moss HE, Stamatakis EA, Tyler LK (2006): Binding crossmodal object features in perirhinal cortex. Proc Natl Acad Sci USA 102:8239-8244.
68. Van Atteveldt N, Formisano E, Goebel R, Blomert L (2004): Integration of letters and speech sounds in the human brain. Neuron 43:271-282.
69. Van Atteveldt NM, Formisano E, Blomert L, Goebel R (2007a): The effect of temporal asynchrony on the multisensory integration of letters and speech sounds. Cereb Cortex 17:962-974.
70. Van Atteveldt NM, Formisano E, Goebel R, Blomert L (2007b): Top-down task effects overrule automatic multisensory responses to letter-sound pairs in auditory association cortex. Neuroimage 36:1345-1360.
71. Van Atteveldt NM, Roebroeck A, Goebel R (2009): Interaction of speech and script in human auditory cortex: Insights from neuro-imaging and effective connectivity. Hear Res 258:152-164.
72. van Wassenhove V, Grant K, Poeppel D (2005): Visual speech speeds up the neural processing of auditory speech. Proc Natl Acad Sci 102:1181-1186.
73. von Kriegstein K, Griffiths T, Thompson S, McAlpine D (2008): Responses to interaural time delay in human cortex. J Neurophysiol 100:2712-2718.
74. Vroomen J, Stekelenburg J (2010): Visual anticipatory information modulates multisensory interactions of artificial audiovisual stimuli. J Cogn Neurosci 22:1583-1596.
75. Wang Y, Celebrini S, Trotter Y, Barone P (2008): Visuo-auditory interactions in the primary visual cortex of the behaving monkey: Electrophysiological evidence. BMC Neurosci 9:79.
76. Welford AT (1980): Choice reaction time: Basic concepts. In: Welford AT, editor. Reaction Times. New York: Academic Press. pp 73-128.
77. Werner S, Noppeney U (2010): Distinct functional contributions of primary sensory and association areas to audiovisual integration in object categorization. J Neurosci 30:2662-2675.
78. Woodworth RS, Schlosberg H (1954): Experimental Psychology. New York: Henry Holt.
79. Zhang N, Chen W (2006): A dynamic fMRI study of illusory double-flash effect on human visual cortex. Exp Brain Res 172:57-66.