Neuronal mechanisms, response dynamics and perceptual functions of multisensory interactions in auditory cortex

Hearing Research

Volumn 258, Issue 1-2, 2009, Pages 72-79

  Musacchia, Gabriella ^a   Schroeder, Charles E ^a,b  

^a NATHAN S KLINE INSTITUTE FOR PSYCHIATRIC RESEARCH   (United States)

^b Columbia University College of Physicians and Surgeons ^*  (United States)

Author keywords

Auditory cortex; Multisensory; Neuroanatomy; Oscillations; Somatosensory

Indexed keywords

ARTICLE; ASSOCIATION CORTEX; AUDITORY CORTEX; AUDITORY DISCRIMINATION; AUDITORY STIMULATION; HUMAN; LATENT PERIOD; MULTISENSORY INTEGRATION; MUSIC; NERVE CELL NETWORK; NERVE PROJECTION; NEUROMODULATION; NONHUMAN; POSITIVE FEEDBACK; PRIORITY JOURNAL; PSYCHOPHYSIOLOGY; RESPONSE TIME; SOMATOSENSORY STIMULATION; SOMATOSENSORY SYSTEM; STIMULUS RESPONSE; VISUAL INFORMATION; VISUAL STIMULATION;

ACOUSTIC STIMULATION; AUDITORY CORTEX; AUDITORY PATHWAYS; AUDITORY PERCEPTION; BRAIN MAPPING; HUMANS; MODELS, NEUROLOGICAL; NEURONS; OSCILLOMETRY; PERCEPTION; SOMATOSENSORY CORTEX; TIME FACTORS; VISUAL PATHWAYS; VISUAL PERCEPTION;

EID: 70449631673     PISSN: 03785955     EISSN: None     Source Type: Journal    
DOI: 10.1016/j.heares.2009.06.018     Document Type: Article

References (93)

1. Angelucci A., and Bullier J. Reaching beyond the classical receptive field of V1 neurons: horizontal or feedback axons?. J. Physiol. Paris 97 2-3 (2003) 141-154
2. Brosch M., Selezneva E., and Scheich H. Nonauditory events of a behavioral procedure activate auditory cortex of highly trained monkeys. J. Neurosci. 25 29 (2005) 6797-6806
3. Calvert G.A., Bullmore E.T., Brammer M.J., Campbell R., Williams S.C., McGuire P.K., Woodruff P.W., Iversen S.D., and David A.S. Activation of auditory cortex during silent lipreading. Science 276 5312 (1997) 593-596
4. Cappe C., and Barone P. Heteromodal connections supporting multisensory integration at low levels of cortical processing in the monkey. Eur. J. Neurosci. 22 11 (2005) 2886-2902
5. Cappe, C., Morel, A., Barone, P., Rouiller, E.M., 2009. The thalamocortical projection systems in primate: an anatomical support for multisensory and sensorimotor interplay. Cereb. Cortex. PMID:19150924.
6. Celesia G.G. Auditory evoked responses. Intracranial and extracranial average evoked responses. Arch. Neurol. 19 4 (1968) 430-437
7. Chandrasekaran C., Trubanova A., Stillitano S., Caplier A., and Ghazanfar A.A. The natural statistics of audiovisual speech. PLoS Comput. Biol. 5 7 (2009) e1000436 PMID:19609344
8. Chen C.M., Lakatos P., Shah A.S., Mehta A.D., Givre S.J., Javitt D.C., and Schroeder C.E. Functional anatomy and interaction of fast and slow visual pathways in macaque monkeys. Cereb. Cortex 17 7 (2007) 1561-1569
9. Cheung S.W., Nagarajan S.S., Bedenbaugh P.H., Schreiner C.E., Wang X., and Wong A. Auditory cortical neuron response differences under isoflurane versus pentobarbital anesthesia. Hear. Res. 156 1-2 (2001) 115-127
10. Davies P.W., Eruklar S.D., and Rose J.E. Single unit activity in the auditory cortex of the cat. Bull. Johns Hopkins Hosp. 99 2 (1956) 55-86
11. de la Mothe L.A., Blumell S., Kajikawa Y., and Hackett T.A. Cortical connections of the auditory cortex in marmoset monkeys: core and medial belt regions. J. Comp. Neurol. 496 1 (2006) 27-71
12. de la Mothe L.A., Blumell S., Kajikawa Y., and Hackett T.A. Thalamic connections of the auditory cortex in marmoset monkeys: core and medial belt regions. J. Comp. Neurol. 496 1 (2006) 72-96
13. Driver J., and Noesselt T. Multisensory interplay reveals crossmodal influences on 'sensory-specific' brain regions, neural responses, and judgments. Neuron 57 1 (2008) 11-23
14. Elbert T., Pantev C., Wienbruch C., Rockstroh B., and Taub E. Increased cortical representation of the fingers of the left hand in string players. Science 270 5234 (1995) 305-307
15. Falchier A., Clavagnier S., Barone P., and Kennedy H. Anatomical evidence of multimodal integration in primate striate cortex. J. Neurosci. 22 13 (2002) 5749-5759
16. Foxe J.J., and Schroeder C.E. The case for feedforward multisensory convergence during early cortical processing. Neuroreport 16 5 (2005) 419-423
17. Fu K.M., Johnston T.A., Shah A.S., Arnold L., Smiley J., Hackett T.A., Garraghty P.E., and Schroeder C.E. Auditory cortical neurons respond to somatosensory stimulation. J. Neurosci. 23 20 (2003) 7510-7515
18. Gaese B.H., and Ostwald J. Anesthesia changes frequency tuning of neurons in the rat primary auditory cortex. J. Neurophysiol. 86 2 (2001) 1062-1066
19. Ghazanfar A.A., and Schroeder C.E. Is neocortex essentially multisensory?. Trends Cogn. Sci. 10 6 (2006) 278-285
20. Giard M.H., and Peronnet F. Auditory-visual integration during multimodal object recognition in humans: a behavioral and electrophysiological study. J. Cogn. Neurosci. 11 5 (1999) 473-490
21. Gobbele R., Schurmann M., Forss N., Juottonen K., Buchner H., and Hari R. Activation of the human posterior parietal and temporoparietal cortices during audiotactile interaction. Neuroimage 20 1 (2003) 503-511
22. Gomez Gonzalez C.M., Clark V.P., Fan S., Luck S.J., and Hillyard S.A. Sources of attention-sensitive visual event-related potentials. Brain Topogr. 7 1 (1994) 41-51
23. Grant K.W., Walden B.E., and Seitz P.F. Auditory-visual speech recognition by hearing-impaired subjects: consonant recognition, sentence recognition, and auditory-visual integration. J. Acoust. Soc. Am. 103 5 Pt. 1 (1998) 2677-2690
24. Hackett T.A., de la Mothe L.A., Ulbert I., Karmos G., Smiley J., and Schroeder C.E. Multisensory convergence in auditory cortex. II. Thalamocortical connections of the caudal superior temporal plane. J. Comp. Neurol. 502 6 (2007) 924-952
25. Hackett T.A., Smiley J.F., Ulbert I., Karmos G., Lakatos P., de la Mothe L.A., and Schroeder C.E. Sources of somatosensory input to the caudal belt areas of auditory cortex. Perception 36 10 (2007) 1419-1430
26. Hackett T.A., Stepniewska I., and Kaas J.H. Thalamocortical connections of the parabelt auditory cortex in macaque monkeys. J. Comp. Neurol. 400 2 (1998) 271-286
27. Hairston W.D., Hodges D.A., Burdette J.H., and Wallace M.T. Auditory enhancement of visual temporal order judgment. Neuroreport 17 8 (2006) 791-795
28. Hayes E.A., Tiippana K., Nicol T.G., Sams M., and Kraus N. Integration of heard and seen speech: a factor in learning disabilities in children. Neurosci. Lett. 351 1 (2003) 46-50
29. Hodges D.A., Hairston W.D., and Burdette J.H. Aspects of multisensory perception: the integration of visual and auditory information in musical experiences. Ann. NY Acad. Sci. 1060 (2005) 175-185
30. Jones E.G. A new view of specific and nonspecific thalamocortical connections. Adv. Neurol. 77 (1998) 49-71
31. Jones E.G. The thalamic matrix and thalamocortical synchrony. Trends Neurosci. 24 10 (2001) 595-601
32. Jousmaki V., and Hari R. Parchment-skin illusion: sound-biased touch. Curr. Biol. 8 6 (1998) R190
33. Kaas J.H., and Hackett T.A. Subdivisions of auditory cortex and processing streams in primates. Proc. Natl. Acad. Sci. USA 97 22 (2000) 11793-11799
34. Kajikawa Y., de la M.L., Blumell S., and Hackett T.A. A comparison of neuron response properties in areas A1 and CM of the marmoset monkey auditory cortex: tones and broadband noise. J. Neurophysiol. 93 1 (2005) 22-34
35. Kayser C., and Logothetis N.K. Do early sensory cortices integrate cross-modal information?. Brain Struct. Funct. 212 2 (2007) 121-132
36. King A.J., and Palmer A.R. Integration of visual and auditory information in bimodal neurones in the guinea-pig superior colliculus. Exp. Brain Res. 60 3 (1985) 492-500
37. Lakatos P., Chen C.M., O'Connell M.N., Mills A., and Schroeder C.E. Neuronal oscillations and multisensory interaction in primary auditory cortex. Neuron 53 2 (2007) 279-292
38. Lakatos P., Karmos G., Mehta A.D., Ulbert I., and Schroeder C.E. Entrainment of neuronal oscillations as a mechanism of attentional selection. Science 320 5872 (2008) 110-113
39. Lakatos P., Pincze Z., Fu K.M., Javitt D.C., Karmos G., and Schroeder C.E. Timing of pure tone and noise-evoked responses in macaque auditory cortex. Neuroreport 16 9 (2005) 933-937
40. Lakatos P., Shah A.S., Knuth K.H., Ulbert I., Karmos G., and Schroeder C.E. An oscillatory hierarchy controlling neuronal excitability and stimulus processing in the auditory cortex. J. Neurophysiol. 94 3 (2005) 1904-1911
41. Laurienti P.J., Perrault T.J., Stanford T.R., Wallace M.T., and Stein B.E. On the use of superadditivity as a metric for characterizing multisensory integration in functional neuroimaging studies. Exp. Brain Res. 166 3-4 (2005) 289-297
42. Lewis J.W., and Van Essen D.C. Corticocortical connections of visual, sensorimotor, and multimodal processing areas in the parietal lobe of the macaque monkey. J. Comp. Neurol. 428 1 (2000) 112-137
43. Liegeois-Chauvel C., Musolino A., and Chauvel P. Localization of the primary auditory are in man. Brain 114 (1991) 139-153
44. Linke R. Differential projection patterns of superior and inferior collicular neurons onto posterior paralaminar nuclei of the thalamus surrounding the medial geniculate body in the rat. Eur. J. Neurosci. 11 1 (1999) 187-203
45. Lipton M.L., Fu K.M., Branch C.A., and Schroeder C.E. Ipsilateral hand input to area 3b revealed by converging hemodynamic and electrophysiological analyses in macaque monkeys. J. Neurosci. 26 1 (2006) 180-185
46. Massaro D.W., Cohen M.M., and Smeele P.M. Perception of asynchronous and conflicting visual and auditory speech. J. Acoust. Soc. Am. 100 3 (1996) 1777-1786
47. Massaux A., Dutrieux G., Cotillon-Williams N., Manunta Y., and Edeline J.M. Auditory thalamus bursts in anesthetized and non-anesthetized states: contribution to functional properties. J. Neurophysiol. 91 5 (2004) 2117-2134
48. Maunsell J.H., and Gibson J.R. Visual response latencies in striate cortex of the macaque monkey. J. Neurophysiol. 68 4 (1992) 1332-1344
49. Maunsell J.H., Nealey T.A., and DePriest D.D. Magnocellular and parvocellular contributions to responses in the middle temporal visual area (MT) of the macaque monkey. J. Neurosci. 10 10 (1990) 3323-3334
50. Meredith M.A., Nemitz J.W., and Stein B.E. Determinants of multisensory integration in superior colliculus neurons. I. Temporal factors. J. Neurosci. 7 10 (1987) 3215-3229
51. Meredith M.A., and Stein B.E. Interactions among converging sensory inputs in the superior colliculus. Science 221 4608 (1983) 389-391
52. Miller L.M., and D'Esposito M. Perceptual fusion and stimulus coincidence in the cross-modal integration of speech. J. Neurosci. 25 25 (2005) 5884-5893
53. Morand S., Thut G., de Peralta R.G., Clarke S., Khateb A., Landis T., and Michel C.M. Electrophysiological evidence for fast visual processing through the human koniocellular pathway when stimuli move. Cereb. Cortex 10 8 (2000) 817-825
54. Musacchia G., Sams M., Nicol T., and Kraus N. Seeing speech affects acoustic information processing in the human brainstem. Exp. Brain Res. 168 1-2 (2006) 1-10
55. Musacchia G., Strait D., and Kraus N. Relationships between behavior, brainstem and cortical encoding of seen and heard speech in musicians and non-musicians. Hear. Res. 241 1-2 (2008) 34-42
56. Musacchia, G., Arum, L., Nicol, T., Garstecki, D., Kraus, N., 2009. Audiovisual deficits in older adults with hearing loss: biological evidence. Ear Hear. PMID:19546807.
57. O'Connell, M.N., McGinnis, T., Mills, A., Lakatos, P., Lipton, M.L., Branch, C., Schroeder, C.E., 2004. Spatiotemporal Dynamics of Unisensory and Multisensory Processes in Neocortex. Society for Neuroscience Annual Meeting (Ref. type: Abstract).
58. Oberman L.M., and Ramachandran V.S. Preliminary evidence for deficits in multisensory integration in autism spectrum disorders: the mirror neuron hypothesis. Soc. Neurosci. 3 3-4 (2008) 348-355
59. Pandya D.N., Van Hoesen G.W., and Mesulam M.M. Efferent connections of the cingulate gyrus in the rhesus monkey. Exp. Brain Res. 42 3-4 (1981) 319-330
60. Populin L.C., and Yin T.C. Bimodal interactions in the superior colliculus of the behaving cat. J. Neurosci. 22 7 (2002) 2826-2834
61. Recanzone G.H., Guard D.C., and Phan M.L. Frequency and intensity response properties of single neurons in the auditory cortex of the behaving macaque monkey. J. Neurophysiol. 83 4 (2000) 2315-2331
62. Rockland K.S., and Ojima H. Multisensory convergence in calcarine visual areas in macaque monkey. Int. J. Psychophysiol. 50 1-2 (2003) 19-26
63. Rockland K.S., and Pandya D.N. Laminar origins and terminations of cortical connections of the occipital lobe in the rhesus monkey. Brain Res. 179 1 (1979) 3-20
64. Romanski L.M., Bates J.F., and Goldman-Rakic P.S. Auditory belt and parabelt projections to the prefrontal cortex in the rhesus monkey. J. Comp. Neurol. 403 2 (1999) 141-157
65. Ross L.A., Saint-Amour D., Leavitt V.M., Molholm S., Javitt D.C., and Foxe J.J. Impaired multisensory processing in schizophrenia: deficits in the visual enhancement of speech comprehension under noisy environmental conditions. Schizophr. Res. 97 1-3 (2007) 173-183
66. Saldana H.M., and Rosenblum L.D. Visual influences on auditory pluck and bow judgments. Percept. Psychophys. 54 3 (1993) 406-416
67. Sams M., Aulanko R., Hamalainen M., Hari R., Lounasmaa O.V., Lu S.T., and Simola J. Seeing speech: visual information from lip movements modifies activity in the human auditory cortex. Neurosci. Lett. 127 1 (1991) 141-145
68. Schmolesky M.T., Wang Y., Hanes D.P., Thompson K.G., Leutgeb S., Schall J.D., and Leventhal A.G. Signal timing across the macaque visual system. J. Neurophysiol. 79 6 (1998) 3272-3278
69. Schroeder C., and Foxe J.J. Multisensory convergence in early cortical processing. In: Calvert G., Spence C., and Stein B.E. (Eds). The Handbook of Multisensory Processes (2004), MIT Press, Cambridge 295-309
70. Schroeder C.E., and Foxe J. Multisensory contributions to low-level, 'unisensory' processing. Curr. Opin. Neurobiol. 15 4 (2005) 454-458
71. Schroeder C.E., and Foxe J.J. The timing and laminar profile of converging inputs to multisensory areas of the macaque neocortex. Brain Res. Cogn. Brain Res. 14 1 (2002) 187-198
72. Schroeder C.E., Lakatos P., Kajikawa Y., Partan S., and Puce A. Neuronal oscillations and visual amplification of speech. Trends Cogn. Sci. 12 3 (2008) 106-113
73. Schroeder C.E., Lindsley R.W., Specht C., Marcovici A., Smiley J.F., and Javitt D.C. Somatosensory input to auditory association cortex in the macaque monkey. J. Neurophysiol. 85 3 (2001) 1322-1327
74. Schroeder C.E., Mehta A.D., and Givre S.J. A spatiotemporal profile of visual system activation revealed by current source density analysis in the awake macaque. Cereb. Cortex 8 7 (1998) 575-592
75. Schroeder C.E., Molholm S., Lakatos P., Ritter W., and Foxe J.J. Human-simian correspondence in the early cortical processing of multisensory cues. Cogn. Process. 5 (2004) 140-151
76. Schroeder C.E., Smiley J., Fu K.G., McGinnis T., O'Connell M.N., and Hackett T.A. Anatomical mechanisms and functional implications of multisensory convergence in early cortical processing. Int. J. Psychophysiol. 50 1-2 (2003) 5-17
77. Schroeder C.E., Steinschneider M., Javitt D.C., Tenke C.E., Givre S.J., Mehta A.D., Simpson G.V., Arezzo J.C., and Vaughan Jr. H.G. Localization of ERP generators and identification of underlying neural processes. Electroencephalogr. Clin. Neurophysiol. Suppl. 44 5 (1995) 5-75
78. Schroger E., and Widmann A. Speeded responses to audiovisual signal changes result from bimodal integration. Psychophysiology 35 6 (1998) 755-759
79. Schurmann M., Caetano G., Jousmaki V., and Hari R. Hands help hearing: facilitatory audiotactile interaction at low sound-intensity levels. J. Acoust. Soc. Am. 115 2 (2004) 830-832
80. Sekiyama K., Kanno I., Miura S., and Sugita Y. Auditory-visual speech perception examined by fMRI and PET. Neurosci. Res. 47 3 (2003) 277-287
81. Sherman S.M., and Guillery R.W. The role of the thalamus in the flow of information to the cortex. Philos. Trans. R. Soc. Lond. B Biol. Sci. 357 1428 (2002) 1695-1708
82. Sincich L.C., Park K.F., Wohlgemuth M.J., and Horton J.C. Bypassing V1: a direct geniculate input to area MT. Nat. Neurosci. 7 10 (2004) 1123-1128
83. Smiley J.F., Hackett T.A., Ulbert I., Karmas G., Lakatos P., Javitt D.C., and Schroeder C.E. Multisensory convergence in auditory cortex. I. Cortical connections of the caudal superior temporal plane in macaque monkeys. J. Comp. Neurol. 502 6 (2007) 894-923
84. Stein B.E., and Meredith M.A. The Merging of the Senses (1993), MIT Pess, Cambridge
85. Stein B.E., and Stanford T.R. Multisensory integration: current issues from the perspective of the single neuron. Nat. Rev. Neurosci. 9 4 (2008) 255-266
86. Steinschneider M., Tenke C.E., Schroeder C.E., Javitt D.C., Simpson G.V., Arezzo J.C., and Vaughan Jr. H.G. Cellular generators of the cortical auditory evoked potential initial component. Electroencephalogr. Clin. Neurophysiol. 84 2 (1992) 196-200
87. Sumby W.H., and Pollack I. Visual contribution to speech intelligibility in noise. J. Acoust. Soc. Am. 26 (1954) 212-215
88. Thompson W.F., Graham P., and Russo F.A. Seeing music performance: visual influences on perception and experience. Semiotica 156 (2005) 203-227
89. van Wassenhove V., Grant K.W., and Poeppel D. Visual speech speeds up the neural processing of auditory speech. Proc. Natl. Acad. Sci. USA 102 4 (2005) 1181-1186
90. Vines B.W., Krumhansl C.L., Wanderley M.M., and Levitin D.J. Cross-modal interactions in the perception of musical performance. Cognition 101 1 (2006) 80-113 PMID:16289067
91. Wallace M.T., Meredith M.A., and Stein B.E. Multisensory integration in the superior colliculus of the alert cat. J. Neurophysiol. 80 2 (1998) 1006-1010
92. White A.J., Solomon S.G., and Martin P.R. Spatial properties of koniocellular cells in the lateral geniculate nucleus of the marmoset Callithrix jacchus. J. Physiol. 533 Pt. 2 (2001) 519-535
93. Zurita P., Villa A.E., de R.Y., de R.F., and Rouiller E.M. Changes of single unit activity in the cat's auditory thalamus and cortex associated to different anesthetic conditions. Neurosci. Res. 19 3 (1994) 303-316