Development of multisensory integration from the perspective of the individual neuron

Nature Reviews Neuroscience

Volumn 15, Issue 8, 2014, Pages 520-535

  Stein, Barry E ^a   Stanford, Terrence R ^a   Rowland, Benjamin A ^a  

^a WAKE FOREST SCHOOL OF MEDICINE   (United States)

Author keywords

[No Author keywords available]

Indexed keywords

ACOUSTIC NERVE FIBER; ASSOCIATION CORTEX; AUDITORY STIMULATION; BINOCULAR CONVERGENCE; BRAIN; DARKNESS; HUMAN; INFORMATION PROCESSING; MASKING; MULTISENSORY INTEGRATION; NERVE CELL; NERVE CELL DIFFERENTIATION; NERVE CELL PLASTICITY; NOISE; NONHUMAN; ORIENTING RESPONSE; PERCEPTION; POSTNATAL DEVELOPMENT; PRIORITY JOURNAL; REARING; RECEPTIVE FIELD; REVIEW; SENSORIMOTOR INTEGRATION; SENSORY STIMULATION; SIGNAL DETECTION; SOMATOSENSORY CORTEX; SOMATOSENSORY SYSTEM; STATE DEPENDENT LEARNING; STIMULUS; SUPERIOR COLLICULUS; TRIGEMINAL NUCLEUS; VISUAL DEPRIVATION; VISUAL FIELD; VISUAL STIMULATION;

ANIMALS; BRAIN; NEURONS; PERCEPTION; SENSATION;

EID: 84904464505     PISSN: 1471003X     EISSN: 14710048     Source Type: Journal    
DOI: 10.1038/nrn3742     Document Type: Review

References (171)

1. Stein, B. E. &Meredith, M. A. The Merging of the Senses (MIT Press, 1993).
2. Alais, D. &Burr, D. The ventriloquist effect results from near-optimal bimodal integration. Curr. Biol. 14, 257-262 (2004).
3. Ernst, M. O. &Banks, M. S. Humans integrate visual and haptic information in a statistically optimal fashion. Nature 415, 429-433 (2002).
4. Shams, L., Ma, W. J. &Beierholm, U. Sound-induced flash illusion as an optimal percept. Neuroreport 16, 1923-1927 (2005).
5. Lewkowicz, D. J. &Lickliter, R. (eds) The Development of Intersensory Perception: Comparative Perspectives (Lawrence Erlbaum Associates, 1994).
6. Stein, B. (ed.) The New Handbook of Multisensory Processing (MIT Press, 2012).
7. Stein, B. E., Stanford, T. R., Ramachandran, R., Perrault, T. J. Jr &Rowland, B. A. Challenges in quantifying multisensory integration: alternative criteria, models, and inverse effectiveness. Exp. Brain Res. 198, 113-126 (2009).
8. Binns, K. E. &Salt, T. E. Importance of NMDA receptors for multimodal integration in the deep layers of the cat superior colliculus. J. Neurophysiol. 75, 920-930 (1996).
9. Wallace, M. T., Meredith, M. A. &Stein, B. E. Multisensory integration in the superior colliculus of the alert cat. J. Neurophysiol. 80, 1006-1010 (1998).
10. Bolognini, N., Leo, F., Passamonti, C., Stein, B. E. &Làdavas, E. Multisensory-mediated auditory localization. Perception 36, 1477-1485 (2007).
11. Corneil, B. D. &Munoz, D. P. The influence of auditory and visual distractors on human orienting gaze shifts. J. Neurosci. 16, 8193-8207 (1996).
12. Diederich, A. &Colonius, H. Bimodal and trimodal multisensory enhancement: effects of stimulus onset and intensity on reaction time. Percept. Psychophys. 66, 1388-1404 (2004).
13. Fetsch, C. R., Pouget, A., DeAngelis, G. C. &Angelaki, D. E. Neural correlates of reliability-based cue weighting during multisensory integration. Nature Neurosci. 15, 146-154 (2012).
14. Foxe, J. J. &Schroeder, C. E. The case for feedforward multisensory convergence during early cortical processing. Neuroreport 16, 419-423 (2005).
15. Frens, M. A. &Van Opstal, A. J. Visual-auditory interactions modulate saccade-related activity in monkey superior colliculus. Brain Res. Bull. 46, 211-224 (1998).
16. Hughes, H. C., Reuter-Lorenz, P. A., Nozawa, G. &Fendrich, R. Visual-auditory interactions in sensorimotor processing: saccades versus manual responses. J. Exp. Psychol. Hum. Percept. Perform. 20, 131-153 (1994).
17. Naumer, M. J. &Kaiser, J. (eds) Multisensory Object Perception in the Primate Brain (Springer, 2010).
18. Sánchez-García, C., Alsius, A., Enns, J. T. &Soto-Faraco, S. Cross-modal prediction in speech perception. PLoS ONE 6, e25198 (2011).
19. Senkowski, D., Talsma, D., Grigutsch, M., Herrmann, C. S. &Woldorff, M. G. Good times for multisensory integration: effects of the precision of temporal synchrony as revealed by gamma-band oscillations. Neuropsychologia 45, 561-571 (2007).
20. Stein, B. E., Meredith, M. A., Huneycutt, W. S. &McDade, L. Behavioral indices of multisensory integration: orientation to visual cues is affected by auditory stimuli. J. Cogn. Neurosci. 1, 12-24 (1989).
21. Talsma, D., Doty, T. J. &Woldorff, M. G. Selective attention and audiovisual integration: is attending to both modalities a prerequisite for early integration? Cereb. Cortex 17, 679-690 (2007).
22. Woods, T. M. &Recanzone, G. H. Visually induced plasticity of auditory spatial perception in macaques. Curr. Biol. 14, 1559-1564 (2004).
23. Zhang, M., Weisser, V. D., Stilla, R., Prather, S. C. &Sathian, K. Multisensory cortical processing of object shape and its relation to mental imagery. Cogn. Affect. Behav. Neurosci. 4, 251-259 (2004).
24. Goldring, J. E., Dorris, M. C., Corneil, B. D., Ballantyne, P. A. &Munoz, D. P. Combined eye-head gaze shifts to visual and auditory targets in humans. Exp. Brain Res. 111, 68-78 (1996).
25. Sperdin, H. F., Cappe, C., Foxe, J. J. &Murray, M. M. Early, low-level auditory-somatosensory multisensory interactions impact reaction time speed. Front. Integr. Neurosci. 3, 2 (2009).
26. Von Saldern, S. &Noppeney, U. Sensory and striatal areas integrate auditory and visual signals into behavioral benefits during motion discrimination. J. Neurosci. 33, 8841-8849 (2013).
27. Bremner, A. J., Lewkowicz, D. J. &Spence, C. (eds) Multisensory Development (Oxford Univ. Press, 2012).
28. Bizley, J. K. &King, A. J. Visual-auditory spatial processing in auditory cortical neurons. Brain Res. 1242, 24-36 (2008).
29. Brett-Green, B., Fifková, E., Larue, D. T., Winer, J. A. &Barth, D. S. A multisensory zone in rat parietotemporal cortex: intra- and extracellular physiology and thalamocortical connections. J. Comp. Neurol. 460, 223-237 (2003).
30. Driver, J. &Noesselt, T. Multisensory interplay reveals crossmodal influences on 'sensory-specific' brain regions, neural responses, and judgments. Neuron 57, 11-23 (2008).
31. Ghazanfar, A. A. &Schroeder, C. E. Is neocortex essentially multisensory? Trends Cogn. Sci. 10, 278-285 (2006).
32. Graziano, M., Gross, C., Taylor, C. &Moore, T. in Crossmodal Space and Crossmodal Attention (eds Spence, C. &Driver, J.) 51-67 (Oxford Univ. Press, 2004).
33. King, A. J. &Palmer, A. R. Integration of visual and auditory information in bimodal neurones in the guinea-pig superior colliculus. Exp. Brain Res. 60, 492-500 (1985).
34. Romanski, L. M. Representation and integration of auditory and visual stimuli in the primate ventral lateral prefrontal cortex. Cereb. Cortex 17 (Suppl. 1), i61-i69 (2007).
35. Schroeder, C. E. &Foxe, J. Multisensory contributions to low-level, 'unisensory' processing. Curr. Opin. Neurobiol. 15, 454-458 (2005).
36. Wallace, M., Ramachandran, R. &Stein, B. A revised view of sensory cortical parcellation. Proc. Natl Acad. Sci. USA 101, 2167-2172 (2004).
37. Winkowski, D. E. &Knudsen, E. I. Top-down control of multimodal sensitivity in the barn owl optic tectum. J. Neurosci. 27, 13279-13291 (2007).
38. Macaluso, E. &Driver, J. in The Handbook of Multisensory Processes (eds Calvert, G. A., Spence, C. &Stein, B. E.) 529-548 (MIT Press, 2004).
39. Cohen, Y. E. &Andersen, R. A. in The Handbook of Multisensory Processes (eds Calvert, G. A., Spence, C. &Stein, B. E.) 463-479 (MIT Press, 2004).
40. Burnett, L. R., Stein, B. E., Chaponis, D. &Wallace, M. T. Superior colliculus lesions preferentially disrupt multisensory orientation. Neuroscience 124, 535-547 (2004).
41. Burnett, L. R., Stein, B. E., Perrault, T. J. Jr &Wallace, M. T. Excitotoxic lesions of the superior colliculus preferentially impact multisensory neurons and multisensory integration. Exp. Brain Res. 179, 325-338 (2007).
42. Groh, J. M. &Sparks, D. L. Saccades to somatosensory targets. II. Motor convergence in primate superior colliculus. J. Neurophysiol. 75, 428-438 (1996).
43. Groh, J. M. &Sparks, D. L. Saccades to somatosensory targets. III. Eye-position-dependent somatosensory activity in primate superior colliculus. J. Neurophysiol. 75, 439-453 (1996).
44. Harrington, L. K. &Peck, C. K. Spatial disparity affects visual-auditory interactions in human sensorimotor processing. Exp. Brain Res. 122, 247-252 (1998).
45. Jay, M. F. &Sparks, D. L. Sensorimotor integration in the primate superior colliculus. I. Motor convergence. J. Neurophysiol. 57, 22-34 (1987).
46. Jay, M. F. &Sparks, D. L. Sensorimotor integration in the primate superior colliculus. II. Coordinates of auditory signals. J. Neurophysiol. 57, 35-55 (1987).
47. Munoz, D. P. &Wurtz, R. H. Saccade-related activity in monkey superior colliculus. I. Characteristics of burst and buildup cells. J. Neurophysiol. 73, 2313-2333 (1995).
48. Munoz, D. P. &Wurtz, R. H. Saccade-related activity in monkey superior colliculus. II. Spread of activity during saccades. J. Neurophysiol. 73, 2334-2348 (1995).
49. Lomber, S. G., Payne, B. R. &Cornwell, P. Role of the superior colliculus in analyses of space: superficial and intermediate layer contributions to visual orienting, auditory orienting, and visuospatial discriminations during unilateral and bilateral deactivations. J. Comp. Neurol. 441, 44-57 (2001).
50. Levine, M. S., Hull, C. D. &Buchwald, N. A. Development of motor activity in kittens. Dev. Psychobiol. 13, 357-371 (1980).
51. Norton, T. T. Receptive-field properties of superior colliculus cells and development of visual behavior in kittens. J. Neurophysiol. 37, 674-690 (1974).
52. Meredith, M. A., Wallace, M. T. &Stein, B. E. Visual, auditory and somatosensory convergence in output neurons of the cat superior colliculus: multisensory properties of the tecto-reticulo-spinal projection. Exp. Brain Res. 88, 181-186 (1992).
53. Sparks, D. L. Translation of sensory signals into commands for control of saccadic eye movements: role of primate superior colliculus. Physiol. Rev. 66, 118-171 (1986).
54. Wurtz, R. H. &Goldberg, M. E. The role of the superior colliculus in visually-evoked eye movements. Bibl. Ophthalmol. 82, 149-158 (1972).
55. Moschovakis, A. K. et al. An anatomical substrate for the spatiotemporal transformation. J. Neurosci. 18, 10219-10229 (1998).
56. Knill, D. C. &Pouget, A. The Bayesian brain: the role of uncertainty in neural coding and computation. Trends Neurosci. 27, 712-719 (2004).
57. Patton, P. E. &Anastasio, T. J. Modeling cross-modal enhancement and modality-specific suppression in multisensory neurons. Neural Comput. 15, 783-810 (2003).
58. Rowland, B., Stanford, T. &Stein, B. A Bayesian model unifies multisensory spatial localization with the physiological properties of the superior colliculus. Exp. Brain Res. 180, 153-161 (2007).
59. Stein, B. E., Labos, E. &Kruger, L. Sequence of changes in properties of neurons of superior colliculus of the kitten during maturation. J. Neurophysiol. 36, 667-679 (1973).
60. Stein, B. E., Labos, E. &Kruger, L. Determinants of response latency in neurons of superior colliculus in kittens. J. Neurophysiol. 36, 680-689 (1973).
61. Benedetti, F. Differential formation of topographic maps on the cerebral cortex and superior colliculus of the mouse by temporally correlated tactile- tactile and tactile-visual inputs. Eur. J. Neurosci. 7, 1942-1951 (1995).
62. Meredith, M. A., Clemo, H. R. &Stein, B. E. Somatotopic component of the multisensory map in the deep laminae of the cat superior colliculus. J. Comp. Neurol. 312, 353-370 (1991).
63. Meredith, M. A. &Stein, B. E. The visuotopic component of the multisensory map in the deep laminae of the cat superior colliculus. J. Neurosci. 10, 3727-3742 (1990).
64. Middlebrooks, J. C. &Knudsen, E. I. A neural code for auditory space in the cat's superior colliculus. J. Neurosci. 4, 2621-2634 (1984).
65. Stein, B. E., Magalhaes-Castro, B. &Kruger, L. Superior colliculus: visuotopic-somatotopic overlap. Science 189, 224-226 (1975).
66. Chalupa, L. M. &Rhoades, R. W. Responses of visual, somatosensory, and auditory neurones in the golden hamster's superior colliculus. J. Physiol. 270, 595-626 (1977).
67. Dräger, U. C. &Hubel, D. H. Physiology of visual cells in mouse superior colliculus and correlation with somatosensory and auditory input. Nature 253, 203-204 (1975).
68. Dräger, U. C. &Hubel, D. H. Topography of visual and somatosensory projections to mouse superior colliculus. J. Neurophysiol. 39, 91-101 (1976).
69. Gordon, B. Receptive fields in deep layers of cat superior colliculus. J. Neurophysiol. 36, 157-178 (1973).
70. Graham, J., Pearson, H. E., Berman, N. &Murphy, E. H. Laminar organization of superior colliculus in the rabbit: a study of receptive-field properties of single units. J. Neurophysiol. 45, 915-932 (1981).
71. King, A. J., Schnupp, J. W. &Thompson, I. D. Signals from the superficial layers of the superior colliculus enable the development of the auditory space map in the deeper layers. J. Neurosci. 18, 9394-9408 (1998).
72. Knudsen, E. I. Auditory and visual maps of space in the optic tectum of the owl. J. Neurosci. 2, 1177-1194 (1982).
73. Stein, B. E. &Dixon, J. P. Superior colliculus cells respond to noxious stimuli. Brain Res. 158, 65-73 (1978).
74. Stein, B. E., Magalhäes-Castro, B. &Kruger, L. Relationship between visual and tactile representations in cat superior colliculus. J. Neurophysiol. 39, 401-419 (1976).
75. Alvarado, J. C., Vaughan, J. W., Stanford, T. R. &Stein, B. E. Multisensory versus unisensory integration: contrasting modes in the superior colliculus. J. Neurophysiol. 97, 3193-3205 (2007).
76. Jiang, W., Jiang, H. &Stein, B. E. Neonatal cortical ablation disrupts multisensory development in superior colliculus. J. Neurophysiol. 95, 1380-1396 (2006).
77. Wallace, M. T. &Stein, B. E. Development of multisensory neurons and multisensory integration in cat superior colliculus. J. Neurosci. 17, 2429-2444 (1997).
78. Wallace, M. T. &Stein, B. E. Onset of cross-modal synthesis in the neonatal superior colliculus is gated by the development of cortical influences. J. Neurophysiol. 83, 3578-3582 (2000).
79. Xu, J., Yu, L., Rowland, B. A., Stanford, T. R. &Stein, B. E. Incorporating cross-modal statistics in the development and maintenance of multisensory integration. J. Neurosci. 32, 2287-2298 (2012).
80. Meredith, M. A. &Stein, B. E. Interactions among converging sensory inputs in the superior colliculus. Science 221, 389-391 (1983).
81. Kadunce, D. C., Vaughan, J. W., Wallace, M. T. &Stein, B. E. The influence of visual and auditory receptive field organization on multisensory integration in the superior colliculus. Exp. Brain Res. 139, 303-310 (2001).
82. Meredith, M. A., Nemitz, J. W. &Stein, B. E. Determinants of multisensory integration in superior colliculus neurons. I. Temporal factors. J. Neurosci. 7, 3215-3229 (1987).
83. Meredith, M. A. &Stein, B. E. Spatial factors determine the activity of multisensory neurons in cat superior colliculus. Brain Res. 365, 350-354 (1986).
84. Kadunce, D. C., Vaughan, J. W., Wallace, M. T., Benedek, G. &Stein, B. E. Mechanisms of within- and cross-modality suppression in the superior colliculus. J. Neurophysiol. 78, 2834-2847 (1997).
85. Pluta, S. R., Rowland, B. A., Stanford, T. R. &Stein, B. E. Alterations to multisensory and unisensory integration by stimulus competition. J. Neurophysiol. 106, 3091-3101 (2011).
86. Larson, M. A. &Stein, B. E. The use of tactile and olfactory cues in neonatal orientation and localization of the nipple. Dev. Psychobiol. 17, 423-436 (1984).
87. Kao, C. Q., McHaffie, J. G., Meredith, M. A. &Stein, B. E. Functional development of a central visual map in cat. J. Neurophysiol. 72, 266-272 (1994).
88. Hubel, D. H. &Wiesel, T. N. Receptive fields of cells in striate cortex of very young, visually inexperienced kittens. J. Neurophysiol. 26, 994-1002 (1963).
89. Horton, J. C. &Hocking, D. R. An adult-like pattern of ocular dominance columns in striate cortex of newborn monkeys prior to visual experience. J. Neurosci. 16, 1791-1807 (1996).
90. Hubel, D. H. &Wiesel, T. N. Receptive fields and functional architecture of monkey striate cortex. J. Physiol. 195, 215-243 (1968).
91. Rakic, P. Prenatal development of the visual system in rhesus monkey. Philos. Trans. R. Soc. Lond. B 278, 245-260 (1977).
92. Wallace, M. T., McHaffie, J. G. &Stein, B. E. Visual response properties and visuotopic representation in the newborn monkey superior colliculus. J. Neurophysiol. 78, 2732-2741 (1997).
93. Wallace, M. T. &Stein, B. E. Sensory organization of the superior colliculus in cat and monkey. Prog. Brain Res. 112, 301-311 (1996).
94. Edwards, S. B., Ginsburgh, C. L., Henkel, C. K. &Stein, B. E. Sources of subcortical projections to the superior colliculus in the cat. J. Comp. Neurol. 184, 309-329 (1979).
95. Huerta, M. F. &Harting, J. K. The projection from the nucleus of the posterior commissure to the superior colliculus of the cat: patch-like endings within the intermediate and deep grey layers. Brain Res. 238, 426-432 (1982).
96. Meredith, M. A. &Clemo, H. R. Auditory cortical projection from the anterior ectosylvian sulcus (Field AES) to the superior colliculus in the cat: an anatomical and electrophysiological study. J. Comp. Neurol. 289, 687-707 (1989).
97. Stein, B. E., Spencer, R. F. &Edwards, S. B. Corticotectal and corticothalamic efferent projections of SIV somatosensory cortex in cat. J. Neurophysiol. 50, 896-909 (1983).
98. Scannell, J. W. et al. Visual motion processing in the anterior ectosylvian sulcus of the cat. J. Neurophysiol. 76, 895-907 (1996).
99. Alvarado, J. C., Stanford, T. R., Rowland, B. A., Vaughan, J. W. &Stein, B. E. Multisensory integration in the superior colliculus requires synergy among corticocollicular inputs. J. Neurosci. 29, 6580-6592 (2009).
100. Alvarado, J. C., Stanford, T. R., Vaughan, J. W. &Stein, B. E. Cortex mediates multisensory but not unisensory integration in superior colliculus. J. Neurosci. 27, 12775-12786 (2007).
101. Jiang, W., Wallace, M. T., Jiang, H., Vaughan, J. W. &Stein, B. E. Two cortical areas mediate multisensory integration in superior colliculus neurons. J. Neurophysiol. 85, 506-522 (2001).
102. Jiang, W., Jiang, H. &Stein, B. E. Two corticotectal areas facilitate multisensory orientation behavior. J. Cogn. Neurosci. 14, 1240-1255 (2002).
103. Jiang, W. &Stein, B. E. Cortex controls multisensory depression in superior colliculus. J. Neurophysiol. 90, 2123-2135 (2003).
104. Stein, B. E., Wallace, M. W., Stanford, T. R. &Jiang, W. Cortex governs multisensory integration in the midbrain. Neuroscientist 8, 306-314 (2002).
105. Stein, B. E. The development of a dialogue between cortex and midbrain to integrate multisensory information. Exp. Brain Res. 166, 305-315 (2005).
106. Wallace, M. T. &Stein, B. E. Cross-modal synthesis in the midbrain depends on input from cortex. J. Neurophysiol. 71, 429-432 (1994).
107. Clemo, H. R. &Stein, B. E. Topographic organization of somatosensory corticotectal influences in cat. J. Neurophysiol. 51, 843-858 (1984).
108. Mucke, L., Norita, M., Benedek, G. &Creutzfeldt, O. Physiologic and anatomic investigation of a visual cortical area situated in the ventral bank of the anterior ectosylvian sulcus of the cat. Exp. Brain Res. 46, 1-11 (1982).
109. Olson, C. R. &Graybiel, A. M. Ectosylvian visual area of the cat: location, retinotopic organization, and connections. J. Comp. Neurol. 261, 277-294 (1987).
110. Wallace, M. T., Meredith, M. A. &Stein, B. E. Converging influences from visual, auditory, and somatosensory cortices onto output neurons of the superior colliculus. J. Neurophysiol. 69, 1797-1809 (1993).
111. Alvarado, J. C., Rowland, B. A., Stanford, T. R. &Stein, B. E. A neural network model of multisensory integration also accounts for unisensory integration in superior colliculus. Brain Res. 1242, 13-23 (2008).
112. McHaffie, J. G. et al. in The New Handbook of Multisensory Processing 31-47 (ed. Stein, B.) (MIT Press, 2012).
113. Fuentes-Santamaria, V., Alvarado, J. C., Stein, B. E. &McHaffie, J. G. Cortex contacts both output neurons and nitrergic interneurons in the superior colliculus: direct and indirect routes for multisensory integration. Cereb. Cortex 18, 1640-1652 (2008).
114. Fuentes-Santamaria, V., McHaffie, J. G. &Stein, B. E. Maturation of multisensory integration in the superior colliculus: expression of nitric oxide synthase and neurofilament SMI-32. Brain Res. 1242, 45-53 (2008).
115. McHaffie, J. G., Kruger, L., Clemo, H. R. &Stein, B. E. Corticothalamic and corticotectal somatosensory projections from the anterior ectosylvian sulcus (SIV cortex) in neonatal cats: an anatomical demonstration with HRP and 3H-leucine. J. Comp. Neurol. 274, 115-126 (1988).
116. Jiang, W., Jiang, H., Rowland, B. A. &Stein, B. E. Multisensory orientation behavior is disrupted by neonatal cortical ablation. J. Neurophysiol. 97, 557-562 (2007).
117. Wilkinson, L. K., Meredith, M. A. &Stein, B. E. The role of anterior ectosylvian cortex in cross-modality orientation and approach behavior. Exp. Brain Res. 112, 1-10 (1996).
118. Rauschecker, J. P. Developmental plasticity and memory. Behav. Brain Res. 66, 7-12 (1995).
119. Wiesel, T. N. &Hubel, D. H. Single-cell responses in striate cortex of kittens deprived of vision in one eye. J. Neurophysiol. 26, 1003-1017 (1963).
120. Wallace, M. T., Perrault, T. J. Jr, Hairston, W. D. &Stein, B. E. Visual experience is necessary for the development of multisensory integration. J. Neurosci. 24, 9580-9584 (2004).
121. Feldman, D. E. The spike-timing dependence of plasticity. Neuron 75, 556-571 (2012).
122. Crair, M. C. Neuronal activity during development: permissive or instructive? Curr. Opin. Neurobiol. 9, 88-93 (1999).
123. Hua, J. Y. &Smith, S. J. Neural activity and the dynamics of central nervous system development. Nature Neurosci. 7, 327-332 (2004).
124. Kirkwood, A., Lee, H. K. &Bear, M. F. Co-regulation of long-term potentiation and experience-dependent synaptic plasticity in visual cortex by age and experience. Nature 375, 328-331 (1995).
125. Knudsen, E. I. &Brainard, M. S. Visual instruction of the neural map of auditory space in the developing optic tectum. Science 253, 85-87 (1991).
126. Withington, D. J. The effect of binocular lid suture on auditory responses in the guinea-pig superior colliculus. Neurosci. Lett. 136, 153-156 (1992).
127. Rauschecker, J. P. &Harris, L. R. Auditory compensation of the effects of visual deprivation in the cat's superior colliculus. Exp. Brain Res. 50, 69-83 (1983).
128. Champoux, F., Bacon, B. A., Lepore, F. &Guillemot, J.-P. Effects of early binocular enucleation on auditory and somatosensory coding in the superior colliculus of the rat. Brain Res. 1191, 84-95 (2008).
129. Mundiñano, I. C. &Martínez-Millán, L. Somatosensory cross-modal plasticity in the superior colliculus of visually deafferented rats. Neuroscience 165, 1457-1470 (2010).
130. Merabet, L. B. &Pascual-Leone, A. Neural reorganization following sensory loss: the opportunity of change. Nature Rev. Neurosci. 11, 44-52 (2010).
131. Chang, E. F. &Merzenich, M. M. Environmental noise retards auditory cortical development. Science 300, 498-502 (2003).
132. Efrati, A. &Gutfreund, Y. Early life exposure to noise alters the representation of auditory localization cues in the auditory space map of the barn owl. J. Neurophysiol. 105, 2522-2535 (2011).
133. Xu, J., Yu, L., Rowland, B. A., Stanford, T. R. &Stein, B. E. Noise-rearing disrupts the maturation of multisensory integration. Eur. J. Neurosci. 39, 602-613 (2014).
134. Wallace, M. T. &Stein, B. E. Early experience determines how the senses will interact. J. Neurophysiol. 97, 921-926 (2007).
135. Benedetti, F. &Ferro, I. The effects of early postnatal modification of body shape on the somatosensory-visual organization in mouse superior colliculus. Eur. J. Neurosci. 7, 412-418 (1995).
136. Benedetti, F. Orienting behaviour and superior colliculus sensory representations in mice with the vibrissae bent into the contralateral hemispace. Eur. J. Neurosci. 7, 1512-1519 (1995).
137. Brainard, M. S. &Knudsen, E. I. Sensitive periods for visual calibration of the auditory space map in the barn owl optic tectum. J. Neurosci. 18, 3929-3942 (1998).
138. King, A. J. Neural plasticity: how the eye tells the brain about sound location. Curr. Biol. 12, R393-R395 (2002).
139. Beauchamp, M. S., Lee, K. E., Argall, B. D. &Martin, A. Integration of auditory and visual information about objects in superior temporal sulcus. Neuron 41, 809-823 (2004).
140. De Gelder, B. &Bertelson, P. Multisensory integration, perception and ecological validity. Trends Cogn. Sci. 7, 460-467 (2003).
141. King, A. J. &Calvert, G. A. Multisensory integration: perceptual grouping by eye and ear. Curr. Biol. 11, R322-R325 (2001).
142. Baier, B., Kleinschmidt, A. &Müller, N. G. Cross-modal processing in early visual and auditory cortices depends on expected statistical relationship of multisensory information. J. Neurosci. 26, 12260-12265 (2006).
143. Fiebelkorn, I. C., Foxe, J. J. &Molholm, S. Dual mechanisms for the cross-sensory spread of attention: how much do learned associations matter? Cereb. Cortex 20, 109-120 (2010).
144. Barraclough, N. E., Xiao, D., Baker, C. I., Oram, M. W. &Perrett, D. I. Integration of visual and auditory information by superior temporal sulcus neurons responsive to the sight of actions. J. Cogn. Neurosci. 17, 377-391 (2005).
145. Yu, L., Rowland, B. A., Xu, J. &Stein, B. E. Multisensory plasticity in adulthood: cross-modal experience enhances neuronal excitability and exposes silent inputs. J. Neurophysiol. 109, 464-474 (2013).
146. Rowland, B., Jiang, W. &Stein, B. Brief cortical deactivation early in life has long-lasting effects on multisensory behavior. J. Neurosci. 34, 7198-7202 (2014).
147. Bi, G. &Poo, M. Synaptic modification by correlated activity: Hebb's postulate revisited. Annu. Rev. Neurosci. 24, 139-166 (2001).
148. Gerstner, W., Ritz, R. &van Hemmen, J. L. Why spikes? Hebbian learning and retrieval of time-resolved excitation patterns. Biol. Cybern. 69, 503-515 (1993).
149. Cuppini, C., Magosso, E., Rowland, B., Stein, B. &Ursino, M. Hebbian mechanisms help explain development of multisensory integration in the superior colliculus: a neural network model. Biol. Cybern. 106, 691-713 (2012).
150. Grossberg, S., Roberts, K., Aguilar, M. &Bullock, D. A neural model of multimodal adaptive saccadic eye movement control by superior colliculus. J. Neurosci. 17, 9706-9725 (1997).
151. Yu, L., Stein, B. E. &Rowland, B. A. Adult plasticity in multisensory neurons: short-term experience-dependent changes in the superior colliculus. J. Neurosci. 29, 15910-15922 (2009).
152. Mégevand, P., Molholm, S., Nayak, A. &Foxe, J. J. Recalibration of the multisensory temporal window of integration results from changing task demands. PLoS ONE 8, e71608 (2013).
153. Yu, L., Rowland, B. A. &Stein, B. E. Initiating the development of multisensory integration by manipulating sensory experience. J. Neurosci. 30, 4904-4913 (2010).
154. Royal, D. W., Krueger, J., Fister, M. C. &Wallace, M. T. Adult plasticity of spatiotemporal receptive fields of multisensory superior colliculus neurons following early visual deprivation. Restor. Neurol. Neurosci. 28, 259-270 (2010).
155. Smith, A. L. et al. An investigation of the role of auditory cortex in sound localization using muscimol-releasing Elvax. Eur. J. Neurosci. 19, 3059-3072 (2004).
156. Putzar, L., Gondan, M. &Röder, B. Basic multisensory functions can be acquired after congenital visual pattern deprivation in humans. Dev. Neuropsychol. 37, 697-711 (2012).
157. Schorr, E. A., Fox, N. A., van Wassenhove, V. &Knudsen, E. I. Auditory-visual fusion in speech perception in children with cochlear implants. Proc. Natl Acad. Sci. USA 102, 18748-18750 (2005).
158. Putzar, L., Hötting, K. &Röder, B. Early visual deprivation affects the development of face recognition and of audio-visual speech perception. Restor. Neurol. Neurosci. 28, 251-257 (2010).
159. Rouger, J. et al. Evidence that cochlear-implanted deaf patients are better multisensory integrators. Proc. Natl Acad. Sci. USA 104, 7295-7300 (2007).
160. Carriere, B. N. et al. Visual deprivation alters the development of cortical multisensory integration. J. Neurophysiol. 98, 2858-2867 (2007).
161. Gori, M., Giuliana, L., Sandini, G. &Burr, D. Visual size perception and haptic calibration during development. Dev. Sci. 15, 854-862 (2012).
162. Gori, M., Sandini, G. &Burr, D. Development of visuo-auditory integration in space and time. Front. Integr. Neurosci. 6, 77 (2012).
163. Lewkowicz, D. J. Perception of dynamic and static audiovisual sequences in 3- and 4-month-old infants. Child Dev. 79, 1538-1554 (2008).
164. Neil, P. A., Chee-Ruiter, C., Scheier, C., Lewkowicz, D. J. &Shimojo, S. Development of multisensory spatial integration and perception in humans. Dev. Sci. 9, 454-464 (2006).
165. Barutchu, A. et al. Audiovisual integration in noise by children and adults. J. Exp. Child Psychol. 105, 38-50 (2010).
166. Foxe, J. J. et al. Severe multisensory speech integration deficits in high-functioning school-aged children with autism spectrum disorder (ASD) and their resolution during early adolescence. Cereb. Cortex http://dx.doi.org/10. 1093/cercor/bht213 (2013).
167. Hairston, W. D., Burdette, J. H., Flowers, D. L., Wood, F. B. &Wallace, M. T. Altered temporal profile of visual-auditory multisensory interactions in dyslexia. Exp. Brain Res. 166, 474-480 (2005).
168. Stevenson, R. A. et al. Multisensory temporal integration in autism spectrum disorders. J. Neurosci. 34, 691-697 (2014).
169. Stanford, T. R., Quessy, S. &Stein, B. E. Evaluating the operations underlying multisensory integration in the cat superior colliculus. J. Neurosci. 25, 6499-6508 (2005).
170. Cuppini, C., Ursino, M., Magosso, E., Rowland, B. A. &Stein, B. E. An emergent model of multisensory integration in superior colliculus neurons. Front. Integr. Neurosci. 4, 6 (2010).
171. Fetsch, C. R., Deangelis, G. C. &Angelaki, D. E. Bridging the gap between theories of sensory cue integration and the physiology of multisensory neurons. Nature Rev. Neurosci. 14, 429-442 (2013).