Neurocomputational approaches to modelling multisensory integration in the brain: A review

Neural Networks

Volumn 60, Issue , 2014, Pages 141-165

  Ursino, Mauro ^a   Cuppini, Cristiano ^a   Magosso, Elisa ^a  

^a UNIVERSITY OF BOLOGNA   (Italy)

Author keywords

Auto associative and hetero associative networks; Bayesian and biologically inspired models; Feedforward and feedback architectures; Inverse effectiveness principle; Multisensory enhancement and suppression; Synaptic learning mechanisms

Indexed keywords

BAYESIAN NETWORKS; BIOMIMETICS; BRAIN; CONFORMAL MAPPING; INTEGRATION; LEARNING SYSTEMS; MEMORY ARCHITECTURE; NETWORK ARCHITECTURE; OBJECT RECOGNITION; PSYCHOPHYSIOLOGY; SELF ORGANIZING MAPS;

BIOLOGICALLY INSPIRED MODELS; FEEDBACK ARCHITECTURE; HETERO-ASSOCIATIVE; INVERSE EFFECTIVENESS PRINCIPLE; LEARNING MECHANISM; MULTISENSORY;

PHYSIOLOGICAL MODELS;

ARTICLE; BAYESIAN LEARNING; BRAIN CORTEX; BRAIN DEVELOPMENT; BRAIN FUNCTION; BRAIN MATURATION; COGNITION; FEEDBACK SYSTEM; HUMAN; KERNEL METHOD; LEARNING ALGORITHM; NERVE CELL NETWORK; NERVE CELL PLASTICITY; NEUROPHYSIOLOGY; NONHUMAN; NONLINEAR SYSTEM; PERCEPTION; SEMANTIC MEMORY; SENSORIMOTOR INTEGRATION; SUPERIOR COLLICULUS; VISUAL CORTEX; VISUAL MEMORY; BIOLOGICAL MODEL; BRAIN; HEARING; MEMORY; NERVE CELL; PHYSIOLOGY; SYSTEM ANALYSIS; VISION;

AUDITORY PERCEPTION; BRAIN; HUMANS; MEMORY; MODELS, NEUROLOGICAL; NEURONS; SYSTEMS INTEGRATION; VISUAL PERCEPTION;

EID: 84907537291     PISSN: 08936080     EISSN: 18792782     Source Type: Journal    
DOI: 10.1016/j.neunet.2014.08.003     Document Type: Article

References (213)

1. Alais D., Burr D. The ventriloquist effect results from near-optimal bimodal integration. Current Biology: CB 2004, 14:257-262.
2. Alais D., Newell F.N., Mamassian P. Multisensory processing in review: from physiology to behaviour. Seeing and Perceiving 2010, 23:3-38.
3. Alvarado J.C., Rowland B.A., Stanford T.R., Stein B.E. A neural network model of multisensory integration also accounts for unisensory integration in superior colliculus. Brain Research 2008, 1242:13-23.
4. Anastasio T.J., Patton P.E. A two-stage unsupervised learning algorithm reproduces multisensory enhancement in a neural network model of the corticotectal system. Journal of Neuroscience 2003, 23:6713-6727.
5. Anastasio T.J., Patton P.E., Belkacem-Boussaid K. Using Bayes rule to model multisensory enhancement in the superior colliculus. Neural Computation 2000, 12:1165-1187.
6. Andersen T.S., Tiippana K., Sams M. Factors influencing audiovisual fission and fusion illusions. Cognitive Brain Research 2004, 21:301-308.
7. Barsalou L.W. Perceptual symbol systems. Behavioral and Brain Sciences 1999, 22:577-609.
8. Barsalou L.W. Grounded cognition. Annual Review of Psychology 2008, 59:617-645.
9. Barsalou L.W., Kyle Simmons W., Barbey A.K., Wilson C.D. Grounding conceptual knowledge in modality-specific systems. Trends in Cognitive Sciences 2003, 7:84-91.
10. Bastos A.M., Usrey W.M., Adams R.A., Mangun G.R., Fries P., Friston K.J. Canonical microcircuits for predictive coding. Neuron 2012, 76:695-711.
11. Battaglia P.W., Jacobs R.A., Aslin R.N. Bayesian integration of visual and auditory signals for spatial localization. Journal of the Optical Society of America A. Optics, Image Science, and Vision 2003, 20:1391-1397.
12. Bauer, J., Weber, C., & Wermter, S. 2012, A SOM-based model for multi-sensory integration in the superior colliculus. In Neural Networks (IJCNN), The 2012 international joint conference on (pp. 1-8).
13. Beierholm U.R., Quartz S.R., Shams L. Bayesian priors are encoded independently from likelihoods in human multisensory perception. Journal of Vision 2009, 9:23.
14. Bell A.H., Corneil B.D., Meredith M.A., Munoz D.P. The influence of stimulus properties on multisensory processing in the awake primate superior colliculus. Canadian Journal of Experimental Psychology 2001, 55:123-132.
15. Bertelson P., Radeau M. Cross-modal bias and perceptual fusion with auditory-visual spatial discordance. Attention, Perception, & Psychophysics 1981, 29:578-584.
16. Binder J.R., Desai R.H. The neurobiology of semantic memory. Trends in Cognitive Sciences 2011, 15:527-536.
17. Binns K.E., Salt T.E. Importance of NMDA receptors for multimodal integration in the deep layers of the cat superior colliculus. Journal of Neurophysiology 1996, 75:920-930.
18. Bizley J.K., King A.J. Visual influences on ferret auditory cortex. Hearing Research 2009, 258:55-63.
19. Bolognini N., Senna I., Maravita A., Pascual-Leone A., Merabet L.B. Auditory enhancement of visual phosphene perception: The effect of temporal and spatial factors and of stimulus intensity. Neuroscience Letters 2010, 477:109-114.
20. Borghi A.M., Cimatti F. Embodied cognition and beyond: Acting and sensing the body. Neuropsychologia 2010, 48:763-773.
21. Borisyuk R., Kazanovich Y. Oscillatory model of attention-giuded object selection and novelty detection. Neural Networks 2004, 17:899-915.
22. Bremmer F., Schlack A., Shah N.J., Zafiris O., Kubischik M., Hoffmann K., Zilles K., Fink G.R. Polymodal motion processing in posterior parietal and premotor cortex: a human fMRI study strongly implies equivalencies between humans and monkeys. Neuron 2001, 29:287-296.
23. Bresciani J.P., Dammeier F., Ernst M.O. Vision and touch are automatically integrated for the perception of sequences of events. Journal of Vision 2006, 6.
24. Bruce C., Desimone R., Gross C.G. Visual properties of neurons in a polysensory area in superior temporal sulcus of the macaque. Journal of Neurophysiology 1981, 46:369-384.
25. Buzsáki G. Rhythms of the brain 2006, Oxford University Press, New York.
26. Caclin A., Soto-Faraco S., Kingstone A., Spence C. Tactile "capture" of audition. Perception & psychophysics 2002, 64:616-630.
27. Calvert G.A., Spence C., Stein B.E. The handbook of multisensory processes 2004, MIT Press, Cambridge, MA, US.
28. Canzoneri E., Magosso E., Serino A. Dynamic sounds capture the boundaries of peripersonal space representation in humans. PLoS One 2012, 7:e44306.
29. Cappe C., Barone P. Heteromodal connections supporting multisensory integration at low levels of cortical processing in the monkey. European Journal of Neuroscience 2005, 22:2886-2902.
30. Casey M.C., Pavlou A., Timotheou A. Audio-visual localization with hierarchical topographic maps: Modeling the superior colliculus. Neurocomputing 2012, 97:344-356.
31. Chen C.C., Henson R.N., Stephan K.E., Kilner J.M., Friston K.J. Forward and backward connections in the brain: A DCM study of functional asymmetries. Neuroimage 2009, 45:453-462.
32. Colonius H., Diederich A. Why aren't all deep superior colliculus neurons multisensory? A Bayes' ratio analysis. Cognitive Affective and Behavior Neuroscience 2004, 4:344-353.
33. Cree G.S., McNorgan C., McRae K. Distinctive features hold a privileged status in the computation of word meaning: Implications for theories of semantic memory. Journal of Experimental Psychology: Learning, Memory, and Cognition 2006, 32:643-658.
34. Cuppini C., Magosso E., Bolognini N., Vallar G., Ursino M. A neurocomputational analysis of the sound-induced flash illusion. Neuroimage 2014, 92:248-266.
35. Cuppini C., Magosso E., Rowland B., Stein B., Ursino M. Hebbian mechanisms help explain development of multisensory integration in the superior colliculus: a neural network model. Biological Cybernetics 2012, 106:691-713.
36. Cuppini C., Magosso E., Ursino M. A neural network model of cortical auditory-visual interactions a neurocomputational analysis of the shams-illusion. IJCCI 2012-4th International Joint Conference on Computational Intelligence 2012, 639-642. SciTePress-Science and Technology Publications, Barcelona, Spain. A. Rosa, A. Dourado, K. Madani, J. Filipe, J. Kacprzyk (Eds.).
37. Cuppini C., Stein B.E., Rowland B.A., Magosso E., Ursino M. A computational study of multisensory maturation in the superior colliculus (SC). Experimental Brain Research 2011, 213:341-349.
38. Cuppini C., Ursino M., Magosso E., Rowland B.A., Stein B.E. An emergent model of multisensory integration in superior colliculus neurons. Frontiers in Integrative Neuroscience 2010, 4:1-15.
39. Damasio A.R. The brain binds entities and events by multiregional activation from convergence zones. Neural Computation 1989, 1:123-132.
40. Damasio A.R. Time-locked multiregional retroactivation: a systems level proposal for the neural substrates of recall and recognition. Cognition 1989, 33:25-62.
41. Damasio A.R. Self comes to mind: constructing the conscious brain 2010, Pantheon Book, New York.
42. di Pellegrino G., Ladavas E., Farnè A. Seeing where your hands are. Nature 1997, 388:730.
43. Duhamel J.R., Colby C.L., Goldberg M.E. Ventral intraparietal area of the macaque: congruent visual and somatic response properties. Journal of Neurophysiology 1998, 79:126-136.
44. Eckhorn R., Reitboeck H.J., Arndt M., Dicke P.W. Feature-linking via synchronization among distributed assemblies: simulation of results from cat cortex. Neural Computation 1990, 2:293-307.
45. Edwards S.B., Ginsburgh C.L., Henkel C.K., Stein B.E. Sources of subcortical projections to the superior colliculus in the cat. Journal of Comparative Neurology 1979, 184:309-330.
46. Ernst M.O., Banks M.S. Humans integrate visual and haptic information in a statistically optimal fashion. Nature 2002, 415:429-433.
47. Farnè A., Ladavas E. Dynamic size-change of hand peripersonal space following tool use. Neuroreport 2000, 11:1645-1649.
48. Farnè A., Ladavas E. Auditory peripersonal space in humans. Journal of Cognitive Neuroscience 2002, 14:1030-1043.
49. Fetsch C.R., Pouget A., DeAngelis G.C., Angelaki D.E. Neural correlates of reliability-based cue weighting during multisensory integration. Nature Neuroscience 2012, 15:146-154.
50. Fetsch C.R., Turner A.H., DeAngelis G.C., Angelaki D.E. Dynamic reweighting of visual and vestibular cues during self-motion perception. The Journal of Neuroscience 2009, 29:15601-15612.
51. Fodor J. The Modularity of mind [electronic resource]: an essay on faculty psychology 1983, The MIT Press.
52. Fogassi L., Gallese V., Fadiga L., Luppino G., Matelli M., Rizzolatti G. Coding of peripersonal space in inferior premotor cortex (area F4). Journal of Neurophysiology 1996, 76:141-157.
53. Freeman E., Driver J. Direction of visual apparent motion driven solely by timing of a static sound. Current Biology 2008, 18:1262-1266.
54. Frens M.A., Van Opstal A.J., Van der Willigen R.F. Spatial and temporal factors determine auditory-visual interactions in human saccadic eye movements. Perception and Psychophysiology 1995, 57:802-816.
55. Fries P., Nikolic D., Singer W. The gamma cycle. Trends in Neurosciences 2007, 30:309-316.
56. Frissen I., Vroomen J., de Gelder B., Bertelson P. The aftereffects of ventriloquism: generalization across sound-frequencies. Acta Psychologica 2005, 118:93-100.
57. Friston K. A theory of cortical responses. Philosophical Transactions of the Royal Society B: Biological Sciences 2005, 360:815-836.
58. Friston K. The free-energy principle: a unified brain theory?. Natural Review of Neuroscience 2010, 11:127-138.
59. Friston K., Kiebel S. Cortical circuits for perceptual inference. Neural Networks 2009, 22:1093-1104.
60. Gallese V., Lakoff G. The Brain's concepts: the role of the Sensory-motor system in conceptual knowledge. Cognitive Neuropsychology 2005, 22:455-479.
61. Gallese V., Sinigaglia C. What is so special about embodied simulation?. Trends in Cognitive Sciences 2011, 15:512-519.
62. Garrido M.I., Kilner J.M., Kiebel S.J., Friston K.J. Evoked brain responses are generated by feedback loops. Proceedings of the National Academy of Sciences 2007, 104:20961-20966.
63. Getzmann S. The effect of brief auditory stimuli on visual apparent motion. Perception 2007, 36:1089-1103.
64. Ghazanfar A.A., Schroeder C.E. Is neocortex essentially multisensory?. Trends in Cognitive Sciences 2006, 10:278-285.
65. Graham J., Vaughan J., Schaafsma S., Wallace M., Stein B. Role of NMDA receptors in multisensory integration in the superior colliculus. In soc neurosci abstr 1996, 22:637.
66. Graziano M.S., Gross C.G. The representation of extrapersonal space: a possible role for bimodal, visual-tactile neurons. The cognitive neurosciences 1995, 1021-1034. MIT Press, Cambridge. M.S. Gazzaniga (Ed.).
67. Graziano M.S., Hu X.T., Gross C.G. Visuospatial properties of ventral premotor cortex. Journal of Neurophysiology 1997, 77:2268-2292.
68. Gu Y., Angelaki D.E., DeAngelis G.C. Neural correlates of multisensory cue integration in macaque MSTd. Nature Neuroscience 2008, 11:1201-1210.
69. Guest S., Catmur C., Lloyd D., Spence C. Audiotactile interactions in roughness perception. Experimental Brain Research 2002, 146:161-171.
70. Hairston W.D., Wallace M.T., Vaughan J.W., Stein B.E., Norris J.L., Schirillo J.A. Visual localization ability influences cross-modal bias. Journal of Cognitive Neuroscience 2003, 15:20-29.
71. Harrison L.M., Stephan K.E., Rees G., Friston K.J. Extra-classical receptive field effects measured in striate cortex with fMRI. Neuroimage 2007, 34:1199-1208.
72. Hart J., Segal J.B., Adkins E., Kraut M. Neural substrates of semantics. Handbook of language disorders 2002, Psychology Press, Philadelphia. A. Hillis (Ed.).
73. Helbig H.B., Ernst M.O. Optimal integration of shape information from vision and touch. Experimental Brain Research 2007, 179:595-606.
74. Holmes N.P. The law of inverse effectiveness in neurons and behaviour: multisensory integration versus normal variability. Neuropsychologia 2007, 45:3340-3345.
75. Holmes N.P. Inverse effectiveness, multisensory integration, and the bodily self: some statistical considerations. Consciousness and Cognition 2009, 18:762-765.
76. Hoshino O. Coherent ongoing subthreshold state of a cortical neural network regulated by slow- and fast-spiking interneurons. Network-Computation in Neural Systems 2006, 17:351-371.
77. Hoshino O. An ongoing subthreshold neuronal state established through dynamic coassembling of cortical cells. Neural Computation 2008, 20:3055-3086.
78. Hoshino O. GABA transporter preserving ongoing spontaneous neuronal activity at firing subthreshold. Neural Computation 2009, 21:1683-1713.
79. Hoshino O. Neuronal responses below firing threshold for subthreshold cross-modal enhancement. Neural Computation 2011, 23:958-983.
80. Hoshino O. Regulation of ambient GABA levels by neuron-glia signaling for reliable perception of multisensory events. Neural Computation 2012, 24:2964-2993.
81. Huang Y., Rao R.P.N. Predictive coding. Wiley Interdisciplinary Reviews: Cognitive Science 2011, 2:580-593.
82. Huerta M.F., Harting J.K. The mammalian superior colliculus: studies of its morphology and connections. Comparative neurology of the optic tectum 1984, 687-773. Plenum, New York. H. Vanega (Ed.).
83. Hyvarinen J. Regional distribution of functions in parietal association area 7 of the monkey. Brain Research 1981, 206:287-303.
84. Innes-Brown H., Crewther D. The impact of spatial incongruence on an auditory-visual illusion. PLoS One 2009, 4:e6450.
85. Iriki A., Tanaka M., Iwamura Y. Coding of modified body schema during tool use by macaque postcentral neurones. Neuroreport 1996, 7:2325-2330.
86. Ishibashi H., Hihara S., Takahashi M., Heike T., Yokota T., Iriki A. Tool-use learning selectively induces expression of brain-derived neurotrophic factor, its receptor trkB, and neurotrophin 3 in the intraparietal multisensorycortex of monkeys. Brain Research. Cognitive Brain Research 2002, 14:3-9.
87. Jiang W., Jiang H., Stein B.E. Two corticotectal areas facilitate multisensory orientation behavior. Journal of Cognitive Neuroscience 2002, 14:1240-1255.
88. Jiang W., Jiang H., Stein B.E. Neonatal cortical ablation disrupts multisensory development in superior colliculus. Journal of Neurophysiology 2006, 95:1380-1396.
89. Jiang W., Wallace M.T., Jiang H., Vaughan J.W., Stein B.E. Two cortical areas mediate multisensory integration in superior colliculus neurons. Journal of Neurophysiology 2001, 85:506-522.
90. Johnson-Laird P.N., Shafir E. The interaction between reasoning and decision making: an introduction. Cognition 1993, 49:1-9.
91. Kadunce D.C., Vaughan J.W., Wallace M.T., Benedek G., Stein B.E. Mechanisms of within- and cross-modality suppression in the superior colliculus. J Neurophysiol 1997, 78:2834-2847.
92. Kadunce D.C., Vaughan J.W., Wallace M.T., Stein B.E. The influence of visual and auditory receptive field organization on multisensory integration in the superior colliculus. Experimental Brain Research 2001, 139:303-310.
93. Kayser C., Petkov C.I., Logothetis N.K. Visual modulation of neurons in auditory cortex. Cerebral Cortex 2008, 18:1560-1574.
94. Kennett S., Taylor-Clarke M., Haggard P. Noninformative vision improves the spatial resolution of touch in humans. Current Biology 2001, 11:1188-1191.
95. King A.J., Walker K.M. Integrating information from different senses in the auditory cortex. Biological Cybernetics 2012, 106:617-625.
96. Körding K.P., Beierholm U., Ma W.J., Quartz S., Tenenbaum J.B., Shams L. Causal Inference in Multisensory Perception. PLoS One 2007, 2:e943.
97. Kraut M.A., Pitcock J.A., Hart J. Neural mechanisms of semantic memory. Current Neurology and Neuroscience Reports 2004, 4:461-465.
98. Ladavas E., di Pellegrino G., Farnè A., Zeloni G. Neuropsychological evidence of an integrated visuotactile representation of peripersonal space in humans. Journal of Cognitive Neuroscience 1998, 10:581-589.
99. Ladavas E., Farnè A., Zeloni G., di Pellegrino G. Seeing or not seeing where your hands are. Experimental Brain Research 2000, 131:458-467.
100. Lakatos P., Chen C.-M, O'Connell M.N., Mills A., Schroeder C.E. Neuronal oscillations and multisensory interaction in primary auditory cortex. Neuron 2007, 53:279-292.
101. Landauer T.K., Dumais S.T. A solution to Plato's problem: the latent semantic analysis theory of acquisition, induction, and representation of knowledge. Psychological Review 1997, 104:211.
102. Lewald J. Rapid adaptation to auditory-visual spatial disparity. Learning & Memory 2002, 9:268-278.
103. Lim H.K., Keniston L.P., Cios K.J. Modeling of multisensory convergence with a network of spiking neurons: a reverse engineering approach. IEEE Transactions on Biomedical Engineering 2011, 58:1940-1949.
104. Lim H.K., Keniston L.P., Shin J.H., Allman B.L., Meredith M.A., Cios K.J. Connectional parameters determine multisensory processing in a spiking network model of multisensory convergence. Experimental Brain Research 2011, 213:329-339.
105. Ma W.J., Beck J.M., Latham P.E., Pouget A. Bayesian inference with probabilistic population codes. Natural Neuroscience 2006, 9:1432-1438.
106. Ma W.J., Zhou X., Ross L.A., Foxe J.J., Parra L.C. Lip-reading aids word recognition most in moderate noise: a Bayesian explanation using high-dimensional feature space. PLoS One 2009, 4:e4638.
107. Ma W.J., Rahmati M. Towards a neural implementation of causal inference in cue combination. Multisensory Research 2012, 26:159-176.
108. Magosso E. Integrating information from vision and touch: a neural network modeling study. IEEE Transactions on Information Technology in Biomedicine 2010, 14:598-612.
109. Magosso E., Cona F., Ursino M. A neural network model can explain ventriloquism aftereffect and its generalization across sound frequencies. Biomedical Research International 2013, 2013:1-17. Article ID 475427.
110. Magosso E., Cuppini C., Serino A., Di Pellegrino G., Ursino M. A theoretical study of multisensory integration in the superior colliculus by a neural network model. Neural Networks 2008, 21:817-829.
111. Magosso E., Cuppini C., Ursino M. A neural network model of ventriloquism effect and aftereffect. PLoS One 2012, 7:e42503.
112. Magosso E., Ursino M., di Pellegrino G., Ladavas E., Serino A. Neural bases of peri-hand space plasticity through tool-use: insights from a combined computational-experimental approach. Neuropsychologia 2010, 48:812-830.
113. Magosso E., Zavaglia M., Serino A., di Pellegrino G., Ursino M. Visuotactile representation of peripersonal space: a neural network study. Neural Computation 2010, 22:190-243.
114. Mahon B.Z., Caramazza A. A critical look at the embodied cognition hypothesis and a new proposal for grounding conceptual content. Journal of Physiology-Paris 2008, 102:59-70.
115. Makin T.R., Holmes N.P., Zohary E. Is that near my hand? Multisensory representation of peripersonal space in human intraparietal sulcus. The Journal of Neuroscience 2007, 27:731-740.
116. Maravita A., Husain M., Clarke K., Driver J. Reaching with a tool extends visual-tactile interactions into far space: evidence from cross-modal extinction. Neuropsychologia 2001, 39:580-585.
117. Martin A. The representation of object concepts in the brain. Annual Review of Psychology 2007, 58:25-45.
118. Martin A., Chao L.L. Semantic memory and the brain: structure and processes. Current Opinion in Neurobiology 2001, 11:194-201.
119. Martin J.G., Meredith M.A., Ahmad K. Modeling multisensory enhancement with self-organizing maps. Frontiers in Computational Neuroscience 2009, 3:1-10.
120. McRae K. Semantic memory: some insights from feature-based connectionist attractor networks. The psychology of learning and motivation: advances in research and theory 2004, 41-86. Academic Press, San Diego, CA. B.H. Ross (Ed.).
121. McRae K., de Sa V.R., Seidenberg M.S. On the nature and scope of featural representations of word meaning. Journal of Experimental Psychology Genetics 1997, 126:99-130.
122. Meredith M.A., Clemo H.R. Auditory cortical projection from the anterior ectosylvian sulcus (Field AES) to the superior colliculus in the cat: an anatomical and electrophysiological study. Journal of Comparative Neurology 1989, 289:687-707.
123. Meredith M.A., Nemitz J.W., Stein B.E. Determinants of multisensory integration in superior colliculus neurons. I. Temporal factors. Journal of Neuroscience 1987, 7:3215-3229.
124. Meredith M.A., Stein B.E. Interactions among converging sensory inputs in the superior colliculus. Science 1983.
125. Meredith M.A., Stein B.E. Spatial factors determine the activity of multisensory neurons in cat superior colliculus. Brain Research 1986, 365:350-354.
126. Meredith M.A., Stein B.E. Visual, auditory, and somatosensory convergence on cells in superior colliculus results in multisensory integration. Journal of Neurophysiology 1986, 56:640-662.
127. Meredith M.A., Stein B.E. Spatial determinants of multisensory integration in cat superior colliculus neurons. Journal of Neurophysiology 1996, 75:1843-1857.
128. Meteyard L., Cuadrado S.R., Bahrami B., Vigliocco G. Coming of age: A review of embodiment and the neuroscience of semantics. Cortex 2012, 48:788-804.
129. Miikkulainen R. Subsymbolic natural language processing: an integrated model of scripts, lexicon, and memory 1993, MIT Press, Cambridge, MA.
130. Miikkulainen R. Dyslexic and category-specific aphasic impairments in a self-organizing feature map model of the lexicon. Brain and Language 1997, 59:334-366.
131. Morgan M.L., DeAngelis G.C., Angelaki D.E. Multisensory integration in macaque visual cortex depends on cue reliability. Neuron 2008, 59:662-673.
132. Morrell F. Visual system's view of acoustic space. Nature 1972, 238:44-46.
133. Musacchia G., Schroeder C.E. Neuronal mechanisms, response dynamics and perceptual functions of multisensory interactions in auditory cortex. Hearing Research 2009, 258:72-79.
134. O'Connor C.M., Cree G.S., McRae K. Conceptual hierarchies in a flat attractor network: dynamics of learning and computations. Cognitive Science 2009, 33:665-708.
135. Ohshiro T., Angelaki D.E., DeAngelis G.C. A normalization model of multisensory integration. Nature Neuroscience 2011, 14:775-782.
136. Parise C., Spence C. Synesthetic congruency modulates the temporal ventriloquism effect. Neuroscience Letters 2008, 442:257-261.
137. Patton P.E., Anastasio T.J. Modelling cross-modal enhancement and modality-specific suppression in multisensory neurons. Neural Computation 2003, 15:783-810.
138. Patton P.E., Belkacem-Boussaid K., Anastasio T.J. Multimodality in the superior colliculus: an information theoretic analysis. Brain Research. Brain Research 2002, 14:10-19.
139. Pavlou, A., & Casey, M. 2010, Simulating the effects of cortical feedback in the superior colliculus with topographic maps. In Neural Networks (IJCNN), The 2010 International Joint Conference on (pp. 1-8).
140. Perrault T.J., Vaughan J.W., Stein B.E., Wallace M.T. Neuron-specific response characteristics predict the magnitude of multisensory integration. Journal of Neurophysiology 2003, 90:4022-4026.
141. Perrault T.J., Vaughan J.W., Stein B.E., Wallace M.T. Superior colliculus neurons use distinct operational modes in the integration of multisensory stimuli. Journal of Neurophysiology 2005, 93:2575-2586.
142. Poremba A., Saunders R.C., Crane A.M., Cook M., Sokoloff L., Mishkin M. Functional mapping of the primate auditory system. Science 2003, 299:568-572.
143. Pouget A., Beck J.M., Ma W.J., Latham P.E. Probabilistic brains: knowns and unknowns. Nature Neuroscience 2013, 16:1170-1178.
144. Press C., Taylor-Clarke M., Kennett S., Haggard P. Visual enhancement of touch in spatial body representation. Experimental Brain Research 2004, 154:238-245.
145. Pulvermüller F. Brain mechanisms linking language and action. Natural Review Neuroscience 2005, 6:576-582.
146. Pulvermüller F., Lutzenberger W., Preissl H. Nouns and verbs in the intact brain: evidence from event-related potentials and high-frequency cortical responses. Cerebral Cortex 1999, 9:497-506.
147. Pylyshyn, Z.W. 1984, Computation and cognition: Towards a foundation for cognitive science. Cambridge, MA.
148. Radeau M., Bertelson P. Auditory-visual interaction and the timing of inputs - Thomas (1941) revisited. Psychological Research 1987, 49:17-22.
149. Ralph M.A.L. Neurocognitive insights on conceptual knowledge and its breakdown. Philosophical Transactions of the Royal Society B: Biological Sciences 2014, 369:20120392.
150. Rao R.P.N., Ballard D.H. Predictive coding in the visual cortex: a functional interpretation of some extra-classical receptive-field effects. Natural Neuroscience 1999, 2:79-87.
151. Recanzone G.H. Rapidly induced auditory plasticity: the ventriloquism aftereffect. Proceedings of the National Academy of Sciences of the United States of America 1998, 95:869-875.
152. Recanzone G.H. Auditory influences on visual temporal rate perception. Journal of Neurophysiology 2003, 89:1078-1093.
153. Rizzolatti G., Scandolara C., Matelli M., Gentilucci M. Afferent properties of periarcuate neurons in macaque monkeys. II. Visual responses. Behavioural Brain Research 1981, 2:147-163.
154. Roach N.W., Heron J., McGraw P.V. Resolving multisensory conflict: a strategy for balancing the costs and benefits of audio-visual integration. Proceedings of the Royal Society B: Biological Sciences 2006, 273:2159-2168.
155. Rock I., Victor J. Vision and touch: an experimentally created conflict between the two senses. Science 1964, 143:594-596.
156. Rogers T.T., Lambon Ralph M.A., Garrard P., Bozeat S., McClelland J.L., Hodges J.R., Patterson K. Structure and deterioration of semantic memory: a neuropsychological and computational investigation. Psychological Review 2004, 111:205-235.
157. Ross L.A., Saint-Amour D., Leavitt V.M., Javitt D.C., Foxe J.J. Do you see what I am saying? Exploring visual enhancement of speech comprehension in noisy environments. Cerebral Cortex 2007, 17:1147-1153.
158. Rowland B.A., Stanford T.R., Stein B.E. A Bayesian model unifies multisensory spatial localization with the physiological properties of the superior colliculus. Experimental Brain Research 2007, 180:153-161.
159. Rowland B.A., Stanford T.R., Stein B.E. A model of the neural mechanisms underlying multisensory integration in the superior colliculus. Perception 2007, 36:1431-1443.
160. Sambo C.F., Forster B. An ERP investigation on visuotactile interactions in peripersonal and extrapersonal space: evidence for the spatial rule. Journal of Cognitive Neuroscience 2009, 21:1550-1559.
161. Sato Y., Toyoizumi T., Aihara K. Bayesian inference explains perception of unity and ventriloquism aftereffect: identification of common sources of audiovisual stimuli. Neural Computation 2007, 19:3335-3355.
162. Schroeder C.E., Foxe J. Multisensory contributions to low-level, 'unisensory' processing. Current Opinion in Neurobiology 2005, 15:454-458.
163. Schröger E., Widmann A. Speeded responses to audiovisual signal changes result from bimodal integration. Psychophysiology 1998, 35:755-759.
164. Serino A., Bassolino M., Farnè A., Ladavas E. Extended multisensory space in blind cane users. Psychological Science 2007, 18:642-648.
165. Serino A., Farnè A., Rinaldesi M.L., Haggard P., Ladavas E. Can vision of the body ameliorate impaired somatosensory function?. Neuropsychologia 2007, 45:1101-1107.
166. Shams L., Beierholm U.R. Causal inference in perception. Trends in Cognitive Sciences 2010, 14:425-432.
167. Shams L., Iwaki S., Chawla A., Bhattacharya J. Early modulation of visual cortex by sound: an MEG study. Neuroscience Letters 2005, 378:76-81.
168. Shams L., Kamitani Y., Shimojo S. Visual illusion induced by sound. Cognitive Brain Research 2002, 14:147-152.
169. Shams L., Kamitani Y., Shimojo S. Illusions: what you see is what you hear. Nature 2000, 408:788.
170. Shams L., Kim R. Crossmodal influences on visual perception. Physics of Life Reviews 2010, 7:269-284.
171. Shams L., Ma W.J., Beierholm U. Sound-induced flash illusion as an optimal percept. Neuroreport 2005, 16:1923-1927.
172. Shipley T. Auditory flutter-driving of visual flicker. Science 1964, 145:1328-1330.
173. Silberman Y., Bentin S., Miikkulainen R. Semantic boost on episodic associations: an empirically-based computational model. Cognitive Science 2007, 31:645-671.
174. Slutsky D.A., Recanzone G.H. Temporal and spatial dependency of the ventriloquism effect. Neuroreport 2001, 12:7-10.
175. Sompolinsky H., Golomb D., Kleinfeld D. Global processing of visual stimuli in a neural network of coupled oscillators. Proceedingsof the National Academy of Sciences of the United Statesof America 1990, 87:7200-7204.
176. Sparks D.L. Translation of sensory signals into commands for control of saccadic eye movements: role of primate superior colliculus. Physiol Rev 1986, 66:118-171.
177. Srinivasan M.V., Laughlin S.B., Dubs A. Predictive coding: a fresh view of inhibition in the retina. Proceedings of the Royal Society of London. Series B. Biological Sciences 1982, 216:427-459.
178. Stanford T.R., Quessy S., Stein B.E. Evaluating the operations underlying multisensory integration in the cat superior colliculus. Journal of Neuroscience 2005, 25:6499-6508.
179. Stein B.E., Meredith M.A. The merging of the senses 1993, MIT Press, Cambridge, MA.
180. Stein B.E., Meredith M.A., Huneycutt W.S., McDade L. Behavioral indices of multisensory integration: orientation to visual cues is affected by auditory stimuli. Journal of Cognitive Neuroscience 1989, 1:12-24.
181. Stein B.E., Spencer R.F., Edwards S.B. Corticotectal and corticothalamic efferent projections of SIV somatosensory cortex in cat. Journal of Neurophysiology 1983, 50:896-909.
182. Taylor-Clarke M., Kennett S., Haggard P. Vision modulates somatosensory cortical processing. Current Biology 2002, 12:233-236.
183. Tipper S.P., Lloyd D., Shorland B., Dancer C., Howard L.A., McGlone F. Vision influences tactile perception without proprioceptive orienting. Neuroreport 1998, 9:1741-1744.
184. Tyler L.K., Moss H.E. Towards a distributed account of conceptual knowledge. Trends Cognitive Science 2001, 5:244-252.
185. Ursino M., Cuppini C., Magosso E. A computational model of the lexical-semantic system based on a grounded cognition approach. Frontiers in Psychology 2010, 1:221.
186. Ursino M., Cuppini C., Magosso E. An integrated neural model of semantic memory, lexical retrieval and category formation, based on a distributed feature representation. Cognitive Neurodynamics 2011, 5:183-207.
187. Ursino M., Cuppini C., Magosso E. The formation of categories and the representation of feature saliency: Analysis with a computational model trained with an Hebbian paradigm. Journal of Integrative Neuroscience 2013, 12:401-425.
188. Ursino M., Cuppini C., Magosso E. A neural network for learning the meaning of objects and words from a featural representation. Neural Networks 2014, (submitted for publication).
189. Ursino M., Cuppini C., Magosso E., Serino A., Di Pellegrino G. Multisensory integration in the superior colliculus: a neural network model. Journal of Computational Neuroscience 2009, 26:55-73.
190. Ursino M., La Cara G.E., Sarti A. Binding and segmentation of multiple objects through neural oscillators inhibited by contour information. Biol. Cybern. 2003, 89:56-70.
191. Ursino M., Magosso E., Cuppini C. Recognition of abstract objects via neural oscillators: interaction among topological organization, associative memory and gamma band synchronization. IEEE Transactions on Neural Networks 2009, 20:316-335.
192. Vaishnavi S., Calhoun J., Chatterjee A. Binding personal and peripersonal space: evidence from tactile extinction. Journal of Cognitive Neuroscience 2001, 13:181-189.
193. Varela F., Lachaux J.P., Rodriguez E., Martinerie J. The brainweb: phase synchronization and large-scale integration. Natural Review Neuroscience 2001, 2:229-239.
194. Vigliocco G., Vinson D.P., Lewis W., Garrett M.F. Representing the meanings of object and action words: The featural and unitary semantic space hypothesis. Cognitive Psychology 2004, 48:422-488.
195. von der Malsburg C., Buhmann J. Sensory segmentation with coupled neural oscillators. Biological Cybernetics 1992, 67:233-242.
196. Vroomen J., de Gelder B. Temporal ventriloquism: sound modulates the flash-lag effect. Journal of Experimental Psychology: Human Perception and Performance 2004, 30:513-518.
197. Wallace M.T., Meredith M.A., Stein B.E. Converging influences from visual, auditory, and somatosensory cortices onto output neurons of the superior colliculus. Journal of Neurophysiology 1993, 69:1797-1809.
198. Wallace M.T., Meredith M.A., Stein B.E. Multisensory integration in the superior colliculus of the alert cat. Journal of Neurophysiology 1998, 80:1006-1010.
199. Wallace M.T., Perrault T.J., Hairston W.D., Stein B.E. Visual experience is necessary for the development of multisensory integration. Journal of Neuroscience 2004, 24:9580-9584.
200. Wallace M.T., Roberson G.E., Hairston W.D., Stein B.E., Vaughan J.W., Schirillo J.A. Unifying multisensory signals across time and space. Experimental Brain Research 2004, 158:252-258.
201. Wallace M.T., Stein B.E. Development of multisensory neurons and multisensory integration in cat superior colliculus. Journal of Neuroscience 1997, 17:2429-2444.
202. Wallace M.T., Stein B.E. Onset of cross-modal synthesis in the neonatal superior colliculus is gated by the development of cortical influences. Journal of Neurophysiology 2000, 83:3578-3582.
203. Wallace M.T., Stein B.E. Sensory and multisensory responses in the newborn monkey superior colliculus. Journal of Neuroscience 2001, 21:8886-8894.
204. Wallace M.T., Stein B.E. Early experience determines how the senses will interact. Journal of Neurophysiology 2007, 97:921-926.
205. Wang D. The time dimension for scene analysis. IEEE Transactions on Neural Networks 2005, 16:1401-1426.
206. Wang D., Terman D. Image segmentation based on oscillatory correlation. Neural Computation 1997, 9:805-836.
207. Wilkinson L.K., Meredith M.A., Stein B.E. The role of anterior ectosylvian cortex in cross-modality orientation and approach behavior. Experimental Brain Research 1996, 112:1-10.
208. Wozny D.R., Beierholm U.R., Shams L. Human trimodal perception follows optimal statistical inference. Journal of Vision 2008, 8:24.
209. Yildirim I., Jacobs R.A. A rational analysis of the acquisition of multisensory representations. Cognitive Science 2012, 36:305-332.
210. Yildirim I., Jacobs R.A. Transfer of object category knowledge across visual and haptic modalities: Experimental and computational studies. Cognition 2013, 126:135-148.
211. Yu L., Rowland B.A., Stein B.E. Initiating the development of multisensory integration by manipulating sensory experience. The Journal of Neuroscience 2010, 30:4904-4913.
212. Zhou Y.D., Fuster J.M. Visuo-tactile cross-modal associations in cortical somatosensory cells. roceedings of the National Academy of Sciences of the United States of America 2000, 97:9777-9782.
213. Zhou Y.D., Fuster J.M. Somatosensory cell response to an auditory cue in a haptic memory task. Behavioural Brain Research 2004, 153:573-578.