Audio-visual localization with hierarchical topographic maps: Modeling the superior colliculus

Neurocomputing

Volumn 97, Issue , 2012, Pages 344-356

  Casey, M C ^a   Pavlou, A ^a   Timotheou, A ^a  

^a UNIVERSITY OF SURREY   (United Kingdom)

Author keywords

Audio visual localization; Computational neuroscience; Multisensory integration; Superior colliculus; Topographic maps

Indexed keywords

AUDIO-VISUAL; COMPUTATIONAL NEUROSCIENCE; MULTISENSORY INTEGRATION; SUPERIOR COLLICULUS; TOPOGRAPHIC MAP;

EYE MOVEMENTS; MAPS;

AUDIO ACOUSTICS;

ARTICLE; AUDITORY FEEDBACK; AUDITORY ORIENTATION; AUDITORY STIMULATION; BRAIN MAPPING; BRAIN REGION; ELECTROENCEPHALOGRAM; NERVE CELL NETWORK; NEUROFEEDBACK; NEUROPHYSIOLOGY; NOISE; PRIORITY JOURNAL; SOUND DETECTION; SPATIAL SUMMATION; SUPERIOR COLLICULUS; VISUAL ACUITY; VISUAL FEEDBACK; VISUAL FIELD; VISUAL ORIENTATION; VISUAL STIMULATION;

EID: 84865335126     PISSN: 09252312     EISSN: 18728286     Source Type: Journal    
DOI: 10.1016/j.neucom.2012.05.015     Document Type: Article

References (74)

1. McGurk H., MacDonald J. Hearing lips and seeing voices. Nature 1976, 264:746-748.
2. Botvinick M., Cohen J. Rubber hands 'feel' touch that eyes see. Nature 1998, 391:756.
3. Ramachandran V.S., Rogers-Ramachandran D., Cobb S. Touching the phantom limb. Nature 1995, 377:489-490.
4. Shams L., Kamitani Y., Shimojo S. Modulations of visual perception by sound. The Handbook of Multisensory Processes. A Bradford Book 2004, 27-33. MIT Press. G.A. Calvert, C. Spence, B.E. Stein (Eds.).
5. Calvert G.A., Thesen T. Multisensory integration: methodological approaches and emerging principles in the human brain. J. Physiol. Paris 2004, 98:191-205.
6. Stein B.E., Meredith M.A. The Merging of the Senses. A Bradford Book 1993, MIT Press, Cambridge, MA.
7. King A.J. The superior colliculus. Curr. Biol. 2004, 14:R335-R338.
8. Katyal S., Zughni S., Greene C., Ress D. Topography of covert visual attention in human superior colliculus. J. Neurophysiol. 2010, 104:3074-3083.
9. Stein B.E., Stanford T.R. Multisensory integration: current issues from the perspective of the single neuron. Nat. Rev. Neurosci. 2008, 9:255-266.
10. May P.J. The mammalian superior colliculus: laminar structure and connections. Progress in Brain Research: Neuroanatomy of the Oculomotor System 2009, vol. 151:321-378. Elsevier.
11. Yu L., Rowland B.A., Stein B.E. Initiating the development of multisensory integration by manipulating sensory experience. J. Neurosci. 2010, 30:4904-4913.
12. Alvarado J.C., Stanford T.R., Rowland B.A., Vaughan J.W., Stein B.E. Multisensory integration in the superior colliculus requires synergy among corticocollicular inputs. J. Neurosci. 2009, 29:6580-6592.
13. Moschovakis A.K., Scudder C.A., Highstein S.M. The microscopic anatomy and physiology of the mammalian saccadic system. Prog. Neurobiol. 1996, 50:133-254.
14. E.K.C. Tsang, B.E. Shi, Active binocular gaze control inspired by superior colliculus, in: Proceedings of the International Joint Conference on Neural Networks (IJCNN) 2006, IEEE, 2006, pp. 7-14.
15. M.C. Casey, D.L. Hickman, J.R.E. Sadler, Small-scale anomaly detection in panoramic imaging using neural models of low-level vision, in: Proceedings of SPIE Defense, Security, and Sensing Conference 2011 on Enhanced and Synthetic Vision, vol. 8042B, SPIE, 2011.
16. Lee J.S., Park C.H. Adaptive decision fusion for audio-visual speech recognition. Speech Recognition 2008, 275-296. InTech. F. Mihelic, J. Zibert (Eds.).
17. Ritter P., Villringer A. Simultaneous EEG-fMRI. Neurosci. Biobehav. Rev. 2006, 30:823-838.
18. Navalpakkam V., Itti L. Modeling the influence of task on attention. Vision Res. 2005, 45:205-231.
19. Anastasio T.J., Patton P.E., Belkacem-Boussaid K. Using Bayes' rule to model multisensory enhancement in the superior colliculus. Neural Comput. 2000, 12:1165-1187.
20. Patton P.E., Anastasio T.J. Modeling cross-modal enhancement and modality-specific suppression in multisensory neurons. Neural Comput. 2003, 15:783-810.
21. Rowland B.A., Stanford T.R., Stein B.E. A model of the neural mechanisms underlying multisensory integration in the superior colliculus. Perception 2007, 36:1431-1443.
22. Magosso E., Cuppini C., Serino A., Pellegrino G.D., Ursino M. A theoretical study of multisensory integration in the superior colliculus by a neural network model. Neural Networks 2008, 21:817-829.
23. Cuppini C., Ursino M., Magosso E., Rowland B.A., Stein B.E. An emergent model of multisensory integration in superior colliculus neurons. Front. Integr. Neurosci. 2010, 4.
24. A. Pavlou, M.C. Casey, Simulating the effects of cortical feedback in the superior colliculus with topographic maps, in: Proceedings of the International Joint Conference on Neural Networks (IJCNN) 2010, IEEE, 2010.
25. M.C. Casey, A. Pavlou, A behavioral model of sensory alignment in the superficial and deep layers of the superior colliculus, in: Proceedings of the International Joint Conference on Neural Networks (IJCNN) 2008, IEEE, 2008, pp. 2751-2756.
26. J. Huo, A. Murray, L. Smith, Y. Zhijun, Adaptation of barn owl localization system with spike timing dependent plasticity, in: Proceedings of the International Joint Conference on Neural Networks (IJCNN) 2008, IEEE, 2008, pp. 155-160.
27. Huo J., Murray A. The adaptation of visual and auditory integration in the barn owl superior colliculus with spike timing dependent plasticity. Neural Networks 2009, 22:913-921.
28. Grossberg S., Roberts K., Aguilar M., Bullock D. A neuronal model of multimodal adaptive saccadic eye movement control by superior colliculus. J. Neurosci. 1997, 17:9706-9725.
29. Cutsuridis V., Smyrnis N., Evdokimidis I., Perantonis S. A neural model of decision-making by the superior colliculus in an antisaccade task. Neural Networks 2007, 20:690-704.
30. Srinivasa N., Grossberg S. A head-neck-eye system that learns fault-tolerant saccades to 3-D targets using a self-organizing neural model. Neural Networks 2008, 21:1380-1391.
31. Itti L., Koch C., Nieber E. A model of saliency-based visual attention for rapid scene analysis. IEEE Trans. Pattern Anal. Mach. Intell. 1998, 20:1254-1259.
32. Siagian C., Chang C.K., Voorhies R., Itti L. Beobot 2.0: cluster architecture for mobile robotics. J. Field Robot. 2011, 28:278-302.
33. Markram H. The blue brain project. Nat. Rev. Neurosci. 2006, 7:153-160.
34. Izhikevich E., Edelman G.M. Large-scale model of mammalian thalamocortical systems. Proc. Natl. Acad. Sci. U.S.A. 2008, 105:3593-3598.
35. Seth A.K., Edelman G.M. Distinguishing causal interactions in neural populations. Neural Comput. 2007, 19:910-933.
36. de Garis H., Shuo C., Goertzel B., Ruiting L. A world survey of artificial brain projects, Part I: large-scale brain simulations. Neurocomputing 2010, 74:3-29.
37. Goertzel B., Lian R., Arel I., de Garis H., Chen S. A world survey of artificial brain projects, Part II: biologically inspired cognitive architectures. Neurocomputing 2010, 74:30-49.
38. Lettvin J.Y., Maturana H.R., Mcculloch W.S., Pitts W.H. What the frog's eye tells the frog's brain. Proc. IRE 1959, 47:1940-1951.
39. Sterling P., Wickelgren B.G. Visual receptive fields in the superior colliculus of the cat. J. Neurophysiol. 1969, 32:1-15.
40. Rauschecker J.P., Harris L.R. Auditory and visual neurons in the cat's superior colliculus selective for the direction of apparent motion stimuli. Brain Res. 1989, 490:56-63.
41. Wallace M.T., McHaffie J.G., Stein B.E. Visual response properties and visuotopic representation in the newborn monkey superior colliculus. J. Neurophysiol. 1997, 78:2732-2741.
42. Prévost F., Lepore F., Guillemot J.P. Spatio-temporal receptive field properties of cells in the rat superior colliculus. Brain Res. 2007, 1142:80-91.
43. Schneider K.A., Kastner S. Visual responses of the human superior colliculus: a high-resolution functional magnetic resonance imaging study. J. Neurophysiol. 2005, 94:2491-2503.
44. Knudsen E.I., Konishi M. A neural map of auditory space in the owl. Science 1978, 200:795-797.
45. Middlebrooks J.C., Knudsen E.I. A neural code for auditory space in the cat's superior colliculus. J. Neurosci. 1984, 4:2621-2634.
46. Grothe B., Pecka M., McAlpine D. Mechanisms of sound localization in mammals. Physiol. Rev. 2010, 90:983-1012.
47. King A.J., Palmer A.R. Cells responsive to free-field auditory stimuli in guinea-pig superior colliculus: distribution and response properties. J. Physiol. 1983, 342:361-381.
48. Sterbing S., Hartung K., Hoffmann K.P. Representation of sound source direction in the superior colliculus of the guinea pig in a virtual auditory environment. Exp. Brain Res. 2002, 142:570-577.
49. Campbell R.A.A., Doubell T.P., Nodal F.R., Schnupp J.W.H., King A.J. Interaural timing cues do not contribute to the map of space in the ferret superior colliculus: a virtual acoustic space study. J. Neurophysiol. 2006, 95:242-254.
50. Zella J.C., Brugge J.F., Schnupp J.W.H. Passive eye displacement alters auditory spatial receptive fields of cat superior colliculus neurons. Nat. Neurosci. 2001, 4:1167-1169.
51. King A.J., Schnupp W.H., Thompson I.D. Signals from the superficial layers of the superior colliculus enable the development of the auditory space map in the deeper layers. J. Neurosci. 1998, 18:9394-9408.
52. Opstal A.J.V., Munoz D.P. Auditory-visual interactions subserving primate gaze orienting. The Handbook of Multisensory Processes. A Bradford Book 2004, 373-393. MIT Press. G.A. Calvert, C. Spence, B.E. Stein (Eds.).
53. Linsker R. Self-organization in a perceptual network. Computer 1988, 21:105-117.
54. Armony J.L., Servan-Schreiber D., Cohen J.D., LeDoux J.E. An anatomically constrained neural network model of fear conditioning. Behav. Neurosci. 1995, 109:246-257.
55. Armony J.L., Servan-Schreiber D., Romanski L.M., Cohen J.D., LeDoux J.E. Stimulus generalization of fear responses: effects of auditory cortex lesions in a computational model and in rats. Cereb. Cortex 1997, 7:157-165.
56. Miikkulainen R., Bednar J.A., Choe Y., Sirosh J. Computational Maps in the Visual Cortex 2005, Springer Science+Business Media, New York.
57. Rumelhart D.E., Zipser D. Feature discovery by competitive learning. Parallel Distributed Processing: Explorations in the Microstructure of Cognition, Foundations 1986, vol. 1:151-193. MIT Press.
58. Hebb D.O. The Organization of Behavior: A Neuropsychological Theory 1949, John Wiley and Sons, New York.
59. A. Pavlou, M.C. Casey, Identifying emotions using topographic conditioning maps, in: M. Koeppen, N. Kasabov, G. Coghill (Eds.), Advances in Neuro-Information Processing: Proceedings of the 15th International Conference on Neuro-Information Processing, Lecture Notes in Computer Science, vol. 5506, Springer-Verlag, 2009, pp. 40-47.
60. M.C. Casey, A. Pavlou, A. Timotheou, Mind the (computational) gap, in: Proceedings of the UK Workshop on Computational Intelligence (UKCI 2010), IEEE, 2010.
61. Kohonen T. Self-organized formation of topologically correct feature maps. Biol. Cybern. 1982, 43:59-69.
62. Clifford C.W.G., Ibbotson M.R. Fundamental mechanisms of visual motion detection: models, cells and functions. Prog. Neurobiol. 2002, 68:409-437.
63. Enroth-Cugell C., Robson J.G. The contrast sensitivity of retinal ganglion cells of the cat. J. Physiol. 1966, 187:517-552.
64. Microsoft Corporation, Kinect for Windows SDK from Microsoft Research [WWW page], 2011. URL http://research.microsoft.com/en-us/um/redmond/projects.
65. Cox H., Zeskind R., Owen M. Robust adaptive beamforming. IEEE Trans. Acoust. Speech Signal Process. 1987, 35:1365-1376.
66. S.T. Birchfield, R. Gangishetty, Acoustic localization by interaural level difference, in: Proceedings of the IEEE International Conference on Acoustics, Speech, and Signal Processing (ICASSP 2005), vol. 4, 2005, pp. iv/1109-iv/1112.
67. Stein B.E., Meredith M.A., Huneycutt W.S., McDade L. Behavioral indices of multisensory integration: orientation to visual cues is affected by auditory stimuli. J. Cognitive Neurosci. 1989, 1:12-24.
68. Knudsen E.I., Brainard M.S. Creating a unified representation of visual and auditory space in the brain. Annu. Rev. Neurosci. 1995, 18:19-43.
69. Ernst M.O., Banks M.S. Humans integrate visual and haptic information in a statistically optimal fashion. Nature 2002, 415:429-433.
70. Battaglia P.W., Jacobs R.A., Aslin R.N. Bayesian integration of visual and auditory signals for spatial localization. J. Opt. Soc. Am. A 2003, 20:1391-1397.
71. Schrader S., Gewaltig M.O., Körner U., Körner E. Cortext: a columnar model of bottom-up and top-down processing in the neocortex. Neural Networks 2009, 22:1055-1070.
72. Shi C., Davis M. Visual pathways involved in fear conditioning measured with fear-potentiated startle: behavioral and anatomic studies. J. Neurosci. 2001, 21:9844-9855.
73. Liu J., Perez-Gonzalez D., Rees A., Erwin H., Wermter S. A biologically inspired spiking neural network model of the auditory midbrain for sound source localisation. Neurocomputing 2010, 74:129-139.
74. Monaci G., Vandergheynst P., Sommer F.T. Learning bimodal structure in audiovisual data. IEEE Trans. Neural Networks 2009, 20:1898-1910.