Cortical dynamics of navigation and steering in natural scenes: Motion-based object segmentation, heading, and obstacle avoidance

Neural Networks

Volumn 22, Issue 10, 2009, Pages 1383-1398

  Andrew Browning, N ^a   Grossberg, Stephen ^a   Mingolla, Ennio ^a  

^a BOSTON UNIVERSITY   (United States)

Author keywords

Motion segmentation; MST; MT; Navigation; Object tracking; Optic flow; Steering

Indexed keywords

APERTURE PROBLEMS; CORTICAL AREAS; FEED-BACK LOOP; HUMAN PERFORMANCE; INPUT STREAMS; LOCAL MOTIONS; MOTION INFORMATION; MOTION SEGMENTATION; MOVING OBJECTS; NATURAL SCENES; NEURAL MODELS; OBJECT INFORMATION; OBJECT MOTION; OBJECT SEGMENTATION; OBJECT TRACKING; OBSTACLE AVOIDANCE; OPTIC FLOW; SELF MOTION; STEERING DECISIONS; VIRTUAL ENVIRONMENTS;

DISPLAY DEVICES; MAMMALS; NAVIGATION; TRACKING (POSITION); VIRTUAL REALITY;

TELLURIC PROSPECTING;

ACCURACY; ARTICLE; BRAIN CORTEX; COMPUTER MODEL; FEEDBACK SYSTEM; HUMAN; INFORMATION; MATHEMATICAL COMPUTING; MOTION ANALYSIS SYSTEM; NONHUMAN; PRIORITY JOURNAL; QUANTITATIVE ANALYSIS; RETINA; SIMULATION; VELOCITY; VIDEORECORDING; VIRTUAL REALITY; VISUAL CORTEX;

ALGORITHMS; HUMANS; MODELS, NEUROLOGICAL; MODELS, STATISTICAL; MOTION PERCEPTION; NERVOUS SYSTEM PHYSIOLOGICAL PHENOMENA; ORIENTATION; PSYCHOPHYSICS; RETINALDEHYDE; SOFTWARE; TEMPORAL LOBE; VISUAL CORTEX; VISUAL PATHWAYS; VISUAL PERCEPTION;

EID: 71749098937     PISSN: 08936080     EISSN: None     Source Type: Journal    
DOI: 10.1016/j.neunet.2009.05.007     Document Type: Article

References (116)

1. Adelson E.H., and Bergen J.R. Spatiotemporal energy models for the perception of motion. Journal of the Optical Society of America, A 2 (1985) 284-299
2. Aggarwal J., and Nandhakumar N. On the computation of motion from sequences of images-A review. Proceedings of the IEEE 76 8 (1988) 917-935 10.1109/5.5965
3. Albright T.D. Direction and orientation selectivity of neurons in visual area MT of the macaque. Journal of Neurophysiology 52 6 (1984) 1106-1130
4. Allman J., Miezin F., and McGuinness E.L. Stimulus specific responses from beyond the classical receptive field: Neurophysiological mechanisms for local-global comparisons in visual neurons. Annual Review of Neuroscience 8 (1985) 407-430
5. AMD, (2003). AMD. Sunnyvale, CA 94088
6. Baloch A.A., and Grossberg S. A neural model of high-level motion processing: Line motion and formotion dynamics. Vision Research 37 21 (1997) 3037-3059
7. Baloch A.A., Grossberg S., Mingolla E., and Nogueira C.A.M. Neural model of first-order and second-order motion perception and magnocellular dynamics. Journal of the Optical Society of America A 16 (1999) 953-978
8. Banks M.S., Ehrlich S.M., Backus B.T., and Crowell J.A. Estimating heading during real and simulated eye movements. Vision Research 36 (1996) 431-443
9. Barlow H., and Levick W. The mechanism of directionally selective units in rabbit's retina. Journal of Physiology 178 (1965) 477-504
10. Bayerl P., and Neumann H. Disambiguating visual motion through contextual feedback modulation. Neural Computation 16 (2004) 2041-2066
11. Beardsley S.A., and Vaina L.M. A laterally interconnected neural architecture in MST accounts for psychophysical discrimination of complex motion patterns. Journal of Computational Neuroscience 10 (2001) 255-280
12. Benardete E., and Kaplan E. The dynamics of primate M retinal ganglion cells. Visual Neuroscience 16 (1999) 355-368
13. Berzhanskaya J., Grossberg S., and Mingolla E. Laminar cortical dynamics of visual form and motion interactions during coherent object motion perception. Spatial Vision 20 (2007) 337-395
14. Born R.T. Center-surround interactions in the middle temporal visual area of the owl monkey. Journal of Neurophysiology 84 (2000) 2658-2669
15. Born R.T., and Bradley D.C. Structure and function of visual area MT. Annual Review of Neuroscience 28 (2005) 157-189
16. Born R.T., and Tootell R.B.H. Segregation of global and local motion processing in primate middle temporal visual area. Nature 357 (1992) 497-499
17. Bradley D.C., and Andersen R.A. Center-surround antagonism based on disparity in primate area MT. Journal of Neuroscience 18 (1998) 7552-7565
18. Bradski G.R., and Kaehler A. Learning OpenCV: Computer vision with the OpenCV library (2008), O'Reilly Press, Cambridge, MA
19. Browning N.A., Mingolla E., and Grossberg S. Heading from optic flow of natural scenes during motion processing by cortical areas MT and MST. Program no. 337.12. 2007 Neuroscience meeting planner (2007), Society for Neuroscience, San Diego, CA
20. Browning, N. A., Mingolla, E., & Grossberg, S. (2007b). Heading from optic flow in a neural model of primate motion processing and navigation. In Eleventh international conference on cognitive and neural systems proceedings
21. Browning, N. A., Mingolla, E., & Grossberg, S. (2007c). Computing heading and optic flow in MT-MST: A neural model of primate motion processing. In Computational cognitive neuroscience conference
22. Browning, N. A., Mingolla, E., & Grossberg, S. (2008). Visually guided navigation and steering: Motion based object segmentation and heading estimation in primates. In Twelfth international conference on cognitive and neural systems proceedings
23. Browning, N. A., Mingolla, E., & Grossberg, S. (2009). A neural model of how the brain computes heading from optic flow in realistic scenes. Cognitive Psychology (in press)
24. Callaway E.M. Structure and function of parallel pathways in the primate early visual system. Journal of Physiology 566 (2005) 13-19
25. Cameron S., Grossberg S., and Guenther F.H. A self-organizing neural network architecture for navigation using optic flow. Neural Computation 10 (1998) 313-352
26. Cao Y., Grossberg S., and Zaydens E. A laminar cortical model of stereopsis and 3D surface perception of complex natural scenes. Journal of Vision 8 850-850a (2008). http://journalofvision.org/8/6/850/ 10.1167/8.6.850
27. Carpenter G.A., Grossberg S., and Rosen D.B. Fuzzy ART: Fast stable learning and categorization of analog patterns by an adaptive resonance system. Neural Networks 4 (1991) 759-771
28. Chelian, S., & Carpenter, G. A. (2005). DISCOV: A neural model of colour vision, with applications to image processing and classification. In Proceedings of AIC05: 10th congress of the international colour association
29. Chey J., Grossberg S., and Mingolla E. Neural dynamics of motion grouping: From aperture ambiguity to object speed and direction. Journal of the Optical Society of America 14 (1997) 2570-2594
30. Chey J., Grossberg S., and Mingolla E. Neural dynamics of motion processing and speed discrimination. Vision Research 38 (1998) 2769-2786
31. Cleland B.G., Dubin M.W., and Levick W.R. Sustained and transient neurones in the cat's retina and lateral geniculate nucleus. Journal of Physiology 217 (1971) 473-496
32. DeAngelis G.C., Ohzawa I., and Freeman R.D. Receptive-field dynamics in the central visual pathways. Trends in Neurosciences 18 (1995) 451-458
33. Del Viva M.M., and Morrone M.C. Motion analysis by feature tracking. Vision Research 38 (1998) 3633-3653
34. Duffy C.J. MST neurons respond to optic flow and translational movement. Journal of Neurophysiology 80 (1998) 1816-1827
35. Duffy C.J., and Wurtz R.H. Response of monkey MST neurons to optic flow stimuli with shifted centers of motion. Journal of Neuroscience 15 (1995) 5192-5208
36. Duffy C.J., and Wurtz R.H. Planar directional contributions to optic flow responses in MST neurons. Journal of Neurophysiology 77 (1997) 782-796
37. Elder, D. M., Grossberg, S., & Mingolla, E. (2005). A neural model of visually-guided steering, obstacle avoidance, and route selection. In Annual meeting of the Society for Neuroscience, Abstract viewer and itinerary planner CD-ROM, Prog.
38. Elder, D. M., Grossberg, S., & Mingolla, E. (2009). A neural model of visually guided steering, obstacle avoidance, and route selection. Journal of Experimental Psychology: Human Perception & Performance (in press)
39. Enroth-Cugell C., and Robson J.G. The contrast sensitivity of retinal ganglion cells of the cat. Journal of Physiology 187 (1966) 517-552
40. Fajen B.R., and Warren W.H. Behavioral dynamics of steering, obstacle avoidance, and route selection. Journal of Experimental Psychology: Human Perception & Performance 29 (2003) 343-362
41. Fajen B.R., and Warren W.H. Visual guidance of intercepting a moving target on foot. Perception 33 (2004) 689-715
42. Fazl A., Grossberg S., and Mingolla E. View-invariant object category learning, recognition, and search: How spatial and object attention are coordinated using surface-based attentional shrouds. Cognitive Psychology 58 (2009) 1-48
43. Francis G., and Grossberg S. Cortical dynamics of form and motion integration: Persistence, apparent motion, and illusory contours. Vision Research 36 (1996) 149-173
44. Fried S.I., Münch T.A., and Werblin F.S. Mechanisms and circuitry underlying directional selectivity in the retina. Nature 420 (2002) 411-4
45. Fried S.I., Münch T.A., and Werblin F.S. Directional selectivity is formed at multiple levels by laterally offset inhibition in the rabbit retina. Neuron 46 (2005) 117-27
46. Gibson J.J. The perception of the visual world (1950), Houghton Mifflin Boston
47. Greve D., Grossberg S., Guenther F., and Bullock D. Neural representations for sensory-motor control, I: Head-centered 3-D target positions from opponent eye commands. Acta Psychologica 82 (1993) 115-138
48. Grossberg S. Some nonlinear networks capable of learning a spatial pattern of arbitrary complexity. Proceedings of the National Academy of Sciences 59 (1968) 368-372
49. Grossberg S. Contour enhancement, short term memory, and constancies in reverbarating neural networks. Studies in Applied Mathematics 52 (1973) 213-257
50. Grossberg S. Adaptive pattern classification and universal recoding: I. Parallel development and coding of neural feature detectors. Biological Cybernetics 23 3 (1976) 121-134
51. Grossberg S. A theory of human memory: Self-organization and performance of sensory-motor codes, maps, and plans. In: Rosen R., and Snell F. (Eds). Progress in theoretical biology Vol. 5 (1978), Academic Press, New York 233-374
52. Grossberg S. How does a brain build a cognitive code?. Psychological Review 87 (1980) 1-51
53. Grossberg S. Why do parallel cortical systems exist for the perception of static form and moving form?. Perception & Psychophysics 49 (1991) 117-141
54. Grossberg S. 3-D vision and figure-ground separation by visual cortex. Perception & Psychophysics 55 (1994) 48-121
55. Grossberg S. The complementary brain: Unifying brain dynamics and modularity. Trends in Cognitive Sciences 4 (2000) 233-246
56. Grossberg S., Mingolla E., and Pack C.C. A neural model of motion processing and visual navigation by cortical area MST. Cerebral Cortex 9 (1999) 878-895
57. Grossberg S., Mingolla E., and Viswanathan L. Neural dynamics of motion integration and segmentation within and across apertures. Vision Research 41 (2001) 2521-2553
58. Grossberg, S., Srihasam, K., & Bullock, D. (2009). Neural dynamics of saccadic and smooth pursuit eye movement coordination during visual tracking of unpredictably moving targets (submitted for publication)
59. Hildreth E.C. Recovering heading for visually-guided navigation. Vision Research 32 (1992) 1177-1192
60. Hodgkin A.L. The conduction of the nerve impulse (1964), C.C. Thomas, Springfield, Illinois
61. Hubel D.H., and Wiesel T.N. Receptive fields of single neurones in the cat's striate cortex. Journal of Physiology 148 (1959) 574-591
62. Hubel D.H., and Wiesel T.N. Receptive fields, binocular interaction and functional architecture in the cat's visual cortex. Journal of Physiology 160 (1962) 106-154
63. Hubel D.H., and Wiesel T.N. Receptive fields and functional architecture of monkey striate cortex. Journal of Physiology 195 (1968) 215-243
64. Institut fuer Algorithmen und Kognitive Systeme. Retrieved May 23, 2008, from http://i21www.ira.uka.de/image_sequences/#taxi
65. Kandel E.R., Schwartz J.H., and Jessell T.M. Principles of neural science (2000), Appleton & Lange
66. Kaplan E., and Benardete E. The dynamics of primate retinal ganglion cells. Progress in Brain Research 134 (2001) 17-34
67. Kelly J.W., Beall A.C., Loomis J.M., Smith R.S., and Macuga K.L. Simultaneous measurement of steering performance and perceived heading on a curving path. ACM Transactions in Applied Perception 3 (2006) 83-94
68. Kelly F., and Grossberg S. Neural dynamics of 3-D surface perception: Figure-ground separation and lightness perception. Perception & Psychophysics 62 (2000) 1596-1618
69. Langer M.S., and Mann R. Optical snow. International Journal of Computer Vision 55 (2003) 55-71
70. Li L., and Warren W.H. Perception of heading during rotation: Sufficiency of dense motion parallax and reference objects. Vision Research 40 (2000) 3873-3894
71. Li L., and Warren W.H. Path perception during rotation: Influence of instructions, depth range, and dot density. Vision Research 44 (2004) 1879-1889
72. Lidén L., and Pack C.C. The role of terminators and occlusion in motion integration and segmentation: A neural solution. Vision Research 39 (1999) 3301-3320
73. Livingstone M.S. Mechanisms of direction selectivity in macaque V1. Neuron 20 (1998) 509-526
74. Livingstone M.S., and Hubel D.H. Psychophysical evidence for separate channels for the perception of form, color, movement, and depth. Journal of Neuroscience 7 (1987) 3416-3468
75. Longuet-Higgins H.C., and Prazdny K. The interpretation of a moving retinal image. Proceedings of the Royal Society of London. Series B, Biological Sciences 208 (1980) 385-397
76. Lucas, B. D., & Kanade, T. (1981). An iterative image registration technique with an application to stereo vision. Proceedings of the DARPA image understanding workshop (pp. 121-130)
77. Mann, R., & Langer, M. S. (2002). Optical snow and the aperture problem. In International conference on pattern recognition
78. Marr D., and Ullman S. Directional selectivity and its use in early visual processing. Proceedings of the Royal Society of London. Series B, Biological Sciences 211 (1981) 151-180
79. Maunsell J.H., and Van Essen D.C. Functional properties of neurons in middle temporal visual area of the macaque monkey. I. Selectivity for stimulus direction, speed, and orientation. Journal of Neurophysiology 49 (1983) 1127-1147
80. Microsoft (2003). Redmond, WA 98052: Microsoft Corporation
81. Mingolla E., Todd J., and Norman J. The perception of globally coherent motion. Vision Research 32 (1992) 1015-1031
82. Mishkin M., Ungerleider L.G., and Macko K.A. Object vision and spatial vision: Two cortical pathways. Trends in Neurosciences 6 (1983) 414-417
83. Nakayama K., and Loomis J.M. Optical velocity patterns, velocity-sensitive neurons, and space perception: A hypothesis. Perception 3 (1974) 63-80
84. Nowlan S.J., and Sejnowski T. Filter selection model for generating visual motion signals. Advances in Neural Information Processing Systems 5 (1993) 369-376
85. Nowlan S.J., and Sejnowski T.J. A selection model for motion processing in area MT of primates. Journal of Neuroscience 15 (1995) 1195-1214
86. Pack C.C., and Born R.T. Temporal dynamics of a neural solution to the aperture problem in visual area MT of macaque brain. Nature 409 (2001) 1040-2
87. Pack C.C., Livingstone M.S., Duffy K.R., and Born R.T. End-stopping and the aperture problem: Two-dimensional motion signals in Macaque V1. Neuron 39 (2003) 671-680
88. Pack C.C., Grossberg S., and Mingolla E. A neural model of smooth pursuit control and motion perception by cortical area MST. Journal of Cognitive Neuroscience 13 (2001) 102-120
89. Page W.K., and Duffy C.J. MST neuronal responses to heading direction during pursuit eye movements. Journal of Neurophysiology 81 (1999) 596-610
90. Ponce C.R., Lomber S.G., and Born R.T. Integrating motion and depth via parallel pathways. Nature Neuroscience 11 (2008) 216-223
91. Rieger J., and Lawton D. Processing differential image motion. Optical Society of America, Journal, A 2 (1985) 354-360
92. Royden C.S., Crowell J.A., and Banks M.S. Estimating heading during eye movements. VisionRresearch 34 (1994) 3197-3214
93. Rumelhart D.E., and Zipser D. Feature discovery by competitive learning. In: Rumelhart, and McClelland (Eds). Parallel distributed processing: Explorations in the microstructure of cognition, vol. 1: Foundations (1986), MIT Press, Cambridge, MA, USA 151-193
94. Rushton S.K., Bradshaw M.F., and Warren P.A. The pop out of scene-relative object movement against retinal motion due to self-movement. Cognition 105 (2007) 237-245
95. Rushton S.K., and Warren P.A. Perception of object movement during self-movement. Proceedings of SPIE 5666 (2005) 473
96. Rushton S.K., and Warren P.A. Moving observers, relative retinal motion and the detection of object movement. Current Biology 15 (2005) 542-543
97. van Santen J.P.H., and Sperling G. Elaborated Reichardt detectors. Journal of the Optical Society of America, A 2 (1985) 300-320
98. Schiller P.H., Finlay B.L., and Volman S.F. Quantitative studies of single-cell properties in monkey striate cortex. I. Spatiotemporal organization of receptive fields. Journal of Neurophysiology 39 (1976) 1288-1319
99. Schneider G.E. Contrasting visuomotor functions of tectum and cortex in the golden hamster. Psychological Research 31 (1967) 52-62
100. Schwartz E.L. Spatial mapping in the primate sensory projection: Analytic structure and relevance to perception. Biological Cybernetics 25 (1977) 181-94
101. Srihasam, K., Bullock, D., & Grossberg, S. (2009). Target selection by the frontal cortex during coordinated saccadic and smooth-pursuit eye movements. Journal of Cognitive Neuroscience (in press)
102. Stone L.S., and Perrone J.A. A role for MST neurons in heading estimation. RECON no. 20010116589 (1994), Society for Neuroscience Abstracts, Miami Beach, FL, United States
103. Stone L.S., and Perrone J.A. Quantitative simulations of MST visual receptive field properties using a template model of heading estimation. Society for Neuroscience Abstracts 23 (1997) 1126
104. Stone L.S., and Perrone J.A. Human heading estimation during visually simulated curvilinear motion. Vision Research 37 (1997) 573-590
105. Tanaka K., Sugita Y., Moriya M., and Saito H. Analysis of object motion in the ventral part of the medial superior temporal area of the macaque visual cortex. Journal of Neurophysiology 69 (1993) 128-142
106. Valois R., Albrecht R., and Thorell L.G. Spatial frequency selectivity of cells in macaque visual cortex. Vision Research 22 (1982) 545-559
107. Wagner R.E., Polimeni J.R., and Schwartz E.L. Gibson, meet topography: The dipole structure of extra striate cortex facilitates navigation via optical flow [Abstract]. Journal of Vision 5 (2005) 895
108. Wallach H. On the visually perceived direction of motion. Psychologische Forschung 20 (1935) 325-380
109. Wang R. A network model of motion processing in area MT of primates. Journal of Computational Neuroscience 4 (1997) 287-308
110. Warren P.A., and Rushton S.K. Perception of object trajectory: Parsing retinal motion into self and object movement components. Journal of Vision 7 (2007) 2
111. Wilkie R.M., and Wann J.P. Driving as night falls: The contribution of retinal flow and visual direction to the control of steering. Current Biology 12 (2002) 2014-2017
112. Wilkie R.M., and Wann J.P. Eye-movements aid the control of locomotion. Journal of Vision 3 (2003) 677-684
113. Wilkie R.M., and Wann J.P. Controlling steering and judging heading: Retinal flow, visual direction and extra-retinal information. Journal of Experimental Psychology 29 (2003) 363-378
114. Wilkie R.M., and Wann J.P. Judgments of path, not heading, guide locomotion. Journal of Experimental Psychology: Human Perception and Performance 32 (2006) 88-96
115. Wuerger S., Shapley R., and Rubin N. On the visually perceived direction of motion by Hans Wallach: 60 years later. Perception 25 (1996) 1317-1367
116. Zemel R.S., and Sejnowski T.J. A model for encoding multiple object motions and self-motion in area MST of primate visual cortex. Journal of Neuroscience 18 (1998) 531-547