A neural model of how the brain computes heading from optic flow in realistic scenes

Cognitive Psychology

Volumn 59, Issue 4, 2009, Pages 320-356

  Browning, N Andrew ^a   Grossberg, Stephen ^a   Mingolla, Ennio ^a  

^a BOSTON UNIVERSITY   (United States)

Author keywords

Heading; Motion; MST; MT; Navigation; Neural model; Optic flow; V1; Visual cortex

Indexed keywords

ALGORITHM; ARTICLE; BEHAVIOR; BINOCULAR VISION; BIOLOGICAL MODEL; COGNITION; COMPUTER SIMULATION; DEPTH PERCEPTION; ENVIRONMENT; HUMAN; LIGHT; MOVEMENT PERCEPTION; NERVE CELL; ORIENTATION; PHYSIOLOGY; ROTATION; VISION; VISUAL CORTEX; VISUAL FIELD; VISUAL SYSTEM;

ALGORITHMS; COGNITION; COMPUTER SIMULATION; ENVIRONMENT; HUMANS; LIGHT; MODELS, NEUROLOGICAL; MOTION PERCEPTION; NEURONS; ORIENTATION; ROTATION; SPACE PERCEPTION; SPATIAL BEHAVIOR; VISION, BINOCULAR; VISUAL CORTEX; VISUAL FIELDS; VISUAL PATHWAYS; VISUAL PERCEPTION;

EID: 70349978313     PISSN: 00100285     EISSN: None     Source Type: Journal    
DOI: 10.1016/j.cogpsych.2009.07.002     Document Type: Article

References (129)

1. Adelson E.H., and Bergen J.R. Spatiotemporal energy models for the perception of motion. Optical Society of America Journal A: Optics and Image Science 2 (1985) 284-299
2. Adelson E.H., and Movshon J.A. Phenomenal coherence of moving visual patterns. Nature 300 5892 (1982) 523-525
3. Agrawal, M., Konolige, K., & Bolles, R. C. (2007). Localization and mapping for autonomous navigation in outdoor terrains: A stereo vision approach. In Eighth IEEE workshop on applications of computer vision (WACV'07), 2007.
4. Alenyà G., Martinez E., and Torras C. Fusing visual and inertial sensing to recover robot egomotion. Journal of Robotic Systems 21 1 (2004) 23-32
5. AMD (2003). Sunnyvale, CA 94088: AMD.
6. Anderson J.C., Binzegger T., Kahana O., Martin K.A.C., and Segev I. Dendritic asymmetry cannot account for directional responses of neurons in visual cortex. Nature Neuroscience 2 (1999) 820-824
7. Baloch A.A., and Grossberg S. A neural model of high-level motion processing: Line motion and for motion dynamics. Vision Research 37 21 (1997) 3037-3059
8. Baloch A.A., Grossberg S., Mingolla E., and Nogueira C.A.M. Neural model of first-order and second-order motion perception and magnocellular dynamics. Journal of the Optical Society of America A 16 5 (1999) 953-978
9. Banks M.S., Ehrlich S.M., Backus B.T., and Crowell J.A. Estimating heading during real and simulated eye movements. Vision Research 36 3 (1996) 431-443
10. Barlow H., and Levick W. The mechanism of directionally selective units in rabbit's retina. The Journal of Physiology 178 3 (1965) 477-504
11. Bayerl P., and Neumann H. Disambiguating visual motion through contextual feedback modulation. Neural Computation 16 10 (2004) 2041-2066
12. Beardsley S.A., and Vaina L.M. A laterally interconnected neural architecture in MST accounts for psychophysical discrimination of complex motion patterns. Journal of Computational Neuroscience 10 3 (2001) 255-280
13. Beauchemin S.S., and Barron J.L. The computation of optical flow. ACM Computing Surveys 27 3 (1995) 433-466
14. Beintema J.A., and van den Berg A.V. Heading detection using motion templates and eye velocity gain fields. Vision Research 38 14 (1998) 2155-2179
15. Benardete E., and Kaplan E. The dynamics of primate M retinal ganglion cells. Visual Neuroscience 16 02 (1999) 355-368
16. Berzhanskaya J., Grossberg S., and Mingolla E. Laminar cortical dynamics of visual form and motion interactions during coherent object motion perception. Spatial Vision 20 4 (2007) 337-395
17. Black, M. J. (2007). Yosemite sequence FAQs. Michael Black personal web pages. Retrieved 1.10.07.
18. Black, M. J., & Anandan, P. (1991). Robust dynamic motion estimation over time. In Proceedings CVPR'91, IEEE computer society conference on computer vision and pattern recognition (pp. 296-302).
19. Born R.T., and Bradley D.C. Structure and function of visual area MT. Annual Review of Neuroscience 28 (2005) 157-189
20. Born R.T., and Tootell R.B.H. Segregation of global and local motion processing in primate middle temporal visual area. Nature 357 6378 (1992) 497-499
21. Browning, N. A., Grossberg, S., & Mingolla, E. (2008). Visually guided navigation and steering: Motion based object segmentation and heading estimation in primates. In Twelfth international conference on cognitive and neural systems, Boston, MA, May 2008, p. 67.
22. Browning, N. A., Grossberg, S., & Mingolla, E. (2009). ViSTARS: Cortical dynamics of navigation and steering in natural scenes: Motion-based object segmentation, heading, and obstacle avoidance. Neural Networks, in press.
23. Callaway E.M. Structure and function of parallel pathways in the primate early visual system. The Journal of physiology 566 1 (2005) 13-19
24. Calow D., Krüger N., Wörgötter F., and Lappe M. Biologically motivated space-variant filtering for robust optic flow processing. Network: Computation in Neural Systems 16 4 (2005) 323-340
25. Cameron S., Grossberg S., and Guenther F.H. A self-organizing neural network architecture for navigation using optic flow. Neural Computation 10 2 (1998) 313-352
26. Chelian, S., & Carpenter, G. A. (2005). DISCOV: A neural model of colour vision, with applications to image processing and classification. In Proceedings of AIC05: 10th Congress of the international colour association, Granada, Spain, May.
27. Chey J., Grossberg S., and Mingolla E. Neural dynamics of motion grouping: From aperture ambiguity to object speed and direction. Journal of the Optical Society of America 14 10 (1997) 2570-2594
28. Chey J., Grossberg S., and Mingolla E. Neural dynamics of motion processing and speed discrimination. Vision Research 38 18 (1998) 2769-2786
29. Cutting J.E., Springer K., Braren P.A., and Johnson S.H. Wayfinding on foot from information in retinal, not optical, flow. Journal of Experimental Psychology: General 121 1 (1992) 41-72
30. Cutting J.E., and Wang R.F. Heading judgments in minimal environments: The value of a heuristic when invariants are rare. Perception & Psychophysics 62 6 (2000) 1146-1159
31. DeAngelis G.C., Ohzawa I., and Freeman R.D. Receptive-field dynamics in the central visual pathways. Trends in Neurosciences 18 10 (1995) 451-458
32. Duffy C.J. MST neurons respond to optic flow and translational movement. Journal of Neurophysiology 80 4 (1998) 1816-1827
33. Duffy C.J., and Wurtz R.H. Response of monkey MST neurons to optic flow stimuli with shifted centers of motion. Journal of Neuroscience 15 7 (1995) 5192-5208
34. Duffy C.J., and Wurtz R.H. Planar directional contributions to optic flow responses in MST neurons. Journal of Neurophysiology 77 2 (1997) 782-796
35. Ehrlich S.M., Beck D.M., Crowell J.A., Freeman T.C.A., and Banks M.S. Depth information and perceived self-motion during simulated gaze rotations. Vision Research 38 20 (1998) 3129-3145
36. Elder D.M., Grossberg S., and Mingolla E. A neural model of visually-guided steering, obstacle avoidance, and route selection. Society for Neuroscience, Washington, DC (2005) Abstract Viewer and Itinerary Planner CD-ROM, Prog
37. Elder, D. M., Grossberg, S., & Mingolla, E. (2009). A neural model of visually guided steering, obstacle avoidance, and route selection. Journal of Experimental Psychology: Human Perception and Performance, in press.
38. Elston G.N., and Rosa M.G. Morphological variation of layer III pyramidal neurones in the occipitotemporal pathway of the macaque monkey visual cortex. Cerebral Cortex 8 (1997) 278-294
39. Fajen B.R., and Warren W.H. Behavioral dynamics of steering, obstacle avoidance, and route selection. Journal of Experimental Psychology: Human Perception & Performance 29 2 (2003) 343-362
40. Fajen B.R., and Warren W.H. Visual guidance of intercepting a moving target on foot. Perception 33 6 (2004) 689-715
41. Fazl A., Grossberg S., and Mingolla E. View-invariant object category learning, recognition, and search: How spatial and object attention are coordinated using surface-based attentional shrouds. Cognitive Psychology 58 (2009) 1-48
42. Fermüller C., and Aloimonos Y. Direct perception of three-dimensional motion from patterns of visual motion. Science (New York, NY) 270 5244 (1995) 1973-1976
43. Fleet D.J., and Langley K. Recursive filters for optical flow. IEEE Transactions on Pattern Analysis and Machine Intelligence 17 1 (1995) 61-67
44. Fried S.I., Münch T.A., and Werblin F.S. Mechanisms and circuitry underlying directional selectivity in the retina. Nature 420 6914 (2002) 411-414
45. Fried S.I., Münch T.A., and Werblin F.S. Directional selectivity is formed at multiple levels by laterally offset inhibition in the rabbit retina. Neuron 46 1 (2005) 117-127
46. Gibson J.J. The perception of the visual world (1950), Houghton Mifflin, Boston
47. Goodale M.A., and Milner D. Separate visual pathways for perception and action. Trends in Neuroscience 15 (1992) 10-25
48. Graziano M.S., Andersen R.A., and Snowden R.J. Tuning of MST neurons to spiral motions. Journal of Neuroscience 14 1 (1994) 54-67
49. Grossberg S. Some physiological and biochemical consequences of psychological postulates. Proceedings of the National Academy of Sciences 59 (1968) 368-372
50. Grossberg S. Contour enhancement, short term memory, and constancies in reverberating neural networks. Studies in Applied Mathematics 52 (1973) 213-257
51. Grossberg S. Intracellular mechanisms of adaptation and self-regulation in self-organizing networks: The role of chemical transducers. Bulletin of Mathematical Biology 42 3 (1980) 365-396
52. Grossberg S. The complementary brain: Unifying brain dynamics and modularity. Trends in Cognitive Sciences 4 6 (2000) 233-246
53. Grossberg S., Mingolla E., and Pack C.C. A neural model of motion processing and visual navigation by cortical area MST. Cerebral Cortex 9 8 (1999) 878-895
54. Grossberg S., Mingolla E., and Viswanathan L. Neural dynamics of motion integration and segmentation within and across apertures. Vision Research 41 19 (2001) 2521-2553
55. Grossberg S., and Rudd M.E. Cortical dynamics of visual motion perception: Short-range and long-range apparent motion. Psychological Review 99 1 (1992) 78-121
56. Hatsopoulos N.G., and Warren W.H. Visual navigation with a neural network. Neural Networks 4 3 (1991) 303-317
57. Heeger D.J., Boynton G.M., Demb J.B., Seidemann E., and Newsome W.T. Motion opponency in visual cortex. Journal of Neuroscience 19 16 (1999) 7162-7174
58. Heeger D.J., and Jepson A.D. Subspace methods for recovering rigid motion I: Algorithm and implementation. International Journal of Computer Vision 7 2 (1992) 95-117
59. Hildreth E.C. Recovering heading for visually-guided navigation. Vision Research 32 6 (1992) 1177-1192
60. Hodgkin A.L. The conduction of the nerve impulse (1964), CC. Thomas, Springfield, Illinois
61. Horn B.K.P., and Schunck B.G. Determining optical flow. Artificial Intelligence 17 1-3 (1981) 185-203
62. Kaiser, M., Perrone, J. A., Stone, L. S., Banks, M. S., & Crowell, J. A. (1993). Extracting heading and temporal range from optic flow: Human performance issues. In Proceedings of the workshop on augmented visual display (AVID) research (pp. 379-396) (see N 94-25490 06-54).
63. Kaplan E., and Benardete E. The dynamics of primate retinal ganglion cells. Progress in Brain Research 134 (2001) 17-34
64. Kaplan E., and Shapley R.M. X and Y cells in the lateral geniculate nucleus of macaque monkeys. Journal of Physiology 330 1 (1982) 125-143
65. Langer M.S., and Mann R. Optical snow. International Journal of Computer Vision 55 1 (2003) 55-71
66. Lappe M., Bremmer F., and van den Berg A.V. Perception of self-motion from visual flow. Trends in Cognitive Sciences 3 9 (1999) 329-336
67. Lappe M., and Rauschecker J.P. A neural network for the processing of optic flow from ego-motion in man and higher mammals. Neural computation 5 3 (1993) 374-391
68. Lappe M., and Rauschecker J.P. Heading detection from optic flow. Nature 369 6483 (1994) 712-713
69. Lappe M., and Rauschecker J.P. Motion anisotropies and heading detection. Biological Cybernetics 72 3 (1995) 261-277
70. Lappe M., and Rauschecker J.P. An illusory transformation in a model of optic flow processing. Vision Research 35 11 (1995) 1619-1631
71. Li L., Sweet B.T., and Stone L.S. Humans can perceive heading without visual path information. Journal of Vision 6 9 (2006) 874-881
72. Li L., and Warren W.H. Perception of heading during rotation: Sufficiency of dense motion parallax and reference objects. Vision Research 40 28 (2000) 3873-3894
73. Li L., and Warren W.H. Path perception during rotation: Influence of instructions, depth range, and dot density. Vision Research 44 16 (2004) 1879-1889
74. Livingstone M.S. Mechanisms of direction selectivity in macaque V1. Neuron 20 3 (1998) 509-526
75. Longuet-Higgins H.C., and Prazdny K. The interpretation of a moving retinal image. Proceedings of the Royal Society of London Series B, Biological Sciences 208 1173 (1980) 385-397
76. Loomis J.M., and Beall A. Visually controlled locomotion: Its dependence on optic flow, three-dimensional space perception, and cognition. Ecological Psychology 10 3-4 (1998) 271-285
77. Lucas, B. D., & Kanade, T. (1981). An iterative image registration technique with an application to stereo vision. In Proceedings of the DARPA image understanding workshop (pp. 121-130).
78. Mann, R., & Langer, M. S. (2002). Optical snow and the aperture problem. In International conference on pattern recognition.
79. Marr D., and Ullman S. Directional selectivity and its use in early visual processing. Proceedings of the Royal Society of London Series B Biological Sciences 211 1183 (1981) 151-180
80. MathWorks (2005). Natick, Mass: The MathWorks Inc.
81. Microsoft (2003). Redmond, WA 98052: Microsoft Corporation.
82. Mingolla E., Browning N.A., and Grossberg S. Neural dynamics of visually-based object segmentation and navigation in complex environments. Journal of Vision 8 6 (2008) 1154 [Abstract]
83. Mingolla E., Todd J., and Norman J. The perception of globally coherent motion. Vision Research 32 6 (1992) 1015-1031
84. Mishkin M., Ungerleider L.G., and Macko K.A. Object vision and spatial vision: Two cortical pathways. Trends in Neurosciences 6 10 (1983) 414-417
85. Ogden J.M., Adelson E.H., Bergen J.R., and Burt P.J. Pyramid-based computer graphics. RCA Engineer 30 5 (1985) 4-15
86. Olson C.F., Matthies L.H., Schoppers M., and Maimone M.W. Rover navigation using stereo ego-motion. Robotics and Autonomous Systems 43 4 (2003) 215-229
87. Orban G.A., Fize D., Peuskens H., Denys K., Nelissen K., Sunaert S., et al. Similarities and differences in motion processing between the human and macaque brain: Evidence from fMRI. Neuropsychologia 41 13 (2003) 1757-1768
88. Pack C.C., and Born R.T. Temporal dynamics of a neural solution to the aperture problem in visual area MT of macaque brain. Nature 409 6823 (2001) 1040-1042
89. Pack C.C., Grossberg S., and Mingolla E. A neural model of smooth pursuit control and motion perception by cortical area MST. Journal of Cognitive Neuroscience 13 1 (2001) 102-120
90. Page W.K., and Duffy C.J. MST neuronal responses to heading direction during pursuit eye movements. Journal of Neurophysiology 81 2 (1999) 596-610
91. Perrone J.A., and Stone L.S. A model of self-motion estimation within primate extrastriate visual cortex. Vision Research 34 21 (1994) 2917-2938
92. Perrone J.A., and Stone L.S. Emulating the visual receptive-field properties of MST neurons with a template model of heading estimation. Journal of Neuroscience 18 15 (1998) 5958-5975
93. Rieger J., and Lawton D. Processing differential image motion. Optical Society of America Journal A: Optics and Image Science 2 (1985) 354-360
94. Rodieck R.W., Binmoeller K.F., and Dineen J. Parasol and midget ganglion cells of the human retina. The Journal of Comparative Neurology 233 1 (1985) 115-132
95. Royden C.S. Mathematical analysis of motion-opponent mechanisms used in the determination of heading and depth. Journal of the Optical Society of America A 14 9 (1997) 2128-2143
96. Royden C.S. Computing heading in the presence of moving objects: A model that uses motion-opponent operators. Vision Research 42 28 (2002) 3043-3058
97. Royden C.S., Banks M.S., and Crowell J.A. The perception of heading during eye movements. Nature 360 6404 (1992) 583-585
98. Royden C.S., Crowell J.A., and Banks M.S. Estimating heading during eye movements. Vision Research 34 23 (1994) 3197-3214
99. Royden C.S., and Hildreth E.C. Human heading judgments in the presence of moving objects. Perception and Psychophysics 58 6 (1996) 836-856
100. Royden C.S., and Vaina L.M. Is precise discrimination of low level motion needed for heading discrimination. NeuroReport 15 6 (2004) 1013-1017
101. Rushton S.K., Harris J.M., Lloyd M.R., and Wann J.P. Guidance of locomotion on foot uses perceived target location rather than optic flow. Current Biology 8 21 (1998) 1191-1194
102. Saito H., Yukie M., Tanaka K., Hikosaka K., Fukada Y., and Iwai E. Integration of direction signals of image motion in the superior temporal sulcus of the macaque monkey. Journal of Neuroscience 6 1 (1986) 145-157
103. Schneider G.E. Contrasting visuomotor functions of tectum and cortex in the golden hamster. Psychological Research 31 1 (1967) 52-62
104. Schwartz E.L. Spatial mapping in the primate sensory projection: Analytic structure and relevance to perception. Biological cybernetics 25 4 (1977) 181-194
105. Simoncelli E.P., and Heeger D.J. A model of neuronal responses in visual area MT. Vision Research 38 5 (1998) 743-761
106. Singh, A. (1990). An estimation-theoretic framework for image-flow computation. In Proceedings, third international conference on computer vision (pp. 168-177).
107. Snowden R., Treue S., Erickson R., and Andersen R. The response of area MT and V1 neurons to transparent motion. Journal of Neuroscience 11 9 (1991) 2768-2785
108. Sony (2003). New York, NY: Sony Corporation of America.
109. Sperling G. Model of visual adaptation and contrast detection. Perception and Psychophysics 8 (1970) 143-157
110. Stone, L. S., & Perrone, J. A. (1994). A role for MST neurons in heading estimation. In RECON no. 20010116589. Society for Neuroscience, Miami Beach, FL, United States, 13-18 November 1994.
111. Stone L.S., and Perrone J.A. Quantitative simulations of MST visual receptive field properties using a template model of heading estimation. Society of Neuroscience Abstracts 23 (1997) 1126
112. Stone L.S., and Perrone J.A. Human heading estimation during visually simulated curvilinear motion. Vision Research 37 5 (1997) 573-590
113. Tarr M.J., and Warren W.H. Virtual reality in behavioral neuroscience and beyond. Nature Neuroscience 5 Suppl. (2002) 1089-1092
114. Tomasi, C., & Kanade, T. (1991). Detection and tracking of point features. School Comput. Sci., Carnegie Mellon Univ., Pittsburgh, PA, Tech. Rep. CMU-CS-91-132.
115. van den Berg A.V. Perception of heading. Nature 365 6446 (1993) 497-498
116. van den Berg A.V., and Brenner E. Humans combine the optic flow with static depth cues for robust perception of heading. Vision Research 34 16 (1994) 2153-2167
117. van Santen J.P.H., and Sperling G. Elaborated Reichardt detectors. Journal of the Optical Society of America A 2 2 (1985) 300-320
118. Wagner R.E., Polimeni J.R., and Schwartz E.L. Gibson meet topography: The dipole structure of extra striate cortex facilitates navigation via optical flow. Journal of Vision 5 8 (2005) 895 [Abstract]
119. Wallach H. On the visually perceived direction of motion. Psychologische Forschung 20 (1935) 325-380
120. Warren W.H. In: Watanabe T. (Ed). High Level Motion Processing (1998), MIT Press, Cambridge, MA
121. Warren W.H., and Fajen B.R. In: Vaina L.M., Beardsley S.A., and Rushton S.K. (Eds). From optic flow to laws of control. Optic flow and beyond (2004), Kluwer Academic Publishers, Norwell MA 307-337
122. Warren W.H., and Hannon D.J. Direction of self-motion is perceived from optical flow. Nature 336 6195 (1988) 162-163
123. Warren W.H., and Hannon D.J. Eye movements and optical flow. Optical Society of America Journal A: Optics and Image Science 7 (1990) 160-169
124. Warren W.H., Kay B.A., Zosh W.D., Duchon A.P., and Sahuc S. Optic flow is used to control human walking. Nature Neuroscience 4 (2001) 213-216
125. Warren W.H., and Saunders J.A. Perceiving heading in the presence of moving objects. Perception 24 3 (1995) 315-331
126. Wilkie R.M., and Wann J.P. Controlling steering and judging heading: Retinal flow, visual direction and extra-retinal information. Journal of Experimental Psychology 29 2 (2003) 363-378
127. Wilkie R.M., and Wann J.P. Judgments of path, not heading, guide locomotion. Journal of Experimental Psychology: Human Perception and Performance 32 1 (2006) 88-96
128. Wuerger S., Shapley R., and Rubin N. "On the visually perceived direction of motion" by Hans Wallach: 60 years later. Perception 25 1317 (1996) 67
129. Zemel R.S., and Sejnowski T.J. A model for encoding multiple object motions and self-motion in area MST of primate visual cortex. Journal of Neuroscience 18 1 (1998) 531-547