A cortical architecture on parallel hardware for motion processing in real time

Journal of Vision

Volumn 10, Issue 10, 2010, Pages

  Pauwels, Karl ^a   Krüger, Norbert ^b   Lappe, Markus ^c   Wörgötter, Florentin ^d   Van Hulle, Marc M ^a  

^a KU LEUVEN UNIVERSITY OF LEUVEN   (Belgium)

^b UNIVERSITY OF SOUTHERN DENMARK   (Denmark)

^c UNIVERSITY OF MÜNSTER   (Germany)

^d Department for Computational Neuroscience III   (Germany)

Author keywords

Binocular disparity; Dorsal visual stream; Egomotion; Graphics processing unit; Object motion; Optical flow

Indexed keywords

ARTICLE; COMPARATIVE STUDY; COMPUTER INTERFACE; HUMAN; MOVEMENT PERCEPTION; PHYSIOLOGY; VISUAL CORTEX; VISUAL SYSTEM;

HUMANS; MOTION PERCEPTION; USER-COMPUTER INTERFACE; VISUAL CORTEX; VISUAL PATHWAYS;

EID: 79952637211     PISSN: None     EISSN: 15347362     Source Type: Journal    
DOI: 10.1167/10.10.18     Document Type: Article

References (106)

1. Adelson, E. H., & Bergen, J. R. (1985). Spatiotemporal energy models for the perception of motion. Journal of the Optical Society of America A, Optics, Image Science, and Vision, 2, 284-299.
2. Adiv, G. (1985). Determining three-dimensional motion and structure from optical flow generated by several moving objects. IEEE Transactions on Pattern Analysis and Machine Intelligence, 7, 384-401.
3. Argyros, A. A., Trahanias, P. E., & Orphanoudakis, S. C. (1998). Robust regression for the detection of independent 3D motion by a binocular observer. Real-Time Imaging, 4, 125-141.
4. Bergen, J., Anandan, P., Hanna, K., & Hingorani, R. (1992). Hierarchical model-based motion estimation. Proceedings of the Second European Conference on Computer Vision, Italy, 588, 237-252.
5. Bremmer, F., Kubischik, M., Pekel, M., Hoffmann, K. P., & Lappe, M. (2010). Visual selectivity for heading in monkey area MST. Experimental Brain Research, 200, 51-60.
6. Brenner, E. (1991). Judging object motion during smooth pursuit eye-movements: The role of optic flow. Vision Research, 31, 1893-1902.
7. Brenner, E., & van den Berg, A. V. (1996). The special role of distant structures in perceived object velocity. Vision Research, 36, 3805-3814.
8. Browning, N. A., Grossberg, S., & Mingolla, E. (2009). Cortical dynamics of navigation and steering in natural scenes: Motion-based object segmentation, heading, and obstacle avoidance. Neural Networks, 22, 1383-1398.
9. Burt, P. J., & Adelson, E. H. (1983). The Laplacian pyramid as a compact image code. IEEE Transactions on Communications, 31, 532-540.
10. Canny, J. (1986). A computational approach to edge detection. IEEE Transactions on Pattern Analysis and Machine Intelligence, 8, 679-698.
11. Chen, Y. H., & Qian, N. (2004). A coarse-to-fine disparity energy model with both phase-shift and position-shift receptive field mechanisms. Neural Computation, 16, 1545-1577.
12. Chiuso, A., Brockett, R., & Soatto, S. (2000). Optimal structure from motion: Local ambiguities and global estimates. International Journal of Computer Vision, 39, 195-228.
13. Daugman, J. G. (1985). Uncertainty relation for resolution in space, spatial frequency, and orientation optimized by two dimensional visual cortical filters. Journal of the Optical Society of America A, Optics, Image Science, and Vision, 2, 1160-1169.
14. Davison, A. J., Reid, I. D., Molton, N. D., & Stasse, O. (2007). MonoSLAM: Real-time single camera SLAM. IEEE Transactions on Pattern Analysis and Machine Intelligence, 29, 1052-1067.
15. Demirdjian, D., & Horaud, R. (2000). Motion-egomotion discrimination and motion segmentation from imagepair streams. Computer Vision and Image Understanding, 78, 53-68.
16. Duffy, C. J., & Wurtz, R. H. (1995). Response of monkey MST neurons to optic flow stimuli with shifted centers of motion. Journal of Neuroscience, 15, 5192-5208.
17. Durrant-Whyte, H., & Bailey, T. (2006). Simultaneous localization and mapping: Part I. The essential algorithms. IEEE Robotics & Automation Magazine, 13, 99-108.
18. Dyde, R. T., & Harris, L. R. (2008). The influence of retinal and extra-retinal motion cues on perceived object motion during self-motion. Journal of Vision, 8(14):5, 1-10, http://www.journalofvision.org/content/8/14/5, doi:10.1167/8.14.5.
19. Eifuku, S., & Wurtz, R. H. (1999). Response to motion in extrastriate area MSTl: Disparity sensitivity. Journal of Neurophysiology, 82, 2462-2475.
20. Ess, A., Leibe, B., Schindler, K., & Van Gool, L. (2009). Robust multiperson tracking from a mobile platform. IEEE Transactions on Pattern Analysis and Machine Intelligence, 31, 1831-1846.
21. Fleet, D. J., & Jepson, A. D. (1990). Computation of component image velocity from local phase information. International Journal of Computer Vision, 5, 77-104.
22. Fleet, D. J., & Jepson, A. D. (1993). Stability of phase information. IEEE Transactions on Pattern Analysis and Machine Intelligence, 15, 1253-1268.
23. Fleet, D. J., Jepson, A. D., & Jenkin, M. R. M. (1991). Phase-based disparity measurement. CVGIP-Image Understanding, 53, 198-210.
24. Franke, U., & Heinrich, S. (2002). Fast obstacle detection for urban traffic situations. IEEE Transactions on Intelligent Transportation Systems, 3, 173-181.
25. Fukushima, K. (1980). Neocognitron: A self-organizing neural network model for a mechanism of pattern recognition unaffected by shift in position. Biological Cybernetics, 36, 193-202.
26. Gautama, T., & Van Hulle, M. M. (2002). A phase-based approach to the estimation of the optical flow field using spatial filtering. IEEE Transactions on Neural Networks, 13, 1127-1136.
27. Gibson, J. J. (1950). The perception of the visual world. Boston: Houghton Mifflin.
28. Gogel, W. C. (1982). Analysis of the perception of motion concomitant with a lateral motion of the head. Perception & Psychophysics, 32, 241-250.
29. Gogel, W. C. (1990). A theory of phenomenal geometry and its applications. Perception & Psychophysics, 48, 105-123.
30. Granlund, G. H. (1978). In search of a general picture processing operator. Computer Graphics and Image Processing, 8, 155-173.
31. Grossberg, S. (2003). How does the cerebral cortex work? Development, learning, attention, and 3D vision by laminar circuits of visual cortex. Behavioral and Cognitive Neuroscience Reviews, 2, 47-76.
32. Hartley, R. I., & Zisserman, A. (2004). Multiple view geometry in computer vision. Cambridge, UK: Cambridge University Press.
33. Henriksson, L., Hyvärinen, A., & Vanni, S. (2009). Representation of cross frequency spatial phase relationships in human visual cortex. Journal of Neuroscience, 29, 14342-14351.
34. Ilg, U. J., Schumann, S., & Their, P. (2004). Posterior parietal cortex neurons encode target motion in world-centered coordinates. Neuron, 43, 145-151.
35. Irani, M., & Anandan, P. (1998). A unified approach to moving object detection in 2D and 3D scenes. IEEE Transactions on Pattern Analysis and Machine Intelligence, 20, 577-589.
36. Koenderink, J. J., & van Doorn, A. J. (1987). Facts on optic flow. Biological Cybernetics, 56, 247-254.
37. Komatsu, H., & Wurtz, R. H. (1988). Relation of cortical areas MT and MST to pursuit eye movements: 1. Localization and visual properties of neurons. Journal of Neurophysiology, 60, 580-603.
38. Kovesi, P. (1999). Image features from phase congruency. Videre, 1, 1-26.
39. Lappe, M., Bremmer, F., Pekel, M., Thiele, A., & Hoffmann, K. P. (1996). Optic flow processing in monkey STS: A theoretical and experimental approach. Journal of Neuroscience, 16, 6265-6285.
40. Lappe, M., & Rauschecker, J. P. (1993). A neural network for the processing of optic flow from egomotion in man and higher mammals. Neural Computation, 5, 374-391.
41. LeCun, Y., Boser, B., Denker, J. S., Henderson, D., Howard, R. E., Hubbard, W., et al. (1989). Backpropagation applied to handwritten zip code recognition. Neural Computation, 1, 541-551.
42. Leibe, B., Schindler, K., Cornelis, N., & Van Gool, L. (2008). Coupled object detection and tracking from static cameras and moving vehicles. IEEE Transactions on Pattern Analysis and Machine Intelligence, 30, 1683-1698.
43. Lindholm, E., Nickolls, J., Oberman, S., & Montrym, J. (2008). NVIDIA Tesla: A unified graphics and computing architecture. IEEE Micro, 28, 39-55.
44. Little, J. J., Poggio, T., & Gamble, E. B. (1988). Seeing in parallelVThe vision machine. International Journal of Supercomputing Applications and High Performance Computing, 2, 13-28.
45. Logan, D. J., & Duffy, C. J. (2006). Cortical area MSTd combines visual cues to represent 3-D self-movement. Cerebral Cortex, 16, 1494-1507.
46. Longuet-Higgins, H. C., & Prazdny, K. (1980). The interpretation of a moving retinal image. Proceedings of the Royal Society of London B: Biological Sciences, 208, 385-397.
47. Lowe, D. G. (2004). Distinctive image features from scale-invariant keypoints. International Journal of Computer Vision, 60, 91-110.
48. Matsumiya, K., & Ando, H. (2009). World-centered perception of 3D object motion during visually guided self-motion. Journal of Vision, 9(1):15, 1-13, http://www.journalofvision.org/content/9/1/15, doi:10.1167/9.1.15.
49. Maunsell, J. H. R., & Van Essen, D. C. (1983). The connections of the middle temporal visual area (MT) and their relationship to a cortical hierarchy in the macaque monkey. Journal of Neuroscience, 3, 2563-2586.
50. Mechler, F., Reich, D. S., & Victor, J. D. (2002). Detection and discrimination of relative spatial phase by V1 neurons. Journal of Neuroscience, 22, 6129-6157.
51. Mishkin, M., Ungerleider, L. G., & Macko, K. A. (1983). Object vision and spatial visionV2 cortical pathways. Trends in Neurosciences, 6, 414-417.
52. Morrone, M. C., & Burr, D. C. (1988). Feature detection in human vision: A phase dependent energy model. Proceedings of the Royal Society of London B: Biological Sciences, 235, 221-245.
53. Nelson, R. C. (1991). Qualitative detection of motion by a moving observer. International Journal of Computer Vision, 7, 33-46.
54. Nister, D., Naroditsky, O., & Bergen, J. (2006). Visual odometry for ground vehicle applications. Journal of Field Robotics, 23, 3-20.
55. Ogale, A. S., Fermuller, C., & Aloimonos, Y. (2005). Motion segmentation using occlusions. IEEE Transactions on Pattern Analysis and Machine Intelligence, 27, 988-992.
56. Ohzawa, I., DeAngelis, G. C., & Freeman, R. D. (1990). Stereoscopic depth discrimination in the visual cortex: Neurons ideally suited as disparity detectors. Science, 249, 1037-1041.
57. Ohzawa, I., DeAngelis, G. C., & Freeman, R. D. (1997). Encoding of binocular disparity by complex cells in the cat's visual cortex. Journal of Neurophysiology, 77, 2879-2909.
58. Oliensis, J. (2005). The least-squares error for structure from infinitesimal motion. International Journal of Computer Vision, 61, 259-299.
59. Orban, G. A. (2008). Higher order visual processing in macaque extrastriate cortex. Physiological Reviews, 88, 59-89.
60. Pauwels, K., & Van Hulle, M. M. (2009). Optic flow from unstable sequences through local velocity constancy maximization. Image and Vision Computing, 27, 579-587.
61. Perna, A., Tosetti, M., Montanaro, D., & Morrone, M. C. (2008). BOLD response to spatial phase congruency in human brain. Journal of Vision, 8(10):15, 1-15, http://www.journalofvision.org/content/8/10/15, doi:10.1167/8.10.15.
62. Perrone, J. A. (1992). Model for the computation of selfmotion in biological systems. Journal of the Optical Society of America A, Optics, Image Science, and Vision, 9, 177-194.
63. Perrone, J. A., & Krauzlis, R. J. (2008a). Spatial integration by MT pattern neurons: A closer look at pattern-to-component effects and the role of speed tuning. Journal of Vision, 8(9):1, 1-14, http://www. journalofvision.org/content/8/9/1, doi:10.1167/8.9.1.
64. Perrone, J. A., & Krauzlis, R. J. (2008b). Vector subtraction using visual and extraretinal motion signals: A new look at efference copy and corollary discharge theories. Journal of Vision, 8(14):24, 1-14, http://www.journalofvision.org/content/8/14/24, doi:10.1167/8.14.24.
65. Poggio, T., Gamble, E. B., & Little, J. J. (1988). Parallel integration of vision modules. Science, 242, 436-440.
66. Pollen, D., & Ronner, S. (1981). Phase relationships between adjacent simple cells in the visual cortex. Science, 212, 1409-1411.
67. Pugeault, N., Pauwels, K., Pilz, F., Van Hulle, M. M., & Krüger, N. (in press). A three level architecture for model-free detection and tracking of independently moving objects. Proceedings of the International Conference on Computer Vision Theory and Applications, Angers, France.
68. Qian, N., & Mikaelian, S. (2000). Relationship between phase and energy methods for disparity computation. Neural Computation, 12, 279-292.
69. Riesenhuber, M., & Poggio, T. (1999). Hierarchical models of object recognition in cortex. Nature Neuroscience, 2, 1019-1025.
70. Ringach, D. (2002). Spatial structure and symmetry of simple cell receptive fields in macaque primary visual cortex. Journal of Neurophysiology, 88, 455-463.
71. Rokers, B., Cormack, L. K., & Huk, A. C. (2009). Disparity- and velocity-based signals for three-dimensional motion perception in human MT. Nature Neuroscience, 12, 1050-1055.
72. Royden, C. S. (1997). Mathematical analysis of motionopponent mechanisms used in the determination of heading and depth. Journal of the Optical Society of America A: Optics, Image Science, and Vision, 14, 2128-2143.
73. Royden, C. S. (2002). Computing heading in the presence of moving objects: A model that uses motion-opponent operators. Vision Research, 42, 3043-3058.
74. Royden, C. S., & Hildreth, E. C. (1996). Human heading judgments in the presence of moving objects. Perception & Psychophysics, 58, 836-856.
75. Rushton, S. K., Bradshaw, M. F., & Warren, P. A. (2007). The pop out of scene-relative object movement against retinal motion due to self-movement. Cognition, 105, 237-245.
76. Rushton, S. K., Harris, J. M., Lloyd, M. R., & Wann, J. P. (1998). Guidance of locomotion on foot uses perceived target location rather than optic flow. Current Biology, 8, 1191-1194.
77. Rushton, S. K., & Warren, P. A. (2005). Moving observers, relative retinal motion and the detection of object movement. Current Biology, 15, R542-R543.
78. Rust, N. C., Mante, V., Simoncelli, E. P., & Movshon, J. A. (2006). How MT cells analyze the motion of visual patterns. Nature Neuroscience, 9, 1421-1431.
79. Sabatini, S. P., Gastaldi, G., Solari, F., Pauwels, K., Van Hulle, M., Di{dotless}́az, J., et al. (2010). A compact harmonic code for early vision based on anisotropic frequency channels. Computer Vision and Image Understanding, 114, 681-699.
80. Sanger, T. D. (1988). Stereo disparity computation using Gabor filters. Biological Cybernetics, 59, 405-418.
81. Sawhney, H. S., Guo, Y. L., & Kumar, R. (2000). Independent motion detection in 3D scenes. IEEE Transactions on Pattern Analysis and Machine Intelligence, 22, 1191-1199.
82. Schindler, K., Suter, D., & Wang, H. (2008). A modelselection framework for multibody structure-andmotion of image sequences. International Journal of Computer Vision, 79, 159-177.
83. Serre, T., Wolf, L., Bileschi, S., Riesenhuber, M., & Poggio, T. (2007). Robust object recognition with cortex like mechanisms. IEEE Transactions on Pattern Analysis and Machine Intelligence, 29, 411-426.
84. Sharma, R., & Aloimonos, Y. (1996). Early detection of independent motion from active control of normal image flow patterns. IEEE Transactions on Systems, Man, and Cybernetics Part B: Cybernetics, 26, 42-52.
85. Simoncelli, E. P., & Heeger, D. J. (1998). A model of neuronal responses in visual areaMT. Vision Research, 38, 743-761.
86. Tanaka, K., Sugita, Y., Moriya, M., & Saito, H. A. (1993). Analysis of object motion in the ventral part of the medial superior temporal area of the macaque visual cortex. Journal of Neurophysiology, 69, 128-142.
87. Thompson, W. B., & Pong, T. C. (1990). Detecting moving objects. International Journal of Computer Vision, 4, 39-57.
88. Torr, P. H. S. (1998). Geometric motion segmentation and model selection. Philosophical Transactions of the Royal Society of London A, Mathematical, Physical, and Engineering Sciences, 356, 1321-1340.
89. Vaudrey, T., Rabe, C., Klette, R., & Milburn, J. (2008). Differences between stereo and motion behaviour on synthetic and real-world stereo sequences. Proceedings of the 23rd International Conference on Image and Vision Computing, New Zealand, 1, 1-6.
90. Wallis, G., & Rolls, E. T. (1997). Invariant face and object recognition in the visual system. Progress in Neurobiology, 51, 167-194.
91. Wang, C. C., Thorpe, C., Thrun, S., Hebert, M., & Durrant-Whyte, H. (2007). Simultaneous localization, mapping and moving object tracking. International Journal of Robotics Research, 26, 889-916.
92. Wang, W. D., & Duncan, J. H. (1996). Recovering the three-dimensional motion and structure of multiple moving objects from binocular image flows. Computer Vision and Image Understanding, 63, 430-446.
93. Wangsiripitak, S., & Murray, D. W. (2009). Avoiding moving outliers in visual SLAM by tracking moving objects. Proceedings of the International Conference on Robotics and Automation, Kobe, Japan, 1, 375-380.
94. Warren, P. A., & Rushton, S. K. (2007). Perception of object trajectory: Parsing retinal motion into self and object movement components. Journal of Vision, 7(11):2, 1-11, http://www.journalofvision.org/content/7/11/2, doi:10.1167/7.11.2.
95. Warren, P. A., & Rushton, S. K. (2008). Evidence for flow-parsing in radial flow displays. Vision Research, 48, 655-663.
96. Warren, P. A., & Rushton, S. K. (2009a). Perception of scene-relative object movement: Optic flow parsing and the contribution of monocular depth cues. Vision Research, 49, 1406-1419.
97. Warren, P. A., & Rushton, S. K. (2009b). Optic flow processing for the assessment of object movement during ego movement. Current Biology, 19, 1555-1560.
98. Warren, W. H., & Saunders, J. A. (1995). Perceiving heading in the presence of moving objects. Perception, 24, 315-331.
99. Wedel, A., Rabe, C., Meissner, A., Franke, U., & Cremers, D. (2009). Detection and segmentation of independently moving objects from dense scene flow. Proceedings of the 7th International Conference on Energy Minimization Methods in Computer Vision and Pattern Recognition, Bonn, Germany, 5681, 14-27.
100. Wexler, M., Lamouret, I., & Droulez, J. (2001). The stationarity hypothesis: An allocentric criterion in visual perception. Vision Research, 41, 3023-3037.
101. Wexler, M., Panerai, F., Lamouret, I., & Droulez, J. (2001). Self-motion and the perception of stationary objects. Nature, 409, 85-88.
102. Williams, B., & Reid, I. (2010). On combining visual SLAM and visual odometry. Proceedings of the International Conference on Robotics and Automation, Anchorage, Alaska, 1, 3494-3500.
103. Wong, T. T., Leung, C. S., Heng, P. A., & Wang, J. Q. (2007). Discrete wavelet transform on consumer-level graphics hardware. IEEE Transactions on Multimedia, 9, 668-673.
104. Yuan, C., Medioni, G., Kang, J. M., & Cohen, I. (2007). Detecting motion regions in the presence of a strong parallax from a moving camera by multiview geometric constraints. IEEE Transactions on Pattern Analysis and Machine Intelligence, 29, 1627-1641.
105. Zemel, R. S., & Sejnowski, T. J. (1998). A model for encoding multiple object motions and self-motion in area MST of primate visual cortex. Journal of Neuroscience, 18, 531-547.
106. Zhang, T., & Tomasi, C. (2002). On the consistency of instantaneous rigid motion estimation. International Journal of Computer Vision, 46, 51-79.