Tutorial: Visual odometry

IEEE Robotics and Automation Magazine

Volumn 18, Issue 4, 2011, Pages 80-92

  Scaramuzza, Davide ^a   Fraundorfer, Friedrich ^b  

^a UNIVERSITY OF PENNSYLVANIA   (United States)

^b ETH ZURICH   (Switzerland)

Author keywords

[No Author keywords available]

Indexed keywords

APPARENT MOTION; APPLICATION DOMAINS; EGO-MOTION; MULTIPLE CAMERAS; ONBOARD CAMERA; VISUAL ODOMETRY; WEARABLE COMPUTING; WHEEL ODOMETRY;

AUGMENTED REALITY; CAMERAS; ESTIMATION; VEHICLE WHEELS; WEARABLE COMPUTERS;

VEHICLES;

EID: 83455211013     PISSN: 10709932     EISSN: None     Source Type: Journal    
DOI: 10.1109/MRA.2011.943233     Document Type: Article

References (82)

1. D. Nister, O. Naroditsky, and J. Bergen, "Visual odometry," in Proc. Int. Conf. Computer Vision and Pattern Recognition, 2004, pp. 652-659.
2. H. Longuet-Higgins, "A computer algorithm for reconstructing a scene from two projections," Nature, vol. 293, no. 10, pp. 133-135, 1981.
3. C. Harris and J. Pike, "3d positional integration from image sequences," in Proc. Alvey Vision Conf., 1988, pp. 87-90.
4. J.-M. Frahm, P. Georgel, D. Gallup, T. Johnson, R. Raguram, C. Wu, Y.-H. Jen, E. Dunn, B. Clipp, S. Lazebnik, and M. Pollefeys, "Building rome on a cloudless day," in Proc. European Conf. Computer Vision, 2010, pp. 368-381.
5. H. Moravec, "Obstacle avoidance and navigation in the real world by a seeing robot rover," Ph.D. dissertation, Stanford Univ., Stanford, CA, 1980.
6. L. Matthies and S. Shafer, "Error modeling in stereo navigation," IEEE J. Robot. Automat., vol. 3, no. 3, pp. 239-248, 1987.
7. L. Matthies, "Dynamic stereo vision," Ph.D. dissertation, CarnegieMellon Univ., Pittsburgh, PA, 1989.
8. S. Lacroix, A. Mallet, R. Chatila, and L. Gallo, "Rover self localization in planetary-like environments," in Proc. Int. Symp. Articial Intelligence, Robotics, and Automation for Space (i-SAIRAS), 1999, pp. 433-440.
9. C. Olson, L. Matthies, M. Schoppers, and M. W. Maimone, "Robust stereo ego-motion for long distance navigation," in Proc. IEEE Conf. Computer Vision and Pattern Recognition, 2000, pp. 453-458.
10. M. Hannah, "Computer matching of areas in stereo images," Ph.D. dissertation, Stanford Univ., Stanford, CA, 1974.
11. H. Moravec, "Towards automatic visual obstacle avoidance," in Proc. 5th Int. Joint Conf. Artificial Intelligence, Aug. 1977, p. 584.
12. W. Forstner, "A feature based correspondence algorithm for image matching," Int. Arch. Photogrammetry, vol. 26, no. 3, pp. 150-166, 1986.
13. C. Olson, L. Matthies, M. Schoppers, and M. Maimone, "Rover navigation using stereo ego-motion," Robot. Autonom. Syst., vol. 43, no. 4, pp. 215-229, 2003.
14. A. Milella and R. Siegwart, "Stereo-based ego-motion estimation using pixel tracking and iterative closest point," in Proc. IEEE Int. Conf. Vision Systems, pp. 21-24, 2006.
15. A. Howard, "Real-time stereo visual odometry for autonomous ground vehicles," in Proc. IEEE/RSJ Int. Conf. Intelligent Robots and Systems, 2008, pp. 3946-3952.
16. Y. Cheng, M. W. Maimone, and L. Matthies, "Visual odometry on the mars exploration rovers," IEEE Robot. Automat. Mag., vol. 13, no. 2, pp. 54-62, 2006.
17. M. Maimone, Y. Cheng, and L. Matthies, "Two years of visual odometry on the mars exploration rovers: Field reports," J. Field Robot., vol. 24, no. 3, pp. 169-186, 2007. (Pubitemid 46647283)
18. M. A. Fischler and R. C. Bolles, "Random sample consensus: A paradigm for model fitting with applications to image analysis and automated cartography," Commun. ACM, vol. 24, no. 6, pp. 381-395, 1981.
19. C. Tomasi and J. Shi, "Good features to track," in Proc. Computer Vision and Pattern Recognition (CVPR '94), 1994, pp. 593-600.
20. P. Besl and N.McKay, "Amethod for registration of 3-d shapes," IEEE Trans. Pattern Anal. Machine Intell., vol. 14, no. 2, pp. 239-256, 1992.
21. A. Comport, E. Malis, and P. Rives, "Accurate quadrifocal tracking for robust 3d visual odometry," in Proc. IEEE Int. Conf. Robotics and Automation, 2007, pp. 40-45. (Pubitemid 350140162)
22. R. Hartley and A. Zisserman, Multiple View Geometry in Computer Vision, 2nd ed. Cambridge U.K.: Cambridge Univ. Press, 2004.
23. D. Nister, O. Naroditsky, and J. Bergen, "Visual odometry for ground vehicle applications," J. Field Robot., vol. 23, no. 1, pp. 3-20, 2006.
24. P. I. Corke, D. Strelow, and S. Singh, "Omnidirectional visual odometry for a planetary rover," in Proc. IEEE/RSJ Int. Conf. Intelligent Robots and Systems, 2005, pp. 4007-4012.
25. M. Lhuillier, "Automatic structure and motion using a catadioptric camera," in Proc. IEEE Workshop Omnidirectional Vision, 2005, pp. 1-8.
26. R. Goecke, A. Asthana, N. Pettersson, and L. Petersson, "Visual vehicle egomotion estimation using the Fourier-Mellin transform," in Proc. IEEE Intelligent Vehicles Symp., 2007, pp. 450-455.
27. J. Tardif, Y. Pavlidis, and K. Daniilidis, "Monocular visual odometry in urban environments using an omnidirectional camera," in Proc. IEEE/ RSJ Int. Conf. Intelligent Robots and Systems, 2008, pp. 2531-2538.
28. M. J. Milford and G.Wyeth, "Single camera vision-only SLAM on a suburban road network," in Proc. IEEE Int. Conf. Robotics and Automation (ICRA '08), 2008, pp. 3684-3689.
29. D. Scaramuzza and R. Siegwart, "Appearance-guided monocular omnidirectional visual odometry for outdoor ground vehicles," IEEE Trans. Robot. (Special Issue on Visual SLAM), vol. 24, no. 5, pp. 1015-1026, Oct. 2008.
30. E. Mouragnon, M. Lhuillier, M. Dhome, F. Dekeyser, and P. Sayd, "Real time localization and 3d reconstruction," in Proc. Int. Conf. Computer Vision and Pattern Recognition, 2006, pp. 363-370. (Pubitemid 44931352)
31. D. Scaramuzza, F. Fraundorfer, and R. Siegwart, "Real-time monocular visual odometry for on-road vehicles with 1-point RANSAC," in Proc. IEEE Int. Conf. Robotics and Automation (ICRA '09), 2009, pp. 4293-4299.
32. A. Pretto, E. Menegatti, and E. Pagello, "Omnidirectional dense large-scale mapping and navigation based on meaningful triangulation," in Proc. IEEE Int. Conf. Robotics and Automation, 2011, pp. 3289-3296.
33. D. Nister, "An efficient solution to the five-point relative pose problem," in Proc. Int. Conf. Computer Vision and Pattern Recognition, 2003, pp. 195-202.
34. M. Milford, G. Wyeth, and D. Prasser, "RatSLAM: A hippocampal model for simultaneous localization and mapping," in Proc. IEEE Int. Conf. Robotics and Automation (ICRA '04), 2004, pp. 403-408.
35. B. Liang and N. Pears, "Visual navigation using planar homographies," in Proc. IEEE Int. Conf. Robotics and Automation (ICRA '02), 2002, pp. 205-210. (Pubitemid 34916601)
36. Q. Ke and T. Kanade, "Transforming camera geometry to a virtual downward-looking camera: Robust ego-motion estimation and groundlayer detection," in Proc. Computer Vision and Pattern Recognition (CVPR), June 2003, pp. 390-397.
37. H. Wang, K. Yuan, W. Zou, and Q. Zhou, "Visual odometry based on locally planar ground assumption," in Proc. IEEE Int. Conf. Information Acquisition, 2005, pp. 59-64.
38. J. Guerrero, R. Martinez-Cantin, and C. Sagues, "Visual map-less navigation based on homographies," J. Robot. Syst., vol. 22, no. 10, pp. 569-581, 2005. (Pubitemid 41638106)
39. D. Scaramuzza, "1-point-RANSAC structure from motion for vehicle- mounted cameras by exploiting non-holonomic constraints," Int. J. Comput. Vis., vol. 95, no. 1, pp. 74-85, 2011.
40. D. Scaramuzza, F. Fraundorfer, M. Pollefeys, and R. Siegwart, "Absolute scale in structure from motion from a single vehicle mounted camera by exploiting nonholonomic constraints," in Proc. IEEE Int. Conf. Computer Vision (ICCV), Kyoto, Oct. 2009, pp. 1413-1419.
41. F. Fraundorfer, D. Scaramuzza, and M. Pollefeys, "A constricted bundle adjustment parameterization for relative scale estimation in visual odometry," in Proc. IEEE Int. Conf. Robotics and Automation, 2010, pp. 1899-1904.
42. N. Sunderhauf, K. Konolige, S. Lacroix, and P. Protzel, "Visual odometry using sparse bundle adjustment on an autonomous outdoor vehicle," in Tagungsband Autonome Mobile Systeme, Reihe Informatik aktuell. Levi, Schanz, Lafrenz, and Avrutin, Eds. Berlin, Springer-Verlag, 2005, pp. 157-163.
43. K. Konolige, M. Agrawal, and J. Sol, "Large scale visual odometry for rough terrain," in Proc. Int. Symp. Robotics Research, 2007.
44. J. Tardif, M. G. M. Laverne, A. Kelly, and M. Laverne, "A new approach to vision-aided inertial navigation," in Proc. IEEE/RSJ Int. Conf. Intelligent Robots and Systems, 2010, pp. 4161-4168.
45. A. I. Mourikis and S. Roumeliotis, "A multi-state constraint kalman filter for vision-aided inertial navigation," in Proc. IEEE Int. Conf. Robotics and Automation, 2007, pp. 3565-3572. (Pubitemid 350140736)
46. E. Jones and S. Soatto, "Visual-inertial navigation, mapping and localization: A scalable real-time causal approach," Int. J. Robot. Res., vol. 30, no. 4, pp. 407-430, 2010.
47. H. Durrant-Whyte and T. Bailey, "Simultaneous localization and mapping (SLAM): Part I. The essential algorithms," Robot. Automat. Mag., vol. 13, no. 2, pp. 99-110, 2006.
48. T. Bailey and H. Durrant-Whyte, "Simultaneous localisation and mapping (SLAM): Part II. State of the art," Robot. Automat. Mag., vol. 13, no. 3, pp. 108-117, 2006. (Pubitemid 46530408)
49. A. Davison, "Real-time simultaneous localisation and mapping with a single camera," in Proc. Int. Conf. Computer Vision, 2003, pp. 1403-1410.
50. G. Klein and D. Murray, "Parallel tracking and mapping for small ar workspaces," in Proc. Int. Symp. Mixed and Augmented Reality, 2007, pp. 225-234.
51. H. Strasdat, J. Montiel, and A. Davison, "Real time monocular SLAM: Why filter?" in Proc. IEEE Int. Conf. Robotics and Automation, 2010, pp. 2657-2664.
52. A. Chiuso, P. Favaro, H. Jin, and S. Soatto, "3-D motion and structure from 2-D motion causally integrated over time: Implementation," in Proc. European Conf. Computer Vision, 2000, pp. 734-750.
53. M. C. Deans, "Bearing-only localization and mapping," Ph.D. dissertation, Carnegie Mellon Univ., Pittsburgh, 2002.
54. L. A. Clemente, A. J. Davison, I. Reid, J. Neira, and J. D. Tardos, "Mapping large loops with a single hand-held camera," in Proc. Robotics Science and Systems, 2007.
55. T. Lemaire and S. Lacroix, "Vision-based SLAM: Stereo and monocular approaches," Int. J. Computer Vision, vol. 74, no. 3, pp. 343-364, 2006.
56. E. Eade and T. Drummond, "Monocular SLAM as a graph of coalesced observations," in Proc. IEEE Int. Conf. Computer Vision, 2007, pp. 1-8.
57. G. Klein and D. Murray, "Improving the agility of keyframe-based SLAM," in Proc. European Conf. Computer Vision, 2008, pp. 802-815.
58. K. Konolige and M. Agrawal, "FrameSLAM: From bundle adjustment to real-time visual mappping," IEEE Trans. Robot., vol. 24, no. 5, pp. 1066-1077, 2008.
59. A. Handa, M. Chli, H. Strasdat, and A. J. Davison, "Scalable active matching," in Proc. Int. Conf. Computer Vision and Pattern Recognition, 2010, pp. 1546-1553.
60. J. Civera, O. Grasa, A. Davison, and J. Montiel, "1-point RANSAC for ekf filtering: Application to real-time structure from motion and visual odometry," J. Field Robot., vol. 27, no. 5, pp. 609-631, 2010.
61. H. Strasdat, J. Montiel, and A. J. Davison, "Scale drift-aware large scale monocular SLAM," in Proc. Robotics Science and Systems, 2010.
62. C. Mei, G. Sibley, M. Cummins, P. Newman, and I. Reid, "RSLAM: A system for large-scale mapping in constant-time using stereo," Int. J. Computer Vision, vol. 94, no. 2, pp. 198-214, 2010.
63. Y. Ma, S. Soatto, J. Kosecka, and S. Sastry, An Invitation to 3D Vision, from Images to Models. Berlin: Springer-Verlag, 2003.
64. C. Geyer and K. Daniilidis, "A unifying theory for central panoramic systems and practical applications," in Proc. European Conf. Computer Vision, 2000, pp. 445-461.
65. D. Scaramuzza, A. Martinelli, and R. Siegwart, "A flexible technique for accurate omnidirectional camera calibration and structure from motion," in Proc. IEEE Int. Conf. Computer Vision Systems (ICVS) 2006, Jan. 2006, pp. 45-53.
66. D. Scaramuzza, "Omnidirectional vision: From calibration to robot motion estimation" Ph.D. dissertation, ETH Zurich, 2008.
67. D. Scaramuzza, "Omnidirectional camera," in Encyclopedia of Computer Vision, K. Ikeuchi, Ed. Berlin: Springer-Verlag, 2012.
68. J. Bouguet (2011). List of camera calibration toolboxes. [Online]. Available: http://www.vision.caltech.edu/bouguetj/calib.doc/htmls/links.html
69. J. Bouguet. Camera calibration toolbox for Matlab. [Online]. Available: http://www.vision.caltech.edu/bouguetj/calib.doc/index.html
70. D. Scaramuzza. (2006). Ocamcalib toolbox: Omnidirectional camera calibration toolbox for Matlab (uses planar grids) google for "ocamcalib." [Online]. Available: https://sites.google.com/site/ scarabotix/ocamcalibtoolbox
71. C. Mei. (2006). Omnidirectional camera calibration toolbox for Matlab (uses planar grids). [Online]. Available: http://homepages.laas.fr/~cmei/index. php/Toolbox
72. J. Barreto. Omnidirectional camera calibration toolbox for Matlab (uses lines). [Online]. Available: http://www.isr.uc.pt/~jpbar/CatPack/pag1.htm
73. Opencv: Open-source computer vision library. [Online]. Available: http://opencv.willowgarage.com/wiki/
74. T. Huang and A. Netravalli, "Motion and structure from feature correspondences: A review," Proc. IEEE, vol. 82, no. 2, pp. 252-268, 1994.
75. E. Kruppa, "Zur ermittlung eines objektes aus zwei perspektiven mit innerer orientierung," Sitzungsberichte der Akademie der Wissenschaften, Wien, Mathematisch-Naturwissenschaftlichen Klasse, Abteilung IIa, vol. 122, pp. 1939-1948, 1913.
76. D. Nister, "An efficient solution to the five-point relative pose problem," in Proc. Computer Vision and Pattern Recognition (CVPR '03), 2003, pp. II: 195-202.
77. H. Goldstein, ClassicalMechanics. Reading, MA:Addison-Wesley, 1981.
78. K. S. Arun, T. S. Huang, and S. D. Blostein, "Least-squares fitting of two 3-d point sets," IEEE Trans. Pattern Anal. Machine Intell., vol. 9, no. 5, pp. 698-700, 1987.
79. F. Moreno-Noguer, V. Lepetit, and P. Fua, "Accurate non-iterative O(n) solution to the PnP problem," in Proc. IEEE Int. Conf. Computer Vision, 2007, pp. 1-8.
80. L. Kneip, D. Scaramuzza, and R. Siegwart, "A novel parameterization of the perspective-three-point problem for a direct computation of absolute camera position and orientation," in Proc. IEEE Conf. Computer Vision and Pattern Recognition, 2011, pp. 2969-2976.
81. D. Nister, "A minimal solution to the generalised 3-point pose problem," in Proc. IEEE Conf. Computer Vision and Pattern Recognition, 2004, pp. 560-567.
82. C. Harris and M. Stephens, "A combined corner and edge detector," in Proc. Alvey Vision Conf., 1988, pp. 147-151.