Visual odometry: Part II: Matching, robustness, optimization, and applications

IEEE Robotics and Automation Magazine

Volumn 19, Issue 2, 2012, Pages 78-90

  Fraundorfer, Friedrich ^a   Scaramuzza, Davide ^b  

^a ETH ZURICH   (Switzerland)

^b UNIVERSITY OF ZURICH   (Switzerland)

Author keywords

[No Author keywords available]

Indexed keywords

COMPACT DESCRIPTOR; DESCRIPTORS; EPIPOLAR LINE; FEATURE DESCRIPTION; FEATURE DESCRIPTORS; IMAGE PATTERNS; KEYPOINTS; LOCAL FEATURE; MOTION MODELS; MULTIPLE CAMERAS; UNEVEN TERRAIN; VISUAL ODOMETRY; WHEEL ODOMETRY; WHEEL SLIPS;

THREE DIMENSIONAL; WHEELS;

IMAGE TEXTURE;

EID: 84862585064     PISSN: 10709932     EISSN: None     Source Type: Journal    
DOI: 10.1109/MRA.2012.2182810     Document Type: Article

References (91)

1. D. Nister, O. Naroditsky, and J. Bergen, "Visual odometry," in Proc. Int. Conf. Computer Vision and Pattern Recognition, 2004, pp. 652-659.
2. H. Moravec, "Obstacle avoidance and navigation in the real world by a seeing robot rover," Ph.D. dissertation, Stanford University, Stanford, CA, 1980.
3. L. Matthies and S. Shafer, "Error modeling in stereo navigation," IEEE J. Robot. Automat., vol. 3, no. 3, pp. 239-248, 1987.
4. S. Lacroix, A. Mallet, R. Chatila, and L. Gallo, "Rover self localization in planetary-like environments," in Proc. Int. Symp. Artificial Intelligence, Robotics, and Automation for Space (i-SAIRAS), 1999, pp. 433-440.
5. C. Olson, L. Matthies, M. Schoppers, and M. W. Maimone, "Robust stereo ego-motion for long distance navigation," in Proc. IEEE Conf. Computer Vision and Pattern Recognition, 2000, pp. 453-458.
6. M. Lhuillier, "Automatic structure and motion using a catadioptric camera," in Proc. IEEE Workshop Omnidirectional Vision, 2005, pp. 1-8.
7. E. Mouragnon, M. Lhuillier, M. Dhome, F. Dekeyser, and P. Sayd, "Real time localization and 3d reconstruction," in Proc. Int. Conf. Computer Vision and Pattern Recognition, 2006, pp. 363-370. (Pubitemid 44931352)
8. J. Tardif, Y. Pavlidis, and K. Daniilidis, "Monocular visual odometry in urban environments using an omnidirectional camera," in Proc. IEEE/RSJ Int. Conf. Intelligent Robots and Systems, 2008, pp. 2531-2538.
9. D. Scaramuzza, F. Fraundorfer, and R. Siegwart, "Real-time monocular visual odometry for on-road vehicles with 1-point RANSAC," in Proc. IEEE Int. Conf. Robotics and Automation (ICRA), 2009, pp. 4293-4299.
10. W. Forstner, "A feature based correspondence algorithm for image matching," Int. Archiv. Photogram., vol. 26, no. 3, pp. 150-166, 1986.
11. C. Harris and J. Pike, "3d positional integration from image sequences," in Proc. Alvey Vision Conf., 1987, pp. 233-236.
12. C. Tomasi and J. Shi, "Good features to track," in Proc. CVPR, 1994, pp. 593-600.
13. E. Rosten and T. Drummond, "Machine learning for high-speed corner detection," in Proc. European Conf. Computer Vision, 2006, vol. 1, pp. 430-443. (Pubitemid 44029797)
14. D. Lowe, "Distinctive image features from scale-invariant key-points," Int. J. Comput. Vis., vol. 20, no. 2, pp. 91-110, 2003.
15. H. Bay, T. Tuytelaars, and L. V. Gool, "Surf: Speeded up robust features," in Proc. ECCV, 2006, pp. 404-417. (Pubitemid 44029795)
16. M. Agrawal, K. Konolige, and M. Blas, "Censure: Center surround extremas for realtime feature detection and matching," in Proc. European Conf. Computer Vision, 2008, pp. 102-115.
17. R. Siegwart, I. Nourbakhsh, and D. Scaramuzza, Introduction to Autonomous Mobile Robots, 2nd ed. Cambridge, MA, MIT Press, 2011.
18. A. Schmidt, M. Kraft, and A. Kasinski, "An evaluation of image feature detectors and descriptors for robot navigation," in Proc. Int. Conf. Computer Vision and Graphics, 2010, pp. 251-259.
19. N. Govender, "Evaluation of feature detection algorithms for structure from motion," Council for Scientific and Industrial Research, Pretoria, Technical Report, 2009.
20. R. C. Gonzalez and R. E. Woods, Digital Image Processing, 3rd ed. Englewood Cliffs, NJ, Prentice Hall, 2007.
21. R. Zabih and J. Woodfill, "Non-parametric local transforms for computing visual correspondence," in Proc. European Conf. Computer Vision, 1994, pp. 151-158.
22. M. Calonder, V. Lepetit, C. Strecha, and P. Fua, "BRIEF: Binary robust independent elementary features," in Proc. European Conf. Computer Vision, 2010, pp. 778-792.
23. E. Rublee, V. Rabaud, K. Konolige, and G. Bradski, "Orb: An efficient alternative to sift or surf (pdf)," in Proc. IEEE Int. Conf. Computer Vision (ICCV), Barcelona, Nov. 2011, pp. 2564-2571.
24. S. Leutenegger, M. Chli, and R. Siegwart, "Brisk: Binary robust invariant scalable keypoints," in Proc. Int. Conf. Computer Vision, 2011, pp. 2548-2555.
25. M. Maimone, Y. Cheng, and L. Matthies, "Two years of visual odom-etry on the mars exploration rovers: Field reports," J. Field Robot., vol. 24, no. 3, pp. 169-186, 2007. (Pubitemid 46647283)
26. A. Davison, "Real-time simultaneous localisation and mapping with a single camera," in Proc. Int. Conf. Computer Vision, 2003, pp. 1403-1410.
27. T. Lemaire and S. Lacroix, "Vision-based SLAM: Stereo and monocular approaches," Int. J. Comput. Vis., vol. 74, no. 3, pp. 343-364, 2006.
28. G. Klein and D. Murray, "Improving the agility of keyframe-based SLAM," in Proc. European Conf. Computer Vision, 2008, pp. 802-815.
29. H. Strasdat, J. Montiel, and A. Davison, "Real time monocular SLAM: Why filter?" in Proc. IEEE Int. Conf. Robotics and Automation, 2010, pp. 2657-2664.
30. B. Horn and B. Schunck, "Determining optical flow," Artif. Intell., vol. 17, no. 1-3, pp. 185-203, 1981.
31. A. Makadia, C. Geyer, and K. Daniilidis, "Correspondence-free structure from motion," Int. J. Comput. Vis., vol. 75, no. 3, pp. 311-327, 2007. (Pubitemid 47383853)
32. P. Torr and D. Murray, "The development and comparison of robust methods for estimating the fundamental matrix," Int. J. Comput. Vis., vol. 24, no. 3, pp. 271-300, 1997. (Pubitemid 127506557)
33. 1 minimization," IEEE Conf. Computer Vision and Pattern Recognition, 2006, pp. 1230-1237. (Pubitemid 44931461)
34. M. A. Fischler and R. C. Bolles, "RANSAC sample consensus: A paradigm for model fitting with applications to image analysis and automated cartography," Commun. ACM, vol. 24, no. 6, pp. 381-395, 1981.
35. R. Hartley and A. Zisserman, Multiple View Geometry in Computer Vision, 2nd ed. Cambridge, MA, Cambridge Univ. Press, 2004.
36. J. Oliensis, "Exact two-image structure from motion," IEEE Trans. Pattern Anal. Machine Intell., vol. 24, no. 12, pp. 1618-1633, 2002.
37. E. Kruppa, "Zur ermittlung eines objektes aus zwei perspektiven mit innerer orientierung," in Proc. Sitz.-Ber. Akad. Wiss., Wien, Math. Naturw. Kl., Abt. IIa., 1913, vol. 122, pp. 1939-1948.
38. O. Faugeras and S. Maybank, "Motion from point matches: Multiplicity of solutions," Int. J. Comput. Vis., vol. 4, no. 3, pp. 225-246, 1990.
39. J. Philip, "A non-iterative algorithm for determining all essential matrices corresponding to five point pairs," Photogram. Rec., vol. 15, no. 88, pp. 589-599, 1996. (Pubitemid 126457333)
40. B. Triggs, "Routines for relative pose of two calibrated cameras from 5 points," INRIA Rhone-Alpes, Tech. Rep., 2000.
41. D. Nister, "An efficient solution to the five-point relative pose problem," Proc. CVPR03, 2003, pp. II: 195-202.
42. H. Stewenius, C. Engels, and D. Nister, "Recent developments on direct relative orientation," ISPRS J. Photogram. Remote Sens., vol. 60, no. 4, pp. 284-294, 2006. (Pubitemid 43815728)
43. O. Pizarro, R. Eustice, and H. Singh, "Relative pose estimation for instrumented, calibrated imaging platforms," in Proc. DICTA, 2003, pp. 601-612.
44. R. Sturm, "Das problem der projektivitaet und seine anwendung auf die flaechen zweiten grades," Math. Annal., vol. 1, no. 4, pp. 533-573, 1869.
45. C. Geyer and H. Stewenius, "A nine-point algorithm for estimating paracatadioptric fundamental matrices," in Proc. IEEE Conf. Computer Vision and Pattern Recognition (CVPR'07), June 2007, pp. 1-8, 17-22.
46. P. Sturm and J. Barreto, "General imaging geometry for central cata-dioptric cameras," in Proc. 10th European Conf. Computer Vision, Marseille, France, 2008, pp. 609-622.
47. P. Sturm, S. Ramalingam, J. Tardif, S. Gasparini, and J. Barreto, "Camera models and fundamental concepts used in geometric computer vision," Foundat. Trends Comput. Graph. Vis., vol. 6, no. 1-2, pp. 1-183, 2010.
48. J. Lim, N. Barnes, and H. Li, "Estimating relative camera motion from the antipodal-epipolar constraint," IEEE Trans. Pattern Anal. Machine Intell., vol. 32, no. 10, pp. 1907-1914, 2010.
49. F. Fraundorfer, P. Tanskanen, and M. Pollefeys, "A minimal case solution to the calibrated relative pose problem for the case of two known orientation angles," in Proc. European Conf. Computer Vision, 2010, pp. 269-282.
50. O. Naroditsky, X. S. Zhou, J. Gallier, S. I. Roumeliotis, and K. Daniilidis, "Two efficient solutions for visual odometry using directional correspondence," IEEE Trans. Pattern Anal. Machine Intell., no. 99, p. 1, 2011.
51. L. Kneip, M. Chli, and R. Siegwart, "Robust real-time visual odome-try with a single camera and an imu," in Proc. British Machine Vision Conf., 2011.
52. D. Ortin and J. M. M. Montiel, "Indoor robot motion based on monocular images," Robotica, vol. 19, no. 3, pp. 331-342, 2001. (Pubitemid 32576642)
53. D. Scaramuzza, "1-point-RANSAC structure from motion for vehicle-mounted cameras by exploiting non-holonomic constraints," Int. J. Comput. Vis., vol. 95, no. 1, pp. 74-85, 2011.
54. D. Scaramuzza, "Performance evaluation of 1-point ransac visual odometry," J. Field Robot., vol. 28, no. 5, pp. 792-811, 2011.
55. P. Torr and A. Zisserman, "Mlesac: A new robust estimator with application to estimating image geometry," Comput. Vis. Image Understand., vol. 78, no. 1, pp. 138-156, 2000. (Pubitemid 30596414)
56. O. Chum and J. Matas, "Matching with prosac-Progressive sample consensus," in Proc. CVPR, 2005, pp. 220-226.
57. D. Nister, "Preemptive ransac for live structure and motion estimation," Machine Vis. Applicat., vol. 16, no. 5, pp. 321-329, 2005. (Pubitemid 41817973)
58. R. Raguram, J. Frahm, and M. Pollefeys, "Exploiting uncertainty in random sample consensus," in Proc. ICCV, 2009, pp. 2074-2081.
59. P. McIlroy, E. Rosten, S. Taylor, and T. Drummond, "Deterministic sample consensus with multiple match hypotheses," in Proc. British Machine Vision Conf., 2010, pp. 1-11.
60. J. Civera, O. Grasa, A. Davison, and J. Montiel, "1-point RANSAC for ekf filtering: Application to real-time structure from motion and visual odometry," J. Field Robot., vol. 27, no. 5, pp. 609-631, 2010.
61. D. Scaramuzza, A. Censi, and K. Daniilidis, "Exploiting motion priors in visual odometry for vehicle-mounted cameras with non-holonomic constraints," in Proc. IEEE/RSJ Int. Conf. Intelligent Robots and Systems, 2011, pp. 4469-4476.
62. E. Rosten, G. Reitmayr, and T. Drummond, "Improved ransac performance using simple, iterative minimal-set solvers," University of Cambridge, Tech. Rep., arXiv:1007.1432v1, 2010.
63. O. Chum, J. Matas, and J. Kittler, "Locally optimized ransac," in Proc. DAGM-Symp., 2003, pp. 236-243.
64. R. C. Smith, P. Cheeseman. (1986). On the representation, and estimation of spatial uncertainty, Int. J. Robot. Res., [Online], vol. 5, no. 4, pp. 56-68. Available: http://ijr.sagepub.com/content/5/4/56.abstract
65. E. Olson, J. Leonard, and S. Teller, "Fast iterative optimization of pose graphs with poor initial estimates," in Proc. ICRA, 2006, pp. 2262-2269. (Pubitemid 44940403)
66. T. Bailey and H. Durrant-Whyte, "Simultaneous localisation and mapping (SLAM): Part II. State of the art," IEEE Robot. Automat. Mag., vol. 13, no. 3, pp. 108-117, 2006. (Pubitemid 46530408)
67. I. Ulrich and I. Nourbakhsh, "Appearance-based place recognition for topological localization," in Proc. IEEE Int. Conf. Robotics and Automation, Apr. 2000, pp. 1023-1029.
68. M. Jogan and A. Leonardis, "Robust localization using panoramic view-based recognition," in Proc. ICPR, 2000, vol. 4, pp. 136-139.
69. K. Mikolajczyk, T. Tuytelaars, C. Schmid, A. Zisserman, J. Matas, F. Schaffalitzky, T. Kadir, and L. Van Gool, "A comparison of affine region detectors," Int. J. Comput. Vis., vol. 65, no. 1-2, pp. 43-72, 2005. (Pubitemid 43281159)
70. P. Newman, D. Cole, and K. Ho, "Outdoor SLAM using visual appearance and laser ranging," in Proc. IEEE Int. Conf. Robotics and Automation, 2006, pp. 1180-1187. (Pubitemid 44940232)
71. M. Cummins and P. Newman, (2008). "FAB-MAP: Probabilistic localization mapping in the space of appearance," Int. J. Robot. Res. [Online], vol. 27, no. 6, pp. 647-665. Available: http://ijr.sagepub.com/cgi/ content/abstract/27/6/647
72. F. Fraundorfer, C. Engels, and D. Nister, "Topological mapping, localization and navigation using image collections," in Proc. IEEE/RSJ Conf. Intelligent Robots and Systems, 2007, pp. 3872-3877.
73. F. Fraundorfer, C. Wu, J.-M. Frahm, and M. Pollefeys, "Visual word based location recognition in 3d models using distance augmented weighting," in Proc. 4th Int. Symp. 3D Data Processing, Visualization, and Transmission, 2008, pp. 1-8.
74. R. Duda, P. Hart, and D. Stork, Pattern Classification, New York, Wiley, 2001.
75. D. Nister and H. Stewenius, "Scalable recognition with a vocabulary tree," in Proc. IEEE Conf. Computer Vision and Pattern Recognition, New York, 2006, pp. 2161-2168. (Pubitemid 44931581)
76. B. Triggs, P. McLauchlan, R. Hartley, and A. Fitzgibbon, "Bundle adjustment a modern synthesis," in Proc. Int. Workshop Vision Algorithms: Theory and Practice (ICCV'99), 2000, pp. 298-372.
77. G. Sibley, L. Matthies, and G. Sukhatme. (2010). Sliding window filter with application to planetary landing. J. Field Robot, [Online], vol. 27, no. 5, pp. 587-608. Available: http://dx.doi.org/10.1002/rob.20360
78. Dacuda AG. (2011). Dacuda scanner mouse. [Online]. Available: http://www.dacuda.com/
79. Y. Cheng, M. W. Maimone, and L. Matthies, "Visual odometry on the mars exploration rovers," IEEE Robot. Automat. Mag., vol. 13, no. 2, pp. 54-62, 2006.
80. J. Kelly and G. S. Sukhatme. (2007. Sept.). An experimental study of aerial stereo visual odometry. In Proc. IFAC-Int. Federation of Automatic Control Symp. Intelligent Autonomous Vehicles, Toulouse, France. [Online]. Available: http://cres.usc.edu/cgi-bin/print.pub.details.pl?pubid=543
81. S. Weiss, D. Scaramuzza, and R. Siegwart, "Monocular-SLAM-based navigation for autonomous micro helicopters in GPS-denied environments," J. Field Robot., vol. 28, no. 6, pp. 854-874, 2011.
82. M. Dunbabin, J. Roberts, K. Usher, G. Winstanley, and P. Corke, "A hybrid AUV design for shallow water reef navigation," in Proc. IEEE Int. Conf. Robotics and Automation, ICRA, Apr. 2005, pp. 2105-2110. (Pubitemid 46084099)
83. B. P. Foley, K. DellaPorta, D. Sakellariou, B. S. Bingham, R. Camilli, R. M. Eustice, D. Evagelistis, V. L. Ferrini, K. Katsaros, D. Kourkoumelis, A. Mallios, P. Micha, D. A. Mindell, C. Roman, H. Singh, D. S. Switzer, and T. Theodoulou, "The 2005 chios ancient shipwreck survey: New methods for underwater archaeology," Hesperia, vol. 78, no. 2, pp. 269-305, 2009.
84. A. G. Daimler. (2011). 6d vision. [Online]. Available: http://www.6d-vision.com/
85. M. Bertozzi, A. Broggi, E. Cardarelli, R. Fedriga, L. Mazzei, and P. Porta, "Viac expedition toward autonomous mobility [from the field]," IEEE Robot. Automat. Mag., vol. 18, no. 3, pp. 120-124, Sept. 2011.
86. E. Guizzo. (2011). How Google's self-driving car works. [Online]. Available: http://spectrum.ieee.org/automaton/robotics/artificial-intelli-gence/ howfl-google-self-driving-car-works
87. D. Scaramuzza and F. Fraundorfer, "Visual odometry," IEEE Robotics Automat. Mag., vol. 18, no. 4, pp. 80-92, Dec. 2011.
88. C. F. Olson, L. H. Matthies, M. Schoppers, and M. W. Maimone, "Stereo ego-motion improvements for robust rover navigation," in Proc. IEEE Int. Conf. Robotics and Automation (ICRA 2001), 2001, vol. 2, pp. 1099-1104. (Pubitemid 32815006)
89. M. V. Srinivasan, S. W. Zhang, M. Lehrer, and T. S. Collett, "Honeybee navigation en route to the goal: Visual flight control and odometry," J. Exp. Biol., vol. 199, 1996, pp. 237-244. (Pubitemid 126603957)
90. P. Viola and M. Jones, "Robust real time object detection," Int. J. Comput. Vis., vol. 57, no. 2, pp. 137-154, 2001.
91. D. Scharstein and R. Szeliski, "A taxonomy and evaluation of dense two-frame stereo correspondence algorithms," Int. J. Comput. Vis., vol. 47, no. 1-3, pp. 7-42, Apr.-June 2002. (Pubitemid 34428419)