Keyframe-based visual-inertial odometry using nonlinear optimization

International Journal of Robotics Research

Volumn 34, Issue 3, 2015, Pages 314-334

  Leutenegger, Stefan ^a,b   Lynen, Simon ^b   Bosse, Michael ^b   Siegwart, Roland ^b   Furgale, Paul ^b  

^a IMPERIAL COLLEGE LONDON   (United Kingdom)

^b ETH ZURICH   (Switzerland)

Author keywords

bundle adjustment; inertial measurement unit (IMU); keyframes; robotics; sensor fusion; simultaneous localization and mapping (SLAM); stereo camera; Visual inertial odometry

Indexed keywords

AGRICULTURAL ROBOTS; CLONE CELLS; COST FUNCTIONS; MAPPING; NONLINEAR PROGRAMMING; ROBOTICS; STEREO IMAGE PROCESSING; STOCHASTIC SYSTEMS;

BUNDLE ADJUSTMENTS; INERTIAL MEASUREMENT UNIT; KEY-FRAMES; ODOMETRY; SENSOR FUSION; SIMULTANEOUS LOCALIZATION AND MAPPING; STEREO CAMERAS;

SLAM ROBOTICS;

EID: 84924677423     PISSN: 02783649     EISSN: 17413176     Source Type: Journal    
DOI: 10.1177/0278364914554813     Document Type: Article

References (59)

1. AgarwalSMierleK. (2010) Ceres Solver. Available at: http://ceres-solver.org (accessed 27 November 2014).
2. Barfoot T, Forbes JR, Furgale PT. Pose estimation using linearized rotations and quaternion algebra. Acta Astronautica. 2011 ; 68 (1 2). 101-112
3. Bayard DS, Brugarolas PB. An estimation algorithm for vision-based exploration of small bodies in space. Proceedings of the 2005 American control conference ; 2005 ; 2005. 4589.
4. Bryson M, Johnson-Roberson M, Sukkarieh S. Airborne smoothing and mapping using vision and inertial sensors. IEEE international conference on robotics and automation, 2009 (ICRA'09) ; 2009 ; 2009. 2037.
5. Chai L, Hoff WA, Vincent T. Three-dimensional motion and structure estimation using inertial sensors and computer vision for augmented reality. Presence: Teleoperators and Virtual Environments. 2002 ; 11 (5). 474-492
6. Davison A. Real-time simultaneous localisation and mapping with a single camera. Proceedings of the ninth IEEE international conference on computer vision ; 2003 2003 ; 2003 2003. 1403.
7. Dong-Si TC, Mourikis AI. Motion tracking with fixed-lag smoothing: Algorithm and consistency analysis. IEEE international conference on robotics and automation (ICRA) ; 2011 ; 2011. 5655.
8. Dong-Si TC, Mourikis AI. Estimator initialization in vision-aided inertial navigation with unknown camera-imu calibration. IEEE/RSJ international conference on intelligent robots and systems (IROS) ; 2012 ; 2012. 1064.
9. EbcinSVethM (2007) Tightly-coupled image-aided inertial navigation using the unscented kalman filter. Technical report, DTIC Document.
10. FurgaleP (2011) Extensions to the Visual Odometry Pipeline for the Exploration of Planetary Surfaces. PhD Thesis, University of Toronto, Institute for Aerospace Studies.
11. Furgale P, Rehder J, Siegwart R. Unified temporal and spatial calibration for multi-sensor systems. IEEE/RSJ international conference on robots and systems (IROS) ; 2013 ; 2013. 1280.
12. Geiger A, Lenz P, Urtasun R. Are we ready for autonomous driving? The KITTI vision benchmark suite. Proceedings of the IEEE conference on computer vision and pattern recognition (CVPR) ; 2012 ; 2012..
13. Harris C, Stephens M. A combined corner and edge detector. Proceedings of the 4th Alvey vision conference ; 1988 ; 1988. 147.
14. Hesch JA, Kottas DG, Bowman SL, Roumeliotis SI Observability-constrained vision-aided inertial navigation. University of Minnesota, Department of Computer Science and Engineering ; 2012a :
15. Hesch JA, Kottas DG, Bowman SL, Roumeliotis SI. Towards consistent vision-aided inertial navigation. Proceedings of the international workshop on the algorithmic foundations of robotics (WAFR) ; 2012b ; 2012b..
16. Hesch JA, Kottas DG, Bowman SL, Roumeliotis SI. Towards consistent vision-aided inertial navigation. Algorithmic Foundations of Robotics X. 2013 ;: 559-574
17. Huang G, Kaess M, Leonard JJ Cambridge, MA: Computer Science and Artificial Intelligence Laboratory, Massachusetts Institute of Technology ; 2013 :
18. Huang GP, Mourikis AI, Roumeliotis SI. A first-estimates Jacobian EKF for improving SLAM consistency. Experimental Robotics ; 2009 ; 2009. 373.
19. Huang GP, Mourikis AI, Roumeliotis SI. An observability-constrained sliding window filter for SLAM. 2011 IEEE/RSJ international conference on intelligent robots and systems (IROS) ; 2011 ; 2011. 65.
20. Indelman V, Williams S, Kaess M, Dellaert F. Factor graph based incremental smoothing in inertial navigation systems. International conference on information fusion (FUSION) ; 2012 ; 2012..
21. Jia C, Evans BL. Probabilistic 3-D motion estimation for rolling shutter video rectification from visual and inertial measurements. MMSP. 2012 ;: 203-208
22. Jones ES, Soatto S. Visual-inertial navigation, mapping and localization: A scalable real-time causal approach. The International Journal of Robotics Research. 2011 ; 30 (4). 407-430
23. Jung SH, Taylor CJ. Camera trajectory estimation using inertial sensor measurements and structure from motion results. Proceedings of the 2001 IEEE computer society conference on computer vision and pattern recognition (CVPR 2001) ; 2001 ; 2001. II.
24. Kaess M, Johannsson H, Roberts R, Ila V, Leonard JJ, Dellaert F. ISAM2: Incremental smoothing and mapping using the Bayes tree. The International Journal of Robotics Research. 2012 ; 31 (2). 216-235
25. Kelly J, Sukhatme GS. Visual-inertial sensor fusion: localization, mapping and sensor-to-sensor self-calibration. The International Journal of Robotics Research. 2011 ; 30 (1). 56-79
26. Kim J, Sukkarieh S. Real-time implementation of airborne inertial-SLAM. Robotics and Autonomous Systems. 2007 ; 55 (1). 62-71
27. Klein G, Murray D. Parallel tracking and mapping for small AR workspaces. 6th IEEE and ACM international symposium on mixed and augmented reality, 2007 (ISMAR 2007) ; 2007 ; 2007. 225.
28. Kneip L, Furgale PT. OpenGV: A unified and generalized approach to real-time calibrated geometric vision. Proceedings of the IEEE international conference on robotics and automation (ICRA) ; 2014 ; 2014..
29. Konolige K, Agrawal M, Sola J Robotics Research. Springer: ; 2011: 201-212.
30. Kottas DG, Hesch JA, Bowman SL, Roumeliotis SI. On the consistency of vision-aided inertial navigation. Proceedings of the international symposium on experimental robotics (ISER) ; 2012 ; 2012..
31. Leutenegger S, Chli M, Siegwart R. BRISK: Binary robust invariant scalable keypoints. Proceedings of the IEEE international conference on computer vision (ICCV) ; 2011 ; 2011..
32. Leutenegger S, Furgale P, Rabaud V, Chli M, Konolige K, Siegwart R. Keyframe-based visual-inertial SLAM using nonlinear optimization. Proceedings of robotics: science and systems (RSS) ; 2013 ; 2013..
33. Li M, Kim BH, Mourikis AI. Real-time motion tracking on a cellphone using inertial sensing and a rolling-shutter camera. 2013 IEEE international conference on robotics and automation (ICRA) ; 2013 ; 2013. 4712.
34. Li M, Mourikis AI. Improving the accuracy of EKF-based visual-inertial odometry. 2012 IEEE international conference on robotics and automation (ICRA) ; 2012a ; 2012a. 828.
35. Li M, Mourikis AI. Improving the accuracy of EKF-based visual-inertial odometry. Proceedings of the IEEE international conference on robotics and automation (ICRA) ; 2012b ; 2012b..
36. Li M, Mourikis AI. Vision-aided inertial navigation for resource-constrained systems. 2012 IEEE/RSJ international conference on intelligent robots and systems (IROS) ; 2012c ; 2012c. 1057.
37. Li M, Mourikis AI. Optimization-based estimator design for vision-aided inertial navigation. Robotics: Science and Systems. 2013 ;: 241-248
38. Lobo J, Dias J. Relative pose calibration between visual and inertial sensors. The International Journal of Robotics Research. 2007 ; 26 (6). 561-575
39. Lynen S, Achtelik MW, Weiss S, Chli M, Siegwart R. A robust and modular multi-sensor fusion approach applied to MAV navigation. 2013 IEEE/RSJ international conference on intelligent robots and systems (IROS) ; 2013 ; 2013. 3923.
40. Martinelli A. State estimation based on the concept of continuous symmetry and observability analysis: the case of calibration. IEEE Transactions on Robotics. 2011 ; 27 (2). 239-255
41. Martinelli A. Visual-inertial structure from motion: observability vs minimum number of sensors. Proceedings of the IEEE international conference on robotics and automation ; 2014 ; 2014..
42. Mei C, Sibley G, Cummins M, Newman PM, Reid ID. RSLAM: a system for large-scale mapping in constant-time using stereo. International Journal of Computer Vision. 2011 ; 94 (2). 198-214
43. Mirzaei FM, Roumeliotis SI IMU-camera Calibration: Bundle Adjustment Implementation. Department of Computer Science and Engineering, University of Minnesota ; 2007 :
44. Mirzaei FM, Roumeliotis SI. A Kalman filter-based algorithm for IMU-camera calibration: observability analysis and performance evaluation. IEEE Transactions on Robotics. 2008 ; 24 (5). 1143-1156
45. Mourikis AI, Roumeliotis SI. A multi-state constraint Kalman filter for vision-aided inertial navigation. Proceedings of the IEEE international conference on robotics and automation (ICRA) ; 2007 ; 2007..
46. Mourikis AI, Trawny N, Roumeliotis SI, Johnson AE, Ansar A, Matthies L. Vision-aided inertial navigation for spacecraft entry, descent, and landing. IEEE Transactions on Robotics. 2009 ; 25 (2). 264-280
47. Nerurkar ED, Wu KJ, Roumeliotis SI. C-KLAM: constrained keyframe-based localization and mapping. Proceedings of the workshop on "multi-view geometry in robotics" at the robotics: science and systems (RSS) ; 2013 ; 2013..
48. Nikolic J, Rehder J, Michael Burri PG, Leutenegger S, Furgale PT, Siegwart R. A synchronized visual-inertial sensor system with FPGA pre-processing for accurate real-time SLAM. Proceedings of the IEEE international conference on robotics and automation (ICRA) ; 2014 ; 2014..
49. Ranganathan A, Kaess M, Dellaert F. Fast 3D pose estimation with out-of-sequence measurements. Proceedings of the IEEE/RSJ international conference on intelligent robots and systems (IROS) ; 2007 ; 2007..
50. Roumeliotis SI, Johnson AE, Montgomery JF. Augmenting inertial navigation with image-based motion estimation. Proceedings IEEE international conference on robotics and automation (ICRA'02) ; 2002 ; 2002. 4326.
51. Savage PG. Strapdown inertial navigation integration algorithm design part 2: Velocity and position algorithms. Journal of Guidance, Control, and Dynamics. 1998 ; 21 (2). 208-221
52. Shin EH, El-Sheimy N. An unscented Kalman filter for in-motion alignment of low-cost IMUs. Position Location and Navigation Symposium, 2004 (PLANS 2004) ; 2004 ; 2004. 273.
53. Sibley G, Matthies L, Sukhatme G. Sliding window filter with application to planetary landing. Journal of Field Robotics. 2010 ; 27 (5). 587-608
54. Strasdat H, Montiel JMM, Davison AJ. Real-time monocular SLAM: why filter?. Proceedings of the IEEE international conference on robotics and automation (ICRA) ; 2010 ; 2010..
55. StrelowDSinghS (2003) Online motion estimation from image and inertial measurements. Available at: https://www.frc.ri.cmu.edu/projects/buzzard/publications/inervis03.pdf
56. Strelow D, Singh S. Motion estimation from image and inertial measurements. The International Journal of Robotics Research. 2004 ; 23 (12). 1157-1195
57. Thrun S, Montemerlo M. The GraphSLAM algorithm with applications to large-scale mapping of urban structures. The International Journal of Robotics Research. 2006 ; 25 (5). 403-430
58. Weiss S, Achtelik M, Lynen S, Chli M, Siegwart R. Real-time onboard visual-inertial state estimation and self-calibration of MAVs in unknown environments. Proceedings of the IEEE international conference on robotics and automation (ICRA) ; 2012 ; 2012..
59. WeissSM (2012) Vision based navigation for micro helicopters. PhD Thesis, Eidgenössische Technische Hochschule, ETH Zürich, 2012.