Visual-inertial navigation, mapping and localization: A scalable real-time causal approach

International Journal of Robotics Research

Volumn 30, Issue 4, 2011, Pages 407-430

  Jones, Eagle S ^a   Soatto, Stefano ^b  

^a MCKINSEY AND COMPANY   (United States)

^b UNIVERSITY OF CALIFORNIA   (United States)

Author keywords

assisted driving; autonomous robotics; localization; location recognition; loop closure; simultaneous localization and mapping (SLAM); structure from motion; vision aided navigation; vision based navigation; Visual inertial navigation

Indexed keywords

ASSISTED DRIVING; AUTONOMOUS ROBOTICS; LOCALIZATION; LOCATION RECOGNITION; LOOP CLOSURE; SIMULTANEOUS LOCALIZATION AND MAPPING; STRUCTURE FROM MOTION; VISION-BASED NAVIGATION; VISUAL-INERTIAL NAVIGATION;

CALIBRATION; CAMERAS; EMBEDDED SOFTWARE; INERTIAL NAVIGATION SYSTEMS; INVERSE KINEMATICS; MAPPING; PARAMETER ESTIMATION; ROBOTICS; TOPOLOGY;

NAVIGATION;

EID: 79953323515     PISSN: 02783649     EISSN: 17413176     Source Type: Journal    
DOI: 10.1177/0278364910388963     Document Type: Article

References (70)

1. Alspach DL and Sorenson HW (1972) Nonlinear Bayesian estimation using gaussian sum approximation. IEEE Transactions on Automatic Control 17: 439-448.
2. Azarbayejani A. and Pentland AP (1995) Recursive estimation of motion, structure, and focal length. IEEE Transactions on Pattern Analysis and Machine Intelligence 17: 562-575.
3. Baldwin G., Mahony R., Trumpf J., Hamel T. and Cheviron T. (2007) Complementary filter design on the special Euclidean group SE(3). Proceedings of the 2007 European Control Conference.
4. Bosse M., Newman P., Leonard J. and Teller S. (2004) Simultaneous localization and map building in large-scale cyclic environments using the Atlas framework. The International Journal of Robotics Research 23: 1113.
5. Brigo D., Hanzon B. and LeGland F. (1998) A differential geometric approach to nonlinear filtering: the projection filter. IEEE Transactions on Automatic Control 68: 181-188.
6. Chiuso A., Favaro P., Jin H. and Soatto S. (2002) Motion and structure causally integrated over time. EEE Transactions on Pattern Analysis and Machine Intelligence 24: 523-535. (Pubitemid 34448770)
7. Chum O., Perdoch M. and Matas J. (2009) Geometric min-hashing: finding a (thick) needle in a haystack. Proceedings of CVPR 2009.
8. Cummins M. and Newman P. (2008) Accelerated appearance-only SLAM. In Proceedings IEEE International Conference on Robotics and Automation (ICRA'08), Pasadena, CA, April 2008.
9. Dalal N. and Triggs B. (2005) Histograms of oriented gradients for human detection. In Proceedings IEEE Conference on Computer Vision and Pattern Recognition.
10. Davison A. (2009) Real-time simulataneous localisation and mapping with single camera. In Proceedings of the 9th International Conference on Computer Vision.
11. Dickmanns ED and Mysliwetz BD (1992) Recursive 3-D road and relative ego-state estimation. IEEE Transactions on Pattern Analysis and Machine Intelligence 14: 199-213.
12. Eade E. and Drummond T. (2007) Monocular SLAM as a graph of coalesced observations. In IEEE 11th International Conference on Computer Vision, 2007. ICCV 2007, Rio de Janeiro, Brazil, October 2007, pp. 1-8.
13. Euston M., Coote P., Mahony R., Kim J. and Hamel T. (2008) A complementary filter for attitude estimation of a fixed-wing uav. In IEEE/RSJ International Conference on Intelligent Robots and Systems, 2008. IROS 2008, pp. 340-345.
14. Favaro P. (1998) Stima del moto e della struttura della scena tramite visione dinamica. Laurea thesis, University of Padova, 1998.
15. Favaro P., Jin H. and Soatto S. (2001) A semidirect approach to structure from motion. In IEEE International Conference on Image Analysis and Processing, pp. 250-255.
16. Favaro P., Jin H. and Soatto S. (2003) A semi-direct approach to structure from motion. The Visual Computer 19: 1-18.
17. Goedeme T., Tuytelaars T. and Van Gool L. (2004) Fast wide baseline matching for visual navigation. In Proceedings of the 2004 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, 2004. CVPR 2004, volume 1, June/July 2004.
18. Goedeme T., Nuttin M., Tuytelaars T. and VanGool L. (2007) Omnidirectional vision based topological navigation. International Journal of Computer Vision 74: 219-236. (Pubitemid 47142800)
19. Guivant JE and Nebot EM (2001) Optimization of the simultaneous localization and map-building algorithm for real-time implementation. IEEE Transactions on Robotics and Automation 17: 242-257. (Pubitemid 32829732)
20. Ho KL and Newman P. (2007) Detecting loop closure with scene sequences. International Journal of Computer Vision 74: 261-286. (Pubitemid 47142802)
21. Isidori A. (1989) Nonlinear Control Systems. Berlin: Springer Verlag.
22. Jazwinski AH (1970) Stochastic Processes and Filtering Theory. New York: Academic Press.
23. Jin H., Favaro P. and Soatto S. (2000) Real-time 3-D motion and structure from point features: a front-end system for vision-based control and interaction. In Computer Vision and Pattern Recognition, pp. 778-779.
24. Jones E. and Soatto S. (2009) Visual-Inertial Navigation, Localization and Mapping: A Scalable Real-time Large-scale Approach. Technical Report UCLA-CSD-100010, 27 August 2009 (revised 10 May 2010 ).
25. Jones ES, Vedaldi A. and Soatto S. (2007) Inertial structure from motion and autocalibration. In Workshop on Dynamical Vision, October 2007.
26. Julier SJ and Uhlmann JK (1997) A new extension of the kalman filter to nonlinear systems. In International Symposium on Aerospace/Defense Sensing, Simulation and Control.
27. Kailath T. (1980) Linear Systems. Englewood Cliffs, NJ: Prentice Hall.
28. Kayton M. and Fried WR (1996) Avionics Navigation Systems. New York: John Wiley & Sons, Inc.
29. Kelly J. and Sukhatme G. (2009) Fast relative pose calibration for visual and inertial sensors. In Experimental Robotics, pp. 515-524.
30. Klein G. and Murray D. (2007) Parallel tracking and mapping for small AR workspaces. In Proceedings of the 2007 6th IEEE and ACM International Symposium on Mixed and Augmented Reality. Los Alamitos, CA: IEEE Computer Society Press, pp. 1-10.
31. Konolige K. and Agrawal M. (2008) Frameslam: from bundle adjustment to real-time visual mapping. IEEE Transactions on Robotics 24: 1066-1077.
32. Konolige K., Agrawal M. and Sola J. (2007) Large scale visual odometry for rough terrain. In Proceedings of the International Symposium on Robotics Research.
33. Kosecká J., Li F. and Yang X. (2005) Global localization and relative positioning based on scale-invariant keypoints. Robotics and Autonomous Systems 52: 27-38. (Pubitemid 40851848)
34. Lazebnik S., Schmid C. and Ponce J. (2006) Beyond bag of features: Spatial pyramid matching for recognizing natural scene categories. In Proceedings IEEE Conference on Computer Vision and Pattern Recognition.
35. Lee T. and Soatto S. (2010) An End-to-End Visual Recognition System. Technical Report UCLA-CSD-100008, 2010.
36. Lowe DG Distinctive image features from scale-invariant keypoints. International Journal of Computer Vision 2 (60). 91-110.
37. Lucas BD and Kanade T. (1981) An iterative image registration technique with an application to stereo vision. In Image Understanding Workshop, pp. 121-130.
38. Ma Y., Soatto S., Kosecka J. and Sastry S. (2003) An Invitation to 3D Vision, from Images to Geometric Models. Berlin: Springer Verlag.
39. McLauchlan PF (1999) Gauge invariance in projective 3d reconstruction. In IEEE Workshop on Multi-View Modeling and Analysis of Visual Scenes, Fort Collins, CO, June 1999.
40. Mei C., Sibley G., Cummins M., Newman P. and Reid I. (2009) A constant time efficient stereo slam system. In BMVC, 2009.
41. Mirzaei FM and Roumeliotis SI (2007) A Kalman filter-based algorithm for IMU-camera calibration. In Proceedings of IEEE/RSJ International Conference on Intelligent Robots and Systems.
42. Morikis AI and Roumeliotis SI (2006) A Multi-state Constraint Kalman Filter for Vision-aided Inertial Navigation. Technical report, Department of Computer Science and Engineering, University of Minnesota, 2006, http://www.cs.umn.edu/amourikis/tech-reports/TR-MSCKF.pdf.
43. Mouragnon E., Dekeyser F., Sayd P., Lhuillier M. and Dhome M. (2006) Real time localization and 3d reconstruction. In 2006 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, Volume 1, 2006.
44. Mourikis AI and Roumeliotis SI (2007) A multi-state constraint Kalman filter for vision-aided inertial navigation. In Proceedings of the IEEE International Conference on Robotics and Automation, Rome, Italy, 10-14 April 2007, pp. 3565-3572.
45. Mourikis AI and Roumeliotis SI (2008) A dual-layer estimator architecture for long-term localization. In Proceedings of the Workshop on Visual Localization for Mobile Platforms, Anchorage, AK, June 2008, pp. 1-8.
46. Murray RM, Li Z. and Sastry SS (1994) A Mathematical Introduction to Robotic Manipulation. Boca Raton, FL: CRC Press.
47. Nebot E. and Durrant-Whyte H. (1999) Initial calibration and alignment of low-cost inertial navigation units for land vehicle applications. Journal of Robotic Systems 16: 81-92.
48. Nister D. (2003) Preemptive ransac for live structure and motion estimation. In Proceedings of the 9th International Conference on Computer Vision.
49. Nister D., Naroditsky O. and Bergen J. (2006) Visual odometry for ground vehicle applications. Journal of Fields Robotics 23: 3-20.
50. Nister D. and Stewenius H. (2006) Scalable recognition with a vocabulary tree. In Proceedings of IEEE International Conference on Computer Vision and Pattern Recognition (CVPR).
51. Qian G., Chellappa R. and Zheng Q. (2001) Robust structure from motion estimation using inertial data. Journal of the Optical Society of America A 18: 2982-2997.
52. Roumeliotis SI, Johnson AE and Montgomery JF (2002) Augmenting inertial navigation with image-based motion estimation. In IEEE International Conference on Robotics and Automation.
53. Salton G. and Buckley C. (1988) Term-weighting approaches in automatic text retrieval. Information Processing and Management.
54. Schindler G., Brown M. and Szeliski R. (2007) City-scale location recognition. In IEEE Conference on Computer Vision and Pattern Recognition, 2007. CVPR '07, June 2007, pp. 1-7.
55. Sivic J. and Zisserman A. (2003) Video Google: A text retrieval approach to object matching in videos. In Proceedings of the 9th International Conference on Computer Vision.
56. Soatto S. (1997) 3-D structure from visual motion: modeling, representation and observability. Automatica 33: 1287-1312. (Pubitemid 127433510)
57. Soatto S. (2009) Actionable information in vision. In Proceedings of the International Conference on Computer Vision, October 2009.
58. Soatto S. (2010) Steps Towards a Theory of Visual Information. NIPS Tutorial Lecture Notes, (also course lectures for UCLA CS269, W10).
59. Sundaramoorthi G., Petersen P., Varadarajan VS and Soatto S. (2009) On the set of images modulo viewpoint and contrast changes. In Proceedings IEEE Conference on Computer Vision and Pattern Recognition, June 2009.
60. Tsai R. (1987) A versatile camera calibration technique for high accuracy 3D machine vision metrology using off-the-shelf tv cameras and lenses. IEEE Journal on Robotics and Automation, 3: 323-344. (Pubitemid 18521694)
61. Vedaldi A. and Soatto S. (2005) Features for recognition: viewpoint invariance for non-planar scenes. In Proceedings of the International Conference of Computer Vision, October 2005, pp. 1474-1481.
62. Vedaldi A., Jin H., Favaro P. and Soatto S. (2005) Kalmansac: Causal inference of dynamical processes in the presence of outliers. In Proceedings of the International Conference on Computer Vision, October 2005.
63. Veth M. and Raquet J. (2006) Fusion of low-cost imaging and inertial sensors for navigation. In Proceedings of the ION meeting on Global Navigation Satellite Systems.
64. Veth M., Raquet J. and Pachter M. (2006) Stochastic constraints for efficient image correspondence search. IEEE Transactions on Aerospace and Electronic Systems 42: 973-982. (Pubitemid 46405530)
65. Wang J., Cipolla R. and Zha H. (2005) Vision-based global localization using a visual vocabulary. In Proceedings of the 2005 IEEE International Conference on Robotics and Automation, 2005. ICRA 2005, April 2005, pp. 4230-4235.
66. Williams B., Klein G. and Reid I. (2007) Real-time SLAM relocalisation. In IEEE 11th International Conference on Computer Vision, 2007. ICCV 2007, Rio de Janeiro, Brazil, October 2007, pp. 1-8.
67. Yang R. and Pollefeys M. (2003) Multi-resolution real-time stereo on commodity graphics hardware. In Proceedings IEEE Conference on Computer Vision and Pattern Recognition (CVPR'03), pp. 211-218.
68. Zhang J., Marszalek M., Lazebnik S. and Schmid C. (2006) Local features and kernels for classification of texture and object categories: A comprehensive study. International Journal of Computer Vision 73: 213-238.
69. Zhang W and Kosecka J. Image based localization in urban environments. Proceedings of 3DPVT, pp. 33-40.
70. Zhu Z., Oskiper T., Samarasekera S., Kumar R. and Sawhney H. (2007) Ten-fold improvement in visual odometry using landmark matching. In IEEE 11th International Conference on Computer Vision, 2007. ICCV 2007, Rio de Janeiro, Brazil, October 2007, pp. 1-8.