Gyro-aided feature tracking for a moving camera: Fusion, auto-calibration and GPU implementation

International Journal of Robotics Research

Volumn 30, Issue 14, 2011, Pages 1755-1774

  Hwangbo, Myung ^a   Kim, Jun Sik ^a   Kanade, Takeo ^a  

^a CARNEGIE MELLON UNIVERSITY   (United States)

Author keywords

Auto calibration; GPU parallel programming; KLT feature tracking; visual inertial sensor fusion

Indexed keywords

AUTO CALIBRATION; AUTO-CALIBRATION METHOD; COMPUTATIONAL BURDEN; CONVERGENCE REGION; FEATURE TRACKER; FEATURE-TRACKING; GPU IMPLEMENTATION; HIGH-ORDER; ILLUMINATION CHANGES; IMAGE WARPING; IN-SITU; INITIAL CONDITIONS; INTER-FRAME; INTRINSIC AND EXTRINSIC PARAMETERS; KLT FEATURE TRACKING; MICRO ELECTRO MECHANICAL SYSTEM; MOVING CAMERAS; NATURAL LANDMARK; ON-THE-FLY; PARALLELIZATIONS; REAL SCENES; SENSOR MEASUREMENTS; SIMULATION EVALUATION; TRACKING PERFORMANCE; VISUAL/INERTIAL SENSOR FUSION; WARPING MODEL;

CALIBRATION; GYROSCOPES; MEMS; MICROELECTROMECHANICAL DEVICES; PARALLEL PROGRAMMING; PRINCIPAL COMPONENT ANALYSIS; ROTATION; SENSORS;

CAMERAS;

EID: 83755172644     PISSN: 02783649     EISSN: 17413176     Source Type: Journal    
DOI: 10.1177/0278364911416391     Document Type: Article

References (35)

1. ADXRS-623 ±150°/sec MEMS Yaw Rate Gyroscope (Data Sheet). 2010 ;:
2. Arun KS, Huang TS, Bolstein SD. Least-squares fitting of two 3-D point sets. IEEE Trans Pattern Anal Machine Intell. 1987 ; 9: 698-700
3. Baker S, Matthews I. Lucas-Kanade 20 years on: A unifying framework. Int J Comput Vision. 2004 ; 56: 221-255
4. Bleser G, Wohlleber C, Becker M, Stricker D. Fast and stable tracking for AR fusing video and inertial sensor data. International Conference in Central Europe on Computer Graphics, Visualization and Computer Vision (WSCG) ; 2006 Plzen, Czech Republic ; 2006. 109.
5. Bouguet J. Pyramidal Implementation of the Lucas Kanade Feature Tracker: Description of the Algorithm.. 2000 ;:
6. Chandaria J, Thomas G, Bartczak B, Koeser K, Koch R, Becker M, et al. Realtime camera tracking in the MATRIS project. SMPTE Motion Imaging J. 2007 ; 116: 266-271
7. Corke P, Lobo J, Dias J. An introduction to inertial and visual sensing. Int J Robotics Res. 2007 ; 26: 519-535 (Pubitemid 47011236)
8. Davison A, Reid I, Molton N, Stasse O. MonoSLAM: Real-time single camera SLAM. IEEE Trans Pattern Recognition Machine Intell. 2007 ; 29: 1052-1067 (Pubitemid 46667415)
9. El-Sheimy N, Hou H, Niu X. Analysis and modeling of inertial sensors using Allan variance. IEEE Trans Instrumentation Meas. 2008 ; 57: 140-149
10. Golub GH, Loan CFV Matrix Computations. 1996 ) Matrix Computations, 3rd edn.Baltimore, MD: The Johns Hopkins University Press. Baltimore, MD: The Johns Hopkins University Press ; 1996:
11. Gray J, Veth M. Deeply-integrated feature tracking for embedded navigation. ION International Technical Meeting Program ; 2009 Anaheim, CA ; 2009..
12. Hartley R. Self-calibration of stationary cameras. Int J Comput Vision. 1997 ; 22: 5-23 (Pubitemid 127507388)
13. Hartley RI, Zisserman A Multiple View Geometry in Computer Vision. 2nd edn. Cambridge: Cambridge University Press ; 2004:
14. Hedborg J, Skoglund J, Felsberg M. KLT tracking implementation on the GPU. Proceedings SSBA 2007 ; 2007 Linkoping, Sweden ; 2007..
15. Hol JD, Schön TB, Gustafsson F. Modeling and calibration of inertial and vision sensors. Int J Robotics Res. 2010 ; 29: 231-244
16. Hol JD, Schön TB, Luinge H, Slycke PJ, Gustafsson F. Robust real-time tracking by fusing measurements from inertial and vision sensors. J Real-Time Image Process. 2007 ; 2: 149-160 (Pubitemid 350027503)
17. Hwangbo M, Kanade T. Factorization-based calibration method for mems inertial measurement unit. Proceedings IEEE International Conference on Robotics and Automation ; 2008 San Hose, CA ; 2008..
18. Hwangbo M, Kim J-S, Kanade T. Inertial-aided klt feature tracking for a moving camera. IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS'09) ; 2009 ; 2009. 1909.
19. Jin H, Favaro P, Soatto S. Real-time feature tracking and outlier rejection with changes in illumination. International Conference on Computer Vision ; 2001 ; 2001. 684.
20. Kelly J, Sukhatme GS. Fast relative pose calibration for visual and inertial sensors. 11th International Symposium Experimental Robotics (ISER'08). 2008 ;:
21. Kim J-S, Hwangbo M, Kanade T. Realtime affine-photometric KLT feature tracker on GPU in CUDA framework. The Fifth IEEE Workshop on Embedded Computer Vision in ICCV 2009. 2009 ;: 1306-1311
22. Lang P, Pinz A. Calibration of hybrid vision / inertial tracking system. Proceedings 2nd Workshop on Integration of Vision and Inertial Sensors (INERVIS'05). 2005 ;:
23. Lobo J, Dias J. Vision and inertial sensor cooperation using gravity as a vertical reference. IEEE Trans Pattern Recognition Machine Intell. 2003 ; 25: 1597-1608
24. Lobo J, Dias J. Relative pose calibration between visual and inertial sensors. Int J Robotics Res. 2007 ; 26: 561-575 (Pubitemid 47011239)
25. Lucas BD, Kanade T. An iterative image registration technique with an application to stereo vision. Proceedings of the 7th International Joint Conference on Artificial Intelligence ; 1981 Vancouver, Canada ; 1981. 674.
26. Makadia A, Daniilidis K. Correspondenceless ego-motion estimation using an IMU. Proceedings IEEE International Conference on Robotics and Automation ; 2005 ; 2005. 3534.
27. Mirzaei FM, Roumeliotis SI. A Kalman filter-based algorithm for IMU-camera calibration: Observability analysis and performance evaluation. IEEE Trans Robotics. 2008 ; 24: 1143-1156
28. Ohmer J, Redding N. GPU-accelerated KLT tracking with monte-carlo-based feature reselection. Computing: Techniques and Applications, 2008. DICTA '08. Digital Image. 2008 ;: 234-241
29. SeppWFuchsS (2010) DLR CalLab and CalDe-The DLR Camera Calibration Toolbox. Available at: http://www.robotic.dlr.de/callab/.
30. Shi J, Tomasi C. Good features to track. IEEE Conference on Computer Vision and Pattern Recognition (CVPR'94) ; 1994 Seattle, WA ; 1994..
31. Sinha SN, Frahm J-M, Pollefeys M, Genc Y. Feature tracking and matching in video using programmable graphics hardware. Machine Vision and Applications. 2007 ;:
32. Yokokohji Y, Sugawara Y, Yoshikawa T. Accurate image overlay on video see-through HMDs using vision and accelerometers. Proceedings IEEE Virtual Reality. 2000 ;:
33. You S, Neumann U, Azuma R. Hybrid inertial and vision tracking for augmented reality registration. Proceedings IEEE Virtual Reality. 1999 ;:
34. Zach C, Gallup D, Frahm J-M. Fast gain-adaptive KLT tracking on the gpu. IEEE Computer Society Conference on Computer Vision and Pattern Recognition Workshops, 2008. CVPR Workshops 2008 ; 2008 ; 2008. 1.
35. Zhang C, Chockalingam P, Kumar A, Burt P, Lakshmikumar A. Qualitative assessment of video stabilization and mosaicking systems. IEEE Workshop on Applications of Computer Vision (WACV 2008). 2008 ;: 1-6