RGB-D mapping: Using Kinect-style depth cameras for dense 3D modeling of indoor environments

International Journal of Robotics Research

Volumn 31, Issue 5, 2012, Pages 647-663

  Henry, Peter ^a   Krainin, Michael ^a   Herbst, Evan ^a   Ren, Xiaofeng ^b   Fox, Dieter ^a  

^a University of Washington   (United States)

^b INTEL CORPORATION   (United States)

Author keywords

kinect; localization; mapping; range sensing; RGB D; SLAM; vision

Indexed keywords

KINECT; LOCALIZATION; RANGE SENSING; RGB-D; SLAM;

ALGORITHMS; CAMERAS; MAPPING; SEMANTICS; SENSORS; VISION; VISUAL COMMUNICATION;

THREE DIMENSIONAL;

EID: 84860151074     PISSN: 02783649     EISSN: 17413176     Source Type: Journal    
DOI: 10.1177/0278364911434148     Document Type: Conference Paper

References (53)

1. Akbarzadeh A, Frahm JM, Mordohai P, Clipp B, Engels C, Gallup D, et al. Towards urban 3D reconstruction from video. Proceedings of the Third International Symposium on 3D Data Processing, Visualization and Transmission (3DPVT). 2006 ;:
2. Andreasson H, Lilienthal A. 6D scan registration using depth-interpolated local image features. Robotics and Autonomous Systems. 2010 ; 58 (2). 157-165
3. Besl PJ, McKay ND. A method for registration of 3-D shapes. IEEE Transactions on Pattern Analysis and Machine Intelligence. 1992 ; 14 (2). 239-256
4. BradskiG (2000) The OpenCV Library. Dr. Dobb's Journal of Software Tools. Available from: http://drdobbs.com/open-source/184404319
5. Calonder M, Lepetit V, Fua P. Keypoint signatures for fast learning and recognition. Proceedings of the European Conference on Computer Vision (ECCV) ; 2008 ; 2008. 58.
6. Chen Y, Medioni G. Object modeling by registration of multiple range images. Image and Vision Computing. 1992 ; 10: 145-155
7. Clemente L, Davison A, Reid I, Neira J, Tardós J. Mapping large loops with a single hand-held camera. Proceedings of Robotics: Science and Systems (RSS). 2007 ;:
8. Cui Y, Schuon S, Chan D, Thrun S, Theobalt C. 3D shape scanning with a time-of-flight camera. Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR) ; 2010 ; 2010..
9. Debevec P, Taylor CJ, Malik J. Modeling and rendering architecture from photographs: A hybrid geometry and image-based approach. SIGGRAPH '96, Proceedings of the 23rd Annual Conference on Computer Graphics ; 1996 ; 1996. 11.
10. Droeschel D, May S, Holz D, Ploeger P, Behnke S. Robust ego-motion estimation with ToF cameras. European Conference on Mobile Robots (ECMR) ; 2009 ; 2009..
11. Du H, Henry P, Ren X, Cheng M, Goldman DB, Seitz SM, et al. Interactive 3D modeling of indoor environments with a consumer depth camera. International Conference on Ubiquitous Computing (UbiComp) ; 2011 ; 2011..
12. Fischler MA, Bolles RC. Random sample consensus: A paradigm for model fitting with applications to image analysis and automated cartography. Communications of the ACM. 1981 ; 24: 381-395
13. Furukawa Y, Ponce J. Accurate, Dense, and Robust Multiview Stereopsis. Pattern Analysis and Machine Intelligence, IEEE Transactions on. 2010 ; 32 (8). 1362-1376
14. Furukawa Y, Curless B, Seitz S, Szeliski R. Reconstructing building interiors from images. Proceedings of the International Conference on Computer Vision (ICCV) ; 2009 ; 2009..
15. Grisetti G, Grzonka S, Stachniss C, Pfaff P, Burgard W. Estimation of accurate maximum likelihood maps in 3D. Proceedings of the IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) ; 2007a ; 2007a..
16. Grisetti G, Stachniss C, Grzonka S, Burgard W. A tree parameterization for efficiently computing maximum likelihood maps using gradient descent. Proceedings of Robotics: Science and Systems (RSS). 2007b ;:
17. Henry P, Krainin M, Herbst E, Ren X, Fox D. RGB-D mapping: Using depth cameras for dense 3D modeling of indoor environments. Proceedings of the International Symposium on Experimental Robotics (ISER) ; 2010 ; 2010..
18. Horn BKP. Closed-form solution of absolute orientation using unit quaternions. Journal of Optical Society of America A. 1987 ; 4: 629-642
19. Johnson A, Kang SB. Registration and integration of textured 3-D data. International Conference on 3-D Digital Imaging and Modeling (3DIM) ; 1997 ; 1997. 234.
20. Kim YM, Theobalt C, Diebel J, Kosecka J, Micusik B, Thrun S. Multi-view image and ToF sensor fusion for dense 3D reconstruction. International Conference on 3-D Digital Imaging and Modeling (3DIM) ; 2009 ; 2009..
21. Konolige K. Large-scale map making. Proceedings of the National Conference on Artificial Intelligence (AAAI) ; 2004 ; 2004..
22. Konolige K. Projected texture stereo. Proceedings of the IEEE International Conference on Robotics and Automation (ICRA) ; 2010a ; 2010a..
23. Konolige K. Sparse sparse bundle adjustment. Proceedings of the British Machine Vision Conference (BMVC) ; 2010b ; 2010b..
24. Konolige K, Agrawal M. FrameSLAM: From bundle adjustment to real-time visual mapping. IEEE Transactions on Robotics. 2008 ; 25 (5). 1066-1077
25. Krainin M, Henry P, Ren X, Fox D. Manipulator and object tracking for in-hand 3D object modeling. The International Journal of Robotics Research. 2011 ; 30 (11). 1311-1327
26. Lourakis M, Argyros A. SBA: A software package for generic sparse bundle adjustment. ACM Transactions on Mathematical Software. 2009 ; 36: 1-30
27. Lowe D. Discriminative image features from scale-invariant keypoints. International Journal of Computer Vision. 2004 ; 60 (2). 91-110
28. Lu F, Milios E. Globally consistent range scan alignment for environment mapping. Autonomous Robots. 1997 ; 4: 333-349 (Pubitemid 127508781)
29. May S, Dröschel D, Holz D, Fuchs E, Malis S, Nüchter A, et al. Three-dimensional mapping with time-of-flight cameras. Journal of Field Robotics. 2009 ; 26 (11-12). 934-965
30. Newcombe R, Davison A. Live dense reconstruction with a single moving camera. Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR) ; 2010 ; 2010. 1498.
31. Newman P, Sibley G, Smith M, Cummins M, Harrison A, Mei C, et al. Navigating, recognising and describing urban spaces with vision and laser. The International Journal of Robotics Research. 2009 ; 28 (11-12). 1406-1433
32. Nister D. An efficient solution to the five-point relative pose problem. IEEE Transactions on Pattern Analysis and Machine Intelligence. 2004 ; 26: 756-777
33. Nister D, Naroditsky O, Bergen J. Visual odometry. Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR) ; 2004 ; 2004..
34. Nister D, Stewenius H. Scalable recognition with a vocabulary tree. Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR) ; 2006 ; 2006..
35. Pfister H, Zwicker M, van Baar J, Gross M. Surfels: Surface elements as rendering primitives. SIGGRAPH 2000, Proceedings of the 27th Annual Conference on Computer Graphics ; 2000 ; 2000. 335.
36. Pollefeys M, Nister D, Frahm J-M, Akbarzadeh A, Mordohai P, Clipp B, et al. Detailed real-time urban 3D reconstruction from video. International Journal of Computer Vision. 2008 ; 72: 143-167 (Pubitemid 351406110)
37. Prusak A, Melnychuk O, Roth H, Schiller I, Koch R. Pose estimation and map building with a time-of-flight-camera for robot navigation. International Journal of Intelligent Systems Technologies and Applications. 2008 ; 5: 355-364
38. Ramos F, Fox D, Durrant-Whyte H. CRF-matching: Conditional random fields for feature-based scan matching. Proceedings of Robotics: Science and Systems (RSS). 2007 ;:
39. Rosten E, Drummond T. Machine learning for high-speed corner detection. Proceedings of the European Conference on Computer Vision (ECCV) ; 2006 ; 2006..
40. Ruhnke M, Kuemmerle R, Grisetti G, Burgard W. Highly accurate maximum likelihood laser mapping by jointly optimizing laser points and robot poses. Proceedings of the IEEE International Conference on Robotics and Automation (ICRA) ; 2011 ; 2011..
41. Rusinkiewicz S, Levoy M. Efficient variants of the ICP algorithm. International Conference on 3-D Digital Imaging and Modeling (3DIM) ; 2001 ; 2001..
42. Se S, Jasiobedzki P. Stereo-vision based 3D modeling and localization for unmanned vehicles. International Journal of Intelligent Control and Systems. 2008 ; 13: 47-58
43. Segal A, Haehnel D, Thrun S. Generalized-ICP. Proceedings of Robotics: Science and Systems (RSS). 2009 ;:
44. Sharp GC, Lee SW, Wehe DK. ICP registration using invariant features. IEEE Transactions on Pattern Analysis and Machine Intelligence. 2002 ; 24: 90-102 (Pubitemid 34325618)
45. Shotton J, Fitzgibbon A, Cook M, Sharp T, Finocchio M, Moore R, et al. Real-time human pose recognition in parts from single depth images. Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR) ; 2011 ; 2011..
46. Snavely N, Seitz S, Szeliski R. Photo tourism: Exploring photo collections in 3D. ACM Transactions on Graphics. 2006 ; 25 (3). 835-846 (Pubitemid 44481045)
47. Strobl K, Mair E, Bodenmuller T, Kielhofer S, Sepp W, Suppa M, et al. The self-referenced DLR 3D-modeler. Proceedings of the IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) ; 2009 ; 2009. 21.
48. Thrun S, Burgard W, Fox D. A real-time algorithm for mobile robot mapping with applications to multi-robot and 3D mapping. Proceedings of the IEEE International Conference on Robotics and Automation (ICRA) ; 2000 ; 2000..
49. Thrun S, Burgard W, Fox D Probabilistic Robotics. Cambridge, MA: MIT Press ; 2005 :
50. Torr PHS, Zisserman A. MLESAC: A new robust estimator with application to estimating image geometry. Computer Vision and Image Understanding. 1996 ; 78 (1). 138-156
51. Triebel R, Burgard W. Improving simultaneous mapping and localization in 3D using global constraints. Proceedings of the National Conference on Artificial Intelligence (AAAI) ; 2005 ; 2005..
52. Triggs B, McLauchlan P, Hartley R, Fitzgibbon A. Bundle adjustment-a modern synthesis. Vision Algorithms: Theory and Practice. 2000 ; 1883: 153-177
53. WuC. (2007). SiftGPU: A GPU implementation of scale invariant feature transform (SIFT). Available at: http://cs.unc.edu/∼ccwu/siftgpu.