HyperDepth: Learning depth from structured light without matching

Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition

Volumn 2016-December, Issue , 2016, Pages 5441-5450

  Fanello, Sean Ryan ^a   Rhemann, Christoph ^a   Tankovich, Vladimir ^a   Kowdle, Adarsh ^a   Escolano, Sergio Orts ^a   Kim, David ^a   Izadi, Shahram ^a  

^a MICROSOFT RESEARCH   (United States)

Author keywords

[No Author keywords available]

Indexed keywords

CAMERAS; COMPUTER VISION; DECISION TREES; IMAGE MATCHING; PATTERN RECOGNITION; PIXELS; PROBLEM SOLVING;

ALGORITHM FOR SOLVING; CLASSIFICATION REGRESSION; CORRESPONDENCE PROBLEMS; DEPTH FROM STRUCTURED LIGHTS; GPU-BASED ALGORITHMS; LOCAL STEREO MATCHING; REFERENCE PATTERNS; STRUCTURED-LIGHT SENSORS;

STEREO IMAGE PROCESSING;

EID: 84986245728     PISSN: 10636919     EISSN: None     Source Type: Conference Proceeding    
DOI: 10.1109/CVPR.2016.587     Document Type: Conference Paper

References (54)

1. Atos-industrial 3d scanning technology. http://www.gom.com/metrology-systems/3d-scanner.html.
2. C. Barnes, E. Shechtman, A. Finkelstein, and D. Goldman. PatchMatch: A randomized correspondence algorithm for structural image editing. ACM SIGGRAPH and Transaction On Graphics, 2009.
3. J. Batlle, E. Mouaddib, and J. Salvi. Recent progress in coded structured light as a technique to solve the correspondence problem: A survey. Pattern recognition, 31 (7): 963-982, 1998.
4. P. J. Besl. Active, optical range imaging sensors. Machine vision and applications, 1 (2): 127-152, 1988.
5. M. Bleyer and C. Breiteneder. Stereo matchingstate-of-the-art and research challenges. In Advanced Topics in Computer Vision, pages 143-179. Springer, 2013.
6. M. Bleyer and M. Gelautz. Simple but effective tree structures for dynamic programming-based stereo matching. In VISAPP (2), pages 415-422. Citeseer, 2008.
7. M. Bleyer, C. Rhemann, and C. Rother. PatchMatch Stereo-Stereo Matching with Slanted Support Windows. In BMVC, 2011.
8. A. Criminisi and J. Shotton. Decision Forests for Computer Vision and Medical Image Analysis. Springer, 2013.
9. J. Davis, R. Ramamoorthi, and S. Rusinkiewicz. Spacetime stereo: A unifying framework for depth from triangulation. In Computer Vision and Pattern Recognition, 2003. Proceedings. 2003 IEEE Computer Society Conference on, volume 2, pages II-359. IEEE, 2003.
10. L. De-Maeztu, S. Mattoccia, A. Villanueva, and R. Cabeza. Linear stereo matching. In Computer Vision (ICCV), 2011 IEEE International Conference on, pages 1708-1715. IEEE, 2011.
11. D. Eigen, C. Puhrsch, and R. Fergus. Depth map prediction from a single image using a multi-scale deep network. In NIPS, 2014.
12. I. Ernst and H. Hirschmüller. Mutual information based semiglobal stereo matching on the gpu. In Advances in Visual Computing, pages 228-239. Springer, 2008.
13. S. R. Fanello, C. Keskin, S. Izadi, P. Kohli, D. Kim, D. Sweeney, A. Criminisi, J. Shotton, S. Kang, and T. Paek. Learning to be a depth camera for close-range human capture and interaction. ACM SIGGRAPH and Transaction On Graphics, 2014.
14. B. Freedman, A. Shpunt, M. Machline, and Y. Arieli. Depth mapping using projected patterns, Apr. 3 2012. US Patent 8, 150, 142.
15. A. Geiger, M. Roser, and R. Urtasun. Efficient large-scale stereo matching. In ACCV, 2010.
16. J. Geng. Structured-light 3d surface imaging: A tutorial. Advances in Optics and Photonics, 3 (2): 128-160, 2011.
17. M. Hansard, S. Lee, O. Choi, and R. P. Horaud. Time-of-flight cameras: principles, methods and applications. Springer Science & Business Media, 2012.
18. H. Hirschmüller. Stereo processing by semiglobal matching and mutual information. Pattern Analysis and Machine Intelligence, IEEE Transactions on, 30 (2): 328-341, 2008.
19. E. Horn and N. Kiryati. Toward optimal structured light patterns. Image and Vision Computing, 17 (2): 87-97, 1999.
20. I. Ishii, K. Yamamoto, T. Tsuji, et al. High-speed 3d image acquisition using coded structured light projection. In Intelligent Robots and Systems, 2007. IROS 2007. IEEE/RSJ International Conference on, pages 925-930. IEEE, 2007.
21. S. Izadi, D. Kim, O. Hilliges, D. Molyneaux, R. Newcombe, P. Kohli, J. Shotton, S. Hodges, D. Freeman, A. Davison, and A. Fitzgibbon. KinectFusion: Real-time 3D reconstruction and interaction using a moving depth camera. In ACM UIST, 2011.
22. M. Ju and H. Kang. Constant time stereo matching. pages 13-17, 2009.
23. K. Konolige. Projected texture stereo. In Robotics and Automation (ICRA), 2010 IEEE International Conference on, pages 148-155. IEEE, 2010.
24. J. J. Le Moigne and A. M. Waxman. Structured light patterns for robot mobility. Robotics and Automation, IEEE Journal of, 4 (5): 541-548, 1988.
25. J. Lu, H. Yang, D. Min, and M. Do. Patch match filter: Efficient edge-aware filtering meets randomized search for fast correspondence field estimation. In CVPR, 2013.
26. M. Martinez and R. Stiefelhagen. Kinect unleashed: getting control over high resolution depth maps. In Proceedings of the 13. IAPR International Conference on Machine Vision Applications, MVA, pages 247-250, 2013.
27. M. Maruyama and S. Abe. Range sensing by projecting multiple slits with random cuts. Pattern Analysis and Machine Intelligence, IEEE Transactions on, 15 (6): 647-651, 1993.
28. S. Mattoccia. Stereo vision algorithms for fpgas. In Computer Vision and Pattern Recognition Workshops (CVPRW), 2013 IEEE Conference on, pages 636-641. IEEE, 2013.
29. P. McIlroy, S. Izadi, and A. Fitzgibbon. Kinectrack: 3d pose estimation using a projected dense dot pattern. IEEE Trans. Vis. Comput. Graph., 20 (6): 839-851, 2014.
30. M. Michael, J. Salmen, J. Stallkamp, and M. Schlipsing. Realtime stereo vision: Optimizing semi-global matching. In Intelligent Vehicles Symposium (IV), 2013 IEEE, pages 1197-1202. IEEE, 2013.
31. D. Min, J. Lu, and M. N. Do. A revisit to cost aggregation in stereo matching: How far can we reduce its computational redundancy In Computer Vision (ICCV), 2011 IEEE International Conference on, pages 1567-1574. IEEE, 2011.
32. C. Nguyen, S. Izadi, and D. Lovell. Modeling kinect sensor noise for improved 3d reconstruction and tracking. In 3DIMPVT, 2012.
33. H. Nishihara. Prism: A practical real-time imaging stereo matcher mit ai memo no. 780. Cambridge, Mass., USA, 1984.
34. K. Papadimitriou, S. Thomas, and A. Dollas. An fpga-based real-time system for 3d stereo matching, combining absolute differences and census with aggregation and belief propagation. In VLSI-SoC: At the Crossroads of Emerging Trends, pages 168-187. Springer, 2015.
35. J. Posdamer and M. Altschuler. Surface measurement by space-encoded projected beam systems. Computer graphics and image processing, 18 (1): 1-17, 1982.
36. V. Pradeep, C. Rhemann, S. Izad, C. Zach, M. Bleyer, and S. Bathiche. Monofusion: Real-time 3d reconstruction of small scenes with a single web camera. 2013.
37. C. Rhemann, A. Hosni, M. Bleyer, C. Rother, and M. Gelautz. Fast cost-volume filtering for visual correspondence and beyond. In CVPR, pages 3017-3024, 2011.
38. C. Richardt, D. Orr, I. Davies, A. Criminisi, and N. A. Dodgson. Real-time spatiotemporal stereo matching using the dual-cross-bilateral grid. In Computer Vision-ECCV 2010, pages 510-523. Springer, 2010.
39. J. Salvi, S. Fernandez, T. Pribanic, and X. Llado. A state of the art in structured light patterns for surface profilometry. Pattern recognition, 43 (8): 2666-2680, 2010.
40. J. Salvi, J. Pages, and J. Batlle. Pattern codification strategies in structured light systems. Pattern Recognition, 37 (4): 827-849, 2004.
41. K. Sato. Range imaging system utilizing nematic liquid crystal mask. In Proc. 1st Int. Conf. Comp. Vision, 1987, pages 657-661, 1987.
42. A. Saxena, M. Sun, and A. Y. Ng. Make3d: Learning 3d scene structure from a single still image. PAMI, 31 (5): 824-840, 2009.
43. D. Scharstein and R. Szeliski. A taxonomy and evaluation of dense two-frame stereo correspondence algorithms. IJCV, 47 (1-3), Apr. 2002.
44. J. Shotton, A. W. Fitzgibbon, M. Cook, T. Sharp, M. Finocchio, R. Moore, A. Kipman, and A. Blake. Real-time human pose recognition in parts from a single depth image. In Proc. CVPR, 2011.
45. P. Vuylsteke and A. Oosterlinck. Range image acquisition with a single binary-encoded light pattern. Pattern Analysis and Machine Intelligence, IEEE Transactions on, 12 (2): 148-164, 1990.
46. P. M. Will and K. S. Pennington. Grid coding: A preprocessing technique for robot and machine vision. Artificial Intelligence, 2 (3): 319-329, 1972.
47. Q. Yang. Stereo matching using tree filtering. PAMI, 37 (4): 834-846, 2015.
48. Q. Yang, L. Wang, and N. Ahuja. A constant-space belief propagation algorithm for stereo matching. In Computer vision and pattern recognition (CVPR), 2010 IEEE Conference on, pages 1458-1465. IEEE, 2010.
49. J. Zbontar and Y. LeCun. Computing the stereo matching cost with a convolutional neural network. ArXiv preprint arXiv: 1409. 4326, 2014.
50. K. Zhang, G. Lafruit, R. Lauwereins, and L. Van Gool. Joint integral histograms and its application in stereo matching. In Image Processing (ICIP), 2010 17th IEEE International Conference on, pages 817-820. IEEE, 2010.
51. L. Zhang, B. Curless, and S. M. Seitz. Rapid shape acquisition using color structured light and multi-pass dynamic programming. In 3D Data Processing Visualization and Transmission, 2002. Proceedings. First International Symposium on, pages 24-36. IEEE, 2002.
52. L. Zhang, B. Curless, and S. M. Seitz. Spacetime stereo: Shape recovery for dynamic scenes. In Computer Vision and Pattern Recognition, 2003. Proceedings. 2003 IEEE Computer Society Conference on, volume 2, pages II-367. IEEE, 2003.
53. S. Zhang. Recent progresses on real-time 3d shape measurement using digital fringe projection techniques. Optics and lasers in engineering, 48 (2): 149-158, 2010.
54. M. Zollhöfer, M. Niesner, S. Izadi, C. Rhemann, C. Zach, M. Fisher, C. Wu, A. Fitzgibbon, C. Loop, C. Theobalt, et al. Real-time non-rigid reconstruction using an rgb-d camera. ACM Transactions on Graphics (TOG), 33 (4): 156, 2014.