Shading-based refinement on volumetric signed distance functions

ACM Transactions on Graphics

Volumn 34, Issue 4, 2015, Pages

  Zollhöfer, Michael ^a,d   Dai, Angela ^b   Innmann, Matthias ^a   Wu, Chenglei ^c   Stamminger, Marc ^a   Theobalt, Christian ^d   Nießner, Matthias ^b  

^a UNIVERSITY OF ERLANGEN NUREMBERG   (Germany)

^b STANFORD UNIVERSITY   (United States)

^c ETH ZURICH   (Switzerland)

^d MAX PLANCK INSTITUTE FOR INFORMATICS   (Germany)

Author keywords

3D reconstruction; Shading based refinement

Indexed keywords

BUDGET CONTROL; CAMERAS; COLOR COMPUTER GRAPHICS; INTERACTIVE COMPUTER GRAPHICS; INVERSE PROBLEMS; REPAIR;

3D RECONSTRUCTION; EFFICIENT RECONSTRUCTION; IMPLICIT SURFACE REPRESENTATION; OBJECTIVE FUNCTIONS; SHADING-BASED REFINEMENT; SIGNED DISTANCE FIELDS; SIGNED DISTANCE FUNCTION; SURFACE REFLECTANCE;

IMAGE RECONSTRUCTION;

EID: 84947228131     PISSN: 07300301     EISSN: 15577368     Source Type: Journal    
DOI: 10.1145/2766887     Document Type: Conference Paper

References (60)

1. AGARWAL, S., FURUKAWA, Y., SNAVELY, N., SIMON, I., CURLESS, B., SEITZ, S. M., AND SZELISKI, R. 2011. Building rome in a day. Communications of the ACM 54, 10, 105-112.
2. AGISOFT, L. 2014. Agisoft photoscan. Professional Edition.
3. BEELER, T., BRADLEY, D., ZIMMER, H., AND GROSS, M. 2012. Improved reconstruction of deforming surfaces by cancelling ambient occlusion. In Proc. ECCV, 30-43.
4. BYLOW, E., STURM, J., KERL, C., KAHL, F., AND CREMERS, D. 2013. Real-time camera tracking and 3d reconstruction using signed distance functions. In Robotics: Science and Systems (RSS) Conference 2013, vol. 9.
5. CARR, J. C., BEATSON, R. K., CHERRIE, J. B., MITCHELL, T. J., FRIGHT, W. R., MCCALLUM, B. C., AND EVANS, T. R. 2001. Reconstruction and representation of 3d objects with radial basis functions. In Proc. SIGGRAPH, ACM, 67-76.
6. CHAN, D., BUISMAN, H., THEOBALT, C., THRUN, S., ET AL. 2008. A noise-aware filter for real-time depth upsampling. In Workshop on Multi-camera and Multi-modal Sensor Fusion Algorithms and Applications-M2SFA2 2008.
7. CHEN, Q., AND KOLTUN, V. 2013. A simple model for intrinsic image decomposition with depth cues. In The IEEE International Conference on Computer Vision (ICCV).
8. CHEN, J., BAUTEMBACH, D., AND IZADI, S. 2013. Scalable real-time volumetric surface reconstruction. ACM TOG 32, 4, 113.
9. CUI, Y., SCHUON, S., CHAN, D., THRUN, S., AND THEOBALT, C. 2010. 3d shape scanning with a time-of-flight camera. In Proc. CVPR, 1173-1180.
10. CURLESS, B., AND LEVOY, M. 1996. A volumetric method for building complex models from range images. In Proceedings of the 23rd annual conference on Computer graphics and interactive techniques, ACM, 303-312.
11. DEBEVEC, P. 2012. The light stages and their applications to photoreal digital actors. In SIGGRAPH Asia Technical Briefs.
12. DIEBEL, J., AND THRUN, S. 2006. An application of Markov Random Fields to range sensing. 291-298.
13. DOLSON, J., BAEK, J., PLAGEMANN, C., AND THRUN, S. 2010. Upsampling range data in dynamic environments. In Proc. CVPR, IEEE, 1141-1148.
14. FUHRMANN, S., AND GOESELE, M. 2014. Floating scale surface reconstruction. ACM TOG 33, 4, 46.
15. GHOSH, A., FYFFE, G., TUNWATTANAPONG, B., BUSCH, J., YU, X., AND DEBEVEC, P. 2011. Multiview face capture using polarized spherical gradient illumination. ACM TOG 30.
16. GOLDLUECKE, B., AUBRY, M., KOLEV, K., AND CREMERS, D. 2014. A super-resolution framework for high-accuracy multiview reconstruction. ijcv 106, 2 (jan), 172-191.
17. HABER, T., FUCHS, C., BEKAER, P., SEIDEL, H.-P., GOESELE, M., AND LENSCH, H. 2009. Relighting objects from image collections. In Computer Vision and Pattern Recognition, 2009. CVPR 2009. IEEE Conference on, 627-634.
18. HAN, Y., LEE, J.-Y., AND KWEON, I. S. 2013. High quality shape from a single rgb-d image under uncalibrated natural illumination. In Proc. ICCV.
19. HASINOFF, S., LEVIN, A., GOODE, P., AND FREEMAN, W. 2011. Diffuse reflectance imaging with astronomical applications. In Proc. ICCV, 185-192.
20. HENRY, P., KRAININ, M., HERBST, E., REN, X., AND FOX, D. 2012. RGB-D mapping: Using Kinect-style depth cameras for dense 3D modeling of indoor environments. Int. J. Robotics Research 31 (apr), 647-663.
21. HERNÁNDEZ, C., VOGIATZIS, G., AND CIPOLLA, R. 2008. Multiview photometric stereo. IEEE PAMI 30, 3, 548-554.
22. HOPPE, H., DEROSE, T., DUCHAMP, T., MCDONALD, J., AND STUETZLE, W. 1992. Surface reconstruction from unorganized points. In Proceedings of the 19th Annual Conference on Computer Graphics and Interactive Techniques, ACM, New York, NY, USA, SIGGRAPH '92, 71-78.
23. HORN, B. K. 1975. Obtaining shape from shading information. The psychology of computer vision, 115-155.
24. IZADI, S., KIM, D., HILLIGES, O., MOLYNEAUX, D., NEWCOMBE, R., KOHLI, P., SHOTTON, J., HODGES, S., FREEMAN, D., DAVISON, A., ET AL. 2011. Kinectfusion: real-time 3d reconstruction and interaction using a moving depth camera. In Proc. UIST, ACM, 559-568.
25. KAZHDAN, M., BOLITHO, M., AND HOPPE, H. 2006. Poisson surface reconstruction. In Proc. SGP.
26. KEHL, W., NAVAB, N., AND ILIC, S. 2014. Coloured signed distance fields for full 3d object reconstruction. In Proc. BMVC.
27. KELLER, M., LEFLOCH, D., LAMBERS, M., IZADI, S., WEYRICH, T., AND KOLB, A. 2013. Real-time 3d reconstruction in dynamic scenes using point-based fusion. In Proc. 3DV, 1-8.
28. KOPF, J., COHEN, M. F., LISCHINSKI, D., AND UYTTENDAELE, M. 2007. Joint bilateral upsampling. ACM TOG 26, 3.
29. LEE, K. J., ZHAO, Q., TONG, X., GONG, M., IZADI, S., LEE, S. U., TAN, P., AND LIN, S. 2012. Estimation of intrinsic image sequences from image+depth video. In Proc. ECCV, 327-340.
30. LEVOY, M., PULLI, K., CURLESS, B., RUSINKIEWICZ, S., KOLLER, D., PEREIRA, L., GINZTON, M., ANDERSON, S., DAVIS, J., GINSBERG, J., SHADE, J., AND FULK, D. 2000. The digital michelangelo project: 3d scanning of large statues. In Proc. SIGGRAPH, 131-144.
31. LINDNER, M., KOLB, A., AND HARTMANN, K. 2007. Datafusion of PMD-based distance-information and high-resolution RGB-images. In Proc. ISSCS, 121-124.
32. LOWE, D. G. 2004. Distinctive image features from scale-invariant keypoints. IJCV 60, 2, 91-110.
33. MULLIGAN, J., AND BROLLY, X. 2004. Surface determination by photometric ranging. In Proc. CVPR Workshops.
34. NAIR, R., RUHL, K., LENZEN, F., MEISTER, S., SCHÄFER, H., GARBE, C. S., EISEMANN, M., MAGNOR, M., AND KONDERMANN, D. 2013. A survey on time-of-flight stereo fusion. In Time-of-Flight and Depth Imaging. Sensors, Algorithms, and Applications. Springer, 105-127.
35. NEHAB, D., RUSINKIEWICZ, S., DAVIS, J., AND RAMAMOORTHI, R. 2005. Efficiently combining positions and normals for precise 3D geometry. Proc. SIGGRAPH 24, 3.
36. NEWCOMBE, R. A., AND DAVISON, A. J. 2010. Live dense reconstruction with a single moving camera. In Computer Vision and Pattern Recognition (CVPR), 2010 IEEE Conference on, IEEE, 1498-1505.
37. NEWCOMBE, R. A., DAVISON, A. J., IZADI, S., KOHLI, P., HILLIGES, O., SHOTTON, J., MOLYNEAUX, D., HODGES, S., KIM, D., AND FITZGIBBON, A. 2011. Kinectfusion: Real-time dense surface mapping and tracking. In Proc. ISMAR, IEEE, 127-136.
38. NIESSNER, M., ZOLLHÖFER, M., IZADI, S., AND STAMMINGER, M. 2013. Real-time 3d reconstruction at scale using voxel hashing. ACM TOG 32, 6, 169.
39. PARK, J., KIM, H., TAI, Y.-W., BROWN, M. S., AND KWEON, I.-S. 2011. High quality depth map upsampling for 3d-tof cameras. In Proc. ICCV, IEEE, 1623-1630.
40. PRADEEP, V., RHEMANN, C., IZADI, S., ZACH, C., BLEYER, M., AND BATHICHE, S. 2013. Monofusion: Real-time 3d reconstruction of small scenes with a single web camera. In Proc. ISMAR.
41. PRADOS, E., AND FAUGERAS, O. 2005. Shape from shading: a well-posed problem? In Proc. CVPR.
42. RAMAMOORTHI, R., AND HANRAHAN, P. 2001. A signalprocessing framework for inverse rendering. In Proc. SIGGRAPH, ACM, 117-128.
43. RICHARDT, C., STOLL, C., DODGSON, N. A., SEIDEL, H.-P., AND THEOBALT, C. 2012. Coherent spatiotemporal filtering, upsampling and rendering of RGBZ videos. CGF (Proceedings of Eurographics) 31, 2 (May).
44. ROTH, H., AND VONA, M. 2012. Moving volume kinectfusion. In BMVC, 1-11.
45. RUSINKIEWICZ, S., HALL-HOLT, O., AND LEVOY, M. 2002. Real-time 3D model acquisition. ACM TOG 21, 3, 438-446.
46. SCHARSTEIN, D., HIRSCHMÜLLER, H., KITAJIMA, Y., KRATHWOHL, G., NEŠIĆ, N., WANG, X., AND WESTLING, P. 2014. High-resolution stereo datasets with subpixel-accurate ground truth. In Pattern Recognition. Springer, 31-42.
47. SCHUON, S., THEOBALT, C., DAVIS, J., AND THRUN, S. 2009. Lidarboost: Depth superresolution for tof 3d shape scanning. In Proc. CVPR, IEEE, 343-350.
48. SEITZ, S., CURLESS, B., DIEBEL, J., SCHARSTEIN, D., AND SZELISKI, R. 2006. A comparison and evaluation of multi-view stereo reconstruction algorithms. In Proc. CVPR, vol. 1, 519-528.
49. SNAVELY, N., SEITZ, S. M., AND SZELISKI, R. 2006. Photo tourism: exploring photo collections in 3d. ACM TOG 25, 3, 835-846.
50. TRIGGS, B., MCLAUCHLAN, P. F., HARTLEY, R. I., AND FITZGIBBON, A. W. 2000. Bundle adjustment-a modern synthesis. In Vision algorithms: theory and practice. Springer, 298-372.
51. WEBER, D., BENDER, J., SCHNOES, M., STORK, A., AND FELLNER, D. 2013. Efficient gpu data structures and methods to solve sparse linear systems in dynamics applications. In CGF, vol. 32, Wiley Online Library, 16-26.
52. WEISE, T., WISMER, T., LEIBE, B., AND VAN GOOL, L. 2009. In-hand scanning with online loop closure. In ICCV Workshops, 1630-1637.
53. WHELAN, T., KAESS, M., FALLON, M., JOHANNSSON, H., LEONARD, J., AND MCDONALD, J. 2012. Kintinuous: Spatially extended kinectfusion.
54. WU, C., VARANASI, K., LIU, Y., SEIDEL, H.-P., AND THEOBALT, C. 2011. Shading-based dynamic shape refinement from multiview video under general illumination. In Proc. ICCV, IEEE, 1108-1115.
55. WU, C., STOLL, C., VALGAERTS, L., AND THEOBALT, C. 2013. On-set performance capture of multiple actors with a stereo camera. ACM TOG (Proc. SIGGRAPh Asia) 32, 6, 161.
56. WU, C., ZOLLHÖFER, M., NIESSNER, M., STAMMINGER, M., IZADI, S., AND THEOBALT, C. 2014. Real-time shading-based refinement for consumer depth cameras. ACM TOG (Proc. SIGGRAPH Asia) 33, 6, 200:1-200:10.
57. YU, L.-F., YEUNG, S.-K., TAI, Y.-W., AND LIN, S. 2013. Shading-based shape refinement of rgb-d images. In Proc. CVPR.
58. ZHANG, Z., TSA, P.-S., CRYER, J. E., AND SHAH, M. 1999. Shape from shading: A survey. IEEE PAMI 21, 8, 690-706.
59. ZHOU, Q.-Y., AND KOLTUN, V. 2014. Color map optimization for 3d reconstruction with consumer depth cameras. ACM Transactions on Graphics (TOG) 33, 4, 155.
60. ZOLLHÖFER, M., NIESSNER, M., IZADI, S., REHMANN, C., ZACH, C., FISHER, M., WU, C., FITZGIBBON, A., LOOP, C., THEOBALT, C., ET AL. 2014. Real-time non-rigid reconstruction using an rgb-d camera. ACM TOG (Proc. SIGGRAPH) 4.