Fusing multiview and photometric stereo for 3d reconstruction under uncalibrated illumination

IEEE Transactions on Visualization and Computer Graphics

Volumn 17, Issue 8, 2011, Pages 1082-1095

  Wu, Chenglei ^a   Liu, Yebin ^a   Dai, Qionghai ^a   Wilburn, Bennett ^b  

^a TSINGHUA UNIVERSITY   (China)

^b MICROSOFT RESEARCH ASIA   (China)

Author keywords

1 minimization; Lambertian reflectance; Multiview stereo; photometric stereo

Indexed keywords

3D MODELS; 3D RECONSTRUCTION; LAMBERTIAN ASSUMPTION; LAMBERTIAN OBJECT; LAMBERTIAN REFLECTANCE; LAMBERTIAN SURFACES; MANUAL INTERACTION; MULTI-VIEW IMAGE; MULTI-VIEW STEREO; MULTI-VIEWS; OPTIMIZATION METHOD; PHOTOMETRIC STEREO; RECONSTRUCTION SYSTEMS; SHAPE PRIORS; SPHERICAL HARMONICS; SURFACE NORMALS; SYNTHETIC DATA;

HARMONIC ANALYSIS; PHOTOMETRY;

THREE DIMENSIONAL;

EID: 79958780720     PISSN: 10772626     EISSN: None     Source Type: Journal    
DOI: 10.1109/TVCG.2010.224     Document Type: Article

References (54)

1. E. de Aguiar, C. Stoll, C. Theobalt, N. Ahmed, H.-P. Seidel, and S. Thrun, "Performance Capture from Sparse Multi-View Video", ACM Trans. Graphics, vol. 27, no. 3, pp. 98:1-98:10, 2008.
2. D. Vlasic, I. Baran, W. Matusik, and J. Popovic, "Articulated Mesh Animation from Multi-View Silhouettes", ACM Trans. Graphics, vol. 27, no. 1, pp. 97(1)-97(9), 2008.
3. D. Vlasic, P. Peers, I. Baran, P. Debevec, J. Popovic, S. Rusinkiewicz, and W. Matusik, "Dynamic Shape Capture Using Multi-View Photometric Stereo", ACM Trans. Graphics, vol. 28, no. 5, Dec. 2009.
4. D. Bradley, T. Popa, A. Sheffer, W. Heidrich, and T. Boubekeur, "Markerless Garment Capture", ACM Trans. Graphics, vol. 27, no. 3, pp. 99-106, 2008.
5. T. Popa, Q. Zhou, D. Bradley, V. Kraevoy, H. Fu, A. Sheffer, and W. Heidrich, "Wrinkling Captured Garments Using Space-Time Data-Driven Deformation", Computer Graphics Forum, vol. 28, no. 2, pp. 427-435, 2009.
6. S. M. Seitz, B. Curless, J. Diebel, D. Scharstein, and R. Szeliski, "A Comparison and Evaluation of Multi-View Stereo Reconstruction Algorithms", Proc. IEEE Int'l Conf. Computer Vision and Pattern Recognition (CVPR'06), vol. 1, pp. 519-528, 2006.
7. J. Starck, G. Miller, and A. Hilton, "Volumetric Stereo with Silhouette and Feature Constraints", Proc. British Machine Vision Conf. (BMVC), Sept. 2006.
8. J. Starck and A. Hilton, "Surface Capture for Performance Based Animation", IEEE Computer Graphics and Applications, vol. 27, no. 3, pp. 21-31, May 2007. (Pubitemid 46767722)
9. G. Vogiatzis, C. H. Esteban, P. H. S. Torr, and R. Cipolla, "Multi-View Stereo via Volumetric Graph-Cuts and Occlusion Robust Photoconsistency", IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 29, no. 12, pp. 2241-2246, Dec. 2007. (Pubitemid 350124071)
10. J.-P. Pons, R. Keriven, and O. Faugeras, "Modelling Dynamic Scenes by Registering Multi-View Image Sequences", Proc. IEEE Int'l Conf. Computer Vision and Pattern Recognition (CVPR'05), vol. 2, pp. 822-827, 2005.
11. D. Bradley, T. Boubekeur, and W. Heidrich, "Accurate Multi-View Reconstruction Using Robust Binocular Stereo and Surface Meshing", Proc. IEEE Int'l Conf. Computer Vision and Pattern Recognition (CVPR'08), 2008.
12. Y. Liu, Q. Dai, and W. Xu, "A Point Cloud Based Multi-View Stereo Algorithm for Free-Viewpoint Video", IEEE Trans. Visualization and Computer Graphics, vol. 16, no. 3, pp. 407-418, May/June 2010.
13. Y. Liu, X. Cao, Q. Dai, and W. Xu, "Continuous Depth Estimation for Multi-View Stereo", Proc. IEEE Int'l Conf. Computer Vision and Pattern Recognition (CVPR'09), 2009.
14. Y. Furukawa and J. Ponce, "Accurate, Dense, and Robust Multiview Stereopsis", Proc. IEEE Int'l Conf. Computer Vision and Pattern Recognition (CVPR'07), 2007.
15. mview, http://vision. middlebury.edu/mview/, 2010.
16. T. Yu, N. Xu, and N. Ahuja, "Shape and View Independent Reflectance Map from Multiple Views", Proc. European Conf. Computer Vision (ECCV), 2004.
17. H. Jin, D. Cremers, D. Wang, E. Prados, A. Yezzi, and S. Soatto, "3-D Reconstruction of Shaded Objects from Multiple Images under Unknown Illumination", Int'l J. Computer Vision, vol. 76, no. 3, pp. 245-256, 2008.
18. R. Zhang, P. S. Tsai, J. E. Cryer, and M. Shah, "Shape from Shading: A Survey", IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 21, no. 8, pp. 690-706, Aug. 1999.
19. R. Basri and D. W. Jacobs, "Lambertian Reflectance and Linear Subspaces", IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 25, no. 2, pp. 218-233, Feb. 2003.
20. R. Ramamoorthi and P. Hanrahan, "On the Relationship between Radiance and Irradiance: Determining the Illumination from Images of Convex Lambertian Object", J. Optical Soc. of America A, vol. 18, pp. 2448-2459, 2001.
21. R. Basri, D. Jacobs, and I. Kemelmacher, "Photometric Stereo with General, Unknown Lighting", Int'l J. Computer Vision, vol. 72, no. 3, pp. 239-257, 2006.
22. C. P. Chen and C. S. Chen, "The 4-Source Photometric Stereo under General Unknown Lighting", Proc. European Conf. Computer Vision (ECCV'06), 2006.
23. S. K. Zhou, G. Aggarwal, R. Chellappa, and D. W. Jacobs, "Appearance Characterization of Linear Lambertian Objects, Generalized Photometric Stereo, and Illumination-Invariant Face Recognition", IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 29, no. 2, pp. 230-245, Feb. 2007. (Pubitemid 46420884)
24. P. Belhumeur, D. Kriegman, and A. Yuille, "The Bas-Relief Ambiquity", Int'l J. Computer Vision, vol. 35, no. 1, pp. 33-44, 1999.
25. D. Simakov, D. Frolova, and R. Basri, "Dense Shape Reconstruction of a Moving Object under Arbitrary, Unknown Lighting", Proc. IEEE Int'l Conf. Computer Vision (ICCV'03), pp. 1202-1209, 2003.
26. D. Frolova, D. Simakov, and R. Basri, "Accuracy of Spherical Harmonic Approximations for Images of Lambertian Objects under Far and Near Lighting", Proc. European Conf. Computer Vision (ECCV'04), 2004.
27. A. Georghiades, "Recovering 3-D Shape and Reflectance from a Small Number of Photographs", Proc. Eurographics Symp. Rendering (EGSR'03), pp. 230-240, 2003.
28. T. Migita, S. Ogino, and T. Shakunaga, "Direct Bundle Estimation for Recovery of Shape, Reflectance Property and Light Position", Proc. European Conf. Computer Vision (ECCV'08), 2008.
29. L. Zhang, B. Curless, and S. M. Seitz, "Spacetime Stereo: Shape Recovery for Dynamic Scenes", Proc. IEEE CS Conf. Computer Vision and Pattern Recognition (CVPR'03), pp. 367-374, 2003.
30. J. Davis, R. Ramamoothi, and S. Rusinkiewicz, "Spacetime Stereo: A Unifying Framework for Depth from Triangulation", Proc. IEEE CS Conf. Computer Vision and Pattern Recognition (CVPR'03), 2003.
31. L. Zhang, B. Curless, A. Hertzmann, and S. M. Seitz, "Shape and Motion under Varying Illumination: Unifying Stucture from Motion, Photometric Stereo, and Multi-View Stereo", Proc. IEEE Int'l Conf. Computer Vision (ICCV'03), vol. 1, p. 618, 2003.
32. J. Lim, J. Ho, M. H. Yang, and D. Kriegman, "Passive Photometric Stereo from Motion", Proc. IEEE Int'l Conf. Computer Vision (ICCV'05), vol. 2, pp. 1635-1642, 2005. (Pubitemid 44042373)
33. N. Joshi and D. Kriegman, "Shape from Varying Illumination and Viewpoint", Proc. IEEE Int'l Conf. Computer Vision (ICCV'07), vol. 2, pp. 1-7, 2007.
34. M. Weber, A. Blake, and R. Cipolla, "Towards a Complete Dense Geometric and Photometric Reconstruction under Varying Pose and Illumination", Proc. British Machine Vision Conf. (BMVC'02), 2002.
35. D. Nehab, S. Rusinkiewicz, J. Davis, and R. Ramamoorthi, "Efficiently Combining Positions and Normals for Precise 3D Geometry", Proc. ACM SIGGRAPH'05, pp. 536-543, 2005.
36. G. Vogiatzis, C. Hernandez, and R. Cipolla, "Reconstruction in the Round Using Photometric Normals", Proc. IEEE Int'l Conf. Computer Vision and Pattern Recognition (CVPR'06), 2006.
37. C. Hernandez, G. Vogiatzis, and R. Cipolla, "Muti-View Photometric Stereo", IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 30, no. 3, pp. 548-554, Mar. 2008.
38. D. Samaras and D. Metaxas, "Incorporating Illumination Constraints in Deformable Models for Shape from Shading and Light Direction Estimation", IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 25, no. 2, pp. 247-264, Feb. 2003.
39. N. Birkbeck, D. Cobzas, P. Sturm, and M. Jagersand, "Variational Shape and Reflectance Estimation under Changing Light and Viewpoints", Proc. European Conf. Computer Vision (ECCV'06), 2006.
40. C. Theobalt, N. Ahmed, H. Lensch, M. Magnor, and H. P. Seidel, "Seeing People in Different Light-Joint Shape, Motion, and Reflectance Capture", IEEE Trans. Visualization and Computer Graphics, vol. 13, no. 4, pp. 663-674, July/Aug. 2007. (Pubitemid 47289543)
41. N. Ahmed, C. Theobalt, P. Dobrev, H.-P. Seidel, and S. Thrun, "Robust Fusion of Dynamic Shape and Normal Capture for High-Quality Reconstruction of Time-Varying Geometry", Proc. IEEE Int'l Conf. Computer Vision and Pattern Recognition (CVPR'08), 2008.
42. W. C. Ma, T. Hawkins, P. Peers, C. F. Chabert, M. Weiss, and P. Debevec, "Rapid Acquisition of Specular and Diffuse Normal Maps from Polarized Spherical Gradient Illumination", Proc. Eurographics Symp. Rendering (EGSR'07), 2007.
43. H. P. A. Lensch, J. Kautz, M. Goesele, W. Heidrich, and H.-P. Seidel, "Image-Based Reconstruction of Spatial Appearance and Geometric Detail", ACM Trans. Graphics, vol. 22, pp. 234-257, 2003. (Pubitemid 44769055)
44. M. Kazhdan, M. Bolitho, and H. Hoppe, "Poisson Surface Reconstruction", Proc. Eurographics Symp. Geometry Processing (EGSG'06), pp. 61-70, 2006.
45. P. Rousseeuw and A. Leroy, Robust Regression and Outlier Detection. John Wiley and Sons, 1987.
46. S. Boyd and L. Vandenberghe, Convex Optimization. Cambridge Univ. Press, 2004.
47. J. Carranza, C. Theobalt, M. Magnor, and H.-P. Seidel, "Free-Viewpoint Video of Human Actors", ACM Trans. Graphics, vol. 22, no. 3, pp. 569-577, 2003.
48. J. Lawrence, A. Ben-Artzi, C. DeCoro, W. Matusik, H. Pfister, R. Ramamoorthi, and S. Rusinkiewicz, "Inverse Shade Trees for Non-Parametric Material Representation and Editing", ACM Trans. Graphics, vol. 25, no. 3, pp. 735-745, 2006. (Pubitemid 44481034)
49. E. Candes and T. Tao, "Decoding by Linear Programming", IEEE Trans. Information Theory, vol. 51, no. 12, pp. 4203-4215, Dec. 2005. (Pubitemid 41800353)
50. R. Ramamoorthi and P. Hanrahan, "An Efficient Representation for Irradiance Environment Maps", Proc. ACM SIGGRAPH'01, 2001.
51. P. Debevec, "Rendering Synthetic Objects into Real Scenes: Bridging Traditional and Image-Based Graphics with Global Illumination and High Dynamic Range Photography", Proc. ACM SIGGRAPH'98, pp. 189-198, July 1998.
52. G. Turk and M. Levoy, "Zippered Polygon Meshes from Range Images", Proc. ACM SIGGRAPH'96, pp. 311-318, 1996.
53. R. H. Woodham, "Photometric Method for Determining Surface Orientation from Multiple Images", Optical Eng., vol. 19, no. 1, pp. 139-144, 1980.
54. Y. Y. Schechner, S. K. Nayar, and P. N. Belhumeur, "A Theory of Multiplexed Illumination", Proc. IEEE Int'l Conf. Computer Vision (ICCV), vol. 2, pp. 808-815, 2003.