An introduction to image-based 3D surface reconstruction and a survey of photometric stereo methods

3D Research

Volumn 2, Issue 3, 2011, Pages 1-17

  Herbort, Steffen ^a   Wöhler, Christian ^a  

^a TU DORTMUND UNIVERSITY   (Germany)

Author keywords

3D surface reconstruction; Photometric Stereo; shape analysis; survey

Indexed keywords

DISTRIBUTION FUNCTIONS; LIGHT SOURCES; PHOTOMETRY; REFLECTION; STEREO IMAGE PROCESSING; SURFACE RECONSTRUCTION; SURVEYING; SURVEYS; THREE DIMENSIONAL COMPUTER GRAPHICS;

3D SHAPE RECONSTRUCTION; 3D SURFACE RECONSTRUCTION; BIDIRECTIONAL REFLECTANCE DISTRIBUTION FUNCTIONS; NON-UNIFORM SURFACE REFLECTANCE; PHOTOMETRIC STEREO; PHOTOMETRIC STEREO METHOD; SHAPE ANALYSIS; SPECULAR REFLECTANCE COMPONENTS;

IMAGE RECONSTRUCTION;

EID: 84856071871     PISSN: None     EISSN: 20926731     Source Type: Journal    
DOI: 10.1007/3DRes.03(2011)4     Document Type: Review

References (108)

1. A. Agrawal, R. Raskar, R. Chellappa (2006) What is the range of surface reconstructions from a gradient field? Proceedings of the European Conference on Computer Vision (ECCV 2006), 1(TR2006-021): 578-591.
2. N. Alldrin, T. Zickler, D. Kriegman (2008) Photometric stereo with non-parametric and spatiallyvarying reflectance, CVPR'08.
3. N. G. Alldrin (2006) Reflectance estimation under natural illumination. Technical report, University of California, San Diego.
4. N. G. Alldrin and D. J. Kriegman (2007) Toward reconstructing surfaces with arbitrary isotropic reflectance: A stratified photometric stereo approach, ICCV'07.
5. N. G. Alldrin, S. P. Mallick, D. J. Kriegman (2007) Resolving the generalized bas-relief ambiguity by entropy minimization, CVPR'07, doi: http://dx. doi. org/10. 1109/CVPR. 2007. 383208.
6. S. Barsky and M. Petrou (2003) The 4-source photometric stereo technique for three-dimensional surfaces in the presence of highlights and shadows, PAMI'03, 25(10): 1239-1252.
7. R. Basri, D. W. Jacobs, I. Kemelmacher (2007) Photometric stereo with general, unknown lighting, IJCV'07, 72(3): 239-257.
8. P. Beckmann and A. Spizzichino (1987) The Scattering of Electromagnetic Waves from Rough Surfaces, Number ISBN-13: 987-0890062382, Artech House Radar Library.
9. P. N. Belhumeur, D. J. Kriegman, A. L. Yuille (1999) The bas-relief ambiguity, IJCV'99, 35(1): 1040-1046, doi: http://dx. doi. org/10. 1023/A: 1008154927611.
10. M. K. Chandraker, F. Kahl, D. Kriegman (2005) Reflections on the generalized bas-relief ambiguity, CVPR'05, 1: 788-795.
11. J. J. Clark (1992) Active photometric stereo, CVPR'92, pages 29-34, doi: http://dx. doi. org/10. 1109/CVPR. 1992. 223231.
12. R. L. Cook and K. E. Torrance (1981) A reflectance model for computer graphics. Proceedings of the 8th annual conference on Computer graphics and interactive techniques, 15(3): 307-316, doi: http://doi. acm. org/10. 1145/800224. 806819.
13. B. L. Curless (1997) New Methods for Surface Reconstruction from Range Images. PhD thesis, Stanford University.
14. d'Angelo and C. Wöhler (2008) Image-based 3d surface reconstruction by combination of photometric, geometric, and real-aperture models, ISPRS Journal of Photo-grammetry and Remote Sensing, 63(3): 297-321, doi: http://dx. doi. org/10. 1016/j. isprsjprs. 2007. 09. 005.
15. P. Debevec (1999) Modeling and rendering architecture from photographs, Technical report.
16. O. Drbohlav and M. Chantler (2005) Can two specular pixels calibrate photometric stereo? ICCV'05, 2: 1850-1857.
17. O. Drbohlav and R. Sara (2002) Specularities reduce ambiguity of uncalibrated photometric stereo, ECCV'02, 2: 46-60.
18. R. O. Dror, E. H. Adelson, A. S. Willsky (2001) Recognition of surface reflectance properties from a single image under unknown real-world illumination, Proceedings of the IEEE Workshop on Identifying Objects Across Variations in Lighting: Psychophysics & Computation.
19. R. O. Dror, E. H. Adelson, A. S. Willsky (2001) Surface reflectance estimation and natural illumination statistics, Proceedings of the IEEE Workshop on Statistical and Computational Theories of Vision.
20. R. O. Dror, E. H. Adelson, A. S. Willsky (2001) Estimating surface reflectance properties from images under unknown illumination, Proceedings of the SPIE Conference on Human Vision and Electronic Imaging IV, 4.
21. R. O. Dror, T. K. Leung, E. H. Adelson, A. S. Willsky (2001) Statistics of real-world illumination, CVPR'01, 2: 164-171.
22. J.-D. Duroua, M. Falconeb, M. Sagona (2007) Numerical methods for shape-from-shading: A new survey with benchmarks, Computer Vision and Image Understanding, 109(1).
23. P. Fechteler, P. Eisert, J. Rurainsky (2007) Fast and high resolution 3d face scanning, ICIP'07, 3: 81-84, doi: http://dx. doi. org/10. 1109/ICIP. 2007. 4379251.
24. W. T. Freeman (1994) The generic viewpoint assumption in a framework for visual perception, Nature, 368: 542-545.
25. J. Garding (1992) Shape from texture for smooth curved surfaces in perspective projection. Journal of Mathematical Imaging and Vision, 2: 630-638.
26. A. S. Georghiades (2003) Incorporating the Torrance and Sparrow model of reflectance in uncalibrated photometric stereo, ICCV'03, 2: 816-823.
27. A. S. Georghiades, P. N. Belhumeur, D. J. Kriegman (2001) From few to many: Illumination cone models for face recognition under variable lighting and pose, PAMI'01, 23: 643-660.
28. D. B. Goldman, B. Curless, A. Hertzmann, S. Seitz (2005) Shape and spatially-varying BRDFs from photometric stereo, ICCV'05, 1: 341-348.
29. B. Goldman, B. Curless, A. Hertzmann, S. Seitz (2010) Shape and spatially varying BRDFs from photometric stereo, PAMI'10, 32(6): 1060-1071.
30. A. Grumpe, S. Herbort, C. Wöhler (2011) 3D reconstruction of non-Lambertian surfaces with nonuniform reflectance parameters by fusion of photometrically estimated surface normal data with active range scanner data, Oldenburger 3D Tage 2011, 10.
31. H. Hayakawa (1994) Photometric stereo under a light source with arbitrary motion. Journal of Optical Society of America A (JOSA A), 11: 3079-3089, doi: 10. 1364/JOSAA. 11. 003079.
32. C. Hernandez and G. Vogiatzis (2010) Selfcalibrating a real-time monocular 3D facial capture system, Fifth International Symposium on 3D Data Processing, Visualization and Transmission (3DPVT2010).
33. C. Hernandez, G. Vogiatzis, R. Cipolla (2008) Shadows in three-source photometric stereo, ECCV'08, pages 290-303, doi: http://dx.doi.org/10.1007/978-3-540-88682-2_23.
34. C. Hernández, G. Vogiatzis, G. J. Brostow, B. Stenger, R. Cipolla (2007) Non-rigid photometric stereo with colored lights, ICCV'07.
35. A. Hertzmann and S. M. Seitz (2003) Shape and materials by example: A photometric stereo approach, CVPR'03, 1: 533-540.
36. A. Hertzmann and S. M. Seitz (2005) Example-based photometric stereo: Shape reconstruction with general, varying BRDFs, PAMI'05, 27(8): 1254-1264.
37. B. K. P. Horn (1970) Shape from shading: A method for obtaining the shape of a smooth opaque object from one view, Technical Report 232, MIT.
38. B. K. P. Horn (1975) Determining Shape from Shading.
39. B. K. P. Horn (1975) Image intensity understanding, Technical Report 335, MIT, Artificial Intelligence Laboratory.
40. B. K. P. Horn (1977) Understanding image intensities, Artificial Intelligence, 11(2): 201-231.
41. B. K. P. Horn (1989) Height and gradient from shading, Technical Report 1105A, MIT, Artificial Intelligence Laboratory.
42. B. K. P. Horn and R. W. Sjoberg (1978) Calculating the reflectance map, Technical Report 498, MIT, Artificial Intelligence Laboratory.
43. K. Ikeuchi (1980) Numerical shape from shading and occluding contours in a single view, Technical Report 566, MIT, Artificial Intelligence Laboratory.
44. K. Ikeuchi (1981) Determining surface orientations of specular surfaces by using the photometric stereo method, PAMI, 3(6): 661-669, doi: http://dx. doi. org/10. 1109/TPAMI. 1981. 4767167.
45. Ikeuchi and B. K. P. Horn (1981) Numerical shape from shading and occluding boundaries, Artificial Intelligence, 17: 141-184.
46. A. B. Israel and T. N. E. Greville (2003) Generalized Inverses Theory & Applications, Springer, 2nd edition.
47. Y. Iwahori, H. Sugie, N. Ishii (1990) Reconstructing shape from shading images under point light source illumination, ICPR'90, 1: 83-87.
48. N. Joshi and D. J. Kriegman (2007) Shape from varying illumination and viewpoint, ICCV'07.
49. R. Kimmel and J. A. Sethian (2001) Optimal algorithm for shape from shading and path planning, Journal of Mathematical Imaging and Vision, 14: 237-244.
50. J.-H. Lambert (1760) Photometria, sive de mensura et gradibus luminis, colorum et umbrae, Vidae Eberhardi Klett.
51. D. Lanman and G. Taubin (2009) Build your own 3D scanner: 3D photography for beginners, Technical Report, Brown University.
52. J. Lawrence, A. Ben-Artzi, C. DeCoro, W. Matusik, H. Pfister, R. Ramamoorthi, S. Rusinkiewicz (2006) Inverse shade trees for non-parametric material representation and editing, ACM Transactions on Graphics (TOG'06), 25(3): 735-745, doi: http://doi. acm. org/10. 1145/1141911. 1141949.
53. J. Lim, J. Ho, M.-H. Yang, D. Kriegman (2005) Passive photometric stereo from motion, ICCV'05, 2: 1635-1642, doi: http://dx. doi. org/10. 1109/ICCV. 2005. 185.
54. S. P. Mallick, T. Zickler, D. J. Kriegman, P. N. Belhumeur (2005) Beyond Lambert: Reconstructing specular surfaces using color, CVPR'05, 1: 619-626.
55. S. Marschner, E. P. F. Lafortune, S. H. Westin, K. E. Torrance, D. P. Greenberg (1999) Image based BRDF measurement, Applied Optics, 39: 16.
56. S. Marschner, S. H. Westin, E. P. F. Lafortune, K. E. Torrance, D. P. Green-berg (1999) Imagebased BRDF measurement including human skin, Proceedings of 10th Eurographics Workshop on Rendering, pages 139-152.
57. S. R. Marschner (1998) Inverse Rendering for Computer Graphics, PhD thesis, Cornell University.
58. W. Matusik, H. Pfister, M. Brand, L. McMillan (2003) A data-driven reflectance model, ACM Transactions on Graphics, 22(3): 759-769.
59. W. Matusik, H. Pfister, M. Brand, L. McMillan (2003) Efficient isotropic BRDF measurement, 14th Eurographics Workshop on Rendering, 44: 241-247.
60. Morel, F. Meriaudeau, C. Stolz, P. Gorria (2005) Polarization imaging applied to 3D reconstruction of specular metallic surfaces.
61. S. K. Nayar, K. Ikeuchi, T. Kanade (1988) Extracting shape and reflectance of Lam-bertian, specular and hybrid surfaces, Technical Report CMU-FU-TR-88-14, The Robotics Institute, Carnegie Mellon University.
62. S. K. Nayar, K. Ikeuchi, T. Kanade (1990) Determining shape and reflectance of hybrid surfaces by photometric sampling, IEEE Transactions on Robotics and Automation, 6(1): 418-431.
63. S. K. Nayar, K. Ikeuchi, T. Kanade (1990) Shape from interreflections, Technical Report CMU-RI-TR-90-14, Carnegie-Mellon University of Pittsburgh, PA, Robotics Institute.
64. S. K. Nayar, K. Ikeuchi, T. Kanade (1991) Surface reflection: Physical and geometrical perspectives, PAMI'99, 13: 611-634.
65. S. K. Nayar, X.-S. Fang, T. Boult (1997) Separation of reflection components using color and polarization, IJCV'97, 21(3): 163-186, doi: 10. 1023/A: 1007937815113.
66. D. Nehab, S. Rusinkiewicz, J. Davis, R. Ramamoorthi (2005) Efficiently combining positions and normals for precise 3d geometry, SIGGRAPH'05, 24(3): 536-543, doi: http://doi. acm. org/10. 1145/1073204. 1073226.
67. F. Nicodemus, J. Richmond, J. Hsia, I. Ginsberg, T. Limperis (1977) Geometrical considerations and nomenclature for reflectance, Technical report, U. S. Department of Commerce, National Bureau of Standards.
68. E. North Coleman, Jr. and R. Jain (1982) Obtaining 3-dimensional shape of textured and specular surfaces using four-source photometry, Computer Graphics and Image Processing, 18: 309-328, doi: http://dx. doi. org/10. 1016/0146-664X(82)90001-6.
69. T. Peng (2006) Algorithms and models for 3-D shape measurement using digital fringe projections, PhD thesis, University of Maryland, Department for Mechanical Engineering.
70. R. Penrose (1955) A generalized inverse for matrices, Proceedings of the Cambridge Philosophical Society, 51: 406-413.
71. B. T. Phong (1975) Illumination for computer generated pictures. Communications of the ACM, 18(6): 311-317, doi: http://doi. acm. org/10. 1145/360825. 360839.
72. E. Prados and O. Faugeras (2003) Perspective shape from shading and viscosity solutions, ICCV'03, 2: 826-831.
73. R. Ramamoorthi (2002) A signal processing framework for forward and inverse rendering, PhD thesis, Stanford University.
74. Y. Sato, M. D. Wheeler, K. Ikeuchi (1997) Object shape and reflectance modeling from observation. Proceedings of the 24th annual conference on Computer graphics and interactive techniques, pages 379-387, doi: http://doi. acm. org/10. 1145/258734. 258885.
75. M. Seitz (1999) An overview of passive vision techniques. Technical report, The Robotics Institute, Carnegie Mellon University.
76. L. Shen and P. Tan (2009) Photometric stereo and weather estimation using internet images, CVPR'09, 1: 1850-1857.
77. B. Shi, Y. Matsushita, Y. Wei, C. Xu, P. Tan (2010) Self-calibrating photometric stereo, CVPR'10.
78. W. M. Silver (1980) Determining shape and reflectance using multiple images, Master's thesis, MIT, Computer Science and Artificial Intelligence Laboratory.
79. D. Simakov, D. Frolova, R. Basri (2003) Dense shape reconstruction of a moving object under arbitrary, unknown lighting, ICCV'03, 2: 1202.
80. M. M. Stark, J. Arvo, B. Smits (2005) Barycentric parameterizations for isotropic BRDFs, IEEE Transactions on Visualization and Computer Graphics, 11(2): 126-138, doi: http://dx. doi. org/10. 1109/TVCG. 2005. 26.
81. R. Szeliski (2010) Computer Vision Algorithms and Applications, online course material.
82. P. Tan and T. Zickler (2009) A projective framework for radiometric image analysis, CVPR 2009, pages 2977-2984, doi: http://dx. doi. org/10. 1109/CVPR. 2009. 5206731.
83. P. Tan, S. Lin, L. Quan, H.-Y. Shum (2003) Highlight removal by illumination-constraint inpainting, ICCV'03, 1: 164-169.
84. P. Tan, S. Lin, L. Quan (2006) Resolution-enhanced photometric stereo, ECCV'06, 3: 58-71.
85. P. Tan, S. P. Mallick, L. Quan, D. J. Kriegman, T. Zickler (2007) Isotropy, reciprocity and the generalized bas-relief ambiguity, CVPR 2007, pages 1-8.
86. P. Tan, S. Lin, L. Quan (2008) Subpixel photometric stereo, PAMI'08, 30(8): 1460-1471.
87. R. T. Tan and K. Ikeuchi (2003) Separating reflection components of textured surfaces using a single image, ICCV 2003, 1: 870-877.
88. R. T. Tan and K. Ikeuchi (2005) Separating reflection components of textured surfaces using a single image, PAMI'05, 27(2): 179-193.
89. A. Tankus, N. Sochen, Y. Yeshurun (2005) Shapefrom-shading under perspective projection, IJCV'05, 63(1): 21-43.
90. D. Thomas and A. Sugimoto (2010) Range image registration of specular objects under complex illumination, Fifth International Symposium on 3D Data Processing, Visualization and Transmission (3DPVT2010).
91. C. Tomasi and T. Kanade (1992) Shape and motion from image streams under orthography: a factorization method, IJCV'92, 9(2): 137-154, doi: http://dx. doi. org/10. 1007/BF00129684.
92. E. Torrance and E. M. Sparrow (1967) Theory for off-specular reflection from roughened surfaces, Journal of the Optical Society of America A (JOSA A), 57(9): 1105-1114.
93. H. von Helmholtz (1924) Handbuch der Physiologischen Optik, Optical Society of America.
94. G. J. Ward (1992) Measuring and modeling anisotropic reflection, ACM SIGGRAPH'92, 26(2): 265-272.
95. T. Weise, B. Leibe, L. van Gool (2007) Fast 3d scanning with automatic motion compensation, CVPR'07, pages 1-8.
96. T. Weyrich, J. Lawrence, H. Lensch, S. Rusinkiewicz, T. Zickler (2008) Principles of appearance acquisition and representation, ACM SIGGRAPH 2008 classes, none: 1-70, doi: http://doi. acm. org/10. 1145/1401132. 1401234.
97. D. R. White, P. Saunders, S. J. Bonsey, J. van de Ven, H. Edgar (1998) Reflectometer for measuring the bidirectional reflectance of rough surfaces, Applied Optics, 37(16): 3450-3454, doi: doi: 10. 1364/AO. 37. 003450.
98. C. Wöhler (2009) 3D Computer Vision - Efficient Methods and Applications, Springer, 1st edition.
99. C. Wöhler and P. d'Angelo (2009) Stereo image analysis of non-Lambertian surfaces, IJCV'09, 81(2): 529-540, doi: http://dx. doi. org/10. 1007/s11263-008-0157-1.
100. L. B. Wolff (1989) Using polarization to separate reflection components, CVPR'89, 1(1): 363-369, doi: http://dx. doi. org/10. 1109/CVPR. 1989. 37873.
101. L. B. Wolff and T. E. Boult (1991) Constraining object features using a polarization reflectance model, PAMI'91, 13(7): 635-657, doi: http://dx. doi. org/10. 1109/34. 85655.
102. R. J. Woodham (1980) Photometric method for determining surface orientation from multiple images, Optical Engineering, 19(1): 139-144.
103. Y. Yu, P. Debevec, J. Malik, T Hawkins (1999) Inverse global illumination: Recovering reflectance models of real scenes from photographs, SIGGRAPH1999, pages 215-224, doi: http://doi. acm. org/10. 1145/311535. 311559.
104. A. L. Yuille, J. M. Coughlan, S. Konishi (2000) The generic viewpoint constraint resolves the generalized bas relief ambiguity, Conference on Information Science and Systems.
105. L. Zhang, B. Curless, A. Hertzmann, S. M. Seitz (2003) Shape and motion under varying illumination: Unifying structure from motion, photometric stereo, and multi-view stereo, ICCV'03, 1: 618-626.
106. R. Zhang, P.-S. Tsai, J. E. Cryer, M. Shah (1999) Shape from shading: A survey, PAMI'99, 21(8): 690-706.
107. Zhou and P. Tan (2010) Ring-light photometric stereo, ECCV'10, pages 1-14.
108. T. Zickler, P. N. Belhumeur, D. J. Kriegman (2002) Helmholtz stereopsis: Exploiting reciprocity for surface reconstruction, ECCV'02, 3: 869-884.