Dense photometric stereo: A Markov random field approach

IEEE Transactions on Pattern Analysis and Machine Intelligence

Volumn 28, Issue 11, 2006, Pages 1830-1846

  Wu, Tai Pang ^a,b   Tang, Kam Lun ^b   Tang, Chi Keung ^a,b   Wong, Tien Tsin ^a,c  

^a IEEE   (Hong Kong)

^b HONG KONG UNIVERSITY OF SCIENCE AND TECHNOLOGY   (Hong Kong)

^c CHINESE UNIVERSITY OF HONG KONG   (Hong Kong)

Author keywords

Belief propagation; Graph cuts; Markov random fields; Normal and surface reconstruction; Photometric stereo; Real time relighting; Robust inference

Indexed keywords

BELIEF PROPAGATION; GRAPH CUTS; GRAPHICS PROCESSING UNIT (GPU); MARKOV RANDOM FIELDS; NORMAL AND SURFACE RECONSTRUCTION; PHOTOMETRIC STEREOS; REAL-TIME RELIGHTING; ROBUST INFERENCE;

COMPUTATIONAL COMPLEXITY; COMPUTATIONAL GEOMETRY; IMAGE PROCESSING; IMAGE RECONSTRUCTION; MARKOV PROCESSES; REAL TIME SYSTEMS;

PHOTOMETRY;

ALGORITHM; ARTICLE; ARTIFICIAL INTELLIGENCE; AUTOMATED PATTERN RECOGNITION; COMPUTER ASSISTED DIAGNOSIS; EVALUATION; IMAGE ENHANCEMENT; INFORMATION RETRIEVAL; METHODOLOGY; PHOTOGRAPHY; PHOTOMETRY; PROBABILITY;

ALGORITHMS; ARTIFICIAL INTELLIGENCE; IMAGE ENHANCEMENT; IMAGE INTERPRETATION, COMPUTER-ASSISTED; INFORMATION STORAGE AND RETRIEVAL; MARKOV CHAINS; PATTERN RECOGNITION, AUTOMATED; PHOTOGRAMMETRY; PHOTOMETRY;

EID: 33947277487     PISSN: 01628828     EISSN: None     Source Type: Journal    
DOI: 10.1109/TPAMI.2006.224     Document Type: Article

References (47)

1. E.H. Adelson and J.R. Bergen, "The Plenoptic Function and the Elements of Early Vision," Computational Models of Visual Processing, chapter 1, pp. 3-20, 1991.
2. D.H. Ballard and C.M. Brown, Computer Vision. Prentice Hall, 1982.
3. S. Barsky and M. Petrou, "The 4-Source Photometric Stereo Technique for Three-Dimensional Surfaces in the Presence of Highlights and Shadows," IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 25, no. 10, pp. 1239-1252, Oct. 2003.
4. R. Basri and D.W. Jacobs, "Photometric Stereo with General, Unknown Lighting," Proc. Conf. Computer Vision and Pattern Recognition, pp. II:374-381, 2001.
5. P.M. Belhumeur and D.J. Kriegman, "What Is the Set of Images of an Object under All Possible Lighting Conditions?" Proc. Conf. Computer Vision and Pattern Recognition, pp. 270-277, 1996.
6. Y. Boykov, O. Veksler, and R. Zabih, "Fast Approximate Energy Minimization via Graph Cuts," IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 23, no. 11, pp. 1222-1239, Nov. 2001.
7. E.N. Coleman Jr. and R. Jain, "Obtaining 3-Dimensional Shape of Textured and Specular Surfaces Using Four-Source Photometry," Computer Graphics and Image Processing, vol. 18, no. 4, pp. 309-328, Apr. 1982.
8. T.H. Cormen, C.E. Lieserson, and R.L. Rivest, Introduction to Algorithms. MIT Press, 1990.
9. D.A. Forsyth and J. Ponce, Computer Vision: A Modern Approach. Prentice-Hall, 2003.
10. D.B. Goldman, B. Curlcss, A. Hertzmann, and S.M. Seitz, "Shape and Spatially-Varying BRDFs from Photometric Stereo," Proc. Int'l Conf. Computer Vision, Oct. 2005.
11. P. Haeberli, "Synthetic Lighting for Photography," http://www.sgi.com/grafica/synth/index.html, Jan. 1992.
12. G. Hager and P. Belhumeur, "Real-Time Tracking of Image Regions with Changes in Geometry and Illumination," Proc. IEEE Conf. Computer Vision and Pattern Recognition, June 1996.
13. A. Hertzmann and S.M. Seitz, "Shape and Materials by Example: A Photometric Stereo Approach," Proc. Conf. Computer Vision and Pattern Recognition, pp. 1:533-540, 2003.
14. P.-M. Ho, T.-T. Wong, and C.-S. Leung, "Compressing the Illumination-Adjustable Images with Principal Component Analysis," IEEE Trans. Circuits and Systems for Video Technology, vol. 15, no. 3, pp. 355-364, Mar. 2005.
15. B.K.P. Horn, Robot Vision. McGraw-Hill, 1986.
16. B.K.P. Horn, R.J. Woodham, and W.M. Silver, "Determining Shape and Reflectance Using Multiple Images," MIT AI Memo, 1978.
17. Y. Ju, K. Lee, and S.U. Lee, "Shape from Shading Using Graph Cuts," Proc. Int'l Conf. Image Processing, pp. I:421-424, 2003.
18. G. Kay and T. Caelly, "Estimating the Parameters of an Illumination Model Using Photometric Stereo," Graphical Models and Image Processing, vol. 57, no. 5, pp. 365-388, 1995.
19. V. Kolmogorov and R. Zabih, "Multi-Camera Scene Reconstruction via Graph Cuts," Proc. European Conf. Computer Vision, vol. III, p. 82, 2002.
20. V. Kolmogorov and R. Zabih, "What Energy Functions Can Be Minimized via Graph Cuts?" IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 26, no. 2, pp. 147-159, Feb. 2004.
21. P. Kovesi, "Shapelets Correlated with Surface Normals Produce Surfaces," Proc. Int'l Conf. Computer Vision, pp. 994-1001, 2005.
22. K.M. Lee and C.C.J. Kuo, "Shape Reconstruction from Photometric Stereo," Proc. Conf. Computer Vision and Pattern Recognition, pp. 479-484, 1992.
23. C.-S. Leung, T.-T. Wong, P.-M. Lam, and K.-H. Choy, "An RBF-Based Image Compression Method for Image-Based Rendering," IEEE Trans. Image Processing, vol. 15, no. 4, pp. 1031-1041, Apr. 2006.
24. Y. Li, H.Y. Shum, C.K. Tang, and R. Szeliski, "Stereo Reconstruction from Multiperspective Panoramas," IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 26, no. 1, pp. 45-62, Jan. 2004.
25. T. Malzbender, D. Gelb, and H. Wolters, "Polynomial Texture Maps," Proc. ACM SIGGRAPH Conf., pp. 519-528, 2001.
26. G. Medioni, M.S. Lee, and C.K. Tang, A Computational Framework for Feature Extraction and Segmentation. Elsevier Science, 2000.
27. S.K. Nayar and H. Murase, "Dimensionality of Illumination in Appearance Matching," Proc. IEEE Int'l Conf. Robotics and Automation, pp. 1326-1332, Apr. 1996.
28. S.K. Nayar, K. Ikeuchi, and T. Kanade, "Determining Shape and Reflectance of Hybrid Surfaces by Photometric Sampling," IEEE Trans. Robotics and Automation, vol. 6, no. 4, pp. 418-431, 1990.
29. R. Ng, R. Ramamoorthi, and P. Hanrahan, "All-Frequency Shadows Using Non-Linear Wavelet Lighting Approximation," ACM Trans. Graphics, vol. 22, no. 3, pp. 376-381, 2003.
30. J.S. Nimeroff, E. Simoncelli, and J. Dorsey, "Efficient Re-Rendering of Naturally Illuminated Environments," Proc. Fifth Eurographics Workshop Rendering, pp. 359-373, June 1994.
31. K. Nishino, Y. Sato, and K. Ikeuchi, "Eigen-Texture Method: Appearance Compression Based on 3D Model," Proc. IEEE Conf. Computer Vision and Pattern Recognition, vol. 1, pp. 618-624, June 1999.
32. J. Pearl, Probabilistic Reasoning in Intelligent Systems: Netwoks of Plausible Inference. Morgan Kaufmann, 1988.
33. N. Petrovic, I. Cohen, B.J. Frey, R. Koetter, and T.S. Huang, "Enforcing Integrability for Surface Reconstruction Algorithms Using Belief Propagation in Graphical Models," Proc. Conf. Computer Vision and Pattern Recognition, vol. I, pp. 743-748, 2001.
34. Y. Shan, Z. Liu, and Z. Zhang, "Image-Based Surface Detail Transfer," Proc. Conf. Computer Vision and Pattern Recognition, vol. II, pp. 794-799, 2001.
35. F. Solomon and K. Ikeuchi, "Extracting the Shape and Roughness of Specular Lobe Objects Using Four Light Photometric Stereo," IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 18, no. 4, pp. 449-454, Apr. 1996.
36. J. Sun, N.N. Zheng, and H.Y. Shum, "Stereo Matching Using Belief Propagation," IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 25, no. 7, pp. 787-800, July 2003.
37. H.D. Tagare and R.J. P. deFigueiredo, "A Theory of Photometric Stereo for a Class of Diffuse Non-Lambertian Surfaces," IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 13, no. 2, pp. 133-152, Feb. 1991.
38. K.L. Tang, C.K. Tang, and T.T. Wong, "Dense Photometric Stereo Using Tensorial Belief Propagation," Proc. Conf. Computer Vision and Pattern Recognition, vol. 1, pp. 132-139, June 2005.
39. M.F. Tappen and W.T. Freeman, "Comparison of Graph Cuts with Belief Propagation for Stereo, Using Identical MRF Parameters," Proc. Int'l Conf. Computer Vision, pp. 900-907, 2003.
40. M. Alex, O. Vasilescu, and D. Terzopoulos, "Tensortextures: Multilinear Image-Based Rendering," ACM Trans. Graphics, vol. 23, no. 3, pp. 336-342, 2004.
41. A. Wenger, A. Gardner, C. Tchou, J. Unger, T. Hawkins, and P.E. Debevec, "Performance Relighting and Reflectance Transformation with Time-Multiplexed Illumination," ACM Trans. Graphics, vol. 24, no. 3, pp. 756-764, 2005.
42. T.-T. Wong, C.-W. Fu, P.-A. Heng, and C.-S. Leung, "The Plenoptic Illumination Function," IEEE Trans. Multimedia, vol. 4 no. 3, Sept. 2002.
43. T.-T. Wong, P.-A. Heng, S.-H. Or, and W.-Y. Ng, "Image-Based Rendering with Controllable Illumination," Proc. Eighth Eurographics Workshop Rendering, pp. 13-22, June 1997.
44. R.J. Woodham, "Photometric Method for Determining Surface Orientation from Multiple Images," Optical Eng., vol. 19, no. 1, pp. 139-144, Jan. 1980.
45. R.J. Woodham, "Gradient and Curvature from the Photometric-Stereo Method, Including Local Confidence Estimation," J. Optical Soc. Am., vol. 11, no. 11, pp. 3050-3068, Nov. 1994.
46. T.P. Wu and C.K. Tang, "Dense Photometric Stereo Using a Mirror Sphere and Graph Cut," Proc. Conf. Computer Vision and Pattern Recognition, vol. 1, pp. 140-147, June 2005.
47. Z. Zhang, "Modeling Geometric Structure and Illumination Variation of a Scene from Real Images," Proc. Int'l Conf. Computer Vision, Jan. 1998.