Video normals from colored lights

IEEE Transactions on Pattern Analysis and Machine Intelligence

Volumn 33, Issue 10, 2011, Pages 2104-2114

  Brostow, Gabriel J ^a   Hernandez, Carlos ^b   Vogiatzis, George ^c   Stenger, Bjorn ^d   Cipolla, Roberto ^e  

^a UNIVERSITY COLLEGE LONDON   (United Kingdom)

^b GOOGLE INC   (United States)

^c ASTON UNIVERSITY   (United Kingdom)

^d TOSHIBA CORPORATION   (Japan)

^e UNIVERSITY OF CAMBRIDGE   (United Kingdom)

Author keywords

multispectral; Photometric stereo; single view; video normals.

Indexed keywords

2D TRACKING; 3-D SHAPE; 3D DATA; 3D RECONSTRUCTION; BLUE LIGHT; CAPTURE SYSTEM; COLOR PHOTOMETRIC STEREO; COLORED LIGHT; MONOCULAR VIDEO SEQUENCES; MULTI-SPECTRAL; PHOTOMETRIC STEREO; SINGLE VIEW; STRUCTURED LIGHT; SURFACE FEATURE; VIDEO NORMALS.;

ALGORITHMS; EXPERIMENTS; PHOTOMETRY; THREE DIMENSIONAL; TWO DIMENSIONAL;

THREE DIMENSIONAL COMPUTER GRAPHICS;

EID: 80051997968     PISSN: 01628828     EISSN: None     Source Type: Journal    
DOI: 10.1109/TPAMI.2011.37     Document Type: Article

References (72)

1. J. Pilet, V. Lepetit, and P. Fua, "Real-Time Non-Rigid Surface Detection," Proc. IEEE CS Conf. Computer Vision and Pattern Recognition, 2005.
2. M. Salzmann, S. Ilic, and P. Fua, "Physically Valid Shape Parameterization for Monocular 3-D Deformable Surface Tracking," Proc. British Machine Vision Conf., 2005.
3. V. Scholz, T. Stich, M. Keckeisen, M. Wacker, and M. Magnor, "Garment Motion Capture Using Color-Coded Patterns," Computer Graphics Forum, vol. 24, no. 3, pp. 439-448, Aug. 2005.
4. R. White and D. Forsyth, "Retexturing Single Views Using Texture and Shading," Proc. European Conf. Computer Vision, pp. 70-81, 2006. (Pubitemid 44029621)
5. R. White, K. Crane, and D. Forsyth, "Capturing and Animating Occluded Cloth," Proc. ACM SIGGRAPH, 2007.
6. S. Seitz, B. Curless, J. Diebel, D. Scharstein, and R. Szeliski, "A Comparison and Evaluation of Multi-View Stereo Reconstruction Algorithms," Proc. IEEE CS Conf. Computer Vision and Pattern Recognition, vol. 1, pp. 519-528, 2006.
7. A. Hertzmann and S. Seitz, "Shape and Materials by Example: A Photometric Stereo Approach," Proc. IEEE CS Conf. Computer Vision and Pattern Recognition, pp. I-533-I-540, 2003.
8. C. Hernandez, G. Vogiatzis, and R. Cipolla, "Multiview Photometric Stereo," IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 30, no. 3, pp. 548-554, Mar. 2008. (Pubitemid 351280061)
9. M. Levoy, K. Pulli, B. Curless, S. Rusinkiewicz, D. Koller, L. Pereira, M. Ginzton, S. Anderson, J. Davis, J. Ginsberg, J. Shade, and D. Fulk, "The Digital Michelangelo Project: 3D Scanning of Large Statues," Proc. ACM SIGGRAPH, pp. 131-144, 2000. (Pubitemid 32393974)
10. T. Malzbender, D.G.B. Wilburn, and B. Ambrisco, "Surface Enhancement Using Real-Time Photometric Stereo and Reflectance Transformation," Proc. European Symp. Rendering, 2006.
11. E. de Aguiar, C. Stoll, C. Theobalt, N. Ahmed, H.-P. Seidel, and S. Thrun, "Performance Capture from Sparse Multi-View Video," ACM Trans. Graphics, vol. 27, no. 3, pp. 1-10, 2008.
12. D. Vlasic, I. Baran, W. Matusik, and J. Popovic, "Articulated Mesh Animation from Multi-View Silhouettes," ACM Trans. Graphics, vol. 27, no. 3, pp. 1-9, 2008.
13. C. Hernandez and G. Vogiatzis, "Self-Calibrating a Real-Time Monocular 3D Facial Capture System," Proc. Int'l Symp. 3DPVT, 2010.
14. C. Hernandez, G. Vogiatzis, G.J. Brostow, B. Stenger, and R. Cipolla, "Non-Rigid Photometric Stereo with Colored Lights," Proc. 11th IEEE Int'l Conf. Computer Vision, 2007.
15. D. Pritchard and W. Heidrich, "Cloth Motion Capture," Computer Graphics Forum, vol. 22, no. 3, pp. 263-272, 2003.
16. R. Woodham, "Photometric Method for Determining Surface Orientation from Multiple Images," Optical Eng., vol. 19, no. 1, pp. 139-144, 1980. (Pubitemid 10456398)
17. J. Lim, J. Ho, M.-H. Yang, and D. Kriegman, "Passive Photometric Stereo from Motion," Proc. 10th IEEE Int'l Conf. Computer Vision, pp. 1635-1642, 2005. (Pubitemid 44042373)
18. D.B. Goldman, B. Curless, A. Hertzmann, and S.M. Seitz, "Shape and Spatially-Varying BRDFs from Photometric Stereo," ICCV '05: Proc. 10th IEEE Int'l Conf. Computer Vision, vol. 1, pp. 341-348,2005. (Pubitemid 44054978)
19. A. Hertzmann and S. Seitz, "Shape Reconstruction with General, Varying BRDFs," IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 27, no. 8, pp. 1254-1264, Aug. 2005.
20. W.-C. Ma, T. Hawkins, P. Peers, C.-F. Chabert, M. Weiss, and P. Debevec, "Rapid Acquisition of Specular and Diffuse Normal Maps from Polarized Spherical Gradient Illumination," Proc. Eurographics Symp. Rendering, 2007.
21. D. Vlasic, P. Peers, I. Baran, P. Debevec, J. Popovic, S. Rusinkiewicz, and W. Matusik, "Dynamic Shape Capture Using Multi-View Photometric Stereo," ACM Trans. Graphics, vol. 28, no. 5, pp. 1-11, 2009.
22. W.-C. Ma, A. Jones, J.-Y. Chiang, T. Hawkins, S. Frederiksen, P. Peers, M. Vukovic, M. Ouhyoung, and P. Debevec, "Facial Performance Synthesis Using Deformation-Driven Polynomial Displacement Maps," ACM Trans. Graphics, vol. 27, no. 5, pp. 1-10, 2008.
23. C.A. Wilson, A. Ghosh, P. Peers, J.-Y. Chiang, J. Busch, and P. Debevec, "Temporal Upsampling of Performance Geometry Using Photometric Alignment," ACM Trans. Graphics, vol. 29, no. 2, pp. 1-11, 2010.
24. A. Petrov, "Light, Color and Shape," Proc. Cognitive Processes and Their Simulation, pp. 350-358, 1987.
25. L. Kontsevich, A. Petrov, and I. Vergelskaya, "Reconstruction of Shape from Shading in Color Images," J. Optical Soc. of Am. A, vol. 11, no. 3, pp. 1047-1052, 1994.
26. M.S. Drew and L.L. Kontsevich, "Closed-Form Attitude Determination Under Spectrally Varying Illumination," Proc. IEEE CS Conf. Computer Vision and Pattern Recognition, pp. 985-990, 1994.
27. R.J. Woodham, "Gradient and Curvature from the Photometric-Stereo Method, Including Local Confidence Estimation," J. Optical Soc. of Am. A, vol. 11, no. 11, pp. 3050-3068, 1994.
28. M.K. Johnson and E.H. Adelson, "Retrographic Sensing for the Measurement of Surface Texture and Shape," Proc. Computer Vision and Pattern Recognition, pp. 1070-1077, 2009.
29. K.S. Bhat, CD. Twigg, J.K. Hodgins, P.K. Khosla, Z. Popovic, and S.M. Seitz, "Estimating Cloth Simulation Parameters from Video," Proc. ACM SIGGRAPH/Eurographics Symp. Computer Animation, pp. 37-51, 2003.
30. X. Gu, S. Zhang, P. Huang, L. Zhang, S.-T. Yau, and R. Martin, "Holoimages," Proc. ACM Symp. Solid and Physical Modeling, pp. 129-138, 2006. (Pubitemid 44059985)
31. T. Weise, B. Leibe, and L.V. Gool, "Fast 3D Scanning with Automatic Motion Compensation," Proc. IEEE Conf. Computer Vision and Pattern Recognition, 2007.
32. L. Zhang, N. Snavely, B. Curless, and S.M. Seitz, "Spacetime Faces: High-Resolution Capture for Modeling and Animation," Proc. ACM Ann. Conf. Computer Graphics, pp. 548-558, 2004.
33. P. Sand, L. McMillan, and J. Popovic, "Continuous Capture of Skin Deformation," ACM Trans. Graphics, vol. 22, no. 3, pp. 578-586, 2003.
34. H. Li, B. Adams, L.J. Guibas, and M. Pauly, "Robust Single-View Geometry and Motion Reconstruction," ACM Trans. Graphics, vol. 28, no. 5, p. 175:1-175:10, 2009.
35. D. Bradley, T. Popa, A. Sheffer, W. Heidrich, and T. Boubekeur, "Markerless Garment Capture," ACM Trans. Graphics, vol. 27, no. 3, pp. 1-9, 2008.
36. J. Starck and A. Hilton, "Surface Capture for Performance-Based Animation," IEEE Computer Graphics and Applications, vol. 27, no. 3, pp. 21-31, May/June 2007. (Pubitemid 46767722)
37. N. Ahmed, C. Theobalt, P. Dobre, H.-P. Seidel, and S. Thrun, "Robust Fusion of Dynamic Shape and Normal Capture for High-Quality Reconstruction of Time-Varying Geometry," Proc. IEEE Conf. Computer Vision and Pattern Recognition, pp. 1-8, 2008.
38. Y. Pekelny and C. Gotsman, "Articulated Object Reconstruction and Markerless Motion Capture from Depth Video," Computer Graphics Forum, vol. 27, no. 2, pp. 399-408, Apr. 2008. (Pubitemid 351612857)
39. W. Chang and M. Zwicker, "Automatic Registration for Articulated Shapes," Computer Graphics Forum, vol. 27, no. 5, pp. 1459-1468, 2008.
40. A. Johnson and M. Hebert, "Using Spin Images for Efficient Object Recognition in Cluttered 3D Scenes," IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 21, no. 5, pp. 433-449, May 1999.
41. A. Tagliasacchi, H. Zhang, and D. Cohen-Or, "Curve Skeleton Extraction from Incomplete Point Cloud," ACM Trans. Graphics, vol. 28, no. 3, 2009.
42. Q. Zheng, A. Sharf, A. Tagliasacchi, B. Chen, H. Zhang, A. Sheffer, and D. Cohen-Or, "Consensus Skeleton for Non-Rigid Space-Time Registration," Computer Graphcis Forum, Special Issue of Eurographics, vol. 29, no. 2, pp. 635-644, 2010.
43. A. Sharf, D.A. Alcantara, T. Lewiner, C. Greif, A. Sheffer, N. Amenta, and D. Cohen-Or, "Space-Time Surface Reconstruction Using Incompressible Flow," ACM Trans. Graphics, vol. 27, no. 5, pp. 1-10, 2008.
44. N.J. Mitra, S. Flory, M. Ovsjanikov, N. Gelfand, L. Guibas, and H. Pottmann, "Dynamic Geometry Registration," Proc. Fifth Eurographics Symp. Geometry Processing Computer Graphics Forum, pp. 173-182, 2007.
45. J. Sußmuth, M. Winter, and G. Greiner, "Reconstructing Animated Meshes from Time-Varying Point Clouds," Computer Graphics Forum, vol. 27, no. 5, pp. 1469-1476, 2008.
46. M. Wand, B. Adams, M. Ovsjanikov, A. Berner, M. Bokeloh, P. Jenke, L. Guibas, H.-P. Seidel, and A. Schilling, "Efficient Reconstruction of Nonrigid Shape and Motion from Real-Time 3D Scanner Data," ACM Trans. Graphics, vol. 28, no. 2, p. 15, Apr. 2009.
47. M.S. Drew, "Direct Solution of Orientation-from-Color Problem Using a Modification of Pentland's Light Source Direction Estimator," Computer Vision and Image Understanding, vol. 64, no. 2, pp. 286-299, 1996. (Pubitemid 126365522)
48. J.A. Paterson, D. Claus, and A.W. Fitzgibbon, "BRDF and Geometry Capture from Extended Inhomogeneous Samples Using Flash Photography," Computer Graphics Forum, special Eurographics issue, vol. 24, no. 3, pp. 383-391, 2005. (Pubitemid 43934656)
49. T. Simchony, R. Chellappa, and M. Shao, "Direct Analytical Methods for Solving Poisson Equations in Computer Vision Problems," IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 12, no. 5, pp. 435-446, May 1990. (Pubitemid 20702886)
50. T. Beeler, B. Bickel, P. Beardsley, B. Sumner, and M. Gross, "High-Quality Single-Shot Capture of Facial Geometry," ACM Trans. Graphics, vol. 29, no. 3, 2010.
51. Y. Furukawa and J. Ponce, "Dense 3D Motion Capture for Human Faces," Proc. IEEE Conf. Computer Vision and Pattern Recognition, pp. 1-8, 2009.
52. D. Bradley, W. Heidrich, T. Popa, and A. Sheffer, "High Resolution Passive Facial Performance Capture," ACM Trans. Graphics, vol. 29, no. 3, 2010.
53. R. Hartley and A. Zisserman, Multiple View Geometry in Computer Vision. Cambridge Univ. Press, 2004.
54. Boujou, 2d3 Ltd., http://www.2d3.com, 2009.
55. S.N. Sinha, M. Pollefeys, and L. McMillan, "Camera Network Calibration from Dynamic Silhouettes," Proc. IEEE CS Conf. Computer Vision and Pattern Recognition, pp. 195-202, 2004.
56. C. Hernandez, F. Schmitt, and R. Cipolla, "Silhouette Coherence for Camera Calibration under Circular Motion," IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 29, no. 2, pp. 343-349, Feb. 2007. (Pubitemid 46420892)
57. C. Hernandez and F. Schmitt, "Silhouette and Stereo Fusion for 3D Object Modeling," Computer Vision and Image Understanding, special issue on model-based and image-based 3D scene rRepre-sentation for interactive visualization, vol. 96, no. 3, pp. 367-392, 2004.
58. G. Vogiatzis, C. Hernandez, P.H.S. Torr, and R. Cipolla, "Multi-view Stereo via Volumetric Graph-Cuts and Occlusion Robust Photo-Consistency," IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 29, no. 12, pp. 2241-2246, Dec. 2007. (Pubitemid 350124071)
59. Y. Furukawa and J. Ponce, "Accurate, Dense, and Robust Multi-View Stereopsis," Proc. IEEE Conf. Computer Vision and Pattern Recognition, pp. 1-8, 2007.
60. C. Hernandez, G Vogiatzis, and R. Cipolla, "Shadows in Three-Source Photometric Stereo," Proc. 10th European Conf. Computer Vision, pp. 290-303, 2008.
61. B. Allen, B. Curless, and Z. Popovic, "Articulated Body Deformation from Range Scan Data," Proc. ACM SIGGRAPH, pp. 612-619, 2002.
62. M. Black and P. Anandan, "The Robust Estimation of Multiple Motions: Parametric and Piecewise Smooth Flow Fields," Computer Vision and Image Understanding, vol. 63, no. 1, pp. 75-104, 1996. (Pubitemid 126342808)
63. N. Ahmed, C. Theobalt, C. Rossl, S. Thrun, and H. Seidel, "Dense Correspondence Finding for Parametrization-Free Animation Reconstruction from Video," Proc. IEEE Conf. Computer Vision and Pattern Recognition, pp. 1-8, 2008.
64. M. Botsch and O. Sorkine, "On Linear Variational Surface Deformation Methods," IEEE Trans. Visualization and Computer Graphics, vol. 14, no. 1, pp. 213-230, Jan./Feb. 2008. (Pubitemid 350157892)
65. P. Sinha and E.H. Adelson, "Recovering Reflectance and Illumination in a World of Painted Polyhedra," Proc. Fourth Int'l Conf. Computer Vision, pp. 156-163, 1993. (Pubitemid 23667857)
66. R. Onn and A. Bruckstein, "Integrability Disambiguates Surface Recovery in Two-Image Photometric Stereo," Int'l J. Computer Vision, vol. 5, no. 1, pp. 105-113, 1990.
67. R.T. Frankot and R. Chellappa, "A Method for Enforcing Integrability in Shape from Shading Algorithms," IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 10, no. 4, pp. 439-451, July 1988. (Pubitemid 18665916)
68. J.P. Lewis, M. Cordner, and N. Fong, "Pose Space Deformation: A Unified Approach to Shape Interpolation and Skeleton-Driven Deformation," Proc. ACM SIGGRAPH, pp. 165-172, 2000. (Pubitemid 32393977)
69. X.C. Wang and C. Phillips, "Multi-Weight Enveloping: Least-Squares Approximation Techniques for Skin Animation," SCA '02: Proc. ACM SIGGRAPH/Eurographics Symp. Computer Animation, pp. 129-138, 2002.
70. "CMU Graphics Lab Motion Capture Database," http://mocap.cs.cmu.edu, 2011.
71. O. Mac Aodha, G.J. Brostow, and M. Pollefeys, "Segmenting Video into Classes of Algorithm-Suitability," Proc. IEEE Conf. Computer Vision and Pattern Recognition, 2010.
72. T. Popa, I. South-Dickinson, D. Bradley, A. Sheffer, and W. Heidrich, "Globally Consistent Space-Time Reconstruction," Proc. Eurographics Symp. Geometry Processing, 2010.