Automatic acquisition of high-fidelity facial performances using monocular videos

ACM Transactions on Graphics

Volumn 33, Issue 6, 2014, Pages

  Shi, Fuhao ^a   Wu, Hsiang Tao ^b   Tong, Xin ^b   Chai, Jinxiang ^a  

^a TEXAS A AND M UNIVERSITY   (United States)

^b MICROSOFT RESEARCH ASIA   (China)

Author keywords

Face animation; Facial detection and tracking; Facial editing; Facial modeling; Facial performance capture

Indexed keywords

DEFORMATION; EDGE DETECTION; THREE DIMENSIONAL COMPUTER GRAPHICS;

DETECTION AND TRACKING; FACE ANIMATION; FACIAL EDITING; FACIAL MODEL; PERFORMANCE CAPTURE;

FACE RECOGNITION;

EID: 84914695505     PISSN: 07300301     EISSN: 15577368     Source Type: Journal    
DOI: 10.1145/2661229.2661290     Document Type: Conference Paper

References (38)

1. AMIT, Y., AND GEMAN, D. 1997. Shape quantization and recognition with randomized trees. Neural Computation. 9(7):1545-1588.
2. BASRI, R., AND JACOBS, D. W. 2003. Lambertian reflectance and linear subspaces. IEEE Transactions on Pattern Analysis and Machine Intelligence. 25, 2, 218-233.
3. BEELER, T., BICKEL, B., BEARDSLEY, P., SUMNER, B., AND GROSS, M. 2010. High-quality single-shot capture of facial geometry. ACM Trans. Graph. 29, 4, 40:1-40:9.
4. BEELER, T., HAHN, F., BRADLEY, D., BICKEL, B., BEARDSLEY, P., GOTSMAN, C., SUMNER, R. W., AND GROSS, M. 2011. High-quality passive facial performance capture using anchor frames. ACM Trans. Graph. 30, 4, 75:1-75:10.
5. BELHUMEUR, P. N., JACOBS, D. W., KRIEGMAN, D. J., AND KUMAR, N. 2011. Localizing parts of faces using a consensus of exemplars. In Computer Vision and Pattern Recognition, 545-552.
6. BICKEL, B., BOTSCH, M., ANGST, R., MATUSIK, W., OTADUY, M., PFISTER, H., AND GROSS, M. 2007. Multi-scale capture of facial geometry and motion. ACM Trans. Graph. 26, 3, 33:1-33:10.
7. BLANZ, V., BASSO, C., POGGIO, T., AND VETTER, T. 2003. Reanimating faces in images and video. In Computer Graphics Forum. 22(3):641-650.
8. BOUAZIZ, S., WANG, Y., AND PAULY, M. 2013. Online modeling for realtime facial animation. ACM Trans. Graph. 32, 4 (July), 40:1-40:10.
9. BRADLEY, D., HEIDRICH, W., POPA, T., AND SHEFFER, A. 2010. High resolution passive facial performance capture. ACM Trans. Graph. 29, 4, 41:1-41:10.
10. BUNNELL, M. 2005. Adaptive tessellation of subdivision surfaces with displacement mapping. In GPU Gems 2, M. Pharr, Ed. Addison-Wesley, 109-122.
11. CAO, C., WENG, Y., LIN, S., AND ZHOU, K. 2013. 3d shape regression for real-time facial animation. ACM Trans. Graph. 32, 4 (July), 41:1-41:10.
12. CAO, C., WENG, Y., ZHOU, S., TONG, Y., AND ZHOU, K. 2013. Facewarehouse: a 3d facial expression database for visual computing. IEEE Trans. on Visualization and Computer Graphics. 20, 3, 413-425.
13. COMANICIU, D., AND MEER, P. 2002. Mean shift: A robust approach toward feature space analysis. IEEE Transactions on Pattern Analysis and Machine Intelligence. 24, 5, 603-619.
14. DAI, Y., LI, H., AND HE, M. 2012. A simple prior-free method for non-rigid structure-from-motion factorization. In Computer Vision and Pattern Recognition, 2018-2025.
15. GARRIDO, P., VALGAERT, L., WU, C., AND THEOBALT, C. 2013. Reconstructing detailed dynamic face geometry from monocular video. ACM Trans. Graph. 32, 6 (Nov.), 158:1-158:10.
16. GUENTER, B., GRIMM, C., WOOD, D., MALVAR, H., AND PIGHIN, F. 1998. Making Faces. In Proceedings of ACM SIGGRAPH 1998, 55-66.
17. HORN, B. K., AND BROOKS, M. J. 1986. The variational approach to shape from shading. Computer Vision, Graphics, and Image Processing. 33, 2, 174-208.
18. HORN, B., AND BROOKS, M. 1989. Shape from Shading. MIT Press: Cambridge, MA.
19. HUANG, H., CHAI, J., TONG, X., AND WU , H.-T. 2011. Leveraging motion capture and 3d scanning for high-fidelity facial performance acquisition. ACM Trans. Graph. 30, 4, 74:1-74:10.
20. KEMELMACHER-SHLIZERMAN, I., AND BASRI, R. 2011. 3d face reconstruction from a single image using a single reference face shape. IEEE Transactions on Pattern Analysis and Machine Intelligence. 33, 2, 394-405.
21. LAMDAN, Y., AND WOLFSON, H. 1988. Geometric hashing: A general and efficient model-based recognition scheme. International Conference on Computer Vision, 238-249.
22. LEPETIT, V., AND FUA, P. 2006. Keypoint recognition using randomized trees. IEEE Transactions on Pattern Analysis and Machine Intelligence. 28(9): 1465-1479.
23. LI, H., ADAMS, B., GUIBAS, L. J., AND PAULY, M. 2009. Robust single-view geometry and motion reconstruction. ACM Trans. Graph. 28, 5, 175:1-175:10.
24. LI, H., YU, J., YE, Y., AND BREGLER, C. 2013. Realtime facial animation with on-the-fly correctives. ACM Trans. Graph. 32, 4 (July), 42:1-42:10.
25. LOURAKIS, M. I. A., 2009. levmar: Levenberg-Marquardt nonlinear least squares algorithms in {C}/{C}++.
26. MA, W.-C., JONES, A., CHIANG, J.-Y., HAWKINS, T., FREDERIKSEN, S., PEERS, P., VUKOVIC, M., OUHYOUNG, M., AND DEBEVEC, P. 2008. Facial performance synthesis using deformation-driven polynomial displacement maps. ACM Trans. Graph. 27, 5, 121:1-121:10.
27. MATTHEWS, I., AND BAKER, S. 2004. Active appearance models revisited. International Journal of Computer Vision. 60, 2, 135-164.
28. PÉREZ, P., GANGNET, M., AND BLAKE, A. 2003. Poisson image editing. ACM Trans. Graph. 22, 3 (July), 313-318.
29. ROSS, D. A., LIM, J., LIN, R.-S., AND YANG, M.-H. 2008. Incremental learning for robust visual tracking. International Journal of Computer Vision. 77, 1-3, 125-141.
30. SUMNER, R. W., AND POPOVIĆ, J. 2004. Deformation transfer for triangle meshes. ACM Trans. Graph. 23, 3 (Aug.), 399-405.
31. SUWAJANAKORN, S., KEMELMACHER-SHLIZERMAN, I., AND SEITZ, S. M. 2014. Total moving face reconstruction. In European Conference on Computer Vision. 796-812.
32. VALGAERTS, L., WU, C., BRUHN, A., SEIDEL, H.-P., AND THEOBALT, C. 2012. Lightweight binocular facial performance capture under uncontrolled lighting. ACM Trans. Graph. 31, 6 (Nov.), 187:1-187:11.
33. VLASIC, D., BRAND, M., PFISTER, H., AND POPOVIĆ, J. 2005. Face transfer with multilinear models. ACM Transactions on Graphics 24, 3, 426-433.
34. WEDIN, P. A˚. 1983. On angles between subspaces of a finite dimensional inner product space. In Matrix Pencils. 263-285.
35. WEISE, T., LI, H., VAN GOOL, L., AND PAULY, M. 2009. Face/off: live facial puppetry. In Symposium on Computer Animation, 7-16.
36. WEISE, T., BOUAZIZ, S., LI, H., AND PAULY, M. 2011. Realtime performance-based facial animation. ACM Trans. Graph. 30, 4, 77:1-77:10.
37. WU, C., VARANASI, K., LIU, Y., SEIDEL, H.-P., AND THEOBALT, C. 2011. Shading-based dynamic shape refinement from multi-view video under general illumination. In International Conference on Computer Vision, 1108-1115.
38. ZHANG, L., SNAVELY, N., CURLESS, B., AND SEITZ, S. 2004. Spacetime faces: high resolution capture for modeling and animation. ACM Transactions on Graphics 23, 3, 548-558.