3D shape regression for real-time facial animation

ACM Transactions on Graphics

Volumn 32, Issue 4, 2013, Pages

  Cao, Chen ^a   Weng, Yanlin ^a   Lin, Stephen ^b   Zhou, Kun ^a  

^a ZHEJIANG UNIVERSITY   (China)

^b MICROSOFT RESEARCH ASIA   (China)

Author keywords

3D avatars; Face tracking; Facial performance; Monocular video tracking; User specific blendshapes

Indexed keywords

3D AVATARS; BLENDSHAPES; FACE TRACKING; FACIAL PERFORMANCE; MONOCULAR VIDEO;

ANIMATION; FACE RECOGNITION; REGRESSION ANALYSIS; THREE DIMENSIONAL COMPUTER GRAPHICS;

THREE DIMENSIONAL;

EID: 84880827668     PISSN: 07300301     EISSN: 15577368     Source Type: Journal    
DOI: 10.1145/2461912.2462012     Document Type: Article

References (35)

1. BEELER, T., BICKEL, B., BEARDSLEY, P., SUMNER, R., AND GROSS, M. 2010. High-quality single-shot capture of facial geometry. ACM Trans. Graph. 29, 4, 40:1-40:9.
2. BEELER, T., HAHN, F., BRADLEY, D., BICKEL, B., BEARDS- LEY, P., GOTSMAN, C., SUMNER, R. W., AND GROSS, M. 2011. High-quality passive facial performance capture using an- chor frames. ACM Trans. Graph. 30, 4, 75:1-75:10.
3. BESL, P., AND MCKAY, H. 1992. A method for registration of 3-d shapes. IEEE Trans. Pattern Anal. Mach. Intell. 14, 2, 239-256.
4. BINGHAM, E., AND MANNILA, H. 2001. Random projection in dimensionality reduction: Applications to image and text data. In Knowledge Discovery and Data Mining, 245-250.
5. BLANZ, V., AND VETTER, T. 1999. A morphable model for the synthesis of 3d faces. In Proceedings of SIGGRAPH, 187-194.
6. BRADLEY, D., HEIDRICH, W., POPA, T., AND SHEFFER, A. 2010. High resolution passive facial performance capture. ACM Trans. Graph. 29, 4, 41:1-41:10.
7. BYRD, R. H., LU, P., NOCEDAL, J., AND ZHU, C. 1995. A limited memory algorithm for bound constrained optimization. SIAM J. Sci. Comput. 16, 5 (Sept.), 1190-1208.
8. CAO, X., WEI, Y., WEN, F., AND SUN, J. 2012. Face alignment by explicit shape regression. In IEEE CVPR, 2887-2894.
9. CAO, C., WENG, Y., ZHOU, S., TONG, Y., AND ZHOU, K. 2013. FaceWarehouse: a 3D Facial Expression Database for Visual Computing. IEEE TVCG, under revision.
10. CASTELAN, M., SMITH, W. A., AND HANCOCK, E. R. 2007. A coupled statistical model for face shape recovery from brightness images. IEEE Trans. Image Processing 16, 4, 1139-1151.
11. CHAI, J.-X., XIAO, J., AND HODGINS, J. 2003. Vision-based control of 3d facial animation. In Symp. Comp. Anim., 193-206.
12. COOTES, T. F., IONITA, M. C., LINDNER, C., AND SAUER, P. 2012. Robust and accurate shape model fitting using random forest regression voting. In ECCV, VII:278-291.
13. DECARLO, D., AND METAXAS, D. 2000. Optical flow constraints on deformable models with applications to face tracking. Int. Journal of Computer Vision 38, 2, 99-127.
14. DEMENTHON, D. F., AND DAVIS, L. S. 1995. Model-based object pose in 25 lines of code. Int. J. Comput. Vision 15, 1-2, 123-141.
15. DOLLAR, P., WELINDER, P., AND PERONA, P. 2010. Cascaded pose regression. In IEEE CVPR, 1078-1085.
16. EKMAN, P., AND FRIESEN, W. 1978. Facial Action Coding Sys- tem: A Technique for the Measurement of Facial Movement. Consulting Psychologists Press.
17. ESSA, I., BASU, S., DARRELL, T., AND PENTLAND, A. 1996. Modeling, tracking and interactive animation of faces and heads: Using input from video. In Computer Animation, 68-79.
18. HUANG, D., AND LA TORRE, F. D. 2012. Facial action transfer with personalized bilinear regression. In ECCV, II:144-158.
19. HUANG, H., CHAI, J., TONG, X., AND WU, H.-T. 2011. Lever- aging motion capture and 3d scanning for high-fidelity facial per- formance acquisition. ACM Trans. Graph. 30, 4, 74:1-74:10.
20. KHOLGADE, N., MATTHEWS, I., AND SHEIKH, Y. 2011. Con- tent retargeting using parameter-parallel facial layers. In Symp. Computer Animation, 195-204.
21. LEWIS, J. P., AND ANJYO, K. 2010. Direct manipulation blend- shapes. IEEE CG&A 30, 4, 42-50.
22. LI, H., WEISE, T., AND PAULY, M. 2010. Example-based facial rigging. ACM Trans. Graph. 29, 4, 32:1-32:6.
23. MATTHEWS, I., XIAO, J., AND BAKER, S. 2007. 2D vs. 3D deformable face models: Representational power, construction, and real-time fitting. Int. J. Computer Vision 75, 1, 93-113.
24. PIGHIN, F., HECKER, J., LISCHINSKI, D., SZELISKI, R., AND SALESIN, D. H. 1998. Synthesizing realistic facial expressions from photographs. In Proceedings of SIGGRAPH, 75-84.
25. PIGHIN, F., SZELISKI, R., AND SALESIN, D. 1999. Resynthe- sizing facial animation through 3d model-based tracking. In Int. Conf. Computer Vision, 143-150.
26. SARAGIH, J., LUCEY, S., AND COHN, J. 2011. Real-time avatar animation from a single image. In AFGR, 213-220.
27. SEO, J., IRVING, G., LEWIS, J. P., AND NOH, J. 2011. Com- pression and direct manipulation of complex blendshape models. ACM Trans. Graph. 30, 6.
28. VLASIC, D., BRAND, M., PFISTER, H., AND POPOVIĆ , J. 2005. Face transfer with multilinear models. ACM Trans. Graph. 24, 3, 426-433.
29. WEISE, T., LI, H., GOOL, L. V., AND PAULY, M. 2009. Face/off: Live facial puppetry. In Symp. Computer Animation, 7-16.
30. WEISE, T., BOUAZIZ, S., LI, H., AND PAULY, M. 2011. Realtime performance-based facial animation. ACM Trans. Graph. 30, 4 (July), 77:1-77:10.
31. WILLIAMS, L. 1990. Performance driven facial animation. In Proceedings of SIGGRAPH, 235-242.
32. XIAO, J., CHAI, J., AND KANADE, T. 2006. A closed-form so- lution to non-rigid shape and motion recovery. Int. J. Computer Vision 67, 2, 233-246.
33. YANG, F., WANG, J., SHECHTMAN, E., BOURDEV, L., AND METAXAS, D. 2011. Expression flow for 3D-aware face com- ponent transfer. ACM Trans. Graph. 30, 4, 60:1-60:10.
34. ZHANG, L., SNAVELY, N., CURLESS, B., AND SEITZ, S. M. 2004. Spacetime faces: high resolution capture for modeling and animation. ACM Trans. Graph. 23, 3, 548-558.
35. ZHANG, Z. 2000. A flexible new technique for camera calibration. IEEE Trans. Pattern Anal. Mach. Intell. 22, 11, 1330-1334.