Realtime performance-based facial animation

ACM Transactions on Graphics

Volumn 30, Issue 4, 2011, Pages

  Weise, Thibaut ^a   Bouaziz, Sofien ^a   Pauly, Hao Li Mark ^a  

^a EPFL   (Switzerland)

Author keywords

Blendshape animation; Face animation; Markerless performance capture; Realtime tracking

Indexed keywords

2D IMAGES; 3D SENSOR; ACQUISITION DEVICE; BLENDSHAPE ANIMATION; CHARACTER ANIMATION; DEPTH MAP; FACE ANIMATION; FACE TRACKING; FACIAL ANIMATION; FACIAL EXPRESSIONS; GAMEPLAY; HIGH NOISE LEVELS; LOW QUALITIES; MARKERLESS; MAXIMUM A POSTERIORI ESTIMATION; NATURAL ENVIRONMENTS; NEW APPLICATIONS; NON-INTRUSIVE; PARAMETER SPACES; REAL TIME; REAL TIME TRACKING; SOCIAL INTERACTIONS; TEMPORAL COHERENCE;

INTERACTIVE COMPUTER GRAPHICS; THREE DIMENSIONAL;

ANIMATION;

EID: 80051884182     PISSN: 07300301     EISSN: 15577368     Source Type: Journal    
DOI: 10.1145/1964921.1964972     Document Type: Article

References (40)

1. ALEXANDER, O., ROGERS, M., LAMBETH, W., CHANG, M., AND DEBEVEC, P. 2009. The digital emily project: photoreal facial modeling and animation. ACM SIGGRAPH 2009 Courses.
2. BEELER, T., BICKEL, B., BEARDSLEY, P., SUMNER, B., AND GROSS, M. 2010. High-quality single-shot capture of facial geometry. ACM Trans. Graph. 29, 40:1-40:9.
3. BLACK, M. J., AND YACOOB, Y. 1995. Tracking and recognizing rigid and non-rigid facial motions using local parametric models of image motion. In ICCV, 374-381.
4. BLANZ, V., AND VETTER, T. 1999. A morphable model for the synthesis of 3d faces. In Proc. SIGGRAPH 99.
5. BORSHUKOV, G., PIPONI, D., LARSEN, O., LEWIS, J. P., AND TEMPELAAR-LIETZ, C. 2005. Universal capture - image-based facial animation for "the matrix reloaded". In SIGGRAPH 2005 Courses.
6. BRADLEY, D., HEIDRICH, W., POPA, T., AND SHEFFER, A. 2010. High resolution passive facial performance capture. ACM Trans. Graph. 29, 41:1-41:10.
7. CHAI, J. X., XIAO, J., AND HODGINS, J. 2003. Vision-based control of 3d facial animation. In SCA.
8. CHUANG, E., AND BREGLER, C. 2002. Performance driven facial animation using blendshape interpolation. Tech. rep., Stanford University.
9. COOTES, T., EDWARDS, G., AND TAYLOR, C. 2001. Active appearance models. PAMI 23, 681-685. (Pubitemid 32585279)
10. COVELL, M. 1996. Eigen-points: Control-point location using principle component analyses. In FG '96.
11. DECARLO, D., AND METAXAS, D. 1996. The integration of optical flow and deformable models with applications to human face shape and motion estimation. In CVPR.
12. DECARLO, D., AND METAXAS, D. 2000. Optical flow constraints on deformable models with applications to face tracking. IJCV 38, 99-127.
13. EKMAN, P., AND FRIESEN, W. 1978. Facial Action Coding System: A Technique for the Measurement of Facial Movement. Consulting Psychologists Press.
14. ESSA, I., BASU, S., DARRELL, T., AND PENTLAND, A. 1996. Modeling, tracking and interactive animation of faces and heads using input from video. In Proc. Computer Animation.
15. FURUKAWA, Y., AND PONCE, J. 2009. Dense 3d motion capture for human faces. In CVPR.
16. GROCHOW, K., MARTIN, S. L., HERTZMANN, A., AND POPOVIĆ , Z. 2004. Style-based inverse kinematics. ACM Trans. Graph. 23, 522-531.
17. GUENTER, B., GRIMM, C., WOOD, D., MALVAR, H., AND PIGHIN, F. 1993. Making faces. IEEE Computer Graphics and Applications 13, 6-8.
18. IKEMOTO, L., ARIKAN, O., AND FORSYTH, D. 2009. Generalizing motion edits with gaussian processes. ACM Trans. Graph. 28, 1:1-1:12.
19. LAU, M., CHAI, J., XU, Y.-Q., AND SHUM, H.-Y. 2007. Face poser: interactive modeling of 3d facial expressions using model priors. In SCA.
20. LI, H., ROIVAINEN, P., AND FORCHEIMER, R. 1993. 3-d motion estimation in model-based facial image coding. PAMI 15, 545-555. (Pubitemid 23675489)
21. LI, H., ADAMS, B., GUIBAS, L. J., AND PAULY, M. 2009. Robust single-view geometry and motion reconstruction. ACM Trans. Graph. 28, 175:1-175:10.
22. LI, H., WEISE, T., AND PAULY, M. 2010. Example-based facial rigging. ACM Trans. Graph. 29, 32:1-32:6.
23. LIN, I.-C., AND OUHYOUNG, M. 2005. Mirror mocap: Automatic and efficient capture of dense 3d facial motion parameters from video. The Visual Computer 21, 6, 355-372. (Pubitemid 41202489)
24. LOU, H., AND CHAI, J. 2010. Example-based human motion denoising. IEEE Trans. on Visualization and Computer Graphics 16, 870-879.
25. LU, P., NOCEDAL, J., ZHU, C., BYRD, R. H., AND BYRD, R. H. 1994. A limited-memory algorithm for bound constrained optimization. SIAM Journal on Scientific Computing.
26. MA, W.-C., HAWKINS, T., PEERS, P., CHABERT, C.-F., WEISS, M., AND DEBEVEC, P. 2007. Rapid acquisition of specular and diffuse normal maps from polarized spherical gradient illumination. In EUROGRAPHICS Symposium on Rendering.
27. MCLACHLAN, G. J., AND KRISHNAN, T. 1996. The EM Algorithm and Extensions. Wiley-Interscience.
28. PÉREZ, P., GANGNET, M., AND BLAKE, A. 2003. Poisson image editing. ACM Trans. Graph. 22, 313-318.
29. PIGHIN, F., AND LEWIS, J. P. 2006. Performance-driven facial animation. In ACM SIGGRAPH 2006 Courses.
30. PIGHIN, F., SZELISKI, R., AND SALESIN, D. 1999. Resynthesizing facial animation through 3d model-based tracking. ICCV 1, 143-150. (Pubitemid 32080032)
31. ROBERTS, S. 1959. Control chart tests based on geometric moving averages. In Technometrics, 239-250.
32. TIPPING, M. E., AND BISHOP, C. M. 1999. Probabilistic principal component analysis. Journal of the Royal Statistical Society, Series B.
33. TIPPING, M. E., AND BISHOP, C. M. 1999. Mixtures of probabilistic principal component analyzers. Neural Computation 11.
34. VIOLA, P., AND JONES, M. 2001. Rapid object detection using a boosted cascade of simple features. In CVPR.
35. WEISE, T., LEIBE, B., AND GOOL, L. V. 2008. Accurate and robust registration for in-hand modeling. In CVPR.
36. WEISE, T., LI, H., GOOL, L. V., AND PAULY, M. 2009. Face/off: Live facial puppetry. In SCA.
37. WILLIAMS, L. 1990. Performance-driven facial animation. In Comp. Graph. (Proc. SIGGRAPH 90).
38. WILSON, C. A., GHOSH, A., PEERS, P., CHIANG, J.-Y., BUSCH, J., AND DEBEVEC, P. 2010. Temporal upsampling of performance geometry using photometric alignment. ACM Trans. Graph. 29, 17:1-17:11.
39. ZHANG, S., AND HUANG, P. 2004. High-resolution, real-time 3d shape acquisition. In CVPR Workshop.
40. ZHANG, L., SNAVELY, N., CURLESS, B., AND SEITZ, S. M. 2004. Spacetime faces: high resolution capture for modeling and animation. ACM Trans. Graph. 23, 548-558.