High fidelity facial animation capture and retargeting with contours

Proceedings - SCA 2013: 12th ACM SIGGRAPH / Eurographics Symposium on Computer Animation

Volumn , Issue , 2013, Pages 7-14

  Bhat, Kiran S ^a   Goldenthal, Rony ^a   Ye, Yuting ^a   Mallet, Ronald ^a   Koperwas, Michael ^a  

^a Industrial Light and Magic ^*  (United States)

Author keywords

Blendshape; Contours; Correctives; Curves; Facial animation; Facial retargeting; Performance capture

Indexed keywords

BLENDSHAPE; CONTOURS; CORRECTIVES; CURVES; FACIAL ANIMATION; FACIAL RETARGETING; PERFORMANCE CAPTURE;

INTERACTIVE COMPUTER GRAPHICS;

ANIMATION;

EID: 84883120938     PISSN: None     EISSN: None     Source Type: Conference Proceeding    
DOI: 10.1145/2485895.2485915     Document Type: Conference Paper

References (45)

1. Alexander, O., Rogers, M., Lambeth, W., Chiang, J.-Y., Ma, W.-C., Wang, C.-C., and Debevec, P. 2010. The digital emily project: Achieving a photorealistic digital actor. Computer Graphics and Applications, IEEE 30, 4 (july-aug.), 20-31.
2. Baltrusaitis, T., Robinson, P., and Morency, L.-P. 2012. 3D constrained local model for rigid and non-rigid facial tracking. In Computer Vision and Pattern Recognition (CVPR 2012).
3. Bascle, B., and Blake, A. 1998. Separability of pose and expression in facial tracking and animation. In Computer Vision, 1998. Sixth International Conference on, IEEE, 323-328.
4. Beeler, T., Hahn, F., Bradley, D., Bickel, B., Beardsley, P., Gotsman, C., Sumner, R. W., and Gross, M. 2011. High-quality passive facial performance capture using anchor frames. ACM Trans. Graph. 30 (August), 75:1-75:10.
5. Bickel, B., Botsch, M., Angst, R., Matusik, W., Otaduy, M., Pfister, H., and Gross, M. 2007. Multi-scale capture of facial geometry and motion. ACM Trans. Graph. 26, 3 (July).
6. Black, M. J., AND Yacoob, Y. 1995. Tracking and recognizing rigid and non-rigid facial motions using local parametric models of image motion. In Proceedings of the Fifth Interna-tional Conference on Computer Vision, IEEE Computer Society, Washington, DC, USA, ICCV '95, 374.
7. Borshukov, G., Piponi, D., Larsen, O., Lewis, J. P., AND Tempelaar-lietz, C. 2005. Universal capture-image-based facial animation for "the matrix reloaded". In ACM SIGGRAPH 2005 Courses, ACM, New York, NY, USA, SIGGRAPH '05.
8. Botsch, M., AND Sorkine, O. 2008. On linear variational surface deformation methods. IEEE Transactions on Visualization and Computer Graphics 14, 1 (Jan.), 213-230.
9. Bradley, D., Heidrich, W., Popa, T., AND Sheffer, A. 2010. High resolution passive facial performance capture. ACM Trans. Graph. 29, 4 (July), 41:1-41:10.
10. Bregler, C., Bhat, K., Saltzman, J., AND Allen, B. 2009. Ilm's multitrack: a new visual tracking framework for high-end vfx production. In SIGGRAPH 2009: Talks, ACM, New York, NY, USA, SIGGRAPH '09, 29:1-29:1.
11. Chuang, E., AND Bregler, C. 2002. Performance driven facial animation using blendshape interpolation. Tech. rep., Stanford University.
12. Cootes, T. F., Edwards, G. J., AND Taylor, C. J. 1998. Active appearance models. In IEEE Transactions on Pattern Anal-ysis and Machine Intelligence, Springer, 484-498.
13. Covell, M., AND Bregler, C. 1996. Eigen-points. In Im-age Processing, 1996. Proceedings., International Conference on, vol. 3, IEEE, 471-474.
14. Cunningham, D. W., Kleiner, M., Wallraven, C., AND Bülthoff, H. H. 2005. Manipulating video sequences to determine the components of conversational facial expressions. ACM Trans. Appl. Percept. 2, 3 (July), 251-269.
15. DIMENSIONAL IMAGING LTD. DI4DTM. http://www.di3d.com/.
16. DYNAMIXYZ. HMC & GrabberTM. http://www.dynamixyz.com.
17. Ekman, P. 1993. Facial expression and emotion. American Psy-chologist 48, 4, 384-392.
18. Fyffe, G., Hawkins, T., Watts, C., Ma, W.-C., AND Debevec, P. 2011. Comprehensive facial performance capture. In Eurographics 2011.
19. Ghosh, A., Fyffe, G., Tunwattanapong, B., Busch, J., Yu, X., AND Debevec, P. 2011. Multiview face capture using polarized spherical gradient illumination. ACM Trans. on Graphics (Proc. SIGGRAPH Asia).
20. Horn, B., and Schunck, B. 1981. Determining optical flow. Artificial intelligence 17, 1, 185-203.
21. Huang, H., Chai, J., Tong, X., and Wu, H.-T. 2011. Leveraging motion capture and 3d scanning for high-fidelity facial performance acquisition. ACM Trans. Graph. 30, 4 (July), 74:1-74:10.
22. IMAGE METRICS. Live DriverTM. http://www.image-metrics.com.
23. Lau, M., Chai, J., Xu, Y.-Q., AND Shum, H.-Y. 2009. Face poser: Interactive modeling of 3d facial expressions using facial priors. ACM Trans. Graph. 29, 1 (Dec.), 3:1-3:17.
24. Li, H., Adams, B., Guibas, L. J., and Pauly, M. 2009. Robust single-view geometry and motion reconstruction. ACM Transac-tions on Graphics (Proceedings SIGGRAPH Asia 2009) 28, 5.
25. Li, H., Weise, T., AND Pauly, M. 2010. Example-based facial rigging. ACM Transactions on Graphics (Proceedings SIG-GRAPH 2010) 29, 3 (July).
26. LI, H., Yu, J., YE, Y., AND Bregler, C. 2013. Realtime facial animation with on-the-fly correctives. ACM Transactions on Graphics (Proc. SIGGRAPH) 32, 4.
27. Liu, X., Mao, T., Xia, S., Yu, Y., AND Wang, Z. 2008. Facial animation by optimized blendshapes from motion capture data. Computer Animation and Virtual Worlds 19, 3-4, 235-245.
28. Ma, W.-C., Jones, A., Chiang, J.-Y., Hawkins, T., Frederiksen, S., Peers, P., Vukovic, M., Ouhyoung, M., AND Debevec, P. 2008. Facial performance synthesis using deformation-driven polynomial displacement maps. ACM Trans. on Graphics (Proc. SIGGRAPH Asia).
29. Meyer, M., Desbrun, M., Schröder, P., AND Barr, A. H. 2002. Discrete differential-geometry operators for triangulated 2-manifolds. In Proc. VisMath, 35-57.
30. Miranda, J. C., Alvarez, X., Orvalho, J. A., Gutierrez, D., Sousa, A. A., AND Orvalho, V. 2011. Sketch express: facial expressions made easy. In Proceedings of the Eighth Euro-graphics Symposium on Sketch-Based Interfaces and Modeling, ACM, New York, NY, USA, SBIM '11, 87-94.
31. MOVA. CONTOURTMReality Capture. http://www.mova.com/.
32. Nusseck, M., Cunningham, D. W., Wallraven, C., AND Bülthoff, H. H. 2008. The contribution of different facial regions to the recognition of conversational expressions. Journal of Vision 8, 8.
33. Orvalho, V., Bastos, P., Parke, F., Oliveira, B., AND Alvarez, X. 2012. A facial rigging survey. In in Proc. of the 33rd Annual Conference of the European Association for Computer Graphics-Eurographics, ACM, 10-32.
34. Pighin, F. H., Szeliski, R., AND Salesin, D. 1999. Resynthesizing Facial Animation through 3D Model-based Tracking. In Proc. 7th International Conference on Computer Vision, Kerkyra, Greece, 143-150.
35. Sagar, M. 2006. Facial performance capture and expressive translation for king kong. In ACM SIGGRAPH 2006 Sketches, ACM, New York, NY, USA, SIGGRAPH '06.
36. Seo, J., Irving, G., Lewis, J. P., and Noh, J. 2011. Compression and direct manipulation of complex blendshape models. ACM Trans. Graph. 30, 6 (Dec.), 164:1-164:10.
37. Seol, Y., Lewis, J., Seo, J., Choi, B., Anjyo, K., AND Noh, J. 2012. Spacetime expression cloning for blendshapes. ACM Trans. Graph. 31, 2 (Apr.), 14:1-14:12.
38. Sifakis, E., Neverov, I., AND Fedkiw, R. 2005. Automatic determination of facial muscle activations from sparse motion capture marker data. ACM Trans. Graph. 24, 3 (July), 417-425.
39. Sumner, R. W., AND Popović, J. 2004. Deformation transfer for triangle meshes. ACM Trans. Graph. 23, 3 (Aug.), 399-405.
40. Terzopoulos, D., AND Waters, K. 1993. Analysis and synthesis of facial image sequences using physical and anatomical models. Pattern Analysis and Machine Intelligence, IEEE Trans-actions on 15, 6, 569-579.
41. Weise, T., LI, H., Gool, L. V., AND Pauly, M. 2009.Face/off: Live facial puppetry. In Proceedings of the 2009 ACM SIGGRAPH/Eurographics Symposium on Computer animation (Proc. SCA'09), Eurographics Association, ETH Zurich.
42. Weise, T., Bouaziz, S., LI, H., AND Pauly, M. 2011. Realtime performance-based facial animation. ACM Transactions on Graphics (Proceedings SIGGRAPH 2011) 30, 4 (July).
43. Williams, L. 1990. Performance-driven facial animation. SIG-GRAPH Comput. Graph. 24, 4 (Sept.), 235-242.
44. Zhang, S., AND Huang, P. 2004. High-resolution, realtime 3d shape acquisition. In Proceedings of the 2004 Con-ference on Computer Vision and Pattern Recognition Workshop (CVPRW'04) Volume 3-Volume 03, IEEE Computer Society, Washington, DC, USA, CVPRW '04, 28.
45. ZHANG, L., SNAVELY, N., CURLESS, B., AND SEITZ, S. M. 2004. Spacetime faces: High-resolution capture for modeling and animation. In ACM Annual Conference on Computer Graph-ics, 548-558.