Dynamic 3D avatar creation from hand-held video input

ACM Transactions on Graphics

Volumn 34, Issue 4, 2015, Pages

  Ichim, Alexandru Eugen ^a   Bouazizy, Sofien ^a   Paulyz, Mark ^a  

^a EPFL   (Switzerland)

Author keywords

3D avatar creation; Blendshapes; Face animation; Rigging

Indexed keywords

ANIMATION; INTERACTIVE COMPUTER GRAPHICS;

3D AVATARS; ACQUISITION SYSTEMS; BLENDSHAPES; COMPUTER ANIMATION; FACE ANIMATION; ON-LINE COMMUNICATION; REAL-TIME APPLICATION; RIGGING;

THREE DIMENSIONAL COMPUTER GRAPHICS;

EID: 84947229810     PISSN: 07300301     EISSN: 15577368     Source Type: Journal    
DOI: 10.1145/2766974     Document Type: Conference Paper

References (54)

1. ALEXANDER, O., ROGERS, M., LAMBETH, W., CHIANG, M., AND DEBEVEC, P. 2009. Creating a photoreal digital actor: The digital emily project. In Visual Media Production, 2009. CVMP'09. Conference for.
2. ALEXANDER, O., FYFFE, G., BUSCH, J., YU, X., ICHIKARI, R., JONES, A., DEBEVEC, P., JIMENEZ, J., DANVOYE, E., ANTIONAZZI, B., EHELER, M., KYSELA, Z., AND VON DER PAHLEN, J. 2013. Digital ira: Creating a real-time photoreal digital actor. In ACM SIGGRAPH 2013 Posters.
3. AMBERG, B., BLAKE, A., FITZGIBBON, A. W., ROMDHANI, S., AND VETTER, T. 2007. Reconstructing high quality face-surfaces using model based stereo. In ICCV.
4. BEELER, T., BICKEL, B., BEARDSLEY, P., SUMNER, B., AND GROSS, M. 2010. High-quality single-shot capture of facial geometry. ACM Transactions on Graphics (TOG).
5. BEELER, T., HAHN, F., BRADLEY, D., BICKEL, B., BEARDSLEY, P., GOTSMAN, C., SUMNER, R. W., AND GROSS, M. 2011. High-quality passive facial performance capture using anchor frames. ACM Trans. Graph.
6. BEELER, T., BICKEL, B., NORIS, G., BEARDSLEY, P., MARSCHNER, S., SUMNER, R. W., AND GROSS, M. 2012. Coupled 3d reconstruction of sparse facial hair and skin. ACM Trans. Graph.
7. BÉRARD, P., BRADLEY, D., NITTI, M., BEELER, T., AND GROSS, M. 2014. High-quality capture of eyes. ACM Trans. Graph. 33, 6 (Nov.), 223:1-223:12.
8. BERMANO, A. H., BRADLEY, D., BEELER, T., ZÜND, F., NOWROUZEZAHRAI, D., BARAN, I., SORKINE, O., PFISTER, H., SUMNER, R. W., BICKEL, B., AND GROSS, M. 2014. Facial performance enhancement using dynamic shape space analysis. ACM Trans. Graph.
9. BICKEL, B., LANG, M., BOTSCH, M., OTADUY, M. A., AND GROSS, M. H. 2008. Pose-space animation and transfer of facial details. In Symposium on Computer Animation.
10. BLANZ, V., AND VETTER, T. 1999. A morphable model for the synthesis of 3d faces. In Proceedings of the 26th annual conference on Computer graphics and interactive techniques.
11. BOTSCH, M., KOBBELT, L., PAULY, M., ALLIEZ, P., AND LEVY, B. 2010. Polygon Mesh Processing. AK Peters.
12. BOUAZIZ, S., WANG, Y., AND PAULY, M. 2013. Online modeling for realtime facial animation. ACM Trans. Graph.
13. BOUAZIZ, S., TAGLIASACCHI, A., AND PAULY, M. 2014. Dynamic 2d/3d registration. Eurographics Tutorial.
14. BUNNELL, M. 2005. Dynamic ambient occlusion and indirect lighting. Gpu gems.
15. CAO, X., WEI, Y., WEN, F., AND SUN, J. 2012. Face alignment by explicit shape regression. In CVPR.
16. CAO, C., WENG, Y., LIN, S., AND ZHOU, K. 2013. 3d shape regression for real-time facial animation. ACM Trans. Graph.
17. CAO, C., HOU, Q., AND ZHOU, K. 2014. Displaced dynamic expression regression for real-time facial tracking and animation. ACM Trans. Graph.
18. CAO, C., WENG, Y., ZHOU, S., TONG, Y., AND ZHOU, K. 2014. Facewarehouse: A 3d facial expression database for visual computing. IEEE Transactions on Visualization and Computer Graphics.
19. CHAI, M., ZHENG, C., AND ZHOU, K. 2014. A reduced model for interactive hairs. ACM Transactions on Graphics (July).
20. CHAMBOLLE, A., CASELLES, V., CREMERS, D., NOVAGA, M., AND POCK, T. 2010. An introduction to total variation for image analysis. Theoretical foundations and numerical methods for sparse recovery 9, 263-340.
21. CHARTRAND, R., AND YIN, W. 2008. Iteratively reweighted algorithms for compressive sensing. In Acoustics, speech and signal processing, 2008. ICASSP 2008. IEEE international conference on, IEEE, 3869-3872.
22. DUDA, R. O., AND HART, P. E. 1972. Use of the hough transformation to detect lines and curves in pictures. Commun. ACM.
23. FROLOVA, D., SIMAKOV, D., AND BASRI, R. 2004. Accuracy of spherical harmonic approximations for images of lambertian objects under far and near lighting. In Computer Vision-ECCV 2004.
24. FU, W. J. 1998. Penalized Regressions: The Bridge versus the Lasso. J. Comp. Graph. Stat.
25. FURUKAWA, Y., AND PONCE, J. 2010. Accurate, dense, and robust multiview stereopsis. IEEE Trans. Pattern Anal. Mach. Intell.
26. GARRIDO, P., VALGAERTS, L., WU, C., AND THEOBALT, C. 2013. Reconstructing detailed dynamic face geometry from monocular video. ACM Transactions on Graphics.
27. GHOSH, A., FYFFE, G., TUNWATTANAPONG, B., BUSCH, J., YU, X., AND DEBEVEC, P. 2011. Multiview face capture using polarized spherical gradient illumination. In Proc. of ACM SIGGRAPH Asia.
28. GONZALEZ, R. C., AND WOODS, R. E. 2006. Digital Image Processing (3rd Edition). Prentice-Hall, Inc.
29. GRAY, R. M. 2006. Toeplitz and circulant matrices: A review. now publishers Inc.
30. HU, L., MA, C., LUO, L., AND LI, H. 2014. Robust hair capture using simulated examples. ACM Transactions on Graphics.
31. HUANG, H., CHAI, J., TONG, X., AND WU, H.-T. 2011. Leveraging motion capture and 3d scanning for high-fidelity facial performance acquisition. ACM Trans. Graph. (Proc. SIGGRAPH).
32. JIMENEZ, J., ECHEVARRIA, J. I., OAT, C., AND GUTIERREZ, D. 2011. GPU Pro 2. AK Peters Ltd., ch. Practical and Realistic Facial Wrinkles Animation.
33. KEMELMACHER-SHLIZERMAN, I., AND BASRI, R. 2011. 3d face reconstruction from a single image using a single reference face shape. Pattern Analysis and Machine Intelligence, IEEE Transactions on.
34. LEWIS, J. P., ANJYO, K., RHEE, T., ZHANG, M., PIGHIN, F., AND DENG, Z. 2014. Practice and Theory of Blendshape Facial Models. In EG - STARs.
35. LI, H., ADAMS, B., GUIBAS, L. J., AND PAULY, M. 2009. Robust single-view geometry and motion reconstruction. ACM Trans. Graph.
36. LI, H., YU, J., YE, Y., AND BREGLER, C. 2013. Realtime facial animation with on-the-fly correctives. ACM Transactions on Graphics.
37. LI, J., XU, W., CHENG, Z., XU, K., AND KLEIN, R. 2015. Lightweight wrinkle synthesis for 3d facial modeling and animation. Computer-Aided Design 58, 0, 117-122. Solid and Physical Modeling 2014.
38. MA, W.-C., JONES, A., CHIANG, J.-Y., HAWKINS, T., FREDERIKSEN, S., PEERS, P., VUKOVIC, M., OUHYOUNG, M., AND DEBEVEC, P. 2008. Facial performance synthesis using deformation-driven polynomial displacement maps. Proc. of ACM SIGGRAPH Asia.
39. OAT, C. 2007. Animated wrinkle maps. In ACM SIGGRAPH 2007 courses.
40. PÉREZ, P., GANGNET, M., AND BLAKE, A. 2003. Poisson image editing. ACM Trans. Graph.
41. SARAGIH, J. M., LUCEY, S., AND COHN, J. F. 2009. Face alignment through subspace constrained mean-shifts. In Computer Vision, 2009 IEEE 12th International Conference on.
42. SARAGIH, J. M., LUCEY, S., AND COHN, J. F. 2011. Deformable model fitting by regularized landmark mean-shift. Int. J. Comput. Vision.
43. SHI, F., WU, H.-T., TONG, X., AND CHAI, J. 2014. Automatic acquisition of high-fidelity facial performances using monocular videos. ACM Trans. Graph. 33, 6 (Nov.), 222:1-222:13.
44. SUMNER, R. W., AND POPOVIĆ , J. 2004. Deformation transfer for triangle meshes. ACM Trans. Graph.
45. VALGAERTS, L., WU, C., BRUHN, A., SEIDEL, H.-P., AND THEOBALT, C. 2012. Lightweight binocular facial performance capture under uncontrolled lighting. Proc. of ACM SIGGRAPH Asia.
46. VENKATARAMAN, K., LODHA, S., AND RAGHAVAN, R. 2005. A kinematic-variational model for animating skin with wrinkles. Computers & Graphics.
47. VLASIC, D., BRAND, M., PFISTER, H., AND POPOVIĆ, J. 2005. Face transfer with multilinear models.
48. WEISE, T., LI, H., VAN GOOL, L., AND PAULY, M. 2009. Face/off: Live facial puppetry. ACM Trans. Graph.
49. WEISE, T., BOUAZIZ, S., LI, H., AND PAULY, M. 2011. Realtime performance-based facial animation. In ACM SIGGRAPH 2011 Papers.
50. WU, Y., KALRA, P., AND THALMANN, N. M. 1996. Simulation of static and dynamic wrinkles of skin. In Proc. of IEEE Computer Animation.
51. WU, C., ZOLLHÖFER, M., NIESSNER, M., STAMMINGER, M., IZADI, S., AND THEOBALT, C. 2014. Real-time shading-based refinement for consumer depth cameras. ACM Trans. Graph. 33, 6 (Nov.), 200:1-200:10.
52. WU, C. 2013. Towards linear-time incremental structure from motion. In 3D Vision, 2013 International Conference on.
53. ZACH, C., POCK, T., AND BISCHOF, H. 2007. A duality based approach for realtime tv-l 1 optical flow. In Pattern Recognition. Springer, 214-223.
54. ZHANG, L., SNAVELY, N., CURLESS, B., AND SEITZ, S. M. 2004. Spacetime faces: High-resolution capture for modeling and animation. In ACM Annual Conference on Computer Graphics.