Video face replacement

Proceedings of the 2011 SIGGRAPH Asia Conference, SA'11

Volumn , Issue , 2011, Pages

  Dale, Kevin ^a   Sunkavalli, Kalyan ^a   Johnson, Micah K ^b   Vlasic, Daniel ^c   Matusik, Wojciech ^b,d   Pfister, Hanspeter ^a  

^a HARVARD UNIVERSITY   (United States)

^b MASSACHUSETTS INSTITUTE OF TECHNOLOGY   (United States)

^c Lantos Technologies   (United States)

^d DISNEY RESEARCH   (United States)

Author keywords

Face replacement; Facial animation; Video compositing

Indexed keywords

3D GEOMETRY; FACE REPLACEMENT; FACIAL ANIMATION; MANUAL OPERATIONS; SINGLE CAMERAS; TEMPORAL CONSISTENCY; USER STUDY; VIDEO COMPOSITING; VIDEO FOOTAGE; VISUAL APPEARANCE;

ANIMATION; INTERACTIVE COMPUTER GRAPHICS; VISUALIZATION;

THREE DIMENSIONAL;

EID: 84855459783     PISSN: None     EISSN: None     Source Type: Conference Proceeding    
DOI: None     Document Type: Conference Paper

References (39)

1. AGARWALA, A., DONTCHEVA, M., AGRAWALA, M., DRUCKER, S., COLBURN, A., CURLESS, B., SALESIN, D., AND COHEN, M. 2004. Interactive digital photomontage. ACM Trans. Graphics (Proc. SIGGRAPH) 23, 3, 294-302.
2. ALEXANDER, O., ROGERS, M., LAMBETH, W., CHIANG, M., AND DEBEVEC, P. 2009. The digital emily project: Photoreal facial modeling and animation. In ACM SIGGRAPH 2009 Courses, 12:1-15.
3. BEELER, T., HAHN, F., BRADLEY, D., BICKEL, B., BEARDSLEY, P., GOTSMAN, C., SUMNER, B., AND GROSS, M. 2011 (to appear). High-quality passive facial performance capture using anchor frames. ACM Trans. Graphics (Proc. SIGGRAPH) 3, 27, 75:1-10.
4. BICKEL, B., BOTSCH, M., ANGST, R., MATUSIK, W., OTADUY, M., PFISTER, H., AND GROSS, M. 2007. Multi-scale capture of facial geometry and motion. ACM Trans. Graphics (Proc. SIGGRAPH) 26, 3, 33:1-10.
5. BITOUK, D., KUMAR, N., DHILLON, S., BELHUMEUR, P., AND NAYAR, S. K. 2008. Face swapping: Automatically replacing faces in photographs. ACM Trans. Graphics (Proc. SIGGRAPH) 27, 3, 39:1-8.
6. BLANZ, V., BASSO, C., POGGIO, T., AND VETTER, T. 2003. Reanimating faces in images and video. Computer Graphics Forum 22, 3, 641-650.
7. BLANZ, V., SCHERBAUM, K., VETTER, T., AND SEIDEL, H.-P. 2004. Exchanging faces in images. Computer Graphics Forum (Proc. Eurographics) 23, 3, 669-676.
8. BORSHUKOV, G., PIPONI, D., LARSEN, O., LEWIS, J., AND TEMPELAAR-LIETZ, C. 2003. Universal capture - Image-based facial animation for "The Matrix Reloaded". In ACM SIGGRAPH 2003 Sketches & Applications.
9. BOYKOV, Y., VEKSLER, O., AND ZABIH, R. 2001. Fast approximate energy minimization via graph cuts. IEEE Trans. Pattern Analysis and Machine Intelligence 23, 11, 1222-1239. (Pubitemid 33137959)
10. BRADLEY, D., HEIDRICH, W., POPA, T., AND SHEFFER, A. 2010. High resolution passive facial performance capture. ACM Trans. Graphics (Proc. SIGGRAPH), 4, 41:1-10.
11. BREGLER, C., COVELL, M., AND SLANEY, M. 1997. Video Rewrite: Driving visual speech with audio. In Proc. SIGGRAPH, 353-360.
12. DECARLO, D., AND METAXAS, D. 1996. The integration of optical flow and deformable models with applications to human face shape and motion estimation. In Proc. IEEE Conf. Computer Vision and Pattern Recognition (CVPR), 231-238.
13. ESSA, I., BASU, S., DARRELL, T., AND PENTLAND, A. 1996. Modeling, tracking and interactive animation of faces and heads: Using input from video. In Proc. Computer Animation, 68-79.
14. EVERINGHAM, M., SIVIC, J., AND ZISSERMAN, A. 2006. "Hello! My name is⋯ Buffy" - automatic naming of characters in TV video. In Proc. British Machine Vision Conference (BMVC), 899-908.
15. EZZAT, T., GEIGER, G., AND POGGIO, T. 2002. Trainable vide-orealistic speech animation. ACM Trans. Graphics (Proc. SIGGRAPH) 21, 3, 388-398.
16. FARBMAN, Z., HOFFER, G., LIPMAN, Y., COHENOR, D., AND LISCHINSKI, D. 2009. Coordinates for instant image cloning. ACM Trans. Graphics (Proc. SIGGRAPH) 28, 3, 67:1-9.
17. FLAGG, M., NAKAZAWA, A., ZHANG, Q., KANG, S. B., RYU, Y. K., ESSA, I., AND REHG, J. M. 2009. Human video textures. In Proc. Symp. Interactive 3D Graphics (I3D), 199-206.
18. GUENTER, B., GRIMM, C., WOOD, D., MALVAR, H., AND PIGHIN, F. 1998. Making faces. In Proc. SIGGRAPH, 55-66.
19. JAIN, A., THORMÄHLEN, T., SEIDEL, H.-P., AND THEOBALT, C. 2010. Moviereshape: Tracking and reshaping of humans in videos. ACM Trans. Graphics (Proc. SIGGRAPH Asia) 29, 5, 148:1-10.
20. JIA, J., SUN, J., TANG, C.-K., AND SHUM, H.-Y. 2006. Dragand-drop pasting. ACM Trans. Graphics (Proc. SIGGRAPH) 25, 3, 631-637.
21. JONES, A., GARDNER, A., BOLAS, M., MCDOWALL, I., AND DEBEVEC, P. 2006. Simulating spatially varying lighting on a live performance. In Proc. European Conf. Visual Media Production (CVMP), 127-133.
22. JOSHI, N., MATUSIK, W., ADELSON, E. H., AND KRIEGMAN, D. J. 2010. Personal photo enhancement using example images. ACM Trans. Graphics 29, 2, 12:1-15.
23. KEMELMACHER-SHLIZERMAN, I., SANKAR, A., SHECHTMAN, E., AND SEITZ, S. M. 2010. Being John Malkovich. In Proc. European Conf. Computer Vision (ECCV), 341-353.
24. KWATRA, V., SCHÖDL, A., ESSA, I., TURK, G., AND BOBICK, A. 2003. Graphcut textures: Image and video synthesis using graph cuts. ACM Trans. Graphics (Proc. SIGGRAPH) 22, 3, 277-286.
25. LEYVAND, T., COHENOR, D., DROR, G., AND LISCHINSKI, D. 2008. Data-driven enhancement of facial attractiveness. ACM Trans. Graphics (Proc. SIGGRAPH) 27, 3, 38:1-9.
26. LI, H., ADAMS, B., GUIBAS, L. J., AND PAULY, M. 2009. Robust single-view geometry and motion reconstruction. ACM Trans. Graphics (Proc. SIGGRAPH) 28, 5, 175:1-10.
27. MA, W.-C., JONES, A., CHIANG, J.-Y., HAWKINS, T., FREDERIKSEN, S., PEERS, P., VUKOVIC, M., OUHYOUNG, M., AND DEBEVEC, P. 2008. Facial performance synthesis using deformation-driven polynomial displacement maps. ACM Trans. Graphics (Proc. SIGGRAPH Asia) 27, 5, 121:1-10.
28. PÉREZ, P., GANGNET, M., AND BLAKE, A. 2003. Poisson image editing. ACM Trans. Graphics (Proc. SIGGRAPH) 22, 3, 313-318.
29. PIGHIN, F. H., SZELISKI, R., AND SALESIN, D. 1999. Resynthesizing facial animation through 3d model-based tracking. In Proc. IEEE Int. Conf. Computer Vision (ICCV), 143-150. (Pubitemid 32080032)
30. RABINER, L., AND JUANG, B.-H. 1993. Fundamentals of speech recognition. Prentice-Hall, Inc., Upper Saddle River, NJ, USA.
31. ROBERTSON, B. 2009. What's old is new again. Computer Graphics World 32, 1.
32. SINGULAR INVERSIONS INC., 2011. FaceGen Modeller manual. www.facegen.com.
33. SUNKAVALLI, K., JOHNSON, M. K., MATUSIK, W., AND PFISTER, H. 2010. Multi-scale image harmonization. ACM Trans. Graphics (Proc. SIGGRAPH) 29, 4, 125:1-10.
34. VIOLA, P. A., AND JONES, M. J. 2001. Robust real-time face detection. In Proc. IEEE Int. Conf. Computer Vision (ICCV), 747-755.
35. VLASIC, D., BRAND, M., PFISTER, H., AND POPOVIĆ, J. 2005. Face transfer with multilinear models. ACM Trans. Graphics (Proc. SIGGRAPH) 24, 3, 426-433.
36. WEISE, T., LI, H., GOOL, L. V., AND PAULY, M. 2009. Face/Off: Live facial puppetry. In Proc. SIGGRAPH/Eurographics Symp. Computer Animation, 7-16.
37. WILLIAMS, L. 1990. Performance-driven facial animation. Computer Graphics (Proc. SIGGRAPH) 24, 4, 235-242.
38. YANG, F., WANG, J., SHECHTMAN, E., BOURDEV, L., AND METAXAS, D. 2011. Expression flow for 3D-aware face component transfer. ACM Trans. Graphics (Proc. SIGGRAPH) 27, 3, 60:1-10.
39. ZHANG, L., SNAVELY, N., CURLESS, B., AND SEITZ, S. M. 2004. Spacetime faces: High resolution capture for modeling and animation. ACM Trans. Graphics 23, 3, 548-558.