Automatic creation of a talking head from a video sequence

IEEE Transactions on Multimedia

Volumn 7, Issue 4, 2005, Pages 628-637

  Choi, Kyoung Ho ^a   Hwang, Jenq Neng ^b  

^a Mokpo National University   (South Korea)

^b University of Washington   (United States)

Author keywords

MPEG 4 facial object; Probabilistic approach; Speech driven talking heads; Talking heads; Virtual face

Indexed keywords

MPEG-4 FACIAL OBJECTS; PROBABILISTIC APPROACH; SPEECH-DRIVEN TALKING HEADS; VIRTUAL FACE;

ANIMATION; COMPUTATIONAL COMPLEXITY; LEAST SQUARES APPROXIMATIONS; PROBABILITY; VECTORS; VIDEO CONFERENCING;

REAL TIME SYSTEMS;

EID: 24944588116     PISSN: 15209210     EISSN: None     Source Type: Journal    
DOI: 10.1109/TMM.2005.850964     Document Type: Article

References (30)

1. W.-S. Lee, M. Escher, G. Sannier, and N. Magnenat-Thalmann, "MPEG-4 compatible faces from orthogonal photos," in Proc. Int. Conf. Computer Animation, 1999, pp. 186-194.
2. P. Fua and C. Miccio, "Animated heads from ordinary images: a least-squares approach," Comput. Vis. Image Understand., vol. 75, no. 3, pp. 247-259, 1999.
3. F. Pighin, R. Szeliski, and D. H. Salesin, "Resynthesizing facial animation through 3-D model-based tracking," in Proc. 7th IEEE Int. Conf. Computer Vision, vol. 1, 1999, pp. 143-150.
4. Z. Liu, Z. Zhang, C. Jacobs, and M. Cohen, "Rapid Modeling of Animated Faces From Video,", Tech.l Rep. MSR-TR-2000-11.
5. A. C. A. del Valle and J. Ostermann, "3-D talking head customization by adapting a generic model to one uncalibrated picture," in Proc. IEEE Int. Symp. Circuits and Systems, 2001, pp. 325-328.
6. C. J. Kuo, R.-S. Huang, and T.-G. Lin, "3-D facial model estimation from single front-view facial image," IEEE Trans. Circuits Syst. Video Technol., vol. 12, no. 3, pp. 183-192, Mar. 2002.
7. L. Moccozet and N. Magnenat Thalmann, "Dirichlet free-from deformations and their application to hand simulation," in Proc. Computer Animation 97, 1997, pp. 93-102.
8. V. Blanz and T. Vetter, "A morphable model for the synthesis of 3-D faces," in Computer Graphics, Annu. Conf. Series, SIGGRAPH 1999, pp. 187-194.
9. E. Cosatto and H. P. Graf, "Photo-realistic talking-heads from image samples," IEEE Trans. Multimedia, vol. 2, no. 3, pp. 152-163, Jun. 2000.
10. I.-C. Lin, C.-S. Hung, T.-J. Yang, and M. Ouhyoung, "A speech driven talking head system based on a single face image," in Proc. 7th Pacific Conf. Computer Graphics and Applications, 1999, pp. 43-49.
11. http://www.ananova.com/ [Online]
12. R.-S. Wang and Y. Wang, "Facial feature extraction and tracking in video sequences," in Proc. IEEE Int. Workshop on Multimedia Signal Processing, 1997, pp. 233-238.
13. D. Reisfeld and Y. Yeshurun, "Robust detection of facial features by generalized symmetry," in Proc. 11th IAPR Int. Conf. Pattern Recognition, 1992, pp. 117-120.
14. M. Zobel, A. Gebhard, D. Paulus, J. Denzler, and H. Niemann, "Robust facial feature localization by coupled features," in Proc. 4th IEEE Int. Conf. Automatic Face and Gesture Recognition, 2000, pp. 2-7.
15. Y. Tian, T. Kanade, and J. Cohn, "Robust lip tracking by combining shape, color and motion," in Proc. 4th Asian Conf. Computer Vision, 2000.
16. J. Luettin, N. A. Tracker, and S. W. Beet, "Active shape models for visual speech feature extraction," University of Sheffield, Sheffield, U.K., Electronic Systems Group Rep. 95/44, 1995.
17. C. Kim and J.-N. Hwang, "An integrated scheme for object-based video abstraction," in Proc. ACM Int. Multimedia Conf., 2000.
18. L. G. Farkas, Anthropometry of the Head and Face. New York: Raven, 1994.
19. K. C. Yow and R. Cipolla, "A probabilistic framework for perceptual grouping of features for human face detection," in Proc. IEEE Int. Conf. Automatic Face and Gesture Recognition '96, 1996, pp. 16-21.
20. H. Tao, R. Lopez, and T. Huang, "Tracking facial features using probabilistic network," Auto. Face Gesture Recognit., pp. 166-170, 1998.
21. ISO/IEC FDIS 14496-1 Systems, ISO/IEC JTC1/SC29/WG11 N2501, Nov. 1998.
22. ISO/IEC FDIS 14496-2 Visual, ISO/IEC JTC1/SC29/WG11 N2502, Nov. 1998.
23. Psychological Image Collection at Stirling (PICS). [Online] Available: http://pics.psych.stir.ac.uk/
24. J. Luettin, N. A. Tracker, and S. W. Beet, "Active shape models for visual speech feature extraction," University of Sheffield, Sheffield, U.K., Electronic Systems Group Rep. 95/44, 1995.
25. K. H. Choi and J.-N. Hwang, "Creating 3-D speech-driven talking heads: a probabilistic approach," in Proc. IEEE Int. Conf. Image Processing, 2002, pp. 984-987.
26. F. Lavagetto, "Converting speech into lip movement: A multimedia telephone for hard of hearing people," IEEE Trans. Rehabil. Eng., vol. 3, no. 1, pp. 90-102, Jan. 1995.
27. R. R. Rao, T. Chen, and R. M. Mersereau, "Audio-to-visual conversion for multimedia communication," IEEE Trans. Ind. Electron., vol. 45, no. 1. pp. 15-22, Feb. 1998.
28. F. I. Parke and K. Waters, Computer Facial Animation. Wellesley, MA: A. K. Peters, 1996.
29. Dual Rate Speech Coder for Multimedia Communications Transmitting at 5.3 and 6.3 kbits/s, ITU-T Recommendation G.723.1, Mar. 1996.
30. K. H. Choi and J.-N. Hwang, "A real-time system for automatic creation of 3-D face models from a video sequence," in Proc. IEEE Int. Conf. Acoustics, Speech, and Signal Processing, 2002, pp. 2121-2124.