Dynamic facial expression analysis and synthesis with MPEG-4 facial animation parameters

IEEE Transactions on Circuits and Systems for Video Technology

Volumn 18, Issue 10, 2008, Pages 1383-1396

  Zhang, Yongmian ^a   Ji, Qiang ^a   Zhu, Zhiwei ^b   Yi, Beifang ^c  

^a RENSSELAER POLYTECHNIC INSTITUTE   (United States)

^b SARNOFF CORPORATION   (United States)

^c STATE UNIVERSITY OF NEW YORK AT FREDONIA   (United States)

Author keywords

Bayesian networks (BNs); Facial animation; Facial expression synthesis; MPEG 4 facial animation parameters (FAPs)

Indexed keywords

ANIMATION; COMPUTER KEYBOARDS; DISTRIBUTED PARAMETER NETWORKS; FACE RECOGNITION; GESTURE RECOGNITION; IMAGE CODING; INFERENCE ENGINES; INTELLIGENT NETWORKS; MODAL ANALYSIS; MOTION PICTURE EXPERTS GROUP STANDARDS; PROGRAMMING THEORY; REAL TIME SYSTEMS; SPEECH ANALYSIS; SPEECH RECOGNITION; UNCERTAINTY ANALYSIS; VIDEO SIGNAL PROCESSING;

COHERENT STRUCTURES; DATA STREAMS; DATA TRANSMISSIONS; DYNAMIC BAYESIAN NETWORKS; DYNAMIC EVOLUTIONS; DYNAMIC FACIAL EXPRESSIONS; FACIAL ACTION CODING SYSTEMS; FACIAL ANIMATION; FACIAL ANIMATION PARAMETERS; FACIAL EXPRESSION ANALYSES; FACIAL EXPRESSION SYNTHESIS; FACIAL EXPRESSIONS; HUMAN EXPRESSIONS; MPEG-4 FACIAL ANIMATION PARAMETERS (FAPS); PERCEPTUAL QUALITIES; PROBABILISTIC FRAMEWORKS; SYNTHETIC FACES; VERY LOW BIT RATES;

BAYESIAN NETWORKS;

EID: 55149113939     PISSN: 10518215     EISSN: None     Source Type: Journal    
DOI: 10.1109/TCSVT.2008.928887     Document Type: Article

References (39)

1. ISO/IEC 14496-MPEG-4 International Standard,, 1998.
2. P. Ekman and W. V. Friesen, Facial Action Coding System (FACS): Manual. Palo Alto, CA: Consulting Psychologists Press, 1978.
3. P. Ekman and W. V. Friesen, Unmasking the Face. Englewood Cliffs, NJ: Prentice-Hall, 1975.
4. J. Allman, J. T. Cacioppo, R. J. Davidson, P. Ekman, W. V. Friesen, C. E. Lzard, and M. Phillips, NSF Report - Facial Expression Understanding Human Interaction Lab, Univ. of California, San Francisco, 1992, Tech. rep..
5. Y. Zhang and Q. Ji, "Active and dynamic information fusion for facial expression understanding from image sequence," IEEE Trans. Pattern Anal. Mach. Intell., vol. 27, no. 5, pp. 699-714, May 2005.
6. H. Kobayashi and F. Hara, "Facial interaction between animated 3-D face robot and human beings," in Proc. Int. Conf. Syst., Man, Cybern., 1997, pp. 3732-3737.
7. C. Padgett and G. Cottrell, M. Mozer, M. Jordan, and T. Petsche, Eds., "Representing face images for emotion classification," in Advances in Neural Information Processing Systems, MIT Press, Cambridge, 1997, vol. 9.
8. M. J. Lyons, J. Budynek, and S. Akamatsu, "Automatic classification of single facial images," IEEE Trans. Pattern Anal. Mach. Intell., vol. 21, no. 12, pp. 1357-1362, Dec. 1999.
9. M. Pantic and L. Rothkrantz, "Expert system for automatic analysis of facial expression," J. Image Vis. Computing, vol. 18, no. 11, pp. 881-905, 2000.
10. Y. Yacoob and L. S. Davis, "Recognizing human facial expressions from long image sequences using optical flow," IEEE Trans. Pattern Anal. Mach. Intell, vol. 18, no. 6, pp. 636-642, Jun. 1996.
11. M. J. Black and Y. Yacoob, "Recognizing facial expression in image sequences using local parameterized models of image motion," Int. J. Comput. Vis., vol. 25, no. 1, pp. 23-48, 1997.
12. I. A. Essa and A. P. Pentland, "Coding, analysis, interpretation, and recognition of facial expressions," IEEE Trans. Pattern Anal. Mach. Intell., vol. 19, no. 7, pp. 757-763, Jul. 1997.
13. N. Oliver, A. Pentland, and F. Bérard, "LAFTER: Lips and face real time tracker," in Proc. IEEE Conf. Comput. Vis. Pattern Recogn., 1997, pp. 123-129.
14. Y. Tian, T. Kanade, and J. F. Cohn, "Recognizing action units for facial expression analysis," IEEE Trans. Pattern Anal. Mach. Intell., vol. 23, no. 2, pp. 97-115, Feb. 2001.
15. M. Malciu and F. Prêteux, "Tracking facial features in video sequences using a deformable model-based approach," in Proc. SPIE Int. Soc. Opt. Eng., 2000, vol. 4121, pp. 51-62.
16. J. Chou, Y. Chang, and Y. Chen, "Facial feature point tracking and expression analysis for virtual conferencing systems," in Proc. IEEE Int. Conf. Multimedia and Expo, 2001, pp. 24-27.
17. T. Cootes, G. Edwards, and C. Taylor, "Active appearance models," IEEE Trans. Pattern Anal. Mach. Intell, vol. 23, no. 6, pp. 681-685, Jun. 2001.
18. J. Ahlberg, "An active model for facial feature tracking," EURASIP J. Appl. Signal Process., vol. 2002, no. 6, pp. 566-571, 2002.
19. R. Cowie, E. Dougls-Cowie, N. Tsapatsoulis, G. Votsis, S. Kollias, W. Fellenz, and J. G. Taylor, "Emotion recognition in human-computer interaction," IEEE Signal Process. Mag., vol. 2001, no. 1, pp. 33-80, Jan. 2001.
20. A. Raouzaiou, N. Tsapatsoulis, K. Karpouzis, and S. Kollias, "Parameterized facial expression synthesis based on mpeg-4," EURASIP J. Appl. Signal Process., vol. 2002, no. 10, pp. 1021-1038, 2002.
21. M. Fardas and A. Bonafonte, "Facial animation parameters extraction and expression recognition using hidden markov models," Signal Process.: Image Commun., vol. 17, pp. 675-688, 2002.
22. P. Eister and B. Girod, "Analyzing facial expressions for virtual conferencing," IEEE Comput. Graphics Appl., pp. 70-79, Sep.-Oct. 1998.
23. S. Valente and J.-L. Dougelay, "Face tracking and realistic animations for telecommunicant clones," IEEE Multimedia, pp. 34-43, Jan.-Mar. 2000.
24. H. Tao and T. S. Huang, "Facial animation and video tracking," in Proc. Workshop Modeling and Motion Capture Techniques for Virtual Environments, 1998, pp. 242-253.
25. F. Lavagetto and R. Pockaj, "The facial animation engine: Toward a high-level interface for the design of MPEG-4 compliant animation faces facial," IEEE Trans. Circuits Syst. Video Technol., vol. 9, no. 2, pp. 277-289, Apr. 1999.
26. T. Goto, M. Escher, C. Zanardi, and N. M-Thalmann, "MPEG-4 based animation with face feature tracking," in Proc. Eurographics Workshop Computer Animation and Simulation, 1999, pp. 89-98.
27. S. Kshirsagar, T. Molet, and N. M-Thalmann, "Principal components of expressive speech animation," in Proc. Int. Computer Graphics Conf., 2001, pp. 38-44.
28. G. A. Abrantes and F. Pereira, "MPEG-4 facial animation technology: Survey, implementation, and results," IEEE Trans. Circuits Syst. Video Technol., vol. 9, no. 2, pp. 290-305, Apr. 1999.
29. Y. Zhang, E. C. Prakash, and E. Sung, "A new physical model with multilayer archtecture for facial expression animation using dynamic adaptive mesh," IEEE Trans. Visual. Graphics, vol. 10, no. 3, pp. 339-352, 2004.
30. D. Terzopoulos and K. Waters, "Analysis and synthesis of facial image sequence using physical and anatomical models," IEEE Trans. Pattern Anal. Mach. Intell., vol. 15, no. 6, pp. 569-579, Jun. 1993.
31. K. Waters, "A muscle model for animating three-dimentional facial expression," in Proc. SIGGRAPH, 1987, pp. 17-24.
32. H. Tao, H. H. Chen, W. Wu, and T. S. Huang, "Compression of mpeg-4 facial animation parameters for transmision of talking heads," IEEE Trans. Circuits Syst. Video Technol., vol. 9, no. 2, pp. 264-276, Apr. 1999.
33. J. Ahlberg and H. Li, "Representation and compressing facial animation parameters using facial action basis functions," IEEE Trans. Circuits Syst. Video Technol., vol. 9, no. 3, pp. 405-410, Jun. 1999.
34. P. Wang and Q. Ji, "Learning discriminant feature for multi-view face and eye detection," in Proc. IEEE Conf. Comput. Vis. Pattern Recogn., 2005.
35. Y. Freund and R. E. Schapire, "A decision-theoretic generalization of on-line learning and an application to boosting," in Proc. Eur. Conf. Computational Learning Theory, 1995, pp. 23-37.
36. H. Gu, Q. Ji, and Z. Zhu, "Active facial tracking for fatigue detection," in Proc. IEEE Workshop Applications of Comput. Vis., 2002, pp. 137-142.
37. Z. Zhu and Q. Ji, "Robust pose invariant facial feature detection and tracking in real-time," in Proc. Int. Conf. Pattern Recogn., 2006, pp. 1092-1095.
38. J. Pearl, Probabilistic Reasoning in Intelligent Systems. San Mateo, CA: Morgan Kaufmann, 1988.
39. T. Kanade, J. Cohn, and Y. Tian, "Comprehensive database for facial expression analysis," in Proc. Int. Conf. Automatic Face and Gesture Recognition, 2000, pp. 46-53.