Expressive speech-driven facial animation

ACM Transactions on Graphics

Volumn 24, Issue 4, 2005, Pages 1283-1302

  Cao, Yong ^a,b,c   Tien, Wen C ^b,d   Faloutsos, Petros ^a,c   Pighin, Frédéric ^b,e  

^a UNIVERSITY OF CALIFORNIA   (United States)

^b UNIVERSITY OF SOUTHERN CALIFORNIA   (United States)

^c Department of Civil and Environmental Engineering   (United States)

^d Department of Computer Science ^*  (United States)

^e University of Southern California   (United States)

Author keywords

Expression synthesis; Facial animation; Independent component analysis; Lip synching

Indexed keywords

BEHAVIORAL RESEARCH; DATABASE SYSTEMS; FACE RECOGNITION; LEARNING SYSTEMS; MATHEMATICAL MODELS; SPEECH SYNTHESIS;

EXPRESSION SYNTHESIS; FACIAL ANIMATION; LIP SYNCHING; MOTION SYNTHESIS;

ANIMATION;

EID: 33645777234     PISSN: 07300301     EISSN: 07300301     Source Type: Journal    
DOI: 10.1145/1095878.1095881     Document Type: Article

References (44)

1. ALBRECHT, I., HABER, J., AND PETER SEIDEL, H. 2002. Speech synchronization for physics-based facial animation. In Proceedings of the International Conference on Computer Graphics, Visualization, and Computer Vision (WSCG'02). 9-16.
2. BRAND, M. 1999. Voice puppetry. In Proceedings of ACM SIGGRAPH 1999. ACM Press/Addison-Wesley Publishing Co. 21-28.
3. BREGLER, C., COVELL, M., AND SLANEY, M. 1997. Video rewrite: Driving visual speech with audio. In SIGGRAPH 97 Conference Proceedings. 353-360.
4. BROOK, N. AND SCOTT, S. 1994. Computer graphics animations of talking faces based on stochastic models. In the International Symposium on Speech, Image Processing, and Neural Networkds.
5. BUHMANN, M. D. 2003. Radial Basis Functions : Theory and Implementations. Cambridge University Press, Cambridge, UK.
6. BURGES, C. 1998. A tutorial on support vector machines for pattern recognition. Data Mining Knowl. Disc. 2, 2, 955-974.
7. CAO, Y., FALOUTSOS, P., AND PIGHIN, F. 2003. Unsupervised learning for speech motion editing. In Proceedings of Eurographics/ ACM SIGGRAPH Symposium on Computer Animation. 225-231.
8. CASSELL, J., PELACHAUD, C., BADLER, N., STEEDMAN, M., ACHORN, B., BECKET, W., DOUVILLE, B., PREVOST, S., AND STONE, M. 1994. Animated conversation: Rule-based generation of facial expression, gesture and spoken intonation for multiple conversational agents. In Proceedings of ACM SIGGRAPH 1994.
9. CHANG, C.-C. AND LIN, C.-J. 2001a. LIBSVM: A Library for Support Vector Machines. Software available at http://www.csie.ntu.edu.tw/~cjlin/libsvm.
10. CHANG, C.-C. AND LIN, C.-J. 2001b. Training nu-support vector classifiers: Theory and algorithms. Neural Computation, 2119-2147.
11. CHUANG, E., DESHPANDE, H., AND BREGLER, C. 2002. Facial expression space learning. In Proceedings of Pacific Graphics.
12. COHEN, N. AND MASSARO, D. W. 1993. Modeling coarticulation in synthetic visual speech. In Models and Techniques in Computer Animation, N. M. Thalmann and D. Thalmann, Eds. Springer-Verlang, 139-156.
13. EKMAN, P. AND FRIESEN, W. 1978. Manual for Facial Action Coding System. Consulting Psychologists Press Inc., Palo Alto, CA.
14. EZZAT, T., GEIGER, G., AND POGGIO, T. 2002. Trainable videorealistic speech animation. In Proceedings of ACM SIGGRPAH 2002. ACM Press, 388-398.
15. FASTICA. Helsinki University of Technology, Laboratory of Computer Information Science, Neural Networks Research Centre. Available at www.cis.hut.fi/projects/ica/fastica/.
16. HOUSE OP MOVES INC. Diva software. Available at www.moves.com/moveshack/ diva.htm.
17. HYVARINEN, A., KARHUNEN, J., AND OJA, E. 2001. Independent Component Analysis. John Wiley & Sons.
18. HYVARINEN, A. AND OJA, E. 2000. Independent component analysis: Algorithms and applications. Neural Networks 13, 411-430.
19. INTERNATIONAL COMPUTER SCIENCE INSTITUTE, BERKELEY, CA. Rasta software. Available at www.icsi.berkeley.edu/Speech/rasta.html.
20. JOSHI, P., TIEN, W. C., DESBRUN, M., AND PIGHIN, F. 2003. Learning controls for blend shape based realistic facial animation. In Proceedings of the 2003 ACM SIGGRAPH/Eurographics Symposium on Computer Animation. Eurographics Association, 187-192.
21. KALBERER, G. A., MUELLER, P., AND GOOL, L. V. 2002. Speech animation using viseme space. In Vision, Modeling, and Visualization VMV 2002. Akademische Verlagsgesellschaft Aka GmbH, Berlin, Germany. 463-470.
22. KOVAR, L., GLEICHER, M., AND PIGHIN, F. 2002. Motion graphs. In Proceedings of ACM SIGGRAPH 2002. ACM Press, 473-482.
23. KSHIRSAGAR, S. AND MAGNENAT-THALMANN, N. 2003. Visyllable based speech animation. In Proceedings of Eurographics 2003.
24. LEE, J., CHAI, J., REITSMA, P. S. A., HODGINS, J. K., AND POLLARD, N. S. 2002. Interactive control of avatars animated with human motion data. In Proceedings of the 29th Annual Conference on Computer Graphics and Interactive Techniques. ACM Press, 491-500.
25. LEE, S. P., BADLER, J. B., AND BADLER, N. I. 2003. Eyes alive. In Proceedings of ACM SIGGRAPH 2003. ACM Press, 637-644.
26. LEE, Y., TERZOPOULOS, D., AND WATERS, K. 1995. Realistic modeling for facial animation. In SIGGRAPH 95 Conference Proceedings. ACM SIGGRAPH, 55-62.
27. LEWIS, J. 1991. Autmated lip-sync: Background and techniques. J. Visualiz. Comput. Animat. 2, 118-122.
28. Li, Y., WANG, T., AND SHUM, H.-Y. 2002. Motion texture: A two-level statistical model for character motion synthesis. In Proceedings of the 29th Annual Conference on Computer Graphics and Interactive Techniques. ACM Press, 465-472.
29. LIEN, J., COHN, J., KANADE, T., AND Li, C. 1998. Automatic facial expression recognition based on FACS action units. In Proceedings of the 3rd IEEE International Conference on Automatic Face and Gesture Recognition. 390-395.
30. MASUKO, T., KOBAYASHI, T., TAMURA, M., MASUBUCHI, J., AND TOKUDA., K. 1998. Text-to-visual speech synthesis based on parameter generation from hmm. In Proceedings of the International Conference on Acoustics, Speech, and Signal Processing (ICASSP'98).
31. NOH, J., FIDALEO, D., AND NEUMANN, U. 2000. Animated deformations with radial basis functions. In ACM Symposium on Virtual Realisty Software and Technology. 166-174.
32. PARKE, F. 1975. A model for human faces that allows speech synchronized animation. J. Comput. Graph. 1, 1, 1-4.
33. PELACHAUD, C. 1991. Realistic face animation for speech. Ph.D. thesis, University of Pennsylvania.
34. PIGHIN, F., HECKER, J., LISCHINSKI, D., SZELISKI, R., AND SALESIN, D. 1998. Synthesizing realistic facial expressions from photographs. In SIGGRAPH 98 Conference Proceedings. ACM SIGGRAPH, 75-84.
35. PRESS, W. H., FLANNERY, B. P., TEUKOLSKY, S. A., AND VETTERLING, W. T. Numerical Recipes in C: The Art of Scientific Computing, 2nd ed. Cambridge University Press, Cambridge, UK.
36. PYUN, H., KIM, Y., CHAE, W, KANG, H. W., AND SHIN, S. Y. 2003. An example-based approach for facial expression cloning. In Proceedings of the ACM SIGGRAPH / Eurographics Symposium on Computer Animation. Eurographics Association, 167-176.
37. SAHNI, S. 1999. Data Structures, Algorithms, and Applications in C++. McGraw-Hill Publishing Co., New York, NY.
38. SAISAN, P., BISSACCO, A., CHIUSO, A., AND SOATTO, S. 2004. Modeling and synthesis of facial motion driven by speech. In European Conference on Computer Vision. 456-467.
39. SANKOFF, D. AND KRUSKAL, J. B. 1983. Time Warps, String Edits, and Macromolecules: The Theory and Practice of Sequence Comparison. CSLI Publications, Stanford University, Stanford, CA.
40. SPEECH GROUP, CARNEGIE MELLON UNIVERSITY. Festival software. Available at www.speech.cs.emu.edu/festival.
41. TENENBAUM, J. B. AND FREEMAN, W. T. 1999. Separating style and content with bilinear models. Neural Computat. J. 12, 1247-1283.
42. VASILESCU, M. A. O. AND TERZOPOULOS, D. 2003. Multilinear subspace analysis of image ensembles. In Conference on Computer Vision and Pattern Recognition.
43. WATERS, K. 1987. A muscle model for animating three-dimensional facial expression. In SIGGRAPH 87 Conference Proceedings. ACM SIGGRAPH, Vol. 21. 17-24.
44. WU, T., LIN, C.-J., AND WENG, R. C. 2003. Probability estimates for multi-class classification by pairwise coupling. In Proceedings of Neural Information Processing System (NIPS'03).