Trainable videorealistic speech animation

Proceedings - Sixth IEEE International Conference on Automatic Face and Gesture Recognition

Volumn , Issue , 2004, Pages 57-64

  Ezzat, Tony ^a   Geiger, Gadi ^a   Poggio, Tomaso ^a  

^a MASSACHUSETTS INSTITUTE OF TECHNOLOGY   (United States)

Author keywords

[No Author keywords available]

Indexed keywords

MULTIDIMENSIONAL MORPHABLE MODEL (MMM); SPEECH ANIMATION; TEXT-TO-SPEECH SYSTEM (TTS);

ANIMATION; AUDIO SYSTEMS; EYE MOVEMENTS; LEARNING SYSTEMS; SPEECH ANALYSIS; SPEECH PROCESSING; SPEECH SYNTHESIS; USER INTERFACES; VIDEO CAMERAS; VIDEO RECORDING;

SPEECH RECOGNITION;

EID: 4544344031     PISSN: None     EISSN: None     Source Type: Conference Proceeding    
DOI: 10.1109/AFGR.2004.1301509     Document Type: Conference Paper

References (37)

1. J. L. Barron, D. J. Fleet, and S. S. Beauchemin. Performance of optical flow techniques. International Journal of Computer Vision, 12(1):43-77, 1994.
2. J. Bergen, P. Anandan, K. Hanna, and R. Hingorani. Hierarchical model-based motion estimation. In Proceedings of the European Conference on Computer Vision, pages 237-252, Santa Margherita Ligure, Italy, 1992.
3. D. Beymer and T. Poggio. Image representations for visual learning. Science, 272:1905-1909, 1996.
4. D. Beymer, A. Shashua, and T. Poggio. Example based image analysis and synthesis. Technical Report 1431, MIT AI Lab, 1993.
5. C. M. Bishop. Neural Networks for Pattern Recognition. Clarendon Press, Oxford, 1995.
6. A. Black and P. Taylor. The Festival Speech Synthesis System. University of Edinburgh, 1997.
7. M. Black, D. Fleet, and Y. Yacoob. Robustly estimating changes in image appearance. Computer Vision and Image Understanding, Special Issue on Robust Statistical Techniques in Image Understanding, pages 8-31, 2000.
8. V. Blanz and T. Vetter. A morphable model for the synthesis of 3D faces. In A. Rockwood, editor, Proceedings of SIGGRAPH 2001, Computer Graphics Proceedings, Annual Conference Series, pages 187-194, Los Angeles, 1999. ACM, ACM Press / ACM SIGGRAPH.
9. M. Brand. Voice puppetry. In A. Rockwood, editor, Proceedings of SIGGRAPH 1999, Computer Graphics Proceedings, Annual Conference Series, pages 21-28, Los Angeles, 1999. ACM, ACM Press / ACM SIGGRAPH.
10. C. Bregler, M. Covell, and M. Slaney. Video rewrite: Driving visual speech with audio. In Proceedings of SIGGRAPH 1997, Computer Graphics Proceedings, Annual Conference Series, pages 353-360, Los Angeles, CA, August 1997. ACM, ACM Press / ACM SIGGRAPH.
11. N. Brooke and S. Scott. Computer graphics animations of talking faces based on stochastic models. In Intl. Symposium on Speech, Image Processing, and Neural Networks, Hong Kong, April 1994.
12. M. M. Cohen and D. W. Massaro. Modeling coarticulation in synthetic visual speech. In N. M. Thalmann and D. Thalmann, editors, Models and Techniques in Computer Animation, pages 139-156. Springer-Verlag, Tokyo, 1993.
13. T. F. Cootes, G. J. Edwards, and C. J. Taylor. Active appearance models. In Proceedings of the European Conference on Computer Vision, Freiburg, Germany, 1998.
14. T. H. Cormen, C. E. Leiserson, and R. L. Rivest. Introduction to Algorithms. The MIT Press and McGraw-Hill Book Company, 1989.
15. E. Cosatto and H. Graf. Sample-based synthesis of photorealistic talking heads. In Proceedings of Computer Animation '98, pages 103-110, Philadelphia, Pennsylvania, 1998.
16. T. Ezzat, G. Geiger, and T. Poggio. Trainable videorealistic facial animation. In Proceedings of SIGGRAPH 2002, volume 21, pages 388-398, San Antonio, Texas, 2002.
17. T. Ezzat, G. Geiger, and T. Poggio. Mary101:a trainable videorealistic speech animation. In G. B. . P. P. E. E. Vatiokis-Bateson, editor, Audiovisual Speech Processing. MIT Press, to appear.
18. T. Ezzat and T. Poggio. Visual speech synthesis by morphing visemes. International Journal of Computer Vision, 38:45-57, 2000.
19. F. Girosi, M. Jones, and T. Poggio. Priors, stabilizers, and basis functions: From regularization to radial, tensor, and additive splines. Technical Report 1430, MIT AI Lab, June 1993.
20. B. Guenter, C. Grimm, D. Wood, H. Malvar, and F. Pighin. Making faces. In Proceedings of SIGGRAPH 1998, Computer Graphics Proceedings, Annual Conference Series, pages 55-66, Orlando, FL, 1998. ACM, ACM Press / ACM SIGGRAPH.
21. B. K. P. Horn and B. G. Schunck. Determining optical flow. Artificial Intelligence, 17:185-203, 1981.
22. X. Huang, F. Alleva, H.-W. Hon, M.-Y. Hwang, K.-F. Lee, and R. Rosenfeld. The SPHINX-II speech recognition system: an overview (http://sourceforge.net/projects/cmusphinx/). Computer Speech and Language, 7(2):137-148, 1993.
23. M. Jones and T. Poggio. Multidimensional morphable models: A framework for representing and maching object classes. In Proceedings of the International Conference on Computer Vision, Bombay, India, 1998.
24. S. Y. Lee, G. Wolberg, and S. Y. Shin. Polymorph: An algorithm for morphing among multiple images. IEEE Computer Graphics Applications, 18:58-71, 1998.
25. Y. Lee, D. Terzopoulos, and K. Waters. Realistic modeling for facial animation. In Proceedings of SIGGRAPH 1995, Computer Graphics Proceedings, Annual Conference Series, pages 55-62, Los Angeles, California, August 1995. ACM, ACM Press / ACM SIGGRAPH.
26. B. LeGoff and C. Benoit. A text-to-audiovisual-speech synthesizer for trench. In Proceedings of the International Conference on Spoken Language Processing (ICSLP), Philadelphia, USA, October 1996.
27. T. Masuko, T. Kobayashi, M. Tamura, J. Masubuchi, and K. Tokuda. Text-to-visual speech synthesis based on parameter generation from hmm. In ICASSP, 1998.
28. F. I. Parke. A parametric model of human faces. PhD thesis, University of Utah, 1974.
29. A. Pearce, B. Wyvill, G. Wyvill, and D. Hill. Speech and expression: A computer solution to face animation. In Graphics Interface, 1986.
30. F. Pighin, J. Hecker, D. Lischinski, R. Szeliski, and D. Salesin. Synthesizing realistic facial expressions from photographs. In Proceedings of SIGGRAPH 1998, Computer Graphics Proceedings, Annual Conference Series, pages 75-84, Orlando, FL, 1998. ACM, ACM Press / ACM SIGGRAPH.
31. T. Poggio and T. Vetter. Recognition and structure from one 2D model view: observations on prototypes, object classes and symmetries. Technical Report 1347, Artificial Intelligence Laboratory, Massachusetts Institute of Technology, 1992.
32. S. Roweis. EM algorithms for PC A and SPCA. In M. I. Jordan, M. J. Kearns, and S. A. Solla, editors, Advances in Neural Information Processing Systems, volume 10. The MIT Press, 1998.
33. K. Scott, D. Kagels, S. Watson, H. Rom, J. Wright, M. Lee, and K. Hussey. Synthesis of speaker facial movement to match selected speech sequences. In Proceedings of the Fifth Australian Conference on Speech Science and Technology, volume 2, pages 620-625, December 1994.
34. K. Sjlander and J. Beskow. Wavesurfer - an open source speech tool. In Proc of ICSLP, volume 4, pages 464-467, Beijing, 2000.
35. J. B. Tenenbaum, V. de Silva, and J. C. Langford. A global geometric framework for nonlinear dimensionality reduction. Science, 290:2319-2323, Dec 2000.
36. G. Wahba. Splines Models for Observational Data. Series in Applied Mathematics, vol. 59, SIAM, Philadelphia, 1990.
37. K. Waters. A muscle model for animating three-dimensional facial expressions. In Computer Graphics (Proceedings of ACM SIGGRAPH 87), volume 21(4), pages 17-24. ACM, July 1987.