Realistic speech animation based on observed 3-D face dynamics

IEE Proceedings: Vision, Image and Signal Processing

Volumn 152, Issue 4, 2005, Pages 491-500

  Muller P ^a   Kalberer, G A ^a   Proesmans, M ^c   Van Gool, L ^a,b  

^a ETH ZURICH   (Switzerland)

^b UNIVERSITY OF LEUVEN   (Belgium)

^c Eyetronics Inc   (Belgium)

Author keywords

[No Author keywords available]

Indexed keywords

ADAPTIVE CONTROL SYSTEMS; FACE RECOGNITION; INDEPENDENT COMPONENT ANALYSIS; MATHEMATICAL MODELS; SPEECH PROCESSING; THREE DIMENSIONAL; VIDEO SIGNAL PROCESSING;

3-D FACE DYNAMICS; FACIAL DEFORMATIONS; PHYSIOGNOMY; SPEECH ANIMATION;

ANIMATION;

EID: 24144456967     PISSN: 1350245X     EISSN: None     Source Type: Conference Proceeding    
DOI: 10.1049/ip-vis:20045112     Document Type: Conference Paper

References (37)

1. Parke, F.I.: 'Computer generated animation of faces', ACM National Conf., 1972, pp. 451-457
2. Eben, O.: 'Personal communication' (Pixar Animation Studios, 1997)
3. Bregler, C., Covell, M., and Slaney, M.: 'Video rewrite: driving visual speech with audio', SIGGRAPH, 1997, pp. 353-360
4. Beier, T., and Neely, S.: 'Feature-based image metamorphosis', SIGGRAPH Conf. Proc., 1992, 26, pp. 35-42
5. Bregler, C., and Omohundro, S.: 'Nonlinear image interpolation using manifold learning', NIPS, 1995, 7
6. Ezzat, T., and Poggio, T.: 'Visual speech synthesis by morphing visemes', Int. J. Comput. Vis., 2000, 38, pp. 45-57
7. Ezzat, T., Geiger, G., and Poggio, T.: Trainable videorealistic speech animation'. Proc. SIGGRAPH, 2002, pp. 388-398
8. Beymer, D., and Poggio, T.: 'Image representation for visual learning', Science, 1996, 272, pp. 1905-1909
9. Bichsel, M.: 'Automatic interpolation and recognition of faces by morphing'. Proc. 2nd Int. Conf. on Automatic Face and Gesture Recognition, 1996, pp. 128-135
10. Cosatto, E., and Graf, H.: 'Photo-realistic talking-heads from image samples', IEEE Trans. Multimedia, 2000, 2, pp. 152-163
11. Cosatto, E.: 'Sample-based talking-head synthesis', PhD Thesis, Swiss Federal Institute of Technology Lausanne, Switzerland, 2002
12. Chen, D., State, A., and Banks, D.: 'Interactive shape metamorphosis', SIGGRAPH'95 Conf. Proc., 1995, pp. 43-44
13. Blanz, V., and Vetter, T.: 'A morphable model for the synthesis of 3D faces', Proc. SIGGRAPH, 1999, pp. 187-194
14. Pighin, F., Hecker, J., Lischinski, D., Szeliski, R., and Salesin, D.H.: 'Synthesizing realistic facial expressions from photographs', SIGGRAPH Proc., 1998, pp. 75-84
15. Guenter, B., Grimm, C., Wood, D., Malvar, H., and Pighin, F.: 'Making faces'. Proc. SIGGRAPH, 1998, pp. 55-66
16. Lin, I., Yeh, J., and Ouhyoung, M.: 'Realistic 3D facial animation parameters from mirror-reflected multi-view video'. Proc. Computer Animation 2001 Conf., 2001, pp. 2-11
17. Waters, K., and Frisbie, J.: 'A coordinated muscle model for speech animation', Graphics Interface, 1995
18. Pelachaud, C., Badler, N., and Steedman, M.: 'Generating facial expressions for speech', Cogn. Sci., 1996, 20, (1), pp. 1-46
19. Kahler, K., Haber, J., and Seidel, H.-P.: 'Reanimating the dead: Reconstruction of expressive faces from skull data'. SIGGRAPH Conf. Proc., 2003, pp. 554-561
20. Eyetronics, http://www.eyetronics.com. 2004
21. Alias, http://www.alias.com. 2004
22. Noh, J., and Neumann, U.: 'Expression cloning'. Proc. SIGGRAPH, 2001, pp. 277-288
23. Kalberer, G., Mueller, P., and Van Gool, L.: 'Generating visemes for realistic animation'. Vision, Modelling, and Visualization VMV, 2002, pp. 233-240
24. Turk, M., and Pentland, A.: 'Face recognition using eigenfaces'. Proc. IEEE Conf. Computer Vision and Pattern Recognition, 1991, pp. 586-591
25. Strang.: 'Linear algebra and its application'. Proc. HBJ, 1988
26. Lucas, B., and Kanade, T.: 'An iterative image registration technique with an application to stereo vision'. Proc. Int. Joint Conf. on Artificial Intelligence, 1981, pp. 674-679
27. Tomasi, C., and Kanade, T.: 'Detection and tracking of feature points', Pittsburgh, PA, 1991, Carnegie Mellon University Technical Report CMU-CS-91-132
28. Hyvärinen, A.: 'Independent component analysis by minimizing of mutual information', Rakentajanaukio 2 C, FIN-02150 Espoo, Technical Report A46, Helsinki University of Technology, Helsinki University of Technology, Finland, 1997
29. Scott, K., Kagels, D., Watson, S., Rom, H., Wright, J., Lee, M., and Hussey, K.: 'Synthesis of speaker facial movement to match selected speech sequences'. Proc. Fifth Australian Conf. on Speech Science and Technology, 1994, 2, pp. 620-625
30. Owens, O., and Blazck, B.: 'Visemes observed by haring-impaired and normal-hearing adult viewers', J. Speech Hear. Res., 1985, 28, pp. 381-393
31. Montgomery, A., and Jackson, P.: 'Physical characteristics of the lips underlying vowel lipreading performance', J. Acoust. Soc. Am., 1983, 73, (6), pp. 2134-2144
32. Massaro, D.: 'Perceiving talking faces' (MIT Press, 1998)
33. Kalberer, G., and Van Gool, L: 'Realistic face animation for speech', J. Vis. Comput. Animat., 2002, 13, pp. 97-106
34. Traber, C.: 'SVOX: The implementation of a text-to-speech system', PhD thesis, ETH; No. 11064, 1995
35. Kalberer, G., and Van Gool, L.: 'Lip animation based on observed 3D speech dynamics', Proc. SPIE. Videometrics and Optical Method for 3D Shape Measurement, 2001, 4309, pp. 16-25
36. Kshirsagar, S., Molet, T., and Magenat-Thalmann, N.: 'Principal components of expressive speech animation'. Proc. Computer Graphics Int., 2001, pp. 38-44
37. Okadome, T., Kaburagi, T., and Honda, M.: 'Trajectory formation of articulatory movements for a given sequence of phonemes', Proc. 5th Int. Conf. on Spoken Language Processing, 1998, 7, pp. 3131-3134