Speaker-independent 3D face synthesis driven by speech and text

Signal Processing

Volumn 86, Issue 10, 2006, Pages 2932-2951

  Savran, Arman ^a   Arslan, Levent M ^a   Akarun, Lale ^a  

^a BOGAZICI UNIVERSITY   (Turkey)

Author keywords

3D facial motion capture; Audio visual codebook; MPEG 4 facial animation; Speaker independent; Visual speech synthesis

Indexed keywords

ANIMATION; CORRELATION METHODS; DATA PROCESSING; GESTURE RECOGNITION; INFORMATION RETRIEVAL; NATURAL LANGUAGE PROCESSING SYSTEMS; RECURRENT NEURAL NETWORKS; THREE DIMENSIONAL;

3D FACIAL MOTION CAPTURE; AUDIO-VISUAL CODEBOOKS; MPEG-4 FACIAL ANIMATION; SPEAKER INDEPENDENT; VISUAL SPEECH SYNTHESIS;

SPEECH RECOGNITION;

EID: 33745712098     PISSN: 01651684     EISSN: None     Source Type: Journal    
DOI: 10.1016/j.sigpro.2005.12.007     Document Type: Article

References (25)

1. McGurk H., and MacDonald J. Hearing lips and seeing voices. Nature 264 (1976) 746-748
2. J. Beskow, Rule-based visual speech synthesis, in: Proceedings of the Fourth European Conference on Speech Communication and Technology (Eurospeech '95), Madrid, Spain, 1995, pp. 299-302.
3. Cohen M.M., and Massaro D.W. Modeling coarticulation in synthetic visual speech. In: Thalmann N.M., and Thalmann D. (Eds). Models and Techniques in Computer Animation (1993), Springer, Tokyo 139-156
4. C. Bregler, M. Covell, M. Slaney, Video rewrite: visual speech synthesis from video, in: Proceedings of the Workshop on Audio-Visual Speech Processing, Rhodes, Greece, 1997, pp. 153-156.
5. J.P. Lewis, F.I. Parke, Automatic lip-synch and speech synthesis for character animation, in: Proceedings of the Graphics Interface '86, Canadian Information Processing Society, Calgary, 1986, pp. 136-140.
6. Yamamoto E., Nakamura S., and Shikano K. Lip movement synthesis from speech based on Hidden Markov Models. J. Speech Commun. 28 (1998) 105-115
7. Lavagetto F. Converting speech into lip movements: a multimedia telephone for hard of hearing people. IEEE Trans. Rehabil. Eng. 3 1 (1995) 90-102
8. D.W. Massaro, J. Beskow, M.M. Cohen, Picture my voice: audio to visual speech synthesis using artificial neural networks, in: Proceedings of the AVSP '99, 1999.
9. Hong P., Wen Z., and Huang T.S. Real-time speech-driven face animation with expressions using neural networks. IEEE Trans. Neural Networks 13 1 (2002) 100-111
10. Arslan L.M., and Talkin D. Codebook based face point trajectory synthesis algorithm using speech input. Elsevier Sci. 953 (1998) 01-13
11. Gonzalez R.C., and Woods R.E. Digital Image Processing (2002), Prentice-Hall, Englewood Cliffs, NJ (pp. 295-302, Chapter 6)
12. Shapiro L.G., and Stockman G.C. Computer Vision (2001), Prentice-Hall, Englewood Cliffs, NJ (pp. 74-75, Chapter 3)
13. Arun K.S., Huang S.T., and Blostein S.D. Least-squares fitting of two 3-D points sets. IEEE Trans. Pattern Anal. Mach. Intell. 9 5 (1987) 698-700
14. H. Dutagaci, Statistical language models for large vocabulary Turkish speech recognition, M.S. Thesis, Bogazici University, 2002.
15. T. Robinson, M. Hochberg, S. Renals, The use of recurrent neural networks in continuous speech recognition, 1995, 〈svr-www.eng.cam.ac.uk/~ajr/rnn4csr94/rnn4csr94.html〉.
16. Rabiner L.R., and Schafer R.W. Digital Processing of Speech Signals (1978), Prentice-Hall, Englewood Cliffs, NJ
17. J. Rothweiler, A root-finding algorithm for line spectral frequencies, in: Proceedings of the IEEE ICASSP 1999, Phoenix, AZ, USA, 1999, pp. II-661-II-664.
18. Rothweiler J. On polynomial reduction in the computation of LSP frequencies. IEEE Trans. Speech Audio Process. 7 5 (1999) 592-594
19. Waibel A., Hanazawa T., Hinton G., Shikano K., and Lang. K. Phoneme recognition using time-delay neural networks. IEEE Trans. Acoust. Speech Signal Process. 37 (1989) 328-339
20. Cernansky M., and Benuskova L. Simple recurrent network trained by RTRL and extended Kalman filter algorithms. Neural Network World 13 3 (2003) 223-234
21. Williams R.J., and Zipser D. A learning algorithm for continually running fully recurrent neural networks. Neural Comput. 1 (1989) 270-280
22. Young S., Evermann G., Kershaw D., Moore G., Odell J., Ollason D., Povey D., Valtchev V., and Woodland P. The HTK Book (2002), Entropic Cambridge Engineering
23. Huang X., Acero A., and Hon H.W. Spoken Language Processing: a Guide to Theory, Algorithm, and System Development (2001), Prentice-Hall PTR (pp. 316-318, Chapter 6)
24. Unser M., Aldroubi A., and Eden M. B-spline signal processing: Part I-theory. IEEE Trans. Signal Process. 41 2 (1993) 821-833
25. Lavagetto F., and Pockaj R. The facial animation engine: towards a high-level interface for the design of MPEG-4 compliant animated faces. IEEE Trans. Circuits Syst. Video Technol. 9 2 (1999) 277-289