Design, implementation and evaluation of the Czech realistic audio-visual speech synthesis

Signal Processing

Volumn 86, Issue 12, 2006, Pages 3657-3673

  Železný, Miloš ^a   Krňoul, Zdeněk ^a   Císař, Petr ^a   Matoušek, Jindřich ^a  

^a UNIVERSITY OF WEST BOHEMIA   (Czech Republic)

Author keywords

Audio visual speech processing; Facial animation; Talking head

Indexed keywords

ANIMATION CONTROL; AUDIO VISUAL SPEECH PROCESSING; FACIAL ANIMATION; TALKING HEAD;

ALGORITHMS; ANIMATION; DATABASE SYSTEMS; LINGUISTICS; MATHEMATICAL MODELS; SPEECH RECOGNITION;

SPEECH SYNTHESIS;

EID: 33749437734     PISSN: 01651684     EISSN: None     Source Type: Journal    
DOI: 10.1016/j.sigpro.2006.02.039     Document Type: Article

References (42)

1. Železný M., Císař P., Krňoul Z., and Novák J. Design of an audio-visual speech corpus for the Czech audio-visual speech synthesis. Proceedings of ICSLP 2002. Denver, USA (2002)
2. F.I. Parke, Parameterized models for facial animation, IEEE Comput. Graph. Appl. (November 1982) 61-68.
3. S. Basu, A. Pentland, A three-dimensional model of human lip motions trained from video, M.I.T. media laboratory pereptual computing section, Technical Report No. 441, MIT Media Laboratory, Cambridge, USA, 1997.
4. V. Strnadová, Hádej, co říkám aneb Odezírání je nejisté umění. - Guess What I Am Talking or Lip-Reading is Uncertain Art, Ministerstvo zdravotnictví České republiky, Prague, Czech Republic, 1998.
5. Kricos P.B. Differences in Visual Intelligibility Across Talkers, Speechreading by Humans and Machines (1996), Springer, Berlin
6. J. Matoušek, J. Romportl, D. Tihelka, Z. Tychtl, Recent improvements on ARTIC: Czech text-to-speech system, in: Proceedings of ICSLP 2004, vol. 3, Jeju, Korea, 2004, pp. 1933-1936.
7. Donovan R.E., and Woodland P.C. A hidden Markov-model-based trainable speech synthesizer. Comput. Speech Language 13 1999.0123 (1999) 223-241
8. Matoušek J., Psutka J., and Krůta J. Design of speech corpus for text-to-speech synthesis. Proceedings of EUROSPEECH 2001 vol. 3 (2001), Ålborg, Denmark 2047-2050
9. Matoušek J., Tihelka D., and Psutka J. Automatic segmentation for Czech concatenative speech synthesis using statistical approach with boundary-specific correction. Proceedings of EUROSPEECH 2003. Geneva, Switzerland (2003) 301-304
10. Romportl J., Matoušek J., and Tihelka D. Prosody model and its application to Czech TTS system. Proceedings of UKROBRAZ 2002. Kijiv, Ukraine (2002) 93-96
11. Romportl J., and Matoušek J. Advanced prosody modelling. Proceedings of TSD 2004. Bonn, Heidelberg (2004), Springer, Berlin 441-447
12. Tihelka D., and Matoušek J. The design of Czech language formal listening tests for the evaluation of TTS systems. Proceedings of LREC 2004. Lisbon, Portugal (2004) 2099-2102
13. Tihelka D. Symbolic prosody driven unit selection for highly natural synthetic speech. Proceedings of EUROSPEECH 2005. Lisbon, Portugal (2005) 2525-2528
14. Masuko T., Kobayashi T., Tamura M., Masubuchi J., and Tokuda K. Text-to-visual speech synthesis based on parameter generation from hmm. Proceedings of ICASSP 1998. Seattle, USA (1998)
15. Tamura M., Masuko T., Kobayashi T., and Tokuda K. Visual speech synthesis based on parameter generation from hmm. speech-driven and text-and-speech-driven approaches. Proceedings of AVSP 1998. Sydney, Australia (1998)
16. Galanes F.M., Unverferth J., Arslan L., and Talkin D. Generation of lip-synched synthetic faces from phonetically clustered face movement data. Proceedings of AVSP 1998. Sydney, Australia (1998)
17. Sako S., Tokuda K., Masuko T., Kobayashi T., and Kitamura T. Hmm-based text-to-audio-visual speech synthesis. Proceedings of ICSLP2000. Beijing, China (2000)
18. Young S.J., Jansen J., Odell J.J., Olasson D., and Woodland P.C. The HTK Book (1999), Cambridge University Press, Cambridge, UK
19. Lubker J., and Gay T. Anticipatory labial coarticulation: experimental, biological, and linguistic variables. J. Acoust. Soc. Amer. 71 (1982) 437-448
20. Boyce S.E. Coarticulatory organization for lip rounding in Turkish and English. J. Acoust. Soc. Amer. 88 6 (1990) 2584-2595
21. Krňoul Z., and Železný M. Coarticulation modeling for the Czech audio-visual speech synthesis. Sixth International Workshop on Electronics, Control, Measurement and Signals, ECMS 2003. Liberec, Czech republic (2003) 64-68
22. Löfquist A. Speech as audible gestures. Speech Production and Speech Modelling (1990), Kluwer Academic Press, Dordrecht, Netherlands 289-322
23. Cohen M.M., and Massaro D.W. Text-to-visual speech synthesis based on parameter generation from hmm. Models and Techniques in Computer Animation (1993), Springer, Tokyo, Japan 139-156
24. Neural Network Simulator 1.1, University of Tübingen 〈http://www-ra.informatik.uni-tuebingen.de/SNNS/〉.
25. Novák M., et al. Umělé neuronové sítě teorie a aplikace (1998), C.H.BECK, Prague Czech Republic
26. Magnenat-Thalmann N., Primeau E., and Thalmann D. Abstract muscle action procedures for human face animation. Visual Comput. 3 5 (1988) 290-297
27. Lee Y., Terzopoulos D., and Walters K. Realistic modeling for facial animation. Proceedings of the 22nd Annual Conference on Computer Graphics and Interactive Techniques (1995), ACM Press, New York 55-62
28. Lee W., and Magnenat-Thalmann N. Fast head modeling for animation. Image Vision Comput. 18 4 (2000) 355-364
29. M. Escher, N. Magnenat Thalmann, Automatic 3D cloning and real-time animation of a human face, Comput. Animation (1997) 58.
30. Cyberware Scanning Products 〈http://www.cyberware.com/products/index.html〉.
31. Proesmans M., and Van Gool L. Reading between the lines-a method for extracting dynamic 3D with texture. Proceedings of the ACM Symposium on Virtual Reality Software and Technology. Lausanne, Switzerland (1997), ACM Press, New York 95-102
32. T. Miyasaka, K. Kuroda, M. Hirose, K. Araki, Reconstruction of realistic 3D surface model and 3D animation from range images obtained by real time 3D measurement system, in: IEEE International Conference on Pattern Recognition (ICPR'00), vol. 4, Barcelona, Spain, September 2000, p. 4594.
33. Fua P. Face models from uncalibrated video sequences. Proceedings of the International Workshop on Modelling and Motion Capture Techniques for Virtual Environments (1998), Springer, Berlin
34. Nagel B., Wingbermuhle J., Weik S., and Liedtke C.E. Automated modelling of real human faces for 3D animation. ICPR 98 (1998) 693-696
35. Akimoto T., Suenaga Y., and Wallace R.S. Automatic creation of 3D facial models. IEEE Comput. Graph. Appl. 13 5 (1993) 16-22
36. Lee W., Kalra P., and Magnenat-Thalmann N. Model based face reconstruction for animation. Proceedings of the MMM'97 (1997), World Scientific Press, Singapore 323-338
37. L. Moccozet, N. Magnenat Thalmann, Dirichlet free-form deformations and their application to hand simulation, in: Computer Animation '97, Geneva, Switzerland, June 1997.
38. Ilic S., and Fua P. Using dirichlet free form deformation to fit deformable models to noisy 3-D data. ECCV vol. 2351 (2002), Springer, Berlin 704-717
39. Z. Krňoul, M. Železný, P. Císař, Face model reconstruction for Czech audio-visual speech synthesis, in: Proceedings of SPECOM 2004, Saint Petersburg, Russian Federation, 2004, pp. 47-51, SPIIRAS.
40. Z. Krňoul, M. Železný, Realistic face animation for a Czech Talking Head, in: Conference on TEXT, SPEECH and DIALOGUE, TSD 2004, Springer, Berlin, 2004, pp. 603-610.
41. B. Le Goff, T. Guiard-Marigny, M. Cohen, C. Benoit, Real-time analysis-synthesis and intelligibility of talking faces, in: Second International Conference on Speech Synthesis, Newark (NY), September 1994.
42. D.W. Massaro, J. Beskow, M.M. Cohen, C.L. Fry, T. Rodgriguez, Picture my voice: audio to visual speech synthesis using artificial neural networks, in: AVSP'99, Santa Cruz, CA, USA, 1999.