Trainable articulatory control models for visual speech synthesis

International Journal of Speech Technology

Volumn 7, Issue 4, 2004, Pages 335-349

  Beskow, Jonas ^a  

^a ROYAL INSTITUTE OF TECHNOLOGY   (Sweden)

Author keywords

Artificial neural networks; Coarticulation; Facial animation; Perceptual evaluation; Speech synthesis

Indexed keywords

ANIMATION; INFORMATION ANALYSIS; NEURAL NETWORKS; REAL TIME SYSTEMS; SIGNAL PROCESSING; SPEECH RECOGNITION; VECTORS;

ARTIFICIAL NEURAL NETWORKS (ANN); COARTICULATION; FACIAL ANIMATION; PERCEPTUAL EVALUATION;

SPEECH SYNTHESIS;

EID: 4143072802     PISSN: 13812416     EISSN: None     Source Type: Journal    
DOI: 10.1023/B:IJST.0000037076.86366.8d     Document Type: Article

References (35)

1. Aglefors, E., Beskow, J., Dahlquist, M., Granström, B., Lundeberg, M., Spens, K.-E., and Öhman, T. (1998). Synthetic faces as a lipreading support. Proceedings of the 5th International Conference on Spoken Language Processing (ICSLP'98). Sydney, Australia, pp. 3047-3050.
2. Allen, J., Hunnicut, M.S., and Klatt, D. (1987). From Text to Speech: The MITalk System. Cambridge, MA: Cambridge University Press.
3. Beskow, J. (1995). Rule-based visual speech synthesis. Proceedings of the 4th European Conference on Speech Communication and Technology (Eurospeech'95). Madrid, Spain, pp. 299-302.
4. Beskow, J. (1997). Animation of talking agents. Proceedings of International Conference on Auditory-Visual Speech Processing (AVSP'97). Rhodos, Greece, pp. 149-152.
5. Bladon, R.A. and Al-Bamerini, A. (1976). Coarticulation resistance in English /1/. Journal of Phonetics, 4:137-150.
6. Bregler, C, Covell, M., and Slaney, M. (1997). Video rewrite: Driving visual speech with audio. Proceedings of ACM SIGGRAPH'97. pp. 353-360.
7. Brooke, N.M. and Scott, D.S. (1998). Two- and three-dimensional audio-visual speech snthesis, Proceedings of International Conference on Auditory-Visual Speech Processing (AVSP'98). Terrigal, Australia, pp. 213-218.
8. Carlson, R., Granström, B., and Hunnicutt, S. (1982). A multi-language text-to-speech module. Proceedings of the 7th International Conference on Acoustics, Speech, and Signal Processing (ICASSP'82), Paris, France, Vol. 3, pp. 1604-1607.
9. Cohen, M.M. and Massaro, D.W. (1993). Modelling coarticulation in synthetic visual speech. In N. Magnenat-Thalmann and D. Thalmann (Eds.). Models and Techniques in Computer Animation. Springer Verlag, Tokyo, pp. 139-156.
10. Cole, R., Carmell, T., Connors, P., Macon, M., Wouters, J., de Villiers, J., Tarachow, A., Massaro, D. W., Cohen, M. M., Beskow, J., Yang, J., Meier, U., Waibel, A., Stone, P., Fortier, G., Davis, A., and Soland, C. (1998). Intelligent animated agents for interactive language training. In STiLL: ESCA Workshop on Speech Technology in Language Learning. Stockholm, Sweden, pp. 163-166.
11. Cosi, P., Magno Caldognetto, E., Perin, G., and Zmarich, C. (2002). Labial coarticulation modeling for realistic facial animation, Proceedings of ICMI '02, 4th IEEE International Conference on Multimodal Interfaces 2002. Pittsburgh, PA, USA, pp. 505-510.
12. Ezzat, T, Geiger, G., and Poggio, T. (2002). Trainable videorealistic speech animation. Proceedings of ACM SICGRAPH 2002. San Antonio, TX, pp. 388-398.
13. Gustafson, J. Bell, L., Beskow, J., Boye, J., Carlson, R., Edlund, J., Granström, B., House, D., and Wirén, M. (2000). AdApt - a multimodal conversational dialogue system in an apartment domain. Proceedings of the 6th International Conference on Spoken Language Processing (ICSLP'2000). Bejing, China, pp. 134-137.
14. Hällgren, Å. and Lyberg, B. (1998). Visual speech synthesis with concatenative speech. Proceedings of International Conference on Auditory-Visual Speech Processing (AVSP'98). Terrigal, Australia, pp. 181-183.
15. LeGoff, B. (1997). Automatic modeling of coarticulation in text-to-visual speech synthesis. Proceedings of the 5th European Conference on Speech Communication and Technology (EUROSPEECH'97), Rhodos, Greece, pp. 1667-1670.
16. LeGoff, B., Guiard-Marigny, T., Cohen, M.M., and Benoît, C. (1994). Real-time analysis-synthesis and intelligibility of talking faces. Proceedings of the second ESCA/IEEE Workshop on Speech Synthesis. New Paltz, New York, USA.
17. Lindblom, B. (1983). Economy of speech gestures. In P. MacNeilage (Ed.), The Production of Speech. New York: Springer-Verlag, pp. 217-245.
18. Löfqvist, A. (1990). Speech as audible gestures. In W.J. Hardcastle, and A. Marchal, (Eds.), Speech Production and Speech Modelling. Dordrecht: Kluwer Academic Publishers, pp. 289-322.
19. MacLeod, A. and Summerfield, Q. (1990). A procedure for measuring auditory and audio-visual speech-reception thresholds for sentences in noise: Rationale, evaluation, and recommendations for use. British Journal of Audiology, 24:29-43.
20. MacNeilage, P.D. and DeClerk, J.L. (1969). On the motor control of coarticulation in CVC monosyllables. Journal of the Acoustical Society of America, 45:1217-1233.
21. Massaro, D.W. (1998). Perceiving Talking Faces: From Speech Perception to a behavioral Principle. Cambridge, Massachusetts: MIT Press.
22. Massaro, D.W., Cohen, M.M., Tabain, M., Beskow, J., and Clark, R. (In press). Animated speech: Research progress and applications. In E. Vatiokis-Bateson, G. Bailly and P. Perrier (Eds.), Audiovisual Speech Processing. MIT Press.
23. Öhman, S. (1967). Numerical model of coarticulation. Journal of the Acoustical Society of America, 41: 310-320.
24. Parke, F.I. (1982). Parametrized models for facial animation. IEEE Computer Graphics, 2(9): 61-68.
25. Pelachaud, C. (2002). Visual text-to-speech. In I. Pandzic, and R. Forchheimer, (Eds.), MPEG-4 Facial Animation - the Standard, Implementation and Applications. John Wiley and Sons, pp. 125-140.
26. Pelachaud, C., Badler, N.I., and Steedman, M. (1996). Generating facial expressions for speech, Cognitive science, 20(1): 1-46.
27. Perkell, J.S. (1990). Testing theories of speech production: Implications of some detailed analyses of variable articulatory data. In W.J. Hardcastle and A. Marchal, (Eds.), Speech Production and Speech Modelling. Dordrecht: Kluwer Academic Publishers, pp. 263-288.
28. Press, W.H., Teukolsky, S.A., Vetterling, W.T., and Flannery, B.P. (1992). Numerical Recipes in C: The Art of Scientific Computing. - 2nd ed. Cambridge University Press.
29. Reveret, L., Bailly, G., and Badin, P. (2000). Mother: A New generation of talking heads providing a flexible articulatory control for video-realistic speech animation. Proceedings of the 6th International Conference on Spoken Language Processing (ICSLP '2000). Bejing, China, pp. 755-758.
30. Shannon, R.V., Zeng, F.-G., Kamath, V., Wygonski, J., and Ekelid, M. (1995). Speech recognition with primarily temporal cues. Science, 270: 303-304.
31. Siciliano, C., Williams, G., Beskow, J., and Faulkner, A. (2003). Evaluation of a multilingual synthetic talking face as a communication aid for the hearing impaired. To appear in Proceedings of the 15th International Congress of Phonetic Sciences. Barcelona, Spain.
32. Ström, N. (1997). Phoneme probability estimation with dynamic sparsely connected artificial neural networks. The Free Speech Journal, 1(5).
33. Sumby, W.H. and Pollack, I. (1954). Visual contribution to speech intelligibility in noise. Journal of the Acoustical Society of America, 26: 212-215.
34. Yehia, H., Rubin, P., and Vatikiotis-Bateson, E. (1998). Quantitative association of vocal-tract and facial behaviour. Speech Communication, 26: 23-43.
35. Young, S., Odell, J., Ollason, D., Valtchev, V., and Woodland, P. (1997). The HTK Book. Entropic Cambridge Research Laboratory.