Audio-visual speech synthesis from French text: Eight years of models, designs and evaluation at the ICP

Speech Communication

Volumn 26, Issue 1-2, 1998, Pages 117-129

  Benoît, Christian ^a,b   Le Goff, Bertrand ^a  

^a CNRS   (France)

^b ADVANCED TELECOMMUNICATIONS RESEARCH INSTITUTE INTERNATIONAL   (Japan)

Author keywords

3D lip model; Coarticulation; Face animation; French visemes; Intelligibility; Loudness; Speaking rate; Speechreading; Text to audiovisual speech synthesis

Indexed keywords

SPEECH ANALYSIS; SPEECH COMMUNICATION; SPEECH INTELLIGIBILITY;

AUDIOVISUAL SPEECH SYNTHESIS; COARTICULATION EFFECT; SPEECH READING;

SPEECH SYNTHESIS;

EID: 0032178686     PISSN: 01676393     EISSN: None     Source Type: Journal    
DOI: 10.1016/S0167-6393(98)00045-4     Document Type: Article

References (42)

1. Abry, C., Lallouache, M.T., 1991. Audibility and stability of articulatory movements: Deciphering two experiments on anticipatory rounding in French. In: Proceedings of the XIIth International Congress of Phonetic Sciences, Aix-en-Provence, France, Vol. 1, pp. 220-225.
2. Adjoudani, A., 1993. Elaboration d'un modèle de lèvres 3D pour animation en temps réel. Mémoire de DEA SIP, INPG, Grenoble, France.
3. Alissali, M., 1993. Architecture logicielle pour la synthèse multilingue de la parole, PhD Thesis. Institut National Polytechnique, Grenoble, France.
4. Bailly, G., 1997. Learning to speak. Sensori-motor control of speech movements. Speech Communication 22, 251-267.
5. Bailly, G., Guerti, M., 1991. Synthesis-by-rule for French. In: Proceedings of the 12th International Congress of Phonetic Sciences, Aix-en-Provence, France, Vol. 2, pp. 506-511.
6. Bailly, G., Alissali, M., 1992. COMPOST: a server for multilingual text-to-speech system. Traitement du Signal 9 (4), 359-366.
7. Benguerel, A.P., Cowan, H.A., 1974. Coarticulation of upper lip protrusion in French. Phonetica 30, 41-55.
8. Benoît, C., Lallouache, T., Mohamadi, T., Abry, C., 1992. A set of French visemes for visual speech synthesis. In: Bailly, G., Benoît, C (Eds.), Talking Machines: Theories, Models and Designs. Elsevier, Amsterdam, pp. 485-504.
9. Benoît, C., Mohamadi, T., Kandel, S., 1994. Audio-visual intelligibility of French speech in noise. Journal of Speech and Hearing Research 37, 1195-1203.
10. Benoît, C., Fuster-Duran, A., Le Goff, B., 1996a. An investigation of hypo- and hyper-speech in the visual modality. In: Proceedings of ETRW 96, Autrans, France, pp. 237-240.
11. Benoît, C., Guiard-Marigny, T., Le Goff, B., Adjoudani, A., 1996b. Which components of the face do humans and machines best speechread?. In: Stork, D., Hennecke, M. (Eds.), Speechreading by Humans and Machines, NATO-ASI Series 150 Springer, Berlin, pp. 315-328.
12. Benoît, C., Adjoudani, A., Guiard-Marigny, T., Le Goff, B., Reveret, L., 1997. A multimedia platform for audio-visual speech processing. In: Proceedings of the 5th Eurospeech Conference, ESCA, Rhodes, Greece, Vol. 3, pp. 1671-1674.
13. Beskow, J., 1995. Rule-based visual speech synthesis. In: Proceedings of Eurospeech'95, Madrid, Spain, Vol. 1, pp. 299-302.
14. Brooke, N.M., Summerfield, A.Q., 1983. Analysis, synthesis and perception of visible articulatory movements. Journal of Phonetics 11, 63-76.
15. Charpentier, F., Moulines, E., 1989. Pitch-synchronous wave-form processing technique for text-to-speech synthesis using diphones. In: Proceedings of the First Eurospeech Conference, ESCA, Paris, France, Vol. 2, pp. 13-19.
16. Cohen, M.M., Massaro, D.W., 1990. Synthesis of visible speech. Behavior Research Methods, Instruments and Computers 22, 260-263.
17. Cohen, M.M., Massaro, D.W., 1993. Modeling coarticulation in synthetic visual speech. Models and techniques. In: Thalmann, N.M., Thalmann, D. (Eds.), Computer Animation. Springer, Tokyo, pp. 139-156.
18. Cornett, R.O., 1967. Cued speech. American Annals of the Deaf 112, 3-13.
19. Erber, N.P., 1969. Interaction of audition and vision in the recognition of oral speech stimuli. Journal of Speech and Hearing Research 12, 423-425.
20. Erber, N.P., 1975. Auditory-visual perception of speech. Journal of Speech and Hearing Research 40, 481-492.
21. Fisher, C.G., 1968. Confusion among visually perceived consonants. Journal of Speech and Hearing Research 15, 474-482.
22. Fowler, C., 1983. Converging sources of evidence on spoken and perceived rhythms of speech: Cyclic production of vowels in monosyllabic stress feet. Journal of Experimental Psychology: Human Perception and Performance 112, 386-412.
23. Guiard-Marigny, T., 1992. Animation en temps réel d'un modèle paramétrisé de lèvres. Mémoire de DEA SIP, INPG, Grenoble, France, p. 72.
24. Guiard-Marigny, T., Adjoudani, A., Benoît, C., 1996. 3D models of the lips and jaw for visual speech synthesis. In: van Santen, J.P.H., Sproat, R., Olive, J., Hirshberg, J. (Eds.), Progress in Speech Synthesis. Springer, New York, pp. 247-258.
25. Joos, M., 1948. Acoustic phonetics. Language 24, 1-136.
26. Le Goff, B., 1993. Commandes paramétriques d'un modèle de visage 3D pour animation en temps réel. Mémoire de DEA SIP, INPG, Grenoble, France.
27. Le Goff, B., 1997a. Automatic modeling of coarticulation in text-to-visual speech synthesis. In: Proceedings of the 5th Eurospeech Conference, ESCA, Rhodes, Greece, Vol. 3, pp. 1667-1670.
28. Le Goff, B., 1997b. Synthèse àpartir du texte de visage 3D parlant français. Mémoire de thèse, INP, Grenoble, France, p. 256.
29. Le Goff, B., Benoît, C., 1996. A text-to-audiovisual-speech synthesizer for French. In: Proceedings of the 4th International Conference on Spoken Language Processing, Philadelphia, PA, USA, Vol. 4, pp. 2163-2166.
30. Le Goff, B., Benoît, C., 1997. A French-speaking synthetic head. In: Benoît, C., Campbell, R. (Eds.), Proceedings of the ESCA workshop on Audio-Visual Speech Processing, Rhodes, Greece, pp. 145-148.
31. Le Goff, B., Guiard-Marigny, T., Benoît, C., 1996. Analysis-synthesis and intelligibility of a talking face. In: Van Santen, J.P.H., Sproat, R.W., Olive, J.P., J. Hirschberg (Eds.), Progress in Speech Synthesis. Springer. New York, pp. 235-246.
32. Löfquist, A., 1990. Speech as audible gestures. In: Hardcastle, W.J., Marchal, A. (Eds.), Speech Production and Speech Modeling. Kluwer Academic Publishers, Dordrecht, pp. 289-322.
33. Massaro, D.W., 1997. Perceiving Talking Faces. MIT Press, Cambridge, MA.
34. McGurk, H., MacDonald, J., 1976. Hearing lips and seeing voices. Nature 264, 746-748.
35. Mohamadi, T., 1993. Synthèse àpartir du texte de visages parlants: Réalisation d'un prototype et mesures d'intelligibilité bimodale. Mémoire de thèse, INP, Grenoble, 174 pp.
36. Parke, F.I., 1974. A parametric model for human faces. Ph.D Dissertation, University of Utah, Department of Computer Sciences.
37. Saintourens, M., Tramus, M.H., Huitric, H., Nahas, M., 1990. Creation of a synthetic face speaking in real time with a synthetic voice. In: Benoît, C., Bailly, G. (Eds.), Proceedings of the 1st ESCA Workshop on Speech Synthesis, Autrans, France, pp. 249-252.
38. Saltzman, E., Munhall, K., 1989. A dynamical approach to gestural patterning in speech production. Ecological Psychology 1, 333-382.
39. Sumby, W.H., Pollack, I., 1954. Visual contribution to speech intelligibility in noise. Journal of the Acoustical Society of America 26, 212-215.
40. Summerfield, Q., MacLeod, A., McGrath, M., Brooke, M., 1989. Lips, teeth, and the benefits of lipreading. In: Young, A.W., Ellis, H.D. (Eds.), Handbook of Research on Face Processing. Elsevier, Amsterdam, pp. 223-233.
41. Woodward, P., 1991. Synthèse de visage parlant. DEA Dissertation, ENSERG, INP Grenoble.
42. Woodward, P., Mohamadi, T., Benoît, C., Bailly, G., 1992. Synthèse àpartir du texte d'un visage parlant français. Actes des 19èmes Journées dEtude sur la Parole, GFCP-SFA, Brussels, Belgium, pp. 319-324.