Analysis and synthesis of the three-dimensional movements of the head, face, and hand of a speaker using cued speech

Journal of the Acoustical Society of America

Volumn 118, Issue 2, 2005, Pages 1144-1153

  Gibert, Guillaume ^a   Bailly, Gérard ^a,c   Beautemps, Denis ^a   Elisei, Frédéric ^a   Brun, Rémi ^b  

^a CNRS   (France)

^b Attitude Studio SA   (France)

^c GRENOBLE INSTITUTE OF TECHNOLOGY   (France)

Author keywords

[No Author keywords available]

Indexed keywords

DATA REDUCTION; MATHEMATICAL MODELS; SPEECH ANALYSIS; STATISTICAL METHODS; TRAJECTORIES;

ACOUSTIC SEGMENTATION; CUED SPEECH; DIPHONES; PHONETIC STRUCTURE;

SPEECH RECOGNITION;

ARTICLE; BODY POSTURE; CONSONANT; FACIAL EXPRESSION; FEMALE; GESTURE; HAND MOVEMENT; HEAD MOVEMENT; HEARING IMPAIRMENT; HUMAN; HUMAN EXPERIMENT; LIP READING; NORMAL HUMAN; PRINCIPAL COMPONENT ANALYSIS; PRIORITY JOURNAL; SPEECH ARTICULATION; SPEECH INTELLIGIBILITY; STATISTICAL MODEL; VOWEL;

ACOUSTIC STIMULATION; COMPUTER SIMULATION; CUES; FACE; FEMALE; HAND; HEAD; HUMANS; IMAGING, THREE-DIMENSIONAL; MANUAL COMMUNICATION; MODELS, BIOLOGICAL; MODELS, STATISTICAL; MOVEMENT; NONVERBAL COMMUNICATION; PHONETICS; PHOTIC STIMULATION; SOUND SPECTROGRAPHY; SPEECH PERCEPTION;

EID: 23744449511     PISSN: 00014966     EISSN: None     Source Type: Journal    
DOI: 10.1121/1.1944587     Document Type: Article

References (31)

1. Attina, V., Beautemps, D., and Cathiard, M.-A. (2002). "Coordination of hand and orofacial movements for CV sequences in French Cued Speech," International Conference on Speech and Language Processing, Boulder, pp. 1945-1948.
2. Attina, V., Beautemps, D., and Cathiard, M.-A. (2003a). "Temporal organization of French Cued Speech production," International Conference of Phonetic Sciences, Barcelona, Spain.
3. Attina, V., Beautemps, D., Cathiard, M.-A., and Odisio, M. (2003b). "Towards an audiovisual synthesizer for Cued Speech: rules for CV French syllables," Auditory-Visual Speech Processing, St Jorioz, France, pp. 227-232.
4. Attina, V., Beautemps, D., Cathiard, M.-A., and Odisio, M. (2004). "A pilot study of temporal organization in Cued Speech production of French syllables: rules for a Cued Speech synthesizer," Speech Commun. 44, 197-214.
5. Badin, P., Bailly, G., Revéret, L., Baciu, M., Segebarth, C., and Savariaux, C. (2002). "Three-dimensional linear articulatory modeling of tongue, lips and face based on MRI and video images," J. Phonetics 30(3) 533-553.
6. Bailly, G., and Badin, P. (2002). "Seeing tongue movements from outside," International Conference on Speech and Language Processing, Boulder, Colorado, pp. 1913-1916.
7. Bailly, G., Gibert, G., and Odisio, M. (2002). "Evaluation of movement generation systems using the point-light technique," IEEE Workshop on Speech Synthesis, Santa Monica, CA, pp. 27-30.
8. Bailly, G., Holm, B., and Aubergé, V. (2004). "A trainable prosodic model: learning the contours implementing communicative functions within a superpositional model of intonation," International Conference on Speech and Language Processing, Jeju, Korea, pp. 1425-1428.
9. Bailly, G., and Holm, B. (to be published). "SFC: a trainable prosodic model," Speech Commun.
10. Bailly, G., Bérar, M., Elisei, F., and Odisio, M. (2003). "Audiovisual speech synthesis," International Journal of Speech Technology 6(4) 331-346.
11. Bernstein, L. E., Demorest, M. E., and Tucker, P. E. (2000). "Speech perception without hearing," Percept. Psychophys. 62 233-252.
12. Campbell, N. (1997). "Computing prosody: Computational models for processing spontaneous speech," Synthesizing Spontaneous Speech, edited by Y. Sagisaka, N. Campbell, and N. Higuchi (Springer-Verlag, Berlin), pp. 165-186.
13. Castiello, U., Paulignan, Y., and Jeannerod, M. (1991). "Temporal dissociation of motor responses and subjective awareness," Brain 114, 2639-2655.
14. Cornett, R. O. (1967). "Cued Speech," Am. Ann. Deaf 112, 3-13.
15. Cornett, R. O. (1988). "Cued Speech, manual complement to lipreading, for visual reception of spoken language. Principles, practice and prospects for automation," Acta Otorhinolaryngol. Belg. 42, 375-384.
16. Cornett, R. O., and Daisey, M. E. (1992). The Cued Speech Resource Book for Parents of Deaf Children. Raleigh, NC, The National Cued Speech Association, Inc.
17. Duchnowski, P., Lum, D. S., Krause, J. C., Sexton, M. G., Bratakos, M. S., and Braida, L. D. (2000). "Development of speechreading supplements based on automatic speech recognition," IEEE Trans. Biomed. Eng. 47(4): 487-496.
18. Elisei, F., Odisio, M., Bailly, G., and Badin, P. (2001). "Creating and controlling video-realistic talking heads," Auditory-Visual Speech Processing Workshop, Scheelsminde, Denmark, pp. 90-97.
19. Engwall, O., and Beskow, J. (2003). "Resynthesis of 3D tongue movements from facial data," EuroSpeech, Geneva.
20. Hunt, A. J., and Black, A. W. (1996). "Unit selection in a concatenative speech synthesis system using a large speech database," International Conference on Acoustics, Speech and Signal Processing, Atlanta, GA, pp. 373-376.
21. Jiang, J., Alwan, A., Bernstein, L., Keating, P., and Auer, E. (2000). "On the Correlation between facial movements, tongue movements and speech acoustics," International Conference on Speech and Language Processing, Beijing, China, pp. 42-45.
22. Kuratate, T., Munhall, K. G., Rubin, P. E., Vatikioti-Bateson, E., and Yehia, H. (1999). "Audio-visual synthesis of talking faces from speech production correlates," EuroSpeech, pp. 1279-1282.
23. Leybaert, J. (2000). "Phonology acquired through the eyes and spelling in deaf children," J. Exp. Child Psychol. 75, 291-318.
24. Leybaert, J. (2003). "The role of Cued Speech in language processing by deaf children: An overview," Auditory-Visual Speech Processing, St Jorioz, France, pp. 179-186.
25. Minnis, S., and Breen, A. P. (1998). "Modeling visual coarticulation in synthetic talking heads using a lip motion unit inventory with concatenative synthesis," International Conference on Speech and Language Processing, Beijing, China, pp. 759-762.
26. Nicholls, G., and Ling, D. (1982). "Cued Speech and the reception of spoken language," J. Speech Hear. Res. 25, 262-269.
27. Odisio, M., and Bailly, G. (2004). "Shape and appearance models of talking faces for model-based tracking," Speech Commun. 44, 63-82.
28. Owens, E., and Blazek, B. (1985). "Visemes observed by hearing-impaired and normal-hearing adult viewers," J. Speech Hear. Res. 28, 381-393.
29. Revéret, L., Bailly, G., and Badin, P. (2000). "MOTHER: a new generation of talking heads providing a flexible articulatory control for video-realistic speech animation," International Conference on Speech and Language Processing, Beijing, China, pp. 755-758.
30. Uchanski, R., Delhorne, L., Dix, A., Braida, L., Reed, C., and Durlach, N. (1994). "Automatic speech recognition to aid the hearing impaired: Prospects for the automatic generation of cued speech," J. Rehabil. Res. Dev. 31, 20-41.
31. Yehia, H. C., Rubin, P. E., and Vatikiotis-Bateson, E. (1998). "Quantitative association of vocal-tract and facial behavior," Speech Commun. 26, 23-43.