Quantitative association of vocal-tract and facial behavior

Speech Communication

Volumn 26, Issue 1-2, 1998, Pages 23-43

  Yehia, Hani ^a   Rubin, Philip ^b   Vatikiotis Bateson, Eric ^c  

^a FEDERAL UNIVERSITY OF MINAS GERAIS   (Brazil)

^b YALE UNIVERSITY   (United States)

^c ADVANCED TELECOMMUNICATIONS RESEARCH INSTITUTE INTERNATIONAL   (Japan)

Author keywords

Dynamic time warping (DTW); Facial motion; Line spectrum pair (LSP); Linear estimator; Principal component analysis; Singular value decomposition; Vocal tract motion

Indexed keywords

CORRELATION METHODS; SPEECH ANALYSIS;

DYNAMIC TIME WARPING (DTW); LINE SPECTRUM PAIR (LSP); LINEAR ESTIMATORS; PRINCIPAL COMPONENT ANALYSIS (PCA); SINGULAR VALUE DECOMPOSITION (SVD); SPEECH ACOUSTICS; VOCAL-TRACT MOTION;

SPEECH COMMUNICATION;

EID: 0032178592     PISSN: 01676393     EISSN: None     Source Type: Journal    
DOI: 10.1016/S0167-6393(98)00048-X     Document Type: Article

References (44)

1. Atal, B.S., Chang, J.J., Tukey, J.W., 1978. Inversion of articulatory-to-acoustic transformation in the vocal-tract by a computing sorting technique. J. Acoust. Soc. Amer. 63 (5), 1535-1555.
2. Badin, P., Beautemps, D., Laboissiere, R., Schwartz, J.L., 1995. Recovery of vocal tract geometry from formants for vowels and fricative consonants using a midsagittal-to-area function conversion model. Journal of Phonetics 23, 221-229.
3. Carter, J.N., Shadle, C.H., Davis, C.J., 1996. On the use of structured light in speech research. In: Proceedings of The First ESCA Tutorial and Research Workshop on Speech Production Modeling and Fourth Speech Production Seminar, pp. 229-232.
4. Fant, G., 1960. Acoustic Theory of Speech Production. The Hague.
5. Hogden, J., 1993. An unsupervised method for learning to track tongue position from ana acoustic signal. Status Report on Speech Research SR-115/116, Haskins Laboratories, New Haven, CT, USA.
6. Horn, R., Johnson, C., 1985. Matrix Analysis. Cambridge.
7. Itakura, F., 1975. Line spectrum representation of linear predictive coefficients of speech signals. J. Acoust. Soc. Amer. 57, 535.
8. Kaburagi, T., Honda, M., 1994. Determination of sagittal tongue shape from the positions of points on the tongue surface. J. Acoust. Soc. Amer. 56, 1356-1366.
9. Kelso, J.A.S., Tuller, B., 1984. Converging evidence in support of common dynamic principles for speech and movement coordination. Amer. J. Psychol. 15, R928-R935.
10. Lin, Q., 1990. Speech production theory and articulatory speech synthesis. Dissertatie, Royal Institute of Technology (KTH), Stockholm.
11. Maeda, S., 1982. A digital simulation method of the vocal-tract system. Speech Communication 1 (3,4), 199-229.
12. McGowan, R.S., 1994. Recovering articulatory movement from formant frequency trajectories using task dynamics and a genetic algorithm: Preliminary model tests. Speech Communication 14, 19-48.
13. Mermelstein, P., 1967. Determination of vocal-tract shape from measured formant frequencies. J. Acoust. Soc. Amer. 41 (5), 1283-1294.
14. Mermelstein, P., 1973. Articulatory model for the study of speech production. J. Acoust. Soc. Amer. 53 (4), 1070-1082.
15. Munhall, K.G., 1985. An examination of intra-articulator relative timing. J. Acoust. Soc. Amer. 78, 1548-1553.
16. Nittrouer, S., Munhall, K.G., Kelso, J.A.S., Tuller, B., Harris, K.S., 1988. Patterns of interarticulator phasing and their relation to linguistic structure. J. Acoust. Soc. Amer. 84, 1653-1661.
17. Ostry, D.J., Munhall, K.G., 1994. Control of jaw orientation and position in mastication and speech. Journal of Neurophysiology 71, 1528-1545.
18. Perkell, J.S., Cohen, M.H., Svirsky, M.A., Matthies, M.L., Garabieta, I., Jackson, M.T.T., 1992. Electromagnetic midsagittal articulometer systems for transducing speech articulatory movements. J. Acoust. Soc. Amer. 92 (6), 3078-3096.
19. Rabiner, L., Juang, B.W., 1993. Fundamentals of Speech Recognition. Prentice-Hall, Englewood Cliffs, NJ.
20. Rubin, P.E., Baer, T., Mermelstein, P., 1981. An articulatory synthesizer for perceptual research. J. Acoust. Soc. Amer. 70, 321-328.
21. Saltzman, E., Munhall, K.G., 1989. A dynamical approach to gestural patterning in speech production. Ecological Psychology 1, 333-382.
22. Schroeder, M.R., 1967. Determination of the geometry of the human vocal-tract by acoustical measurements. J. Acoust. Soc. Amer. 41 (4), 1002-1010.
23. Schroeter, J., Sondhi, M., 1991. Speech coding based on physiological models of speech production. In: Sondhi, M.M., Furui, S. (Eds.), Advances in Speech Processing. Marcel Dekker, New York, pp. 231-268.
24. Schroeter, J., Sondhi, M.M., 1994. Techniques for estimating vocal-tract shapes from the speech signal. IEEE Trans. Speech Audio Process. 2 (1), 133-150.
25. Scully, C., 1990. Articulatory synthesis. In: Hardcastle, W.J., Marchal, A. (Eds.), Speech Production and Speech Modelling. Kluwer Academic Publishers, Dordrecht, pp. 151-186.
26. Shirai, K., 1993. Estimation and generation of articulatory motion using neural networks. Speech Communication 13, 45-51.
27. Sondhi, M.M., Schroeter, J., 1987. A hybrid time-frequency domain articulatory speech synthesizer. IEEE Trans. Acoust. Speech Signal Process. 35 (7), 955-967.
28. Stevens, K., House, A., 1955. Development of a quantitative description of vowel articulation. J. Acoust. Soc. Amer. 27 (3), 484-493.
29. Stone, M., 1990. A three-dimensional model of tongue movement based on ultrasound and X-ray microbeam data. J. Acoust. Soc. Amer. 87 (5), 2207-2217.
30. Sugamura, N., Itakura, F., 1986. Speech analysis and synthesis methods developed at ECL in NTT - From LPC to LSP -. Speech Communication 5, 199-215.
31. Tiede, M.K., Vatikiotis-Bateson, E., 1994. Extracting articulator movement parameters from a videodisc-based cineradiographic database. In: Proc. International Conference on Spoken Language Processing, pp. S02-4.1-S02-4.4.
32. Vatikiotis-Bateson, E., Ostry, D.J., 1995. An analysis of the dimensionality of jaw motion in speech. Journal of Phonetics 23, 101-117.
33. Vatikiotis-Bateson, E., Yehia, H., 1996. Physiological modeling of facial motion during speech. H-96 65, The Acoustical Society of Japan.
34. Vatikiotis-Bateson, E., Yehia, H.C., 1997. Unified physiological model of audible-visible speech production. In: Fifth European Conference on Speech Communication and Technology.
35. Vatikiotis-Bateson, E., Munhall, K.G., Kasahara Y., Garcia, F., Yehia, H., 1996a. Characterizing audiovisual information during speech. In: Proceedings of the International Conference on Spoken Language Processing, pp. 1485-1488.
36. Vatikiotis-Bateson, E., Munhall, K.G., Hirayama, M., Lee, Y.C., Terzopoulos, D., 1996b. The dynamics of audiovisual behavior in speech. In: Stork, D., Hennecke, M. (Eds.), Speech Reading by Humans and Machines, Vol. 150, NATO-ASI Series, Series F, Computers and Systems Sciences. Springer, Berlin, pp. 221-232.
37. Vatikiotis-Bateson, E., Kuratate, T., Tiede, M.K., Yehia, H.C., 1998. Kinematics-based synthesis of realistic talking faces. Technical Report TR-H-237, ATR-HIP.
38. Yehia, H., Itakura, F., 1994. Determination of human vocaltract dynamic geometry from formant trajectories using spatial and temporal Fourier analysis. In: Proc. IEEE International Conference on Acoustics, Speech and Signal Processing, pp. 477-480.
39. Yehia, H., Itakura, F., 1996. A method to combine acoustical and morphological constraints in the speech production inverse problem. Speech Communication 18 (2), 151-174.
40. Yehia, H., Takeda, K., Itakura, F., 1996. An acoustically oriented vocal-tract model. IEICE Transactions on Information and Systems E79 (D-8), 1198-1208.
41. Yehia, H., Takeda, K., Itakura, F., in review. An analysis of the acoustic-to-articulatory mapping during speech under morphological and continuity constraints. Speech Communication.
42. Yehia, H.C., Rubin, P., Vatikiotis-Bateson, E., 1997. Quantitative association of orofacial and vocal-tract shapes. In: European Tutorial and Research Workshop on Audio-Visual Speech Processing: Computational and Cognitive Science Approaches.
43. Zacks, S., 1971. The Theory of Statistical Inference. Wiley, New York.
44. Zacks, S., Thomas, T.R., 1994. A new neural network for articulatory speech recognition and its application to vowel identification. Computer Speech and Language 8, 189-209.