Inverting mappings from smooth paths through Rn to paths through Rm: A technique applied to recovering articulation from acoustics

Speech Communication

Volumn 49, Issue 5, 2007, Pages 361-383

  Hogden, John ^a   Rubin, Philip ^b   McDermott, Erik ^c   Katagiri, Shigeru ^d   Goldstein, Louis ^b  

^a LOS ALAMOS NATIONAL LABORATORY   (United States)

^b HASKINS LABORATORIES   (United States)

^c NTT Corporation   (Japan)

^d DOSHISHA UNIVERSITY   (Japan)

Author keywords

Channel normalization; Dimensionality reduction; Speech inverse problem

Indexed keywords

ACOUSTICS; CONFORMAL MAPPING; INVERSE PROBLEMS; SIGNAL ANALYSIS; SPEECH RECOGNITION; TIME SERIES ANALYSIS;

CHANNEL NORMALIZATION; DIMENSIONALITY REDUCTION; SPEECH INVERSE PROBLEM; SPEECH PERCEPTION;

SPEECH ANALYSIS;

EID: 34247647975     PISSN: 01676393     EISSN: None     Source Type: Journal    
DOI: 10.1016/j.specom.2007.02.008     Document Type: Article

References (101)

1. Abuelma'atti M. Harmonic and intermodulation distortion in carbon microphones. Appl. Acoust. 31 (1990) 233-243
2. Aczel J., and Dhombres J. In: Rota G.-C. (Ed). Functional Equations in Several Variables with Applications to Mathematics, Information Theory, and the Natural and Social Sciences. Encyclopedia of Mathematics and its Applications Series Vol. 31 (1989), Cambridge University Press, Cambridge
3. Ahalt S., Krishnamurthy A., Chen P., and Melton D. Competitive learning algorithms for vector quantization. Neural Networks 3 (1990) 277-290
4. Atal B.S., Chang J.J., Mathews M.V., and Tukey J.W. Inversion of articulatory-to-acoustic transformation in the vocal tract by a computer-sorting technique. J. Acoust. Soc. Amer. 63 5 (1978) 1535-1555
5. Badin P., Beautemps D., Laboissière R., and Schwartz J.-L. Recovery of vocal tract geometry from formants for vowels and fricative consonants using a midsagittal-to-area function conversion method. J. Phonetics 23 (1995) 221-229
6. Balchandran R., and Mammone R. Non-parametric estimation and correction of non-linear distortion in speech systems. Proc. Internat. Conf. Acoust. Speech Signal Process. Vol. 2 (1998) 749-752
7. Beautemps D., Badin P., and Bailly G. Linear degrees of freedom in speech production: analysis of cineradio- and labio-film data and articulatory-acoustic modeling. J. Acoust. Soc. Amer. 109 5 (2001) 2165-2180
8. Bendat J. Nonlinear Systems Techniques and Applications (1998), Wiley, New York
9. Beneviste A., and Goursat M. Blind Equalizers. IEEE Trans. Communications 32 (1984) 871-883
10. Blackburn C.S., and Young S. Enhanced speech recognition using an articulatory production model trained on X-ray data. Comput. Speech Language 15 (2001) 195-215
11. Blesser B. Speech perception under conditions of spectral transformation: I. Phonetic characteristics. J. Speech Hear. Res. 15 (1972) 5-41
12. Boe L.J., Perrier P., and Bailly G. The geometric vocal tract variables controlled for vowel production: proposals for constraining acoustic-to-articulatory conversion. J. Phonetics 20 (1992) 27-38
13. Carreira-Perpinan, M.A., 2001. Continuous latent variable models for dimensionality reduction and sequential data reconstruction. Unpublished Ph.D. Dissertation, University of Sheffield, Sheffield, UK.
14. Coker C. A model of articulatory dynamics and control. Proc. IEEE 64 4 (1976) 452-460
15. Dempster A., Laird N., and Rubin D. Maximum likelihood from incomplete data via the EM algorithm (with discussion). J. Roy. Statist. Soc. Ser. B 39 (1977) 1-38
16. Deng L. A dynamic, feature-based approach to the interface between phonology and phonetics for speech modeling and recognition. Speech Comm. 24 (1998) 299-323
17. Deng L., Ramsay G., and Sun D. Production models as a structural basis for automatic speech recognition. Speech Comm. 22 (1997) 93-111
18. Dillon W., and Goldstein M. Multivariate Analysis: Methods and Applications (1984), Wiley, New York
19. Dusan, S., Deng, L., 2000. Acoustic-to-articulator inversion using dynamical and phonological constraints. In: Proc. 5th Seminar on Speech Production, pp. 237-240.
20. Edelman A., Arias T., and Smith S. The geometry of algorithms with orthogonality constraints. SIAM J. Matrix Anal. Appl. 20 2 (1998) 303-353
21. Fant G. Acoustic Theory of Speech Production. second ed. (1970), Mouton and Co, The Hague
22. Flanagan J. Speech Analysis, Synthesis, and Perception. second ed. (1972), Springer, New York
23. Fowler C., and Turvey M. Immediate compensation in bite-block speech. Phonetica 37 (1980) 306-326
24. Frankel J., and King S. ASR - articulatory speech recognition. Proc. Eurospeech (2001) 599-602
25. Frankel, J., King, S., 2001b. Speech recognition in the articulatory domain: Investigating an alternative to acoustic HMMs. In: Proc. Workshop for Innovations in Speech Processing.
26. Gray R. Vector quantization. IEEE Acoust. Speech Signal Process. Mag. (1984) 4-29
27. Guenther F., Hampson M., and Johnson D. A theoretical investigation of reference frames for the planning of speech movements. Psychol. Rev. 105 4 (1998) 611-633
28. Gupta S., and Schroeter J. Pitch-synchronous frame-by-frame and segment-based articulatory analysis by synthesis. J. Acoust. Soc. Amer. 94 5 (1993) 2517-2530
29. Hirayama, M., Vatikiotis-Bateson, E., Honda, K., Koike, Y., Kawato, M., 1992. Physiologically based speech synthesis. Poster presented at Neural Information Processing Systems (92).
30. Hiroya S., and Honda M. Estimation of articulatory movements from speech acoustics using an HMM-based speech production model. IEEE Trans. Speech Audio Process. 12 2 (2004) 175-185
31. Hogden, J., 1991. Low-dimensional phoneme mapping using a continuity constraint. Unpublished Doctoral Dissertation, Stanford University, Stanford, CA.
32. Hogden, J., 1995. A maximum likelihood approach to estimating speech articulator positions from speech acoustics. In: Neural Information Processing Systems (95), Workshop on Neural Networks for Speech Processing, Vail, CO.
33. Hogden, J., 2000. USA Patent # 6,052,662.
34. Hogden, J., Valdez, P., 2000. Bridging the gap between speech production and speech recognition. In: Proc. 5th Seminar on Speech Production: Models and Data, Kloster Seeon, Bavaria, May 1st-4th.
35. Hogden J., Rubin P., and Saltzman E. An unsupervised method for learning to track tongue position from an acoustic signal. J. Acoust. Soc. Amer. 91 4 (1992) 2443 (A)
36. Hogden, J., Saltzman, E., Rubin, P., 1993. Tracking moving objects with unsupervised neural networks. Paper presented at the World Conference on Neural Networks, Portland, OR.
37. Hogden J., Rubin P., and Saltzman E. An unsupervised method for learning to track tongue position from an acoustic signal. Bull. Communication Parlee 3 (1996) 101-116
38. Hogden J., Zlokarnik I., Lofqvist A., Gracco V., Rubin P., and Saltzman E. Accurate recovery of articulator positions from acoustics - new conclusions based on human data. J. Acoust. Soc. Amer. 100 3 (1996)
39. Hogden, J., Nix, D., Valdez, P., 1998. Maximum likelihood continuity mapping: bridging the gap between production and recognition. Paper presented at the 9th Hub-5 Conversational Speech Recognition Workshop, Linthicum Heights, MD, USA, September 24th-25th.
40. Hogden J., Valdez P., Katagiri S., and McDermott E. Blind inversion of multidimensional functions for speech enhancement. Proc. Eurospeech Conf. (2003) 1409-1412
41. Kaburagi T., and Honda M. Dynamic articulatory model based on multidimensional invariant-feature task representation. J. Acoust. Soc. Amer. 110 1 (2001) 441-452
42. Kambhatle N., and Leen T. Dimension reduction by local PCA. Neural Computation 9 (1997) 1-18
43. Kimber, D., 1994. Geometric methods for nonparametric modeling of dynamical systems. Unpublished Ph.D. Dissertation, Stanford University, Stanford, CA.
44. Kirchhoff K. Robust Speech Recognition using Articulatory Information (No. TR-98-037) (1998), International Computer Science Institute, Berkeley, CA
45. Kuc, R., Tutuer, F., Vaisnys, J.R., 1985. Determining vocal tract shape by applying dynamic constraints. In: Proc. Internat. Conf. on Acoustics Speech and Signal Processing, Tampa, FL.
46. Levin D. Representations of sound that are insensitive to spectral filtering and parameterization procedures. J. Acoust. Soc. Amer. 111 5 (2002) 2257-2271
47. Liberman A., Cooper F., Shankweiler D., and Studdert-Kennedy M. Perception of the speech code. Psychol. Rev. 74 6 (1967) 431-461
48. Liberman A., and Mattingly I. The motor theory of speech perception revised. Cognition 21 (1985) 1-36
49. Licklider J., and Pollack I. Effects of differentiation, integration, and infinite peak clipping upon the intelligibility of speech. J. Acoust. Soc. Amer. 20 (1948) 42-51
50. Lindblom B. Role of articulation in speech perception: clues from production. J. Acoust. Soc. Amer. 99 3 (1996) 1683-1692
51. Lindblom B., Lubker J., and Gay T. Formant frequencies of some fixed-mandible vowels and a model of speech motor programming by predictive simulation. J. Phonetics 7 (1979) 146-161
52. Lyons R. Understanding Digital Signal Processing. second ed. (2004), Prentice-Hall, Upper Saddle River, NJ
53. Maeda S. An articulatory model of the tongue based on a statistical analysis. J. Acoust. Soc. Amer. 65 (1979) S22
54. Markel J., and Gray A. Linear Prediction of Speech (1976), Springer, New York
55. Maxson C., and Meyer J. How many subspaces force linearity?. Amer. Math. Monthly 108 6 (2001) 531-536
56. McDermott E., and Nakamura A. Production-oriented models for speech recognition. IEICE Trans. Special Issue on Statistical Modeling for Speech Processing E89-D 3 (2006) 1006-1014
57. McGowan, R., 1987. Articulatory synthesis: numerical solution of a hyperbolic differential equation. Haskins Laboratories Status Report on Speech Research, SR-89/90.
58. McGowan R., and Faber A. Introduction to papers on speech recognition and perception from an articulatory view. J. Acoust. Soc. Amer. 99 3 (1996) 1680-1682
59. McGowan R., and Lee M. Task dynamic and articulatory recovery of lip and velar approximations under model mismatch conditions. J. Acoust. Soc. Amer. 99 1 (1996) 595-608
60. Mermelstein P. Articulatory model for the study of speech production. J. Acoust. Soc. Amer. 53 4 (1973) 1070-1082
61. Moody, J., 1999. Visualizing speech with a recurrent neural network trained on human acoustic-articulator data. Unpublished Ph.D. Dissertation, University of California, San Diego, CA.
62. Morris, R., Clements, M., 2001. Maximum-likelihood compensation of zero-memory nonlinearities in speech signals. In: Proc. Internat. Conf. on Acoustics, Speech, and Signal Processing, pp. 289-292.
63. Muller E., and McLeod G. Perioral biomechanics and its relation to labial motor control. J. Acoust. Soc. Amer. 78 Suppl. 1 (1982) S38
64. Nelson W. Articulatory feature analysis - I. Initial processing considerations (1977), Bell Laboratories, Memorandum
65. Nix, D., 1998. Machine learning methods for inferring vocal-tract articulation from speech acoustics. Unpublished Ph.D. Thesis, University of Colorado, Boulder, CO.
66. Oppenheim A. Speech analysis-synthesis system based on homomorphic filtering. J. Acoust. Soc. Amer. 45 2 (1969) 458-465
67. Ouni S., and Laprie Y. Modeling the articulatory space using a hypercube codebook for acoustic-to-articulatory inversion. J. Acoust. Soc. Amer. 118 1 (2005) 444-460
68. Paliwal K., and Alsteris L. On the usefulness of STFT phase spectrum in human listening tests. Speech Comm. 45 (2005) 153-170
69. Papcun G., Hotchberg J., Thomas T., Laroche F., Zacks J., and Levy S. Inferring articulation and recognizing gestures from acoustics with a neural network trained on X-ray microbeam data. J. Acoust. Soc. Amer. 92 2 (1992) 688-700
70. Perkell J., Cohen M., Svirsky M., Matthies M., Garabieta I., and Jackson M. Electromagnetic midsagittal articulometer systems for transducing speech articulatory movements. J. Acoust. Soc. Amer. 92 6 (1992) 3078-3096
71. Perkell J., Mathies M., Svirsky M., and Jordan M. Trading relations between tongue-body raising and lip rounding in production of the vowel /u/: A pilot 'motor equivalence' study. J. Acoust. Soc. Amer. 93 5 (1993) 2948-2961
72. Perrier P., Labiossiere R., Abry C., and Maeda S. Speech production: models and data. Speech Comm. 22 (1997) 82-92
73. Qiu W., and Hua Y. A GCD method for blind channel identification. Digital Signal Processing 7 (1997) 199-205
74. Quatieri T.F., Reynolds D.A., and O'Leary G.C. Estimation of handset nonlinearity with application to speaker recognition. IEEE Trans. Speech Audio Process. 8 5 (2000) 567-584
75. Remez R.E., Rubin P.E., Pisoni D.B., and Carrell T.D. Speech perception without traditional speech cues. Science 212 (1981) 947-950
76. Reynolds, D., 1996. The effects of handset variability on speaker recognition performance: experiments on the Switchboard corpus. In: Proc. IEEE Internat. Conf. on Acoustics, Speech, and Signal Processing, pp. 113-116.
77. Richards H., and Bridle J. The HDM: a segmental hidden dynamic model of coarticulation. Proc. IEEE Internat. Conf. Acoustics Speech Signal Process. 1 (1999) 357-360
78. Richards H., Bridle J., Hunt M., and Mason J. Dynamic constraint weighting in the context of articulatory parameter estimation. Proc. Eurospeech 97 (1997)
79. Rose R., Schroeter J., and Sondhi M. The potential role of speech production models in automatic speech recognition. J. Acoust. Soc. Amer. 99 3 (1996) 1699-1709
80. Rosenblum L.D. The primacy of multimodal speech perception. In: Pisoni D., and Remez R. (Eds). Handbook of Speech Perception (2005), Blackwell, Malden, MA 51-78
81. Roweis, S., 1999. Data driven production models for speech processing. Unpublished Ph.D. Thesis, California Institute of Technology, Pasadena, CA.
82. Roweis S., and Alwan A. Towards articulatory speech recognition: learning smooth maps to recover articulator information. Proc. Eurospeech 3 (1997) 1227-1230
83. Roweis S., and Saul L. Nonlinear dimensionality reduction by locally linear embedding. Science 290 (2000) 2323-2326
84. Rubin P., Baer T., and Mermelstein P. An articulatory synthesizer for perceptual research. J. Acoust. Soc. Amer. 70 2 (1981) 321-328
85. Saberi, and Perrott. Cognitive restoration of reversed speech. Nature 398 (1999) 760
86. Saltzman E., and Munhall K. A dynamical approach to gestural patterning in speech production. Ecol. Psychol. 1 4 (1989) 333-382
87. Schroeder M. Determination of the geometry of the human vocal tract by acoustic measurements. J. Acoust. Soc. Amer. 41 4 (1967) 1002-1010
88. Schroeter J., and Sondhi M. Techniques for estimating vocal-tract shapes from the speech signal. IEEE Trans. Speech Audio Process. 2 1 (1994) 133-150
89. Scully C. Model prediction and real speech: fricative dynamics. In: Lindblom B., and Ohman S. (Eds). Frontiers of Speech Comm. Research (1979), Academic Press, New York 35-48
90. Shirai K., and Kobayashi T. Estimating articulatory motion from speech wave. Speech Comm. 5 (1986) 159-170
91. Sondhi M., and Resnick J.R. The inverse problem for the vocal tract: numerical methods, acoustical experiments, and speech synthesis. J. Acoust. Soc. Amer. 73 3 (1983) 985-1002
92. Sorokin V., and Trushkin A.V. Articulatory-to-acoustic mapping for inverse problem. Speech Comm. 19 (1996) 105-118
93. Strang G. Linear Algebra and Its Applications. second ed. (1980), Academic Press, New York
94. Strange W., Jenkins J., and Johnston T. Dynamic specification of coarticulated vowels. J. Acoust. Soc. Amer. 74 (1983) 695-705
95. Suzuki, S., Okadome, T., Honda, M., 1998. Determination of articulatory positions from speech acoustics by applying dynamic articulatory constraints. In: Proc. Internat. Conf. on Spoken Language Perception, pp. 2251-2254.
96. Tenenbaum J., Silva V.d., and Langford J. A global geometric framework for nonlinear dimensionality reduction. Science 290 (2000) 2319-2323
97. Tsimbinos, J., 1995. Identification and compensation of nonlinear distortion. Unpublished Ph.D. Dissertation, University of South Australia, The Levels, Australia.
98. Wakita H. Direct estimation of the vocal tract shape by inverse filtering of acoustic speech waveforms. IEEE Trans. Audio Electroacoust. AU-21 5 (1973) 417-427
99. Whalen D., Wiley E., Rubin P., and Cooper F. The Haskins Laboratories' pulse code modulation (PCM) system. Behavioral Res. Methods Instruments Comput. 22 6 (1990) 550-559
100. Yehia H., Rubin P., and Vatikiotis-Bateson E. Quantitative association of vocal-tract and facial behavior. Speech Comm. 22 1-2 (1998) 23-43
101. Zlokarnik I. Adding articulatory features to acoustic features for automatic speech recognition. J. Acoust. Soc. Amer. 97 5 pt. 2 (1995) 3246 (A)