Articulatory information for noise robust speech recognition

IEEE Transactions on Audio, Speech and Language Processing

Volumn 19, Issue 7, 2011, Pages 1913-1924

  Mitra, Vikramjit ^a   Nam, Hosung ^b   Espy Wilson, Carol Y ^a   Saltzman, Elliot ^b,c   Goldstein, Louis ^b,d  

^a UNIVERSITY OF MARYLAND   (United States)

^b HASKINS LABORATORIES   (United States)

^c BOSTON UNIVERSITY   (United States)

^d UNIVERSITY OF SOUTHERN CALIFORNIA   (United States)

Author keywords

Articulatory phonology; articulatory speech recognition; artificial neural networks (ANNs); noise robust speech recognition; speech inversion; task dynamic model; vocal tract variables

Indexed keywords

ARTICULATORY PHONOLOGY; ARTICULATORY SPEECH RECOGNITION; ARTIFICIAL NEURAL NETWORKS (ANNS); NOISE-ROBUST SPEECH RECOGNITION; SPEECH INVERSION; TASK DYNAMIC MODEL; VOCAL-TRACTS;

ACOUSTIC NOISE; FEATURE EXTRACTION; NEURAL NETWORKS; RESEARCH; VOCABULARY CONTROL;

SPEECH RECOGNITION;

EID: 79960545035     PISSN: 15587916     EISSN: None     Source Type: Journal    
DOI: 10.1109/TASL.2010.2103058     Document Type: Article

References (67)

1. P. Lockwood and J. Boudy, "Experiments with a non linear spectral subtractor (NSS), hidden Markov models and the projection, for robust speech recognition in cars," in Proc. Eurospeech, 1991, pp. 79-82.
2. S. Srinivasan and D. L. Wang, "Transforming binary uncertainties for robust speech recognition," IEEE Trans Audio, Speech, Lang. Process., vol. 15, no. 7, pp. 2130-2140, Sep. 2007.
3. O. Deshmukh, C. Espy-Wilson, and L. H. Carney, "Speech enhancement using the modified phase opponency model," J. Acoust. Soc. Amer., vol. 121, no. 6, pp. 3886-3898, 2007. (Pubitemid 46872142)
4. R. Flynn and E. Jones, "Combined speech enhancement and auditory modelling for robust distributed speech recognition," Speech Commun., vol. 50, pp. 797-809, 2008.
5. H. Hermansky and N. Morgan, "RASTA processing of speech," IEEE Trans. Speech Audio Process., vol. 2, no. 4, pp. 578-589, Oct. 1994.
6. C. Chen and J. Bilmes, "MVA processing of speech features," IEEE Trans Audio, Speech, Lang. Process., vol. 15, no. 1, pp. 257-270, Jan. 2007.
7. T. M. Sullivan, "Multi-Microphone correlation-based processing for robust automatic speech recognition," Ph.D. dissertation, Carnegie Mellon Univ., Pittsburgh, PA, 1996.
8. H. You, Q. Zhu, and A. Alwan, "Entropy-based variable frame rate analysis of speech signals and its application to ASR," in Proc. ICASSP, 2004, pp. 549-552.
9. B. Strope and A. Alwan, "A model of dynamic auditory perception and its application to robust word recognition," IEEE Trans. Speech Audio Process., vol. 5, no. 5, pp. 451-464, Sep. 1997. (Pubitemid 127746017)
10. Speech Processing, Transmission and Quality Aspects (STQ); Distributed Speech Recognition; Front-End Feature Extraction Algorithm; Compression Algorithms, ETSI ES 201 108 Ver. 1.1.3, 2003.
11. Speech Processing, Transmission and Quality Aspects (STQ); Distributed Speech Recognition; Adv. Frontend Feature Extraction Algorithm; Compression Algorithms, ETSI ES 202 050 Ver. 1.1.5, 2007.
12. B. Kingsbury, G. Saon, L. Mangu, M. Padmanabhan, and R. Sarikaya, "Robust speech recognition in noisy environments: The 2001 IBM spin evaluation system," in Proc. ICASSP, 2002, vol. 1, pp. I-53-I-56.
13. M. J. F. Gales and S. J. Young, "Robust continuous speech recognition using parallel model combination," IEEE Trans. Speech Audio Process., vol. 4, no. 5, pp. 352-359, Sep. 1996. (Pubitemid 126753023)
14. C. Leggetter and P.Woodland, "Maximum likelihood linear regression for speaker adaptation of continuous density hidden Markov models," Comput., Speech Lang., vol. 9, pp. 171-185, 1995.
15. Z. Zhang and S. Furui, "Piecewise-linear transformation-based HMM adaptation for noisy speech," Speech Commun., vol. 42, no. 1, pp. 43-58, Jan. 2004.
16. M. Cooke, P. Green, L. Josifovski, and A. Vizinho, "Robust automatic speech recognition with missing and uncertain acoustic data," Speech Commun., vol. 34, pp. 267-285, 2001. (Pubitemid 32284867)
17. J. Barker, L. Josifovski, M. P. Cooke, and P. D. Green, "Soft decisions in missing data techniques for robust automatic speech recognition," in Proc. Int. Conf. Spoken Lang. Process., 2000, pp. 373-376.
18. L. Josifovski, M. Cooke, P. Green, and A. Vizinho, "State based imputation of missing data for robust speech recognition and speech enhancement," in Proc. Eurospeech, 1999, vol. 6, pp. 2833-2836.
19. M. Ostendorf, "Moving beyond the 'beads-on-a-string' model of speech," in Proc. IEEE Autom. Speech Recognition Understanding Workshop, CO, 1999, vol. 1, pp. 79-83.
20. J. Sun and L. Deng, "An overlapping-feature-based phonological model incorporating linguistic constraints: Applications to speech recognition," J. Acoust. Soc. Amer., vol. 111, no. 2, pp. 1086-1101, Feb. 2002. (Pubitemid 34127489)
21. D. Jurafsky, W.Ward, Z. Jianping, K. Herold, Y. Xiuyang, and Z. Sen, "What kind of pronunciation variation is hard for triphones to model?," in Proc. ICASSP, 2001, vol. 1, pp. 577-580. (Pubitemid 32839316)
22. K. N. Stevens, "Toward a model for speech recognition," J. Acoust. Soc. Amer., vol. 32, pp. 47-55, 1960.
23. R. Cole, R. M. Stern, and M. J. Lasry, , J. S. Perkell and D. Klatt, Eds., "Performing fine phonetic distinctions: Templates versus features," in Invariance and Variability of Speech Processes. Hillsdale, NJ: Lawrence Erlbaum Assoc., 1986, ch. 15, pp. 325-345.
24. B. Lochschmidt, "Acoustic-phonetic analysis based on an articulatory model," in Automatic Speech Analysis and Recognition, J. P. Hayton, Ed. Dordrecht, The Netherlands: D. Reidel, 1982, pp. 139-152.
25. R. D. Mori, P. Laface, and E. Piccolo, "Automatic detection and description of syllabic features in continuous speech," IEEE Trans. Acoust., Speech, Signal Process., vol. ASSP-24, no. 5, pp. 365-379, Oct. 1976.
26. O. Schmidbauer, "Robust statistic modelling of systematic variabilities in continuous speech incorporating acoustic-articulatory relations," in Proc. ICASSP, 1989, pp. 616-619. (Pubitemid 20604192)
27. L. Deng, "A generalized hidden Markov model with state-conditioned trend functions of time for the speech signal," Signal Process., vol. 27, no. 1, pp. 65-78, 1992.
28. L. Deng and D. Sun, "A statistical approach to ASR using atomic units constructed from overlapping articulatory features," J. Acoust. Soc. Amer., vol. 95, pp. 2702-2719, 1994. (Pubitemid 24152864)
29. K. Erler and L. Deng, "Hidden Markov model representation of quantized articulatory features for speech recognition," Comput., Speech Lang., vol. 7, pp. 265-282, 1993. (Pubitemid 23705305)
30. S. King and P. Taylor, "Detection of phonological features in continuous speech using neural networks," Comput. Speech Lang., vol. 14, no. 4, pp. 333-353, Oct. 2000.
31. N. Chomsky and M. Halle, The Sound Pattern of English. New York: Harper & Row, 1968.
32. J. Harris, English Sound Structure. Oxford, U.K.: Wiley-Blackwell, 1994.
33. S. King, J. Frankel, K. Livescu, E. McDermott, K. Richmond, and M. Wester, "Speech production knowledge in automatic speech recognition," J. Acoust. Soc. Amer., vol. 121, no. 2, pp. 723-742, 2007. (Pubitemid 46192674)
34. K. Richmond, "Estimating articulatory parameters from the acoustic speech signal," Ph.D. dissertation, Univ. of Edinburgh, Edinburgh, U.K., 2001.
35. K. Stevens, Acoustic Phonetics. Cambridge, MA: MIT Press, 2000.
36. B. S. Atal, J. J. Chang, M. V. Mathews, and J. W. Tukey, "Inversion of articulatory-to-acoustic transformation in the vocal tract by a computer sorting technique," J. Acoust. Soc. Amer., vol. 63, pp. 1535-1555, 1978. (Pubitemid 8346208)
37. M. G. Rahim, C. C. Goodyear, W. B. Kleijn, J. Schroeter, and M. Sondhi, "On the use of neural networks in articulatory speech synthesis," J. Acoust. Soc. Amer., vol. 93, no. 2, pp. 1109-1121, 1993. (Pubitemid 23059838)
38. G. Papcun, J. Hochberg, T. R. Thomas, F. Laroche, J. Zachs, and S. Levy, "Inferring articulation and recognizing gestures from acoustics with a neural network trained on x-ray microbeam data," J. Acoust. Soc. Amer., vol. 92, no. 2, pp. 688-700, 1992.
39. P. Ladefoged, R. Harshman, L. Goldstein, and L. Rice, "Generating vocal tract shapes from formant frequencies," J. Acoust. Soc. Amer., vol. 64, no. 4, pp. 1027-1035, 1978. (Pubitemid 9053326)
40. J. Hogden, A. Lofqvist, V. Gracco, I. Zlokarnik, P. Rubin, and E. Saltzman, "Accurate recovery of articulator positions from acoustics: New conclusions based on human data," J. Acoust. Soc. Amer., vol. 100, no. 3, pp. 1819-1834, 1996. (Pubitemid 26307570)
41. T. Okadome, S. Suzuki, and M. Honda, "Recovery of articulatory movements from acoustics with phonemic information," in Proc. 5th Seminar Speech Prod., Bavaria, Germany, 2000, pp. 229-232.
42. C. Qin and M. Á. Carreira-Perpiñán, "An empirical investigation of the nonuniqueness in the acoustic-to-articulatory mapping," in Proc. Interspeech, 2007, pp. 74-77.
43. D. Neiberg, G. Ananthakrishnan, and O. Engwall, "The acoustic to articulation mapping: Non-Linear or non-unique?," in Proc. Interspeech, 2008, pp. 1485-1488.
44. J. Frankel and S. King, "ASR-Articulatory speech recognition," in Proc. Eurospeech, 2001, pp. 599-602.
45. J. Frankel, K. Richmond, S. King, and P. Taylor, "An automatic speech recognition system using neural networks and linear dynamic models to recover and model articulatory traces," in Proc. ICSLP, 2000, vol. 4, pp. 254-257.
46. R. Daniloff and R. Hammarberg, "On defining coarticulation," J. Phon., vol. 1, pp. 239-248, 1973.
47. C. P. Browman and L. Goldstein, "Towards an articulatory phonology," Phonol. Yearbook, vol. 85, pp. 219-252, 1986.
48. C. P. Browman and L. Goldstein, "Articulatory phonology: An overview," Gynecol. Obstet. Invest., vol. 49, pp. 155-180, 1992.
49. E. Saltzman and K. Munhall, "A dynamical approach to gestural patterning in speech production," Ecol. Psychol., vol. 1, no. 4, pp. 332-382, 1989.
50. H. Nam, L. Goldstein, E. Saltzman, and D. Byrd, "TADA: An enhanced, portable task dynamics model in Matlab," J. Acoust. Soc. Amer., vol. 115, no. 5, p. 2430, 2004.
51. J. Westbury, X-ray Microbeam Speech Production Database User's Handbook. Madison, WI: Univ. of Wisconsin, 1994.
52. R. S. McGowan, "Recovering articulatory movement from formant frequency trajectories using task dynamics and a genetic algorithm: Preliminary model tests," Speech Commun., vol. 14, no. 1, pp. 19-48, Feb. 1994.
53. V. Mitra, H. Nam, C. Espy-Wilson, E. Saltzman, and L. Goldstein, "Retrieving tract variables from acoustics: A comparison of different machine learning strategies," IEEE J. Sel. Topics Signal Process., vol. 4, no. 6, pp. 1027-1045, Dec. 2010.
54. K. Kirchhoff, "Robust speech recognition using articulatory information," Ph.D. dissertation, Univ. of Bielefeld, Bielefeld, Germany, 1999.
55. K. Kirchhoff, G. A. Fink, and G. Sagerer, "Combining acoustic and articulatory feature information for robust speech recognition," Speech Commun., vol. 37, no. 3-4, pp. 303-319, Jul. 2002. (Pubitemid 34524845)
56. M. Richardson, J. Bilmes, and C. Diorio, "Hidden-articulator Markov models for speech recognition," Speech Commun., vol. 41, no. 2-3, pp. 511-529, Oct. 2003.
57. V. Mitra, H. Nam, C. Espy-Wilson, E. Saltzman, and L. Goldstein, "Noise robustness of tract variables and their application to speech recognition," in Proc. Interspeech, 2009, pp. 2759-2762.
58. H. M. Hanson and K. N. Stevens, "A quasiarticulatory approach to controlling acoustic source parameters in a Klatt-type formant synthesizer using HLsyn," J. Acoust. Soc. Amer., vol. 112, no. 3, pp. 1158-1182, 2002. (Pubitemid 35006671)
59. D. Pearce and H. G. Hirsch, "The aurora experimental framework for the performance evaluation of speech recognition systems under noisy conditions," in Proc. Autom. Speech Recognition: Challenges For New Millenium, ASR-2000, Paris, France, 2000, pp. 181-188.
60. H. Nam, V. Mitra, M. Tiede, E. Saltzman, L. Goldstein, C. Espy-Wilson, and M. Hasegawa-Johnson, "A procedure for estimating gestural scores from natural speech," in Proc. Interspeech, 2010, pp. 30-33.
61. V. Mitra, I. Özbek, H. Nam, X. Zhou, and C. Espy-Wilson, "From acoustics to vocal tract time functions," in Proc. ICASSP, 2009, pp. 4497-4500.
62. J. Hogden, D. Nix, and P. Valdez, An articulatorily constrained, maximum likelihood approach to speech recognition Los Alamos National Laboratory, Los Alamos, NM, Tech. Rep., LA-UR-96-3945, 1998.
63. L. Gillick and S. Cox, "Some statistical issues in the comparison of speech recognition algorithms," in Proc. ICASSP, 1989, vol. 1, pp. 532-535. (Pubitemid 20604171)
64. V. Mitra, B. J. Borgstrom, C. Espy-Wilson, and A. Alwan, "A noisetype and level-dependent MPO-based speech enhancement architecture with variable frame analysis for noise-robust speech recognition," in Proc. Interspeech, Brighton, U.K., 2009, pp. 2751-2754.
65. Y. Ephraim and D. Malah, "Speech enhancement using a minimum mean square log-spectral amplitude estimator," IEEE Trans. Acoust., Speech, Signal Process., vol. ASSP-33, no. 2, pp. 443-445, Apr. 1985.
66. X. Xiao, J. Li, E. S. Chng, H. Li, and C. Lee, "A study on the generalization capability of acoustic models for robust speech recognition," IEEE Trans. Audio, Speech, Lang. Process., vol. 18, no. 6, pp. 1158-1169, Aug. 2009.
67. X. Cui and Y. Gong, "A study of variable-parameter gaussian mixture hidden Markov modeling for noisy speech recognition," IEEE Trans. Audio, Speech, Lang. Process., vol. 15, no. 4, pp. 1366-1376, May 2007.