Automatic model complexity control using marginalized discriminative growth functions

IEEE Transactions on Audio, Speech and Language Processing

Volumn 15, Issue 4, 2007, Pages 1414-1424

  Liu, Xunying ^a   Gales, Mark ^a  

^a UNIVERSITY OF CAMBRIDGE   (United Kingdom)

Author keywords

Complexity control; Discriminative criteria; Growth functions

Indexed keywords

AUTOMATIC MODELS; BAYESIAN INFORMATION CRITERIONS; BAYESIAN LEARNING; COMPLEXITY CONTROL; CONTROL SCHEMES; CONTROL TECHNIQUES; DISCRIMINATIVE CRITERIA; GROWTH FUNCTIONS; MARGINALIZATION; SPEECH RECOGNITION SYSTEMS; STANDARD PROBLEMS; SYSTEM COMPLEXITY; WORD ERROR RATES;

BAYESIAN NETWORKS; INFORMATION THEORY; LEARNING ALGORITHMS; SPEECH ANALYSIS; STANDARDS;

SPEECH RECOGNITION;

EID: 64149085496     PISSN: 15587916     EISSN: None     Source Type: Journal    
DOI: 10.1109/TASL.2006.889804     Document Type: Article

References (27)

1. A. R. Barron, J. J. Rissanen, and B. Yu, "The minimum description length principle in coding and modeling," IEEE Trans. Inf. Theory, vol. 44, no. 6, pp. 2743-2760, Oct. 1998.
2. W. Chou and W. Reichl, "Decision tree state tying based on penalized Bayesian information criterion," in Proc. ICASSP, Phoenix, AZ, 1999, vol. 1, pp. 345-348.
3. G. Evermann, H. Y. Chan, M. J. F. Gales, T. Hain, X. Liu, D. Mrva, L. Wang, and P. C. Woodland, "Development of the 2003 CU-HTK conversational telephone speech transcription system," in Proc. ICASSP, Montreal, QC, Canada, pp. 249-252.
4. M. J. F. Gales, "Maximum likelihood multiple subspace projection projections for hidden Markov models," IEEE Trans. Speech Audio Process., vol. 10, no. 2, pp. 37-47, Feb. 2002.
5. Z. Ghahramani and M. J. Beal, "Graphical models and variational methods," in Advanced Mean Field Method-Theory and Practice. Cambridge, MA: MIT Press, 2000.
6. V. Goel, S. Kumar, and W. Byrne, "Segmental minimum Bayes-risk decoding for automatic speech recognition," IEEE Trans. Speech Audio Process., vol. 13, no. 3, pp. 234-249, May 2004.
7. P. S. Gopalakrishnan, D. Kanevsky, A. Nadas, and D. Nahamoo, "Generalization of the Baum algorithm to rational objective functions," in Proc. ICASSP, 1989, pp. 631-634.
8. J. Kaiser, B. Horvat, and Z. Kacic, "A novel loss function for the overall risk-criterion based discriminative training of HMM models," in Proc. ICSLP, 2000.
9. X. Liu and M. J. F. Gales, "Automatic model complexity control using marginalized discriminative growth functions," in Proc. ASRU, St. Thomas, U.S. Virgin Islands, 2003, pp. 37-42.
10. X. Liu, M. J. F. Gales, and P. C. Woodland, "Automatic complexity control for HLDA systems," in Proc. ICASSP, Hong Kong, 2003, vol. 1, pp. 132-135.
11. X. Liu and M. J. F. Gales, "Model complexity control and compression using discriminative growth functions," in Proc. ICASSP, Montreal, QC, Canada, 2004, pp. 797-800.
12. [12] --, "Automatic model complexity control using marginalized discriminative growth functions," Cambridge Univ. Eng. Dept., Cambridge, U.K., 2004, Tech. Rep. CUED/F-INFENG/TR-490.
13. A. Nádas, "A decision theorectic formulation of a training problem in speech recognition and a comparision of training by unconditional versus conditional maximum likelihood," IEEE Trans. Acoust., Speech, Signal Process., vol. ASSP-31, no. 4, pp. 814-817, Aug. 1983.
14. R. M. Neal, "Probabilistic inference using markov chain Monte Carlo methods," Dept. Comput. Sci., Univ. Toronto, 1993, Tech. Rep. CGT-TR-93-1.
15. Y. Normandin, "Hidden Markov models maximum mutual information estimation and the speech recognition problem," Ph.D. dissertation, McGill Univ., Montreal, QC, Canada, 1991.
16. M. Padmanabhan and L. R. Bahl, "Model complexity adaptation using a discriminant measure," IEEE Trans. Speech Audio Process., vol. 8, no. 2, pp. 205-208, Mar. 2000.
17. D. Povey and P. C.Woodland, "Minimum phone error and I-smoothing for improved discriminative training," in Proc. ICASSP, 2002, pp. 105-108.
18. D. Povey, "Discriminative training for large vocabulary speech recognition," Ph.D. dissertation, Eng. Dept., Cambridge Univ., Cambridge, U.K., 2003.
19. J. J. Rissanen, "Fisher information and stochastic complexity," IEEE Trans. Inf. Theory, vol. 42, no. 1, pp. 40-47, Jan. 1996.
20. G. Schwartz, "Estimating the dimension of a model," Ann. Statistics, vol. 6, no. 2, pp. 461-464, Feb. 1978.
21. K. Shinoda and T. Watanabe, "Speaker adaptation with autonomous model complexity control by MDL principle," in Proc. ICASSP, Atlanta, GA, 1995, pp. 717-720.
22. [22] --, "Acoustic modeling based on the MDL principle for speech recognition," in Proc. Eurospeech, 1997, vol. 1, pp. 99-102.
23. [23] --, "MDL based context dependent subword modeling for speech recognition," J. Acoust. Soc. Jpn., vol. 21, pp. 79-86, 2000.
24. V. Valtchev, "Discriminative methods for HMM-based speech recognition," Ph.D. dissertation, Eng. Dept., Cambridge Univ., Cambridge, U.K., 1995.
25. S. Watanabe, Y. Minami, A. Nakamura, and N. Ueda, "Variational Bayesian estimation and clustering for speech recognition," IEEE Trans. Speech Audio Process., vol. 12, no. 4, pp. 365-381, Jul. 2004.
26. A. H.Welsh, Aspects of Statistical Inference. NewYork:Wiley, 1996.
27. P. C. Woodland and D. Povey, "Large scale discriminative training of hidden Markov models for speech recognition," Comput. Speech Lang., vol. 16, pp. 25-47, 2002.