Trust region-based optimization for maximum mutual information estimation of HMMs in speech recognition

IEEE Transactions on Audio, Speech and Language Processing

Volumn 19, Issue 8, 2011, Pages 2474-2485

  Liu, Cong ^a   Hu, Yu ^a   Dai, Li Rong ^b   Jiang, Hui ^c  

^a IFLYTEK RESEARCH   (China)

^b UNIVERSITY OF SCIENCE AND TECHNOLOGY OF CHINA   (China)

^c YORK UNIVERSITY   (Canada)

Author keywords

Discriminative training; global optimization; lower bounded auxiliary function; trust region problem

Indexed keywords

AUXILIARY FUNCTIONS; CONVERGENCE BEHAVIORS; DISCRIMINATIVE TRAINING; EFFICIENT GLOBAL OPTIMIZATION; FIRST-ORDER APPROXIMATIONS; GAUSSIAN MIXTURES; LARGE VOCABULARY CONTINUOUS SPEECH RECOGNITION; LOCAL OPTIMA; LOWER BOUNDS; MAXIMUM MUTUAL INFORMATION ESTIMATIONS; OBJECTIVE FUNCTIONS; OPTIMAL POINTS; OPTIMIZATION METHOD; PENALTY TERM; QUADRATIC FUNCTION; RECOGNITION PERFORMANCE; REGION-BASED; STATIONARY POINTS; TRUST REGION;

CONTINUOUS SPEECH RECOGNITION; GLOBAL OPTIMIZATION; HIDDEN MARKOV MODELS; STEREOPHONIC BROADCASTING;

OPTIMIZATION;

EID: 80053182852     PISSN: 15587916     EISSN: None     Source Type: Journal    
DOI: 10.1109/TASL.2011.2144969     Document Type: Article

References (22)

1. T. Anastasakos, J. McDonough, R. Schwarts, and J. Makhoul, "A compact model for speaker-adaptive training," in Proc. Int. Conf. Spoken Lang. Process., 1996, pp. 1137-1140.
2. L. R. Bahl, P. F. Brown, P. V. De Souza, and R. L. Mercer, "Maximum mutual information estimation of hidden Markov model parameters for speech recognition," in Proc. ICASSP'86, Tokyo, Japan, 1986, pp. 49-52.
3. A. P. Dempster, N. M. Laird, and D. B. Rubin, "Maximum likelihood from incomplete data via the EM algorithm (with discussion)," J. R. Statist. Soc. B, vol. 39, pp. 1-38, 1977.
4. X. He, L. Deng, and W. Chou, "Discriminative learning in sequential pattern recognition: A unifying review for optimization-based speech recognition," IEEE Signal Process. Mag., vol. 25, no. 5, pp. 14-14-36, Sep. 2008.
5. H. Jiang, X. Li, and C.-J. Liu, "Large margin hidden Markov models for speech recognition," IEEE Trans. Audio, Speech, Lang. Process., vol. 14, no. 5, pp. 1584-1595, Sep. 2006. (Pubitemid 46552926)
6. H. Jiang and X. Li, A General Approximation-Optimization Approach to Large Margin Estimation of HMMs. Vienna, Austria: I-TECH Education and Publishing, 2007, Robust Speech Recognition and Understanding, ch. 7, pp. 103-120.
7. H. Jiang and X. Li, "Parameter estimation of statistical models using convex optimization: An advanced method of discriminative training for speech and language processing," IEEE Signal Process. Mag. , vol. 27, no. 3, pp. 115-127, May 2010.
8. H. Jiang, "Discriminative training of HMMs for automatic speech recognition: A survey," Comput. Speech Lang., vol. 24, no. 4, pp. 589-608, Oct. 2010.
9. B.-H. Juang, W. Chou, and C.-H. Lee, "Minimum classification error rate methods for speech recognition," IEEE Trans. Speech Audio Process., vol. 5, no. 3, pp. 257-265, May 1997. (Pubitemid 127745998)
10. J. L. M. Yuan and C.-. Lee, "Soft margin estimation of hidden Markov model parameters," in Proc. Interspeech'06, 2006, pp. 2422-2425.
11. P. Liu, C. Liu, H. Jiang, F. Soong, and R.-H. Wang, "A constrained line search optimization method for discriminative training of HMMs," IEEE Trans. Audio, Speech, Lang. Process., vol. 16, no. 1, pp. 900-909, Jan. 2008.
12. P. Liu and F. Soong, "A quadratic optimization approach to discriminative training of CDHMMs," IEEE Signal Process. Lett., vol. 16, no. 1, pp. 149-152, Mar. 2009.
13. E. McDermott, T. Hazen, J. L. Roux, A. Nakamura, and S. Katagiri, "Discriminative training for large-vocabulary speech recognition using minimum classification error," IEEE Trans. Audio, Speech, Lang. Process., vol. 15, no. 1, pp. 203-223, Jan. 2007.
14. J. Nocedal and S. J. Wright, Numerical Optimization, 2nd ed. New York: Springer, 2006.
15. D. S. Pallet, W. M. Fisher, and J. G. Fiscus, "Tools for the analysis of benchmark speech recognition tests," in Proc. ICASSP'90, 1990.
16. D. Povey and P. C.Woodland, "Minimum phone error and I-smoothing for improved discriminative training," in Proc. ICASSP'02, Orlando, FL, 2002, pp. 105-108.
17. D. Povey, "Discriminative Training for Large Vocabulary Speech Recognition," Ph.D. dissertation, Cambridge Univ., Cambridge, U.K., 2004.
18. F. Sha and L. K. Saul, "Large margin hidden Markov models for automatic speech recognition," in Proc. NIPS'07, 2007.
19. D. C. Sorensen, "Newton's method with a model trust region modification," SIAM J. Numer. Anal., vol. 19, no. 2, pp. 409-426, Apr. 1982.
20. Z.-J. Yan, C. Liu, Y. Hu, and H. Jiang, "A trust region based optimization for maximum mutual information estimation of HMMs in speech recognition," in Proc. ICASSP09, 2009, pp. 4521-4524.
21. D. Yu, L. Deng, X. He, and A. Acero, "Large-margin minimum classification error training for large-scale speech recognition tasks," in Proc. ICASSP07, 2007, pp. 1137-1140.
22. P. C. Woodland and D. Povey, "Large scale discriminative training of hidden Markov models for speech recognition," Comput. Speech Lang., vol. 16, no. 1, pp. 25-47, Jan. 2002.