Large Margin Training of Continuous Density Hidden Markov Models

Automatic Speech and Speaker Recognition: Large Margin and Kernel Methods

Volumn , Issue , 2009, Pages 101-114

  Sha, Fei ^a   Saul, Lawrence K ^b  

^a University of Southern California ^*  (United States)

^b UNIVERSITY OF CALIFORNIA   (United States)

Author keywords

Automatic speech recognition (ASR); CD HMMS and large margin training; CD HMMs parameter estimation and ASR and machine learning; CD HMMs trained by margin maximization; Continuous Density Hidden Markov Models (CD HMMs) and ASR; Hamming distance and margin constraints; Large margin training framework; minimum classification error (MCE) training and sequence misclassifications; Phonetic recognition in TIMIT speech corpus

Indexed keywords

EID: 78649306402     PISSN: None     EISSN: None     Source Type: Book    
DOI: 10.1002/9780470742044.ch7     Document Type: Chapter

References (27)

1. Altun Y, Tsochantaridis I and Hofmann T 2003 Hidden Markov support vector machines. Proceedings of the 20th International Conference on Machine Learning (ICML-03) (eds. T Fawcett and N Mishra), Washington, DC. AAAI Press, pp. 3-10.
2. Bahl LR, Brown PF, de Souza PV and Mercer RL 1986 Maximum mutual information estimation of hidden Markov model parameters for speech recognition. Proceedings of the IEEE International Conference on Acoustics, Speech, and Signal Processing, Tokyo, pp. 49-52.
3. Gopalakrishnan PS, Kanevsky D, Nádas A and Nahamoo D 1991 An inequality for rational functions with applications to some statistical estimation problems. IEEE Transactions on Information Theory 37(1), 107-113.
4. Jiang H and Li X 2007 Incorporating training errors for large margin HMMs under semi-definite programming framework. Proceedings of the IEEE International Conference on Acoustics, Speech, and Signal Processing (ICASSP-07), Honolulu, HI, vol. 4, pp. 629-632.
5. Jiang H, Li X and Liu C 2006 Large margin hidden Markov models for speech recognition. IEEE Transactions on Audio, Speech and Language Processing 14(5), 1584-1595.
6. Juang BH and Katagiri S 1992 Discriminative learning for minimum error classification. IEEE Transactions on Signal Processing 40(12), 3043-3054.
7. Kapadia S, Valtchev V and Young S 1993 MMI training for continuous phoneme recognition on the TIMIT database. Proceedings of the IEEE International Conference on Acoustics, Speech, and Signal Processing (ICASSP-93), Minneapolis, MN, vol. 2, pp. 491-494.
8. Keshet J, Shalev-Shwartz S, Bengio S, Singer Y and Chazan D 2006 Discriminative kernel-based phoneme sequence recognition. Proceedings of the International Conference on Spoken Language Processing (ICSLP-06), Pittsburgh, PA.
9. Lafferty J, McCallum A and Pereira FCN 2001 Conditional random fields: Probabilistic models for segmenting and labeling sequence data. Proceedings of the 18th International Conference on Machine Learning (ICML-01). Morgan Kaufmann, San Francisco, CA, pp. 282-289.
10. Lamel LF, Kassel RH and Seneff S 1986 Speech database development: design and analsysis of the acoustic-phonetic corpus. Proceedings of the DARPA Speech Recognition Workshop (ed. LS Baumann), pp. 100-109.
11. Lee KF and Hon HW1988 Speaker-independent phone recognition using hiddenMarkov models. IEEE Transactions on Acoustics, Speech, and Signal Processing 37(11), 1641-1648.
12. Li J, YuanMand Lee C 2007 Approximate test risk bound minimization through soft margin estimation. IEEE Transactions on Speech, Audio and Language Processing 15(8), 2392-2404.
13. Li X, Jiang H and Liu C 2005 Large margin HMMs for speech recognition. Proceedings of the IEEE International Conference on Acoustics, Speech, and Signal Processing (ICASSP-05), Philadelphia, PA, pp. 513-516.
14. McDermott E, Hazen TJ, Roux JL, Nakamura A and Katagiri S 2006 Discriminative training for large vocabulary speech recognition using minimum classification error. IEEE Transactions on Speech and Audio Processing 15(1), 203-223.
15. Nádas A 1983 A decision-theoretic formulation of a training problem in speech recognition and a comparison of training by unconditional versus conditional maximum likelihood. IEEE Transactions on Acoustics, Speech and Signal Processing 31(4), 814-817.
16. Povey D, Kanevsky D, Kingsbury B, Ramabhadran B, Saon G and Visweswariah K 2008 Boosted MMI for model and feature-space discriminative training. Proceedings of the IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP-08), Las Vegas, NV.
17. Povey D and Woodland P 2002 Minimum phone error and i-smoothing for improved discriminative training. Proceedings of the IEEE International Conference on Acoustics, Speech, and Signal Processing (ICASSP-02), Orlando, FL, pp. 105-108.
18. Robinson T 1994 An application of recurrent nets to phone probability estimation. IEEE Transactions on Neural Networks 5(2), 298-305.
19. Roux JL and McDermott E 2005 Optimization methods for discriminative training. Proceedings of 9th European Conference on Speech Communication and Technology (EuroSpeech-05), Lisbon, Portugal, pp. 3341-3344.
20. Sha F 2007 Large margin training of acoustic models for speech recognition University of Pennsylvania, Philadelphia, PA, PhD thesis.
21. Sha F and Saul LK 2006 Large margin Gaussian mixture modeling for phonetic classification and recognition. Proceedings of the IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP-06), Toulouse, France, pp. 265-268.
22. Sha F and Saul LK 2007a Comparison of large margin training to other discriminative methods for phonetic recognition by hidden Markov models. Proceedings of the IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP-07), Honolulu, HI, pp. 313-316.
23. Sha F and Saul LK 2007b Large margin hidden Markov models for automatic speech recognition. Advances in Neural Information Processing Systems 19 (eds. B Schölkopf, J Platt and T Hofmann). MIT Press, Cambridge, MA, pp. 1249-1256.
24. Taskar B, Guestrin C and Koller D 2004 Max-margin Markov networks. Advances in Neural Information Processing Systems (NIPS 16) (eds. S Thrun, L Saul and B Schölkopf). MIT Press, Cambridge, MA, pp. 25-32.
25. Valtchev V, Odell JJ, Woodland PC and Young SJ 1997 MMIE training of large vocabulary recognition systems. Speech Communication 22, 303-314.
26. Woodland PC and Povey D 2002 Large scale discriminative training of hidden Markov models for speech recognition. Computer Speech and Language 16, 25-47.
27. Yu D, Deng L, He X and Acero A 2006 Use of incrementally regulated discriminative margins in MCE training for speech recognition. Proceedings of the International Conference on Spoken Language Processing (ICSLP-06), Pittsburgh, PA, pp. 2418-2421.