Margin-Based discriminative training for string recognition

IEEE Journal on Selected Topics in Signal Processing

Volumn 4, Issue 6, 2010, Pages 917-925

  Heigold, Georg ^a   Dreuw, Philippe ^a   Hahn, Stefan ^a   Schluter, Ralf ^a   Ney, Hermann ^a  

^a RWTH AACHEN UNIVERSITY   (Germany)

Author keywords

Handwriting recognition; large vocabulary continuous speech recognition; margin based training; part of speech tagging

Indexed keywords

ADDITIONAL LOSS; AUDIO DATA; DE FACTO STANDARD; DISCRIMINATIVE TRAINING; GENERALIZATION BOUND; HANDWRITING RECOGNITION; LARGE VOCABULARY CONTINUOUS SPEECH RECOGNITION; LARGE-MARGIN CLASSIFIERS; LOSS FUNCTIONS; MARGIN-BASED TRAINING; MAXIMUM MUTUAL INFORMATION; MINIMUM PHONE ERROR; PART OF SPEECH TAGGING; SMOOTH APPROXIMATION;

CONTINUOUS SPEECH RECOGNITION; STEREOPHONIC BROADCASTING;

CHARACTER RECOGNITION;

EID: 78649262962     PISSN: 19324553     EISSN: None     Source Type: Journal    
DOI: 10.1109/JSTSP.2010.2076110     Document Type: Article

References (34)

1. W. Macherey, L. Haferkamp, R. Schlüter, and H. Ney, "Investigations on error minimizing training criteria for discriminative training in automatic speech recognition," in Proc. Interspeech, Lisbon, Portugal, Sep. 2005.
2. X. He, L. Deng, and W. Chou, "Discriminative learning in sequential pattern recognition\A unifying review for optimization-oriented speech recognition," IEEE Signal Process. Mag., vol. 25, no. 5, pp. 14-36, Sep. 2008.
3. V. Vapnik, The Nature of Statistical Learning Theory. New York: Springer-Verlag, 1995.
4. Y. Yin and H. Jiang, "A fast optimization method for large margin estimation of HMMs based on second order cone programming," in Proc. Interspeech, Antwerp, Belgium, Aug. 2007.
5. F. Sha and L. Saul, "Comparison of large margin training to other discriminative methods for phonetic recognition by hidden Markov models," in Proc. IEEE Int. Conf. Acoust., Speech, Signal Process. (ICASSP), Honolulu, HI, Apr. 2007.
6. H. Jiang and X. Li, "Incorporating training errors for large margin HMMs under semi-definite programming framework," in IEEE Int. Conf. Acoust., Speech, Signal Process. (ICASSP), Honolulu, HI, USA, Apr. 2007, pp. 313-316.
7. J. Li, Z. Yan, C.-H. Lee, and R. Wang, "A study on soft margin estimation for LVCSR," in Proc. IEEE Autom. Speech Recognition Understanding Workshop (ASRU), Kyoto, Japan, Dec. 2007.
8. D. Yu, L. Deng, X. He, and A. Acero, "Large-margin minimum classification error training: A theoretical risk minimization perspective," Comput. Speech Lang., vol. 22, pp. 415-429, 2008.
9. G. Saon and D. Povey, "Penalty function maximization for large margin HMM training," in Proc. Interspeech, Brisbane, Australia, Sep. 2008.
10. Y. Altun, I. Tsochantaridis, and T. Hofmann, "Hidden Markov support vector machines," in Proc. Int. Conf. Machine Learning (ICML), Washington, DC, 2003.
11. B. Taskar, C. Guestrin, and D. Koller, "Max-margin Markov networks," Adv. Neural Inf. Process. Syst. (NIPS), pp. 873-880, 2003.
12. E. McDermott, S. Watanabe, and A. Nakamura, "Margin-space integration of MPE loss via differencing of MMI functionals for generalized error-weighted discriminative training," in Proc. Interspeech, Brighton, U.K., Sep. 2009.
13. G. Heigold, T. Deselaers, R. Schlüter, and H. Ney, "Modified MMI/MPE: A direct evaluation of the margin in speech recognition," in Proc. Int. Conf. Mach. Learn. (ICML), Helsinki, Finland, Jul. 2008.
14. G. Heigold, R. Schlüter, and H. Ney, "Modified MPE/MMI in a transducer-based framework," in Proc. IEEE Int. Conf. Acoust., Speech, Signal Process. (ICASSP), Taipei, Taiwan, Apr. 2009, pp. 3749-3752.
15. J. Zhang, R. Jin, Y. Yang, and A. Hauptmann, "Modified logistic regression: An approximation to SVM and its applications in large-scale text categorization," in Proc. Int. Conf. Mach. Learn. (ICML), Aug. 2003.
16. D. Povey, D. Kanevsky, B. Kingsbury, B. Ramabhadran, G. Saon, and K. Visweswariah, "Boosted MMI for model and feature-space discriminative training," in Proc. IEEE Int. Conf. Acoust., Speech, Signal Process. (ICASSP), Las Vegas, NV, Apr. 2008, pp. 4057-4060.
17. E. McDermott and A. Nakamura, "Flexible discriminative training based on equal error group scores obtained from an error-indexed forward-backward algorithm," in Proc. Interspeech, Brisbane, Australia, Sep. 2008.
18. D. Povey, "Discriminative training for large vocabulary speech recognition," Ph.D. dissertation, Cambridge Univ., Cambridge, U.K., 2004.
19. A. Nádas, D. Nahamoo, and M. Picheny, "On a model-robust training method for speech recognition," IEEE Trans. Acoust., Speech, Signal Process., vol. 36, no. 9, pp. 1432-1436, Sep. 1988.
20. W. Macherey, R. Schlüter, and H. Ney, "Discriminative training with tied covariance matrices," in Proc. Interspeech, Jeju Island, Korea, Oct. 2004.
21. M. Riedmiller and H. Braun, "A direct adaptive method for faster back-propagation learning: The Rprop algorithm," in Proc. IEEE Int. Conf. Neural Networks (ICNN), San Francisco, CA, 1993, pp. 586-592.
22. G. Heigold, P. Lehnen, R. Schlüter, and H. Ney, "On the equivalence of Gaussian and log-linear HMMs," in Proc. Interspeech, Brisbane, Australia, Sep. 2008.
23. L. Devillers, H. Maynard, and S. Rosset et al., "The French Media/Evalda project: The evaluation of the understanding capability of spoken language dialog systems," in Proc. Int. Conf. Lang. Resources Eval. (LREC), Lisbon, Portugal, May 2004.
24. J. Lafferty, A. McCallum, and F. Pereira, "Conditional random fields: Probabilistic models for segmenting and labeling sequence data," in Proc. Int. Conf. Mach. Learn. (ICML), San Francisco, CA, Jun.-Jul. 2001.
25. P. Dreuw, G. Heigold, and H. Ney, "Confidence-based discriminative training for model adaptation in offline Arabic handwriting recognition," in Proc. Int. Conf. Document Anal. Recogn. (ICDAR), Barcelona, Spain, Jul. 2009.
26. M. Bisani and H. Ney, "Bootstrap estimates for confidence intervals in ASR performance evaluation," in Proc. IEEE Int. Conf. Acoust., Speech, Signal Process. (ICASSP), Montreal, QC, Canada, May 2004, pp. 409-412.
27. T. Eisele, R. Haeb-Umbach, and D. Langmann, "A comparative study of linear feature transformation techniques for automatic speech recognition," in Proc. Int. Conf. Spoken Lang. Process. (ICSLP), Philadelphia, PA, Oct. 1996.
28. G. Heigold, D. Rybach, R. Schlüter, and H. Ney, "Investigations on convex optimization using log-linear HMMs for digit string recognition," in Proc. Interspeech, Brighton, U.K., Sep. 2009.
29. J. Lööf, C. Gollan, S. Hahn, G. Heigold, B. Hoffmeister, C. Plahl, D. Rybach, R. Schlüter, and H. Ney, "The RWTH 2007 TC-STAR evaluation system for European English and Spanish," in Proc. Interspeech, Antwerp, Belgium, Aug. 2007.
30. B. Hoffmeister, C. Plahl, P. Fritz, G. Heigold, J. Lööf, R. Schlüter, and H. Ney, "Development of the 2007 RWTH Mandarin LVCSR system," in Proc. IEEE Autom. Speech Recogn. Understanding Workshop (ASRU), Kyoto, Japan, Dec. 2007.
31. S. Hahn, P. Lehnen, G. Heigold, and H. Ney, "Optimizing CRFs for SLU tasks in various languages using modified training criteria," in Proc. Interspeech, Brighton, U.K., Sep. 2009.
32. K. Marasek and R. Gubrynowicz, "Design and data collection for spoken Polish dialogs database," in Proc. Int. Conf. Lang. Resources Eval. (LREC), Marrakech, Morocco, May 2008.
33. M. Pechwitz, S. S. Maddouri, V. Mägner, N. Ellouze, and H. Amiri, "ICDAR 2007 Arabic handwriting recognition competition," in Col-loque Int. Francophone sur l'Ecrit et le Document (CIFED), Ham-mamet, Tunisia, Oct. 2002.
34. V. Märgner and H. Abed, "ICDAR 2007 Arabic handwriting recognition competition," in Proc. Int. Conf. Document Anal. Recogn. (ICDAR), Curitiba, Brazil, Sep. 2007