Discriminative n-gram language modeling

Computer Speech and Language

Volumn 21, Issue 2, 2007, Pages 373-392

  Roark, Brian ^a   Saraclar, Murat ^b   Collins, Michael ^c  

^a OREGON HEALTH AND SCIENCE UNIVERSITY   (United States)

^b BOGAZICI UNIVERSITY   (Turkey)

^c MASSACHUSETTS INSTITUTE OF TECHNOLOGY   (United States)

Author keywords

[No Author keywords available]

Indexed keywords

ALGORITHMS; AUTOMATA THEORY; COMPUTER SIMULATION; ERROR ANALYSIS; PARAMETER ESTIMATION; SPEECH RECOGNITION;

FIRST PASS RECOGNITION SYSTEMS; MULTI PASS RECOGNITION SYSTEMS; PERCEPTRON ALGORITHMS; WORD ERROR RATE (WER);

COMPUTATIONAL LINGUISTICS;

EID: 33750713654     PISSN: 08852308     EISSN: 10958363     Source Type: Journal    
DOI: 10.1016/j.csl.2006.06.006     Document Type: Article

References (40)

1. Allauzen, C., Mohri, M., Roark, B., 2003. Generalized algorithms for constructing language models. In: Proceedings of the 41st Annual Meeting of the Association for Computational Linguistics. pp. 40-47.
2. Bacchiani M., Riley M., Roark B., and Sproat R. MAP adaptation of stochastic grammars. Computer Speech and Language 20 1 (2006) 41-68
3. Bacchiani, M., Roark, B., Saraclar, M., 2004. Language model adaptation with MAP estimation and the perceptron algorithm. In: Proceedings of the Human Language Technology Conference and Meeting of the North American Chapter of the Association for Computational Linguistics (HLT-NAACL). pp. 21-24.
4. Bahl L.R., Brown P.F., de Souza P.V., and Mercer R.L. Estimating hidden Markov model parameters so as to maximize speech recognition accuracy. IEEE Transactions on Speech and Audio Processing 1 1 (1993) 77-83
5. Balay, S., Gropp, W. D., McInnes, L. C., Smith, B. F., 2002. Petsc users manual, technical Report ANL-95/11-Revision 2.1.2, Argonne National Laboratory.
6. Banerjee, S., Mostow, J., Beck, J., Tam, W., December 2003. Improving language models by learning from speech recognition errors in a reading tutor that listens. In: Proceedings of the Second International Conference on Applied Artificial Intelligence. Fort Panhala, Kolhapur, India.
7. Benson, S.J., McInnes, L.C., Moré, J.J., Sarich, J., 2002. Tao users manual, technical Report ANL/MCS-TM-242-Revision 1.4, Argonne National Laboratory.
8. Benson, S.J., Moré, J.J., 2002. A limited memory variable metric method for bound constrained minimization, preprint ANL/ACSP909-0901, Argonne National Laboratory.
9. Chen, Z., Lee, K.-F., Li, M.J., October 2000. Discriminative training on language model. In: Proceedings of the Sixth International Conference on Spoken Language Processing (ICSLP). Beijing, China.
10. Collins, M., 2002. Discriminative training methods for hidden Markov models: Theory and experiments with perceptron algorithms. In: Proceedings of the Conference on Empirical Methods in Natural Language Processing (EMNLP). pp. 1-8.
11. Collins M. Parameter estimation for statistical parsing models: Theory and practice of distribution-free methods. In: Bunt H., Carroll J., and Satta G. (Eds). New Developments in Parsing Technology (2004), Kluwer
12. Collins, M.J., Saraclar, M., Roark, B., 2005. Discriminative syntactic language modeling for speech recognition. In: Proceedings of the 43rd Annual Meeting of the Association for Computational Linguistics. pp. 507-514.
13. Freund Y., and Schapire R. Large margin classification using the perceptron algorithm. Machine Learning 3 37 (1999) 277-296
14. Goel V., and Byrne W. Minimum Bayes-risk automatic speech recognition. Computer Speech and Language 14 2 (2000) 115-135
15. Jelinek, F., 1996. Acoustic sensitive language modeling. Tech. rep., Center for Language and Speech Processing, Johns Hopkins University, Baltimore, MD.
16. Johnson, M., Geman, S., Canon, S., Chi, Z., Riezler, S., 1999. Estimators for stochastic "unification- based" grammars. In: Proceedings of the 37th Annual Meeting of the Association for Computational Linguistics. pp. 535-541.
17. Khudanpur S., and Wu J. Maximum entropy techniques for exploiting syntactic, semantic and collocational dependencies in language modeling. Computer Speech and Language 14 4 (2000) 355-372
18. Kuo, H.-K.J., Fosler-Lussier, E., Jiang, H., Lee, C.-H., May 2002. Discriminative training of language models for speech recognition. In: Proceedings of the International Conference on Acoustics, Speech, and Signal Processing (ICASSP), Orlando, Florida.
19. Lafferty, J., McCallum, A., Pereira, F., 2001. Conditional random fields: Probabilistic models for segmenting and labeling sequence data. In: Proceedings of the 18th International Conference on Machine Learning. pp. 282-289.
20. Ljolje, A., Bocchieri, E., Riley, M., Roark, B., Saraclar, M., Shafran, I., 2003. The AT&T 1xRT CTS system. In: DARPA Rich Transcription Workshop, Boston.
21. Malouf, R., 2002. A comparison of algorithms for maximum entropy parameter estimation. In: Proceedings of the Sixth Conference on Natural Language Learning (CoNLL). pp. 49-55.
22. Mangu L., Brill E., and Stolcke A. Finding consensus in speech recognition: word error minimization and other applications of confusion networks. Computer Speech and Language 14 4 (2000) 373-400
23. Mangu, L., Padmanabhan, M., 2001. Error corrective mechanisms for speech recognition. In: Proceedings of the International Conference on Acoustics, Speech, and Signal Processing (ICASSP).
24. McCallum, A., Li, W., 2003. Early results for named entity recognition with conditional random fields, feature induction and web-enhanced lexicons. In: Seventh Conference on Natural Language Learning (CoNLL).
25. Mohri M., Pereira F.C.N., and Riley M. Weighted finite-state transducers in speech recognition. Computer Speech and Language 16 1 (2002) 69-88
26. NIST, 2000. Speech recognition scoring toolkit (SCTK) version 1.2c. Available from: http://www.nist.gov/speech/tools.
27. Pinto, D., McCallum, A., Wei, X., Croft, W.B., 2003. Table extraction using conditional random fields. In: Proceedings of the ACM SIGIR.
28. Povey, D., Woodland, P., 2002. Minimum phone error and I-smoothing for improved discriminative training. In: Proceedings of the International Conference on Acoustics, Speech, and Signal Processing (ICASSP). pp. 105-108.
29. Ratnaparkhi, A., Roukos, S., Ward, R.T., 1994. A maximum entropy model for parsing. In: Proceedings of the International Conference on Spoken Language Processing (ICSLP). pp. 803-806.
30. Ringger, E.K., Allen, J.F., 1996. Error corrections via a post-processor for continuous speech recognition. In: Proceedings of the International Conference on Acoustics, Speech, and Signal Processing (ICASSP).
31. Roark, B., Saraclar, M., Collins, M., 2004a. Corrective language modeling for large vocabulary ASR with the perceptron algorithm. In: Proceedings of the International Conference on Acoustics, Speech, and Signal Processing (ICASSP). pp. 749-752.
32. Roark, B., Saraclar, M., Collins, M., Johnson, M., 2004b. Discriminative language modeling with conditional random fields and the perceptron algorithm. In: Proceedings of the 42nd Annual Meeting of the Association for Computational Linguistics.
33. Rosenfeld R., Chen S.F., and Zhu X. Whole-sentence exponential language models: a vehicle for linguistic-statistical integration. Computer Speech and Language 15 1 (2001) 55-73
34. Sha, F., Pereira, F., 2003. Shallow parsing with conditional random fields. In: Proceedings of HLT-NAACL. Edmonton, Canada.
35. Stolcke, A., Bratt, H., Butzberger, J., Franco, H., Gadde, V. R. R., Plauche, M., Richey, C., Shriberg, E., Sonmez, K., Weng, F., Zheng, J., 2000. The SRI march 2000 Hub-5 conversational speech transcription system. In: Proceedings of the NIST Speech Transcription Workshop.
36. Stolcke, A., Weintraub, M., 1998. Discriminitive language modeling. In: Proceedings of the 9th Hub-5 Conversational Speech Recognition Workshop.
37. Taskar, B., Guestrin, C., Koller, D., 2003. Max margin Markov networks. In: Proceedings of NIPS 2003.
38. Wainwright, M. J., Jordan, M. I., 2002. Graphical models, exponential families, and variational inference, Technical Report 649, UC Berkeley, Department of Statistics.
39. Wallach, H., 2002. Efficient training of conditional random fields. Master's thesis, University of Edinburgh.
40. Woodland, P., Povey, D., 2000. Large scale discriminative training for speech recognition. In: Proc. ISCA ITRW ASR2000. pp. 7-16.