On the estimation of discount parameters for language model smoothing

Proceedings of the Annual Conference of the International Speech Communication Association, INTERSPEECH

Volumn , Issue , 2011, Pages 1433-1436

  Sundermeyer, Martin ^a   Schlüter, Ralf ^a   Ney, Hermann ^a  

^a RWTH AACHEN UNIVERSITY   (Germany)

Author keywords

Absolute discounting; Kneser Ney method; Language model pruning; Language model smoothing

Indexed keywords

ABSOLUTE DISCOUNTING; BASELINE METHODS; KNESER-NEY METHOD; LANGUAGE MODEL; LOG LIKELIHOOD; PARAMETER ESTIMATE; PROBABILITY ESTIMATE; PRUNING ALGORITHMS; SMOOTHING ALGORITHMS; STATISTICAL LANGUAGE MODELING; TRAINING DATA; WORD ERROR RATE;

ALGORITHMS; COMPUTATIONAL LINGUISTICS; ESTIMATION; PROBABILITY DISTRIBUTIONS;

PARAMETER ESTIMATION;

EID: 84865773873     PISSN: None     EISSN: 19909772     Source Type: Conference Proceeding    
DOI: None     Document Type: Conference Paper

References (17)

1. Chen, S. F., "Shrinking Exponential Language Models", Proc. of HLT-NAACL 2009, pp. 468-476
2. Schwenk, H., "Continuous space language models", Computer Speech and Language 21 (2007), pp. 492-518
3. Mikolov, T., Karafiát, M., Burget, L., Černoczký, J. H., and Khudanpur, S., "Recurrent neural network based language model" Proc. of Interspeech 2010, pp. 1045-1048
4. Kneser, R., and Ney, H., "Improved Backing-Off For M-Gram Language Modeling", Proc. of ICASSP 1995, pp. 181-184
5. Ney, H., Martin, S., and Wessel, F., "Statistical Language Modeling Using Leaving-One-Out", in Young, S., and Bloothooft, G., "Corpus-Based Methods in Language And Speech Processing", Kluwer Acad. Publ. 1997
6. Chen, S. F., and Goodman, J., "An Empirical Study of Smoothing Techniques for Language Modeling", Harvard University, Tech. Rep. TR-10-98, August 1998
7. Stolcke, A., "SRILM - An Extensible Language Modeling Toolkit", Proc. of ICSLP 2002, pp. 901-904
8. Federico, M., Bertoldi, N., and Cettolo, M., "IRSTLM: An Open Source Toolkit for Handling Large Scale Language Models", Proc. of Interspeech 2008, pp. 1618-1621
9. Siivola, V., Hirsimäki, T., and Virpioja, S., "On Growing and Pruning Kneser-Ney Smoothed N-Gram Models", IEEE Trans. on Audio, Speech, and Language Processing, Vol. 15, No. 5, July 2007
10. Boyd, S., and Vandenberghe, L., "Convex Optimization", Cambridge University Press 2004
11. Igel, C., and Hüsken, M., "Empirical evaluation of the improved Rprop learning algorithm", Neurocomputing 50 (2001), pp. 105-123
12. Sundermeyer, M., Nusbaum-Thom, M., Wiesler, S., Plahl, C., El-Desoky Mousa, A., Hahn, S., Nolden, D., Schlüter, R., and Ney., H., "The RWTH 2010 Quaero ASR Evaluation System for English, French, and German", Proc. of ICASSP 2011, accepted for publication
13. Andrés-Ferrer, J., Ney., H., "From empirical Bayes to Leavingone-out", Proc. of IWSM 2011, accepted for publication
14. Andrés-Ferrer, J., Ney., H., "Extensions of Absolute Discounting (Kneser-Ney Method)" Proc. of ICASSP 2009, pp. 4729-4732
15. Stolcke, A., "Entropy-based Pruning of Backoff Language Models", Proc. of ICSLP 2002, pp. 901-903
16. Chelba, C., Brants, T., Neveitt, W., Xu, P., "Study on Interaction between Entropy Pruning and Kneser-Ney Smoothing" Proc. of Interspeech 2010, pp. 2422-2425
17. Stolcke, A., SRILM user mailing list, September 28, 2010