Empirical evaluation and combination of advanced language modeling techniques

Proceedings of the Annual Conference of the International Speech Communication Association, INTERSPEECH

Volumn , Issue , 2011, Pages 605-608

  Mikolov, Tomáš ^a   Deoras, Anoop ^b   Kombrink, Stefan ^a   Burget, Lukáš ^a   Černocký, Jan Honza ^a  

^a BRNO UNIVERSITY OF TECHNOLOGY   (Czech Republic)

^b JOHNS HOPKINS UNIVERSITY   (United States)

Author keywords

Language modeling; Model combination; Neural networks; Speech recognition

Indexed keywords

CLASS-BASED; EMPIRICAL EVALUATIONS; GOOD-TURING; INDIVIDUAL MODELS; LANGUAGE MODEL; LANGUAGE MODELING; LINEAR INTERPOLATION; MAXIMUM ENTROPY MODELS; MODEL COMBINATION; RANDOM FORESTS; STATE OF THE ART; STRUCTURED LANGUAGE; TRI GRAMS;

DECISION TREES; MAXIMUM ENTROPY METHODS; NEURAL NETWORKS; SPEECH RECOGNITION;

COMPUTATIONAL LINGUISTICS;

EID: 84865803833     PISSN: None     EISSN: 19909772     Source Type: Conference Proceeding    
DOI: None     Document Type: Conference Paper

References (20)

1. Joshua T. Goodman (2001). A bit of progress in language modeling, extended version. Technical report MSR-TR-2001-72.
2. Yoshua Bengio, Rejean Ducharme and Pascal Vincent. 2003. A neural probabilistic language model. Journal of Machine Learning Research, 3:1137-1155
3. H. Schwenk. Continuous space language models. Computer Speech and Language, vol. 21, 2007.
4. Tomáš Mikolov, Martin Karafiát, Lukáš Burget, Jan Černocký, Sanjeev Khudanpur: Recurrent neural network based language model, In: Proc. INTERSPEECH 2010.
5. Tomáš Mikolov, Stefan Kombrink, Lukáš Burget, Jan Černocký, Sanjeev Khudanpur: Extensions of recurrent neural network language model. In: Proc. ICASSP 2011.
6. Peng Xu. Random forests and the data sparseness problem in language modeling, Ph.D. thesis, Johns Hopkins University, 2005.
7. Denis Filimonov and Mary Harper. A joint language model with fine-grain syntactic tags. EMNLP 2009.
8. Ahmad Emami, Frederick Jelinek. Exact training of a neural syntactic language model. In: Proc. ICASSP, 2004.
9. Tomáš Mikolov, Jiří Kopecký, Lukáš Burget, Ondřej Glembek and Jan Černocký : Neural network based language models for highly inflective languages, In: Proc. ICASSP 2009.
10. A. Emami, F. Jelinek. Random clusterings for language modeling. In: Proc. ICASSP 2005.
11. S. F. Chen and J. T. Goodman. An empirical study of smoothing techniques for language modeling. In Proceedings of the 34th Annual Meeting of the ACL, 1996.
12. Andreas Stolcke. SRILM - language modeling toolkit. http://www.speech. sri.com/projects/srilm/
13. Tanel Alumäe and Mikko Kurimo. Efficient Estimation of Maximum Entropy Language Models with N-gram features: an SRILM extension, In: Proc. INTERSPEECH 2010.
14. Puyang Xu, D. Karakos, S. Khudanpur. Self-Supervised Discriminative Training of Statistical Language Models. ASRU 2009.
15. D. Klakow. Log-linear interpolation of language models. In: Proc. Internat. Conf. Speech Language Process., 1998.
16. S. Momtazi, F. Faubel, D. Klakow. Within and Across Sentence Boundary Language Model. In: Proc. INTERSPEECH 2010.
17. W.Wang and M. Harper. The SuperARV language model: Investigating the effectiveness of tightly integrating multiple knowledge sources. In: Proc. EMNLP, 2002.
18. H.-S. Le, I. Oparin, A. Allauzen, J.-L. Gauvain, F. Yvon. Structured Output Layer Neural Network Language Model. In: Proc. ICASSP 2011.
19. A. Mnih and G. Hinton. Three new graphical models for statistical language modelling. Proceedings of the 24th international conference on Machine learning, 2007.
20. D. E. Rumelhart, G. E. Hinton, R. J. Williams. 1986. Learning internal representations by back-propagating errors. Nature, 323, 533-536.