Recurrent neural network based language model

Proceedings of the 11th Annual Conference of the International Speech Communication Association, INTERSPEECH 2010

Volumn , Issue , 2010, Pages 1045-1048

  Mikolov, Tomaš ^a,b   Karafiát, Martin ^a   Burget, Lukaš ^a   Jan, Cernocky ^a   Khudanpur, Sanjeev ^b  

^a BRNO UNIVERSITY OF TECHNOLOGY   (Czech Republic)

^b JOHNS HOPKINS UNIVERSITY   (United States)

Author keywords

Language modeling; Recurrent neural networks; Speech recognition

Indexed keywords

COMPUTATIONAL LINGUISTICS; MODELING LANGUAGES; SPEECH COMMUNICATION; SPEECH RECOGNITION;

BACKOFF; LANGUAGE MODEL; N-GRAMS; STATE OF THE ART; WALL STREET JOURNAL; WORD ERROR RATE; % REDUCTIONS; BACKOFFS; NETWORK-BASED; TRAINING COMPLEXITY;

RECURRENT NEURAL NETWORKS;

EID: 79959829092     PISSN: None     EISSN: None     Source Type: Conference Proceeding    
DOI: None     Document Type: Conference Paper

References (14)

1. Mahoney, M. Text Compression as a Test for Artificial Intelligence. In AAAI/IAAI, 486-502, 1999
2. Goodman Joshua T. (2001). A bit of progress in language modeling, extended version. Technical report MSR-TR-2001-72.
3. Yoshua Bengio, Rejean Ducharme and Pascal Vincent. 2003. A neural probabilistic language model. Journal of Machine Learning Research, 3:1137-1155
4. Holger Schwenk and Jean-Luc Gauvain. Training Neural Network Language Models On Very Large Corpora. in Proc. Joint Conference HLT/EMNLP, October 2005.
5. Mikael Bodén. A Guide to Recurrent Neural Networks and Backpropagation. In the Dallas project, 2002.
6. Yoshua Bengio and Patrice Simard and Paolo Frasconi. Learning Long-Term Dependencies with Gradient Descent is Difficult. IEEE Transactions on Neural Networks, 5, 157-166.
7. Jeffrey L. Elman. Finding Structure in Time. Cognitive Science, 14, 179-211
8. Puyang Xu and Damianos Karakos and Sanjeev Khudanpur. Self-Supervised Discriminative Training of Statistical Language Models. ASRU 2009.
9. Denis Filimonov and Mary Harper. 2009. A joint language model with fine-grain syntactic tags. In EMNLP.
10. Bengio, Y. and Senecal, J.-S. Adaptive Importance Sampling to Accelerate Training of a Neural Probabilistic Language Model. IEEE Transactions on Neural Networks.
11. Morin, F. and Bengio, Y. Hierarchical Probabilistic Neural Network Language Model. AISTATS'2005.
12. Tomáš Mikolov, Jiří Kopecký, Lukáš Burget, Ondřej Glembek and Jan Černocký : Neural network based language models for highly inflective languages, In: Proc. ICASSP 2009.
13. T. Hain. et al., "The 2005 AMI system for the transcription of speech in meetings," in Proc. Rich Transcription 2005 Spring Meeting Recognition Evaluation Workshop, UK, 2005.
14. Mahoney, M. 2000. Fast Text Compression with Neural Networks. In Proc. FLAIRS.