A survey on the application of recurrent neural networks to statistical language modeling

Computer Speech and Language

Volumn 30, Issue 1, 2015, Pages 61-98

  De Mulder, Wim ^a   Bethard, Steven ^b   Moens, Marie Francine ^a  

^a UNIVERSITY OF LEUVEN   (Belgium)

^b UNIVERSITY OF ALABAMA AT BIRMINGHAM   (United States)

Author keywords

Language modeling; Machine translation; Natural language processing; Recurrent neural networks; Speech recognition

Indexed keywords

COMPUTATIONAL LINGUISTICS; NATURAL LANGUAGE PROCESSING SYSTEMS; SPEECH RECOGNITION; SPEECH TRANSMISSION; SURVEYS;

CONTEXT-WORD; LANGUAGE MODEL; MACHINE TRANSLATIONS; NATURAL LANGUAGE PROCESSING; RECURRENT NEURAL NETWORK MODEL; STATE-OF-THE-ART TECHNIQUES; STATISTICAL LANGUAGE MODELING; TRAINING TIME;

RECURRENT NEURAL NETWORKS;

EID: 84913601027     PISSN: 08852308     EISSN: 10958363     Source Type: Journal    
DOI: 10.1016/j.csl.2014.09.005     Document Type: Review

References (138)

1. H.E. Abed, V. Märgner, M. Kherallah, and A.M. Alimi Online Arabic handwriting recognition competition Proceedings of the 10th International Conference on Document Analysis and Recognition 2009 1388 1392
2. F. Ahmed, E.W. De Luca, and A. Nürnberger Revised N-gram based automatic spelling correction tool to improve retrieval effectiveness Polibits 40 2009 39 48
3. L.B. Almeida, T. Langlois, J.D. Amaral, and A. Plankhov Parameter adaptation in stochastic optimization D. Saad, On-Line Learning in Neural Networks 1998 Cambridge University Press 111 134
4. E. Arisoy, S.F. Chen, B. Ramabhadran, and A. Sethy Converting neural network language models into back-off language models for efficient decoding in automatic speech recognition IEEE/ACM Trans. Audio Speech Lang. Process. 22 1 2014 184 192
5. L.R. Bahl, F. Jelinek, and R. Mercer A maximum likelihood approach to continuous speech recognition IEEE Trans. Pattern Anal. Mach. Intell. 1983 179 190
6. S.D. Balkin Using recurrent neural network for time series forecasting Proceedings of the International Symposium on Forecasting Barbados 1997
7. Y. Bengio, R. Ducharme, and P. Vincent A neural probabilistic language model J. Mach. Learn. Res. 3 2003 1137 1155
8. A. Blanco, M. Delgado, and M.C. Pegalajar A genetic algorithm to obtain the optimal recurrent neural network Int. J. Approx. Reason. 23 1 2000 67 83
9. L. Breiman Stacked regressions Mach. Learn. 24 1 1996 49 64
10. P.F. Brown, P.V. deSouza, R.L. Mercer, V.J.D. Pietra, and J.C. Lai Class-based N-gram models for natural language Comput. Linguist. 18 1992 467 479
11. P.F. Brown, V.J. Della Pietra, S.A. Della Pietra, and R.L. Mercer The mathematics of statistical machine translation: parameter estimation Comput. Linguist. 19 2 1993 263 311
12. E. Buabin Boosted hybrid recurrent neural classifier for text document classification on the Reuters news text corpus Int. J. Mach. Learn. Comput. 2 5 2012 588 592
13. C. Chelba, T. Brants, W. Neveitt, and P. Xu Study on interaction between entropy pruning and Kneser-Ney smoothing Proceedings of Interspeech 2010 2242 2245
14. S. Chen, D. Beeferman, and R. Rosenfeld Evaluation metrics for language models Proceedings of the DARPA Broadcast News Transcription and Understanding Workshop 1998
15. S.F. Chen, L. Mangu, B. Ramabhadran, R. Sarikaya, and A. Sethy Scaling shrinkage-based language models Proceedings of IEEE Workshop on Automatic Speech Recognition and Understanding (ASRU) 2009 299 304
16. Y. Chen, B. Perozzi, R. Al-Rfou, and S. Skiena The expressive power of word embeddings. Technical report 2013 Cornell University Library arXiv:1301.3226v4
17. K.W. Church A stochastic parts program and noun phrase parser for unrestricted text Proceedings of the Second Conference on Applied Natural Language Processing 1988 136 143
18. P. Clarkson, and T. Robinson The applicability of adaptive language modelling for the broadcast news task Proceedings 5th International Conference on Spoken Language Processing 1998 233 236
19. P. Clarkson, and T. Robinson Improved language modelling through better language model evaluation measures Comput. Speech Lang. 15 1 2001 39 53
20. R. Collobert, and J. Weston A unified architecture for natural language processing: deep neural networks with multitask learning Proceedings of the 25th International Conference on Machine learning 2008 160 167
21. R. Collobert, J. Weston, L. Bottou, M. Karlen, K. Kavukcuoglu, and P. Kuksa Natural language processing (almost) from scratch J. Mach. Learn. Res. 12 2011 2493 2537
22. T. Cover, and J. Thomas Elements of Information Theory 1991 John Wiley and Sons Inc.
23. A. Deoras, D. Filimonov, M. Harper, and F. Jelinek Model combination for speech recognition using empirical Bayes risk minimization Proceedings of the IEEE Workshop on Spoken Language Technology 2010
24. A. Deoras Search and Decoding Strategies For Complex Lexical Modeling in LVCSR 2011 Johns Hopkins University (PhD dissertation)
25. A. Deoras, T. Mikolov, S. Kombrink, M. Karafiat, and S. Khudanpur Variational approximation of long-span language models for LVCSR Proceedings of the IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP) 2011 5532 5535
26. A. Deoras, T. Mikolov, and K. Church A fast re-scoring strategy to capture long-distance dependencies EMNLP'11 Proceedings of the Conference on Empirical Methods in Natural Language Processing 2011 1116 1127
27. A. Deoras, T. Mikolov, M. Karafiát, and S. Khudanpur Variational approximation of long-span language models in LVCSR Proceedings of the IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP) 2011 5532 5535
28. A. Deoras, T. Mikolov, S. Kombrink, and K. Church Approximate inference: a sampling based modeling technique to capture complex dependencies in a language model Speech Commun. 55 1 2013 162 177
29. E.M. de Novais, T.D. Tadeu, and I. Paraboni Improved text generation using N-gram statistics Proceedings of the 12th Ibero-American Conference on Advances in Artificial Intelligence 2010 316 325
30. K. Deschacht, J. De Belder, and M.-F. Moens The latent words language model Comput. Speech Lang. 26 5 2012 384 409
31. N.J. de Vos Reservoir computing as an alternative to traditional artificial neural networks in rainfall-runoff modelling Hydrol. Earth Syst. Sci. 2012 Discussion 9
32. T.G. Dietterich Ensemble methods in machine learning Proceedings of the First International Workshop on Multiple Classifier Systems 2000 1 15
33. J.L. Elman Finding structure in time Cogn. Sci. 14 2 1990 179 211
34. A. Emami, and F. Jelinek Random clusterings for language modeling Proceedings of the International Conference on Acoustics, Speech and Signal Processing 2005 581 584
35. S.E. Fahlman, and C. Lebiere The cascade-correlation learning architecture. Technical Report 1989 Carnegie Mellon University
36. S.E. Fahlman The recurrent cascade-correlation learning architecture. Technical Report 1991 Carnegie Mellon University
37. C. Fellbaum WordNet: An Electronic Lexical Database 1998 MIT Press
38. M. Fernández-Redondo, and C. Hernández-Espinosa A comparison among weight initialization methods for multilayer feedforward networks Proceedings of the IEEE-INNS-ENNS International Joint Conference on Neural Networks 2000 543 548
39. D. Filimonov, and M. Harper A joint language model with fine-grain syntactic tags Proceedings of the 2009 Conference on Empirical Methods in Natural Language Processing 2009 1114 1123
40. A. Finch, P. Dixon, and E. Sumita Integrating models derived from non-parametric Bayesian co-segmentation into a statistical machine transliteration system Proceedings of the Named Entities Workshop 2011 23 27
41. A. Finch, P. Dixon, and E. Sumita Rescoring a phrase-based machine transliteration system with recurrent neural network language models NEWS'12 Proceedings of the 4th Named Entity Workshop 2012 47 51
42. V. Frinken, F. Zamora-Martinez, S. Espana-Boquera, M.J. Castro-Bleda, A. Fischer, and H. Bunke Long-short term memory neural networks language modeling for handwriting recognition Int. Conf. Pattern Recognit. 2012 701 704
43. T. Fukada, M. Schuster, and Y. Sagisaka Phoneme boundary estimation using bidirectional recurrent neural networks and its applications Syst. Comput. Jpn. 30 4 1999 20 30
44. F.A. Gers, and J. Schmidhuber LSTM recurrent networks learn simple context free and context sensitive languages IEEE Trans. Neural Netw. 12 6 2001 1333 1340
45. P. Gil, A. Cardoso, and L. Palma Estimating the number of hidden neurons in recurrent neural networks for nonlinear system identification Proceedings of the IEEE International Symposium on Industrial Electronics 2009 2053 2058
46. J. Glass, T. Hazen, S. Cyphers, I. Malioutov, D. Huynh, and R. Barzilay Recent progress in MIT spoken lecture processing project Proceedings Interspeech 2007
47. A. Graves, and J. Schmidhuber Framewise phoneme classification with bidirectional LSTM and other neural network architectures Neural Netw. 18 5-6 2005 602 610
48. A. Graves, S. Fernández, and J. Schmidhuber Bidirectional LSTM networks for improved phoneme classification and recognition ICANN05 Proceedings of the 15th International Conference on Artificial Neural Networks: Formal Models and their Applications 2005 799 804
49. A. Graves, S. Fernández, F. Gomez, and J. Schmidhuber Connectionist temporal classification: labelling unsegmented sequence data with recurrent neural networks Proceedings of the International Conference on Machine Learning 2006 369 376
50. A. Graves, and J. Schmidhuber Offline handwriting recognition with multidimensional recurrent neural networks D. Koller, D. Schuurmans, Y. Bengio, L. Bottou, Advances in Neural Information Processing Systems 2009 MIT Press
51. A. Graves, M. Liwicki, S. Fernandez, R. Bertolami, H. Bunke, and J. Schmidhuber A novel connectionist system for unconstrained handwriting recognition IEEE Trans. Pattern Anal. Mach. Intell. 31 5 2009 855 868
52. A. Graves Supervised Sequence Labelling with Recurrent Neural Networks 2012 Springer
53. A. Graves, A. Mohamed, and G. Hinton Speech recognition with deep recurrent neural networks Proceedings of IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP 2013) 2013
54. E. Grosicki, and H.E. Abed ICDAR 2009 handwriting recognition competition 10th International Conference on Document Analysis and Recognition 2009 1398 1402
55. T. Hain The 2005 AMI system for the transcription of speech in meetings Proceedings Rich Transcription 2005 Spring Meeting Recognition Evaluation Workshop 2005
56. J. Hammerton Named entity recognition with long short-term memory Proceedings of CoNLL-2003 2003 172 175
57. L. Hansen, and P. Salamon Neural network ensembles IEEE Trans. Pattern Anal. Mach. Intell. 12 10 1990 993 1001
58. S. Hashem Optimal linear combinations of neural networks Neural Netw. 10 4 1997 599 614
59. P.A. Heeman POS tags and decision trees for language modeling Proceedings of the Joint SIGDAT Conference on Empirical Methods in Natural Language Processing and Very Large Corpora 1999 129 137
60. S. Hochreiter, and J. Schmidhuber Long short-term memory Neural Comput. 9 8 1997 1735 1780
61. S. Hochreiter, Y. Bengio, P. Frasconi, and J. Schmiduber Gradient flow in recurrent nets: the difficulty of learning long-term dependencies S.C. Kremer, J.F. Kolen, A Field Guide to Dynamical Recurrent Neural Networks 2001 IEEE Press
62. F. Huang, A. Ahuja, D. Downey, Y. Yang, Y. Guo, and A. Yates Learning representations for weakly supervised natural language processing tasks Comput. Linguist. 40 1 2014 85 120
63. Z. Huang, G. Zweig, M. Levit, B. Dumoulin, B. Oguz, and S. Chang Accelerating recurrent neural network training via two stage classes and parallelization IEEE Workshop on Automatic Speech Recognition and Understanding (ASRU) 2013 326 331
64. R. Iyer, M. Ostendorf, and M. Meteer Analyzing and predicting language model improvements Proceedings of the IEEE Workshop on Automatic Speech Recognition and Understanding 1997
65. H. Jaeger The "echo state" approach to analysing and training recurrent neural networks. GMD Report 148 2001 German National Research Center for Information Technology
66. H. Jaeger Tutorial on training recurrent neural networks, covering BPPT, RTRL, EKF and the "echo state network" approach. GMD Report 159 2002 German National Research Center for Information Technology
67. F. Jelinek Statistical Methods for Speech Recognition 1998 A Bradford Book
68. M. Jordan Serial order: a parallel distributed processing approach. Technical report 1986 University of California San Diego
69. K. Kirchhoff, and M. Yang Improved language modeling for statistical machine translation Proceedings of the ACL Workshop on Building and Using Parallel Texts 2005 125 128
70. S. Kombrink, T. Mikolov, M. Karafiát, and L. Burget Recurrent neural network based language modeling in meeting recognition In Proceedings of Interspeech 2011 2011 2877 2880
71. R. Kneser, and H. Ney Improved backing-off for N-gram language modeling In Proceedings of the IEEE International Conference on Acoustics, Speech and Signal Processing 1995 141 184
72. H.-S. Le, I. Oparin, A. Messaoudi, A. Allauzen, J.-L. Gauvain, and F. Yvon Large vocabulary SOUL neural network language odels Proceedings Interspeech 2011 1469 1472
73. H.-S. Le, I. Oparin, A. Allauzen, J.-L. Gauvain, and F. Yvon Structured output layer neural network language model Proceedings of the International Conference on Acoustics, Speech and Signal Processing 2011 5524 5527
74. H.-S. Le, A. Allauzen, and F. Yvon Measuring the influence of long range dependencies with neural network language models Proceedings of the NAACL-HLT 2012 Workshop 2012 1 10
75. H.-S. Le Continuous Space Models with Neural Networks in Natural Language Processing 2013 Université Paris Sud (PhD dissertation)
76. G. Lecorvé, and P. Motlicek Conversion of recurrent neural network language models to weighted finite state transducers for automatic speech recognition Proceedings of Interspeech 2012 1666 1669
77. Y. LeCun, L. Bottou, G. Orr, and K. Muller Efficient BackProp G. Orr, K. Muller, Neural Networks, Tricks of the Trade 1998 Springer
78. M. Liwicki, and H. Bunke IAM-OnDB - an on-line English sentence database acquired from handwritten text on a whiteboard Proceedings of the 8th International Conference on Document Analysis and Recognition 2005 956 961
79. M. LukǒsevѤcius, and H. Jaeger Reservoir computing approaches to recurrent neural network training Comput. Sci. Rev. 3 3 2009 127 149
80. W. Maass, T. Natschläger, and H. Markram Real-time computing without stable states: a new framework for neural computation based on perturbations Neural Comput. 14 11 2002 2531 2560
81. F.J. Maldonado, and M.T. Manry Optimal pruning of feedforward neural networks based upon the Schmidt procedure Conference Record of the Thirty-Sixth Asilomar Conference on Signals, Systems and Computers 2002 1024 1028
82. C.D. Manning, and H. Schuetze Foundations of Statistical Natural Language Processing 1999 MIT Press Cambridge, MA
83. V. Märgner, and H.E. Abed ICDAR 2009 Arabic handwriting recognition competition Proceedings of the10th International Conference on Document Analysis and Recognition 2009 1383 1387
84. R. Marichal, J.D. Pi neiro, E. González, and J. Torres Novel recurrent neural network weight initialization strategy Proceedings of the World Congress on Engineering and Computer Science 2007 537 548
85. U.-V. Marti, and H. Bunke The IAM-database: an English sentence database for offline handwriting recognition Int. J. Doc. Anal. Recognit. 5 1 2002 39 46
86. S.C. Martin, J. Liermann, and H. Ney Adaptive topic-dependent language modelling using word-based varigrams Proceedings of Eurospeech 1997 1447 1450
87. R. Matignon Neural Network Modeling Using SAS Enterprise Miner 2005 AuthorHouse
88. T. Mikolov Recurrent neural network based language model Proceedings of the 11th Annual Conference of the International Speech Communication Association 2010 1045 1048
89. T. Mikolov, A. Deoras, D. Povey, L. Burget, and J.H. Cernocky Strategies for training large scale neural network language models IEEE Automatic Speech Recognition and Understanding Workshop 2011
90. T. Mikolov, S. Kombrink, L. Burget, J.H. Cernocky, and S. Khudanpur Extensions of recurrent neural network language model ICASSP 2011 2011
91. T. Mikolov, A. Deoras, S. Kombrink, L. Burget, and J.H. Cernocky Empirical evaluation and combination of advanced language modeling techniques Proceedings of Interspeech 2011 605 608
92. T. Mikolov Statistical Language Models Based on Neural Networks 2012 Brno University of Technology (PhD dissertation)
93. T. Mikolov, W.T. Yih, and G. Zweig Linguistic regularities in continuous space word representations Proceedings of the 2013 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies 2013 746 751
94. T. Mikolov, K. Chen, G. Corrado, and J. Dean Efficient estimation of word representations in vector space 2013 Cornell University Library arXiv:1301.3781
95. A. Mnih, and K. Kavukcuoglu Learning word embeddings efficiently with noise-contrastive estimation Adv. Neural Inf. Process. Syst. 26 2013 2265 2273
96. A. Mnih, and Y.W. Teh A fast and simple algorithm for training neural probabilistic language models Proceedings of the 29th International Conference on Machine Learning 2012 1751 1758
97. S. Momtazi, F. Faubel, and D. Klakow Within and across sentence boundary language model Proceedings of Interspeech 2010 1800 1803
98. R.C. Moore Improved smoothing for N-gram language models based on ordinary counts Proceedings of the ACL-IJCNLP 2009 Conference Short Papers 2009 349 352
99. F. Morin, and Y. Bengio Hierarchical probabilistic neural network language model Proceedings eAISTATS'05 2005 246 252
100. J. Niehues, and A. Waibel Continuous space language models using restricted Boltzmann machines Proceedings of the International Workshop on Spoken Language Translation 2012 164 170
101. F.J. Och, and H. Ney Discriminative training and maximum entropy models for statistical machine translation Proceedings of the 40th Annual Meeting of the Association for Computational Linguistics (ACL) 2002 295 302
102. D. Opitz, and R. Macline Popular ensemble methods: an empirical study J. Artif. Intell. Res. 11 1999 169 198
103. G. Panchal, A. Ganatra, Y.P. Kosta, and D. Panchal Behaviour analysis of multilayer perceptrons with multiple hidden neurons and hidden layers Int. J. Comput. Theory Eng. 3 2011 332 337
104. K. Papineni, S. Roukos, T. Ward, and W.-J. Zhu BLEU: a method for automatic evaluation of machine translation Proceedings of the 40th Annual Meeting on Association for Computational Linguistics 2002 311 318
105. J. Peters, and D. Klakow Compact maximum entropy language models Proceedings of the IEEE Workshop on Automatic Speech Recognition and Understanding 1999
106. E. Polak Optimization: Algorithms and Consistent Approximations 1997 Springer-Verlag
107. G. Potamianosa, and F. Jelinek A study of N-gram and decision tree letter language modeling methods Speech Commun. 24 3 1998 171 192
108. L. Prechelt Early stopping - but when? Neural Networks: Tricks of the Trade 1997 Springer-Verlag
109. X. Qiang, G. Cheng, and Z. Wang An overview of some classical growing neural networks and new developments Proceedings of the 2nd International Conference on Education Technology and Computer 2010 351 355
110. R. Reed Pruning algorithms - a survey IEEE Trans. Neural Netw. 4 5 1993 740 747
111. S. Renals, N. Morgan, H. Bourlard, M. Cohen, and H. Franco Connectionist probability estimators in HMM speech recognition IEEE Trans. Speech Audio Process. 2 1 1994 161 174
112. A.J. Robinson An application of recurrent neural nets to phone probability estimation IEEE Trans. Neural Netw. 5 2 1994 298 305
113. R. Rojas Neural Networks: A Systematic Introduction 1996 Springer-Verlag
114. R. Rosenfeld Adaptive Statistical Language Modeling: A Maximum Entropy Approach 1994 Carnegie Mellon University (PhD dissertation)
115. R. Rosenfeld Two decades of statistical language modeling: where do we go from here Proceedings of the IEEE 2000 1270 1278
116. D.E. Rumelhart, G. Hinton, and R.J. Williams Learning internal representations by error propagation D.E. Rumelhart, J.L. McClelland, Parallel Distributed Processing 1986 MIT Press 318 362
117. A.M. Schäfer, and H.G. Zimmerman Recurrent neural networks are universal approximators Int. J. Neural Syst. 17 4 2007 253 263
118. R. Schapire, Y. Freund, P. Bartlett, and W. Lee Boosting the margin: a new explanation for the effectiveness of voting methods Proceedings of the Fourteenth International Conference on Machine Learning 1997 322 330
119. J. Schmidhuber, S. Hochreiter, and Y. Bengio Evaluating benchmark problems by random guessing S.C. Kremer, J.F. Kolen, A Field Guide to Dynamical Recurrent Neural Networks 2001 IEEE Press
120. B. Schrauwen, D. Verstraeten, and J. Van Campenhout An overview of reservoir computing: theory, applications and implementations Proceedings of the 15th European Symposium on Artificial Neural Networks 2007 471 482
121. M. Schuster On supervised learning from sequential data with applications for speech recognition 1999 Nara Institute of Science and Technology (PhD dissertation)
122. H. Schwenk Continuous space language models for statistical machine translation Prague Bull. Math. Linguist. 93 2010 137 146
123. H. Schwenk, and J.-L. Gauvain Training neural network language models on very large corpora Proceedings of the Conference on Human Language Technology and Empirical Methods in Natural Language Processing 2005 201 208
124. S. Seung Perceptron learning. Lecture 2002 http://hebb.mit.edu/courses/9.641/2002/lectures/lecture03.pdf
125. K.G. Sheela, and S.N. Deepa Review on methods to fix number of hidden neurons in neural networks Math. Prob. Eng. 2013 Article ID 425740
126. Y. Shi, W.-Q. Zhang, J. Liu, and M.T. Johnson RNN language model with word clustering and class-based output layer EURASIP J. Audio Speech Music Process. 22 2013
127. M. Snover, B. Dorr, R. Schwartz, L. Micciulla, and J. Makhoul A study of translation edit rate with targeted human annotation Proceedings of Association for Machine Translation in the Americas 2006 223 231
128. A. Stolcke, Y. König, and M. Weintraub Explicit word error minimization in N-best list rescoring Proceedings of the 5th European Conference on Speech Communication and Technology 1997
129. M. Sundermeyer, I. Oparin, J.-L. Gauvain, B. Freiberg, R. Schluter, and H. Ney Comparison of feedforward and recurrent neural network language models Proceedings of the International Conference on Acoustics, Speech and Signal Processing 2013 8430 8434
130. J. Turian, L. Ratinov, and Y. Bengio Word representations: a simple and general method for semi-supervised learning ACL 2010
131. A. Vaswani, Y. Zhao, V. Fossum, and D. Chiang Decoding with large-scale neural language models improves translation Proceedings of Empirical Models in Natural Language Processing 2013 1387 1392
132. D. Verstraeten, B. Schrauwen, and D. Stroobandt Reservoir-based techniques for speech recognition Proceedings of the IEEE International Joint Conference on Neural Networks 2006 1050 1053
133. A. Vinciarelli, S. Bengio, and H. Bunke Offline recognition of unconstrained handwritten texts using HMMs and statistical language models IEEE Trans. Pattern Anal. Mach. Intell. 26 6 2004 709 720
134. S. Wang, D. Schuurmans, F. Peng, and Y. Zhao Combining statistical language models via the latent maximum entropy principle Mach. Learn. 60 1-3 2005 229 250
135. D.W. White GANNet: A Genetic Algorithm for Searching Topology and Weight Spaces in Neural Network Design 1993 University of Maryland (PhD dissertation)
136. P. Xu Random Forests and the Data Sparseness Problem in Language Modeling 2005 Johns Hopkins University (PhD dissertation)
137. P. Xu, D. Karakos, and S. Khudanpur Self-supervised discriminative training of statistical language models Proceedings of the IEEE Workshop on Automatic Speech Recognition & Understanding 2009 317 322
138. W. Xu, and A. Rudnicky Can artificial neural networks learn language models? Proceedings of 6th International Conference on Spoken Language Processing 2000 202 205