LSTM neural networks for language modeling

13th Annual Conference of the International Speech Communication Association 2012, INTERSPEECH 2012

Volumn 1, Issue , 2012, Pages 194-197

  Sundermeyer, Martin ^a   Schlüter, Ralf ^a   Ney, Hermann ^a  

^a RWTH AACHEN UNIVERSITY   (Germany)

Author keywords

Language modeling; LSTM neural networks; Recurrent neural networks

Indexed keywords

FEED-FORWARD NETWORK; LANGUAGE MODELING; LONG SHORT-TERM MEMORY; RECURRENT MODELS; RECURRENT NETWORKS; SPEECH RECOGNITION SYSTEMS;

NETWORK ARCHITECTURE; RECURRENT NEURAL NETWORKS; SPEECH RECOGNITION;

COMPUTATIONAL LINGUISTICS;

EID: 84878402147     PISSN: None     EISSN: None     Source Type: Conference Proceeding    
DOI: None     Document Type: Conference Paper

References (19)

1. Kneser, R., and Ney, H., "Improved Backing-Off For M-Gram Language Modeling", Proc. of ICASSP 1995, pp. 181-184
2. Bengio, Y., Ducharme, R., "A neural probabilistic language model", Proc. of Advances in Neural Information Processing Systems (2001), vol. 13., pp. 932-938.
3. Schwenk, H., "Continuous space language models", Computer Speech and Language 21 (2007), pp. 492-518
4. Oparin, I., Sundermeyer, M., Ney, H., Gauvain, J.-L., "Performance Analysis of Neural Networks in Combination with n-Gram Language Models", Proc. of ICASSP 2012, accepted for publication
5. Mikolov, T., Karafiát, M., Burget, L., Černoczký, J. H., and Khudanpur, S., "Recurrent neural network based language model" Proc. of Interspeech 2010, pp. 1045-1048
6. Elman, J., "Finding Structure in Time", Cognitive Science 14 (1990), pp. 179-211
7. Rumelhart, D. E., Hinton, G. E., Williams, R. J., "Learning representations by back-propagating errors", Nature 323 (1986), pp. 533-536
8. Bengio, Y., Simard, P., Frasconi, P., "Learning long-term dependencies with gradient descent is difficult" IEEE Transactions on Neural Networks 5 (1994), pp. 157-166
9. Martens, J., Sutskever, I., "Learning Recurrent Neural Networks with Hessian-Free Optimization", Proc. of the 28th Int. Conf. on Machine Learning 2011
10. Hochreiter, S., Schmidhuber, J., "Long Short-Term Memory", Neural Computation 9 (8), 1997, pp. 1735-1780
11. Gers, F. A., "Learning to Forget: Continual Prediction with LSTM", Proc. of the 9th Int. Conf. on Artificial Neural Networks, 1999, pp. 850-855
12. Gers, F. A., Schraudolph, N. N., Schmidhuber, J., "Learning Precise Timing with LSTM Recurrent Networks", Journal of Machine Learning Research 3, 2002, pp. 115-143
13. Graves, A., Schmidhuber, J., "Framewise Phoneme Classification with Bidirectional LSTM and Other Neural Network Architectures", Neural Networks, Vol. 18, Issue 5-6, 2005, pp. 602-610
14. Bishop, C., "Neural Networks for Pattern Recognition", Clarendon Press, Oxford, 1995
15. Le, H. S., Allauzen, A., Wisniewski, G., Yvon, F., "Training continuous space language models: some practical issues", Proc. of the 2010 Conf. on Emp. Methods in NLP, pp. 778-788
16. Morin, F., Bengio, Y., "Hierarchical Probabilistic Neural Network Language Model", Proc. of the 10th Int. Workshop on Artificial Intelligence and Statistics
17. Goodman, J., "Classes for fast maximum entropy training", Proc. of the ICASSP, 2001
18. Mikolov, T., Kombrink, S., Burget, L., Černocký, J., Khudanpur, S., "Extensions of Recurrent Neural Network Language Model", Proc. of the ICASSP 2011, pp. 5528-5531
19. Mikolov, T., Kombrink, S., Deoras, A., Burget, L., Černocký , J., "RNNLM - Recurrent Neural Network Language Modeling Toolkit", Proc. of the 2011 ASRU Workshop, pp. 196-201