Comparison of feedforward and recurrent neural network language models

ICASSP, IEEE International Conference on Acoustics, Speech and Signal Processing - Proceedings

Volumn , Issue , 2013, Pages 8430-8434

  Sundermeyer, M ^a   Oparin, I ^b,c   Gauvain, J L ^b   Freiberg, B ^b   Schluter, R ^a   Ney, H ^a,b  

^a RWTH AACHEN UNIVERSITY   (Germany)

^b UFR 919 Laboratoire d'Informatique Pour la Mécanique et les Sciences de l'Ingénieur   (France)

^c LABORATOIRE NATIONAL DE MÉTROLOGIE ET D ESSAIS LNE   (France)

Author keywords

Automatic speech recognition; feedforward neural networks; recurrent neural networks

Indexed keywords

AUTOMATIC SPEECH RECOGNITION; CONTEXT DEPENDENCY; FEED-FORWARD; FIXED NUMBERS; LANGUAGE MODEL; PARALLELIZATIONS; RECURRENT NETWORKS; SPEED UP;

COMPUTATIONAL LINGUISTICS; FEEDFORWARD NEURAL NETWORKS; SIGNAL PROCESSING; SPEECH RECOGNITION;

RECURRENT NEURAL NETWORKS;

EID: 84890480734     PISSN: 15206149     EISSN: None     Source Type: Conference Proceeding    
DOI: 10.1109/ICASSP.2013.6639310     Document Type: Conference Paper

References (15)

1. Kneser, R., and Ney, H., "Improved Backing-Off For M-Gram Language Modeling", Proc. of ICASSP 1995, pp. 181-184
2. Bengio, Y., and Ducharme, R., "A neural probabilistic language model", Proc. of Advances in Neural Information Processing Systems (2001), vol. 13., pp. 932-938
3. Schwenk, H., Gauvain, J.-L., "Connectionist Language Modeling for Large Vocabulary Continuous Speech Recognition", Proc. of ICASSP 2002, pp. 765-768
4. Mikolov, T., Karafiat, M., Burget, L., C? ernocky, J., and Khudanpur, S., "Recurrent neural network based language model", Proc. of Interspeech 2010, pp. 1045-1048
5. Sundermeyer, M., Schluter, R., and Ney, H., "LSTM Neural Networks for Language Modeling", Proc. of Interspeech 2012
6. Mikolov, T., Kombrink, S., Burget, L., C? ernocky, J., and Khudanpur, S., "Extensions of Recurrent Neural Network Language Model", Proc. of ICASSP 2011, pp. 5528-5531
7. Ar?soy, E., Sainath, T. N., Kingsbury, B., and Ramabhadran, B., "Deep Neural Network Language Models", Proc. of NAACL-HLT 2012 Workshop, pp. 20-28
8. Le, H. S., Oparin, I., Messaoudi, A., Allauzen, A., Gauvain, J.-L., and Yvon, F. "Large Vocabulary SOUL Neural Network Language Models", Proc. of Interspeech 2011, pp. 1469-1472
9. Le, H. S., Allauzen, A., and Yvon, F., "Measuring the Influence of Long Range Dependencies with Neural Network Language Models", Proc. of NAACLHLT 2012 Workshop, pp. 1-10
10. Goodman, J. T., "A bit of progress in language modeling", Computer Speech and Language 15 (2001), pp. 403-434
11. Goodman, J. T., "Classes for fast maximum entropy training", Proc. of the ICASSP 2001, pp. 561-564
12. Morin, F., Bengio, Y., "Hierarchical Probabilistic Neural Network Language Model", Proc. of the 10th Int. Workshop on Artificial Intelligence and Statistics 2005
13. Schwenk, H., "Continuous Space Language Models", Computer Speech and Language 21 (2007), pp. 492-518
14. Park, J., Liu, X., Gales, M. J. F., and Woodland, P. C., "Improved Neural Network Based Language Modelling and Adaptation", Proc. of Interspeech 2010, pp. 1041-1044
15. Hoffmeister, B., Schluter, R., and Ney, H., "iCNC and iROVER: The Limits of Improving System Combination with Classification?", Proc. of Interspeech 2008, pp. 232-235