Efficient lattice rescoring using recurrent neural network language models

ICASSP, IEEE International Conference on Acoustics, Speech and Signal Processing - Proceedings

Volumn , Issue , 2014, Pages 4908-4912

  Liu, X ^a   Wang, Y ^a   Chen, X ^a   Gales, M J F ^a   Woodland, P C ^a  

^a UNIVERSITY OF CAMBRIDGE   (United Kingdom)

Author keywords

language model; recurrent neural network; speech recognition

Indexed keywords

COMPUTATIONAL LINGUISTICS; RECURRENT NEURAL NETWORKS; SIGNAL PROCESSING;

CONFUSION NETWORKS; CONVERSATIONAL TELEPHONE SPEECH RECOGNITION; GENERALIZATION PERFORMANCE; INTRINSIC CHARACTERISTICS; LANGUAGE MODEL; RESCORING APPROACH; SPEECH RECOGNITION SYSTEMS; VECTOR REPRESENTATIONS;

SPEECH RECOGNITION;

EID: 84905240726     PISSN: 15206149     EISSN: None     Source Type: Conference Proceeding    
DOI: 10.1109/ICASSP.2014.6854535     Document Type: Conference Paper

References (28)

1. E. Arisoy, S. F. Chen, B. Ramabhadran, and A. Sethy (2013), "Converting neural network language models into back-off language models for efficient decoding in automatic speech recognition, " in Proc. ICASSP, Vancouver, Canada, 2013, pp. 8242-8246.
2. Y. Bengio and R. Ducharme (2003), "A neural probabilistic language model, " Journal of Machine Learning Research, vol. 3, pp. 1137-1155, 2003.
3. I. Bulyko, M. Ostendorf, and A. Stolcke (2003), "Getting more mileage from web text sources for conversational speech language modeling using class-dependent mixtures, " in Proc. HLT, Edmonton, Canada, 2003.
4. A. Deoras, T. Mikolov, S. Kombrink, M. Karafiat, and S. Khudanpur (2011), "Variational approximation of long-span language models for LVCSR, " in Proc. ICASSP, Prague, Czech Republic, 2011, pp. 5532-5535.
5. A. Deoras, T. Mikolov and K. Church (2011), "A fast re-scoring strategy to capture long-distance dependencies", Proc. EMNLP, Edinburgh, UK, 2011, pp. 1116-1127.
6. A. Deoras, T. Mikolov, S. Kombrink, and K. Church (2013), "Approximate inference: A sampling based modeling technique to capture complex dependencies in a language model, " Speech Communication, vol. 55, no. 1, pp. 162-177, January 2013.
7. A. Emami and L. Mangu (2007), "Empirical study of neural network language models for Arabic speech recognition, " in Proc. ASRU, Kyoto, Japan, 2007, pp. 147-152.
8. G. Evermann and P. C. Woodland (2000), "Posterior probability decoding, confidence estimation and system combination, " in Proc. Speech Transcription Workshop, College Park, MD, 2000.
9. G. Evermann, H. Y. Chan, M. J. F. Gales, B. Jia, D. Mrva, P. C.Woodland, and K. Yu (2005), "Training LVCSR systems on thousands of hours of data, " in Proc. ICASSP, Philadelphia, PA, 2005, vol. 1, pp. 209-212.
10. G. Lecorve and P. Motlicek (2012), "Conversion of recurrent neural network language models to weighted finite state transducers for automatic speech recognition, " in Proc. ISCA Interspeech, Portland, OR, 2012.
11. H.-S. Le, I. Oparin, A. Allauzen, J. Gauvain, and F. Yvon (2013), "Structured output layer neural network language models for speech recognition, " IEEE Transactions on Audio, Speech and Language Processing, vol. 21, no. 1, pp. 197-206, 2013.
12. X. Liu, M. J. F. Gales, J. L. Hieronymus, and P. C. Woodland (2010), "Language model combination and adaptation using weighted finite state transducers, " in Proc. ICASSP, Dallas, TX, 2010, pp. 5390-5393.
13. X. Liu, M. J. F. Gales, and P. C. Woodland (2013), "Use of contexts in language model interpolation and adaptation, " Computer Speech &Language, vol. 27, no. 1, pp. 301-321, January 2013.
14. T. Mikolov, M. Karafiat, L. Burget, J. Cernocky, and S. Khudanpur (2010), "Recurrent neural network based language model, " in Proc. ISCA Interspeech, Makuhari, Japan, 2010, pp. 1045-1048.
15. T. Mikolov, S. Kombrink, L. Burget, J. H. Cernocky, and S. Khudanpur (2011), "Extensions of recurrent neural network language model, " in Proc. ICASSP, Prague, Czech Republic, 2011, pp. 5528-5531.
16. T. Mikolov, S. Kombrink, L. Burget, J. H. Cernocky and S. Khudanpur (2011), "RNNLM-Recurrent neural network language modeling toolkit", in demo session of IEEE ASRU2011, Hawaii.
17. F. Morin and Y. Bengio (2005), "Hierarchical probabilistic neural network language model, " in Proc. International workshop on artificial intelligence and statistics, Barbados, 2005, pp. 246-252.
18. M. Mohri (1997), "Finite-state transducers in language and speech processing, " Computational linguistics, vol. 23, no. 2, pp. 269-311, 1997.
19. T. R. Niesler and P. C. Woodland (1996), "A variable-length category-based n-gram language model, " in Proc. ICASSP, Atlanta, GA, 1996, vol. 1, pp. 164-167.
20. H. Ney and S. Ortmanns (1999), "Dynamic programming search for continuous speech recognition, " IEEE Signal Processing Magazine, vol. 16, no. 5, pp. 64-83, 1999.
21. J. J. Odell, V. Valtchev, P. C. Woodland, and S. J. Young (1994), "A one pass decoder design for large vocabulary recognition, " in Proc. HLT, Stroudsburg, PA, 1994, pp. 405-410.
22. J. Park, X. Liu, M. J. F. Gales, and P. C. Woodland (2010), "Improved neural network based language modelling and adaptation, " in Proc. ISCA Interspeech, Makuhari, Japan, 2010, pp. 1041-1044.
23. D. E. Rumelhart, G. E. Hintont, and R. J. Williams (1986), "Learning representations by back-propagating errors, " Nature, vol. 323, no. 6088, pp. 533-536, 1986.
24. H. Schwenk (2007), "Continuous space language models, " Computer Speech &Language, vol. 21, no. 3, pp. 492-518, 2007.
25. Y. Si, Q. Zhang, T. Li, J. Pan, and Y. Yan (2013), "Prefix tree based n-best list re-scoring for recurrent neural network language model used in speech recognition system, " in Proc. ISCA Interspeech, Lyon, France, 2013, pp. 3419-3423.
26. A. Stolcke (1998), "Entropy-based pruning of backoff language models, " in Proc. DARPA Broadcast News Transcription and Understanding Workshop, Landsdowne, VA, 1998, pp. 270-274.
27. M. Sundermeyer, R. Schluter, and H. Ney (2012), "LSTM neural networks for language modeling, " in Proc. ISCA Interspeech, Portland, OR, 2012.
28. M. Sundermeyer, I. Oparin, J. L. Gauvain, B. Freiberg, R. Schluter, and H. Ney (2013), "Comparison of feedforward and recurrent neural network language models, " in Proc. ICASSP, Vancouver, Canada, 2013, pp. 8430-8434