Converting Neural Network Language Models into back-off language models for efficient decoding in automatic speech recognition

ICASSP, IEEE International Conference on Acoustics, Speech and Signal Processing - Proceedings

Volumn , Issue , 2013, Pages 8242-8246

  Arisoy, Ebru ^a   Chen, Stanley F ^a   Ramabhadran, Bhuvana ^a   Sethy, Abhinav ^a  

^a IBM T J WATSON RESEARCH CENTER   (United States)

Author keywords

decoding with neural network language models; Neural network language models

Indexed keywords

ACCURACY IMPROVEMENT; APPROXIMATE METHODS; AUTOMATIC SPEECH RECOGNITION; HIGHER-ORDER MODELS; LANGUAGE MODEL; LARGE VOCABULARY CONTINUOUS SPEECH RECOGNITION; N-GRAM LANGUAGE MODELS; NETWORK LANGUAGE;

CONTINUOUS SPEECH RECOGNITION; DECODING; NEURAL NETWORKS; SIGNAL PROCESSING;

COMPUTATIONAL LINGUISTICS;

EID: 84890469222     PISSN: 15206149     EISSN: None     Source Type: Conference Proceeding    
DOI: 10.1109/ICASSP.2013.6639272     Document Type: Conference Paper

References (20)

1. Yoshua Bengio, Rejean Ducharme, Pascal Vincent, and Christian Jauvin, "A neural probabilistic language model," Journal of Machine Learning Research, vol. 3, pp. 1137-1155, 2003.
2. Holger Schwenk and Jean-Luc Gauvain, "Training neural network language models on very large corpora," in Proceedings of HLT-EMNLP 2005, 2005, pp. 201-208.
3. Holger Schwenk, "Continuous space language models," Computer Speech and Language, vol. 21, no. 3, pp. 492-518, July 2007.
4. Hong-Kwang Jeff Kuo, Ebru Arisoy, Ahmad Emami, and Paul Vozila, "Large scale hierarchical neural network language models," in Proceedings of Interspeech, Portland, Oregon, USA, 2012.
5. Tomas Mikolov, Martin Karafiat, Lukas Burget, Jan Cernocky, and Sanjeev Khudanpur, "Recurrent neural network based language model," in Proceedings of INTERSPEECH 2010, 2010, pp. 1045-1048.
6. Tomas Mikolov, Stefan Kombrink, Lukas Burget, Jan Cernocky, and Sanjeev Khudanpur, "Extensions of recurrent neural network language model," in Proceedings of IEEE International Conference on Acoustic, Speech and Signal Processing, 2011, pp. 5528-5531.
7. Holger Schwenk and Jean-Luc Gauvain, "Connectionist language modeling for large vocabulary continuous speech recognition," in Proceedings of IEEE International Conference on Acoustic, Speech and Signal Processing, Orlando, Florida, USA, 2002, pp. 765-768.
8. Andreas Stolcke, "Entropy-based pruning of backoff language models," in Proceedings of DARPA Broadcast News Transcription and Understanding Workshop, Lansdowne, VA, USA, 1998, pp. 270-274.
9. M. Siu andM. Ostendorf, "Variable n-grams and extensions for conversational speech language modeling," IEEE Transactions on Acoustics, Speech, and Signal Processing, vol. 8, no. 1, pp. 63-75, January 2000.
10. Vesa Siivola and Bryan Pellom, "Growing an n-gram model," in Proceedings of Interspeech, 2005, pp. 1309-1312.
11. Vesa Siivola, Teemu Hirsimaki, and Sami Virpioja, "On growing and pruning kneser-ney smoothed n-gram models," IEEE Transactions on Audio, Speech and Language Processing, vol. 15, no. 5, pp. 1617-1624, 2007.
12. Sami Virpioja and Mikko Kurimo, "Compact n-gram models by incremental growing and clustering of histories," in Proceedings of Interspeech -ICSLP, Pittsburgh, PA, USA, 2006, pp. 1037-1040.
13. Anoop Deoras, Tomas Mikolov, Stefan Kombrink, Martin Karafiat, and Sanjeev Khudanpur, "Variational approximation of long-span language models for LVCSR," in Proceedings of IEEE International Conference on Acoustic, Speech and Signal Processing, Prague, Czech Republic, 2012, pp. 5532-5535.
14. Anoop Deoras, Tomas Mikolov, Stefan Kombrink, and Kenneth Church, "Approximate inference: A sampling based modeling technique to capture complex dependencies in a language model," Speech Communication, vol. 55, no. 1, pp. 162-177, January 2013.
15. Gwenole Lecorve and Petr Motlicek, "Conversion of recurrent neural network language models to weighted finite state transducers for automatic speech recognition," in Proceedings of Interspeech, Portland, Oregon, USA, 2012.
16. Wen Wang, Andreas Stolcke, and Mary P. Harper, "The use of a linguistically motivated language model in conversational speech recognition," in Proceedings of IEEE International Conference on Acoustic, Speech and Signal Processing, 2004, pp. 261-264.
17. Ahmad Emami, A neural syntactic language model, Ph.D. thesis, Johns Hopkins University, Baltimore, MD, USA, 2006.
18. Hong-Kwang Jeff Kuo, L. Mangu, A. Emami, I. Zitouni, and Y-S. Lee, "Syntactic features for Arabic speech recognition," in Proceedings of ASRU 2009, Merano, Italy, 2009, pp. 327-332.
19. S. F. Chen, B. Kingsbury, L. Mangu, D. Povey, G. Saon, H. Soltau, and G. Zweig, "Advances in speech transcription at IBM under the DARPA EARS program," IEEE Transactions on Audio, Speech, and Language Processing, vol. 14, no. 5, pp. 1596-1608, 2006.
20. P. F. Brown, V. J. Della Pietra, P. V. deSouza, J. C. Lai, and R. L. Mercer, "Class-based n-gram models of natural language," Computational Linguistics, vol. 18, no. 4, 1990.