Recurrent neural network language model adaptation for multi-genre broadcast speech recognition

Proceedings of the Annual Conference of the International Speech Communication Association, INTERSPEECH

Volumn 2015-January, Issue , 2015, Pages 3511-3515

  Chen, X ^a   Tan, T ^a,b   Liu, X ^a   Lanchantin, P ^a   Wan, M ^a   Gales, M J F ^a   Woodland, P C ^a  

^a UNIVERSITY OF CAMBRIDGE   (United Kingdom)

^b SHANGHAI JIAO TONG UNIVERSITY   (China)

Author keywords

Language model adaptation; Latent Dirichlet allocation; RNNLM; Speech recognition; Topic model

Indexed keywords

COMPUTATIONAL LINGUISTICS; DATA MINING; RECURRENT NEURAL NETWORKS; SEMANTICS; SPEECH; SPEECH COMMUNICATION; STATISTICS; VARIATIONAL TECHNIQUES;

HIERARCHICAL DIRICHLET PROCESS; LANGUAGE MODEL ADAPTATION; LATENT DIRICHLET ALLOCATION; NETWORK LANGUAGE; PROBABILISTIC LATENT SEMANTIC ANALYSIS; RNNLM; TOPIC MODELING; WORD ERROR RATE REDUCTIONS;

SPEECH RECOGNITION;

EID: 84959155988     PISSN: 2308457X     EISSN: 19909772     Source Type: Conference Proceeding    
DOI: None     Document Type: Conference Paper

References (34)

1. T. Mikolov, M. Karafiát, L. Burget, J. Cernocky, and S. Khudanpur, "Recurrent neural network based language model. " Proc. Interspeech, 2010, pp. 1045-1048.
2. T. Mikolov, S. Kombrink, L. Burget, J. Cernocky, and S. Khudanpur, "Extensions of recurrent neural network language model, " Proc. ICASSP, 2011, pp. 5528-5531.
3. M. Sundermeyer, I. Oparin, J.-L. Gauvain, B. Freiberg, R. Schluter, and H. Ney, "Comparison of feedforward and recurrent neural network language models, " Proc. ICASSP, Vancouver, Canada, May 2013, pp. 8430-8434.
4. K. Yao, G. Zweig, M.-Y. Hwang, Y. Shi, and D. Yu, "Recurrent neural networks for language understanding. " Proc. Interspeech, 2013, pp. 2524-2528.
5. G. Mesnil, X. He, L. Deng, and Y. Bengio, "Investigation of recurrent-neural-network architectures and learning methods for spoken language understanding. " Proc. Interspeech, 2013.
6. C. Chelba, T. Mikolov, M. Schuster, Q. Ge, T. Brants, P. Koehn, and T. Robinson, "One billion word benchmark for measuring progress in statistical language modeling, " Google, Tech. Rep., 2013. [Online]. Available: http: //arxiv. org/abs/1312. 3005
7. X. Chen, M. Gales, K. Knill, C. Breslin, L. Chen, K. Chin, and V. Wan, "An initial investigation of long-term adaptation for meeting transcription, " Proc. Interspeech, 2014.
8. Y. Wu, H. Yamamoto, X. Lu, S. Matsuda, C. Hori, and H. Kashioka, "Factored recurrent neural network language model in TED lecture transcription. " Proc. IWSLT, 2012, pp. 222-228.
9. T. Mikolov and G. Zweig, "Context dependent recurrent neural network language model. " Proc. SLT, 2012, pp. 234-239.
10. T.-H. Wen, A. Heidel, H.-Y. Lee, Y. Tsao, and L.-S. Lee, "Recurrent neural network based personalized language modeling by social network crowdsourcing. " Proc. Interspeech, 2013.
11. Y. Shi, "Language models with meta-information, " Ph. D. dissertation, TU Delft, Delft University of Technology, 2014.
12. O. Tilk and T. Alumae, "Multi-domain recurrent neural network language model for medical speech recognition, " in Proc. HLT, vol. 268, 2014.
13. T. Hofmann, "Probabilistic latent semantic indexing, " in Proc. 22nd ACM SIGIR conference, 1999, pp. 50-57.
14. D. M. Blei, A. Y. Ng, and M. I. Jordan, "Latent Dirichlet allocation, " The Journal of Machine Learning Research, vol. 3, pp. 993-1022, 2003.
15. Y. W. Teh, M. I. Jordan, M. J. Beal, and D. M. Blei, "Hierarchical Dirichlet processes, " Journal of the American Statistical Association, vol. 101, no. 476, 2006.
16. D. E. Rumelhart, G. E. Hinton, and R. J. Williams, Learning representations by back-propagating errors. MIT Press, Cambridge, MA, USA, 1988.
17. H. Schwenk, "Continuous space language models, " Computer Speech & Language, vol. 21, no. 3, pp. 492-518, 2007.
18. A. Emami and L. Mangu, "Empirical study of neural network language models for Arabic speech recognition, " Proc. IEEE Workshop on ASRU, 2007, pp. 147-152.
19. J. Park, X. Liu, M. J. F. Gales, and P. C. Woodland, "Improved neural network based language modelling and adaptation, " Proc. ISCA Interspeech, 2010, pp. 1041-1044.
20. H.-S. Le, I. Oparin, A. Allauzen, J. Gauvain, and F. Yvon, "Structured output layer neural network language models for speech recognition, " IEEE Transactions on Audio, Speech, and Language Processing, vol. 21, no. 1, pp. 197-206, 2013.
21. X. Liu, Y. Wang, X. Chen, M. Gales, and P. C. Woodland, "Efficient lattice rescoring using recurrent neural network language models, " Proc. ICASSP, 2014.
22. X. Chen, Y. Wang, X. Liu, M. Gales, and P. Woodland, "Efficient GPU-based training of recurrent neural network language models using spliced sentence bunch, " Proc. Interspeech, 2014.
23. X. Chen, X. Liu, M. Gales, and P. C. Woodland, "Improving the training and evaluation efficiency of recurrent neural network language models, " Proc. ICASSP, 2015.
24. X. Chen, X. Liu, M. Gales, and P. C. Woodland, "Recurrent neural network language model training with noise contrastive estimation for speech recognition, " Proc. ICASSP, 2015.
25. T. Mikolov and G. Zweig, "Context dependent recurrent neural network language model. " Proc. SLT, 2012, pp. 234-239.
26. D. Mrva and P. C. Woodland, "A PLSA-based language model for conversational telephone speech. " Proc. Interspeech, 2004.
27. D. Gildea and T. Hofmann, "Topic-based language models using EM. " Proc. Eurospeech, 1999.
28. Y. C. Tam and T. Schultz, "Unsupervised language model adaptation using latent semantic marginals. " Proc. Interspeech, 2006.
29. X.-H. Phan and C.-T. Nguyen, "GibbsLDA++: A C/C++ implementation of latent Dirichlet allocation (LDA), " 2007.
30. F. Grezl and P. Fousek, "Optimizing bottle-neck features for LVCSR, " Proc. ICASSP, 2008, pp. 4729-4732.
31. J. Park, F. Diehl, M. J. F. Gales, M. Tomalin, and P. C. Woodland, "The efficient incorporation of MLP features into automatic speech recognition systems, " Computer Speech & Language, vol. 25, no. 3, pp. 519-534, 2011.
32. P. Lanchantin, P. J. Bell, M. J. Gales, T. Hain, X. Liu, Y. Long, J. Quinnell, S. Renals, O. Saz, M. S. Seigel, P. C. Woodland, "Automatic transcription of multi-genre media archives, " Proc. First Workshop on Speech, Language and Audio in Multimedia, 2013.
33. X. Chen, Y. Wang, X. Liu, M. Gales, and P. C. Woodland, "Efficient training of recurrent neural network language models using spliced sentence bunch, " Proc. Interspeech, 2014.
34. L. Mangu, E. Brill, and A. Stolcke, "Finding consensus in speech recognition: word error minimization and other applications of confusion networks, " Computer Speech & Language, vol. 14, no. 4, pp. 373-400, 2000.