Factored Recurrent Neural Network Language Model in TED Lecture Transcription

2012 International Workshop on Spoken Language Translation, IWSLT 2012

Volumn , Issue , 2012, Pages 222-228

  Wu, Youzheng ^a   Yamamoto, Hitoshi ^a   Lu, Xugang ^a   Matsuda, Shigeki ^a   Hori, Chiori ^a   Kashioka, Hideki ^a  

^a NATIONAL INSTITUTE OF INFORMATION AND COMMUNICATIONS TECHNOLOGY   (Japan)

Author keywords

[No Author keywords available]

Indexed keywords

COMPUTATIONAL LINGUISTICS;

DATA SET; LANGUAGE MODEL; N-GRAMS; NETWORK LANGUAGE; NETWORK-BASED; PERFORMANCE; STATE OF THE ART; TEST DATA; WORD ERROR RATE; WORD ERROR RATE REDUCTIONS;

RECURRENT NEURAL NETWORKS;

EID: 84910084714     PISSN: None     EISSN: None     Source Type: Conference Proceeding    
DOI: None     Document Type: Conference Paper

References (23)

1. Alexandrescu, A. and Kirchhoff, K. (2006). Factored neural language models. In Proceedings of the NAACL 2006, pages 1-4, New York, USA.
2. Arisoy, E., Sainath, T. N., Kingsbury, B., and Ramabhadran, B. (2012). Deep neural network language models. In Proceedings of NAACL-HLT 2012 Workshop: Will We Ever Really Replace the N-gram Model? On the Future of Language Modeling for HLT, pages 20-28, Montreal, Canada.
3. Bengio, Y., Ducharme, R., Vincent, P., and Jauvin, C. (2003). A neural probabilistic language model. Journal of Machine Learning Research, pages 1137-1155.
4. Bilmes, J. A. and Kirchhoff, K. (2003). Factored language models and generalized parallel backoff. In Proceedings of NAACL 2003, pages 4-6, USA.
5. Boden, M. (2002). A guide to recurrent neural networks and backpropagation. In The Dallas Project, Sics Technical Report.
6. Chelba, C. and Jelinek, F. (1998). Exploiting syntactic structure for language modeling. In Proceedings of the 36th Annual Meeting of the Association for Computational Linguistics and 17th International Conference on Computational Linguistics, pages 225-231, Montreal, Canada.
7. Duh, K. and Kirchhoff, K. (2004). Automatic learning of language model structure. In Proceedings of COL-ING 2004, pages 148-154, Geneva, Switzerland.
8. Emami, A. and Jelinek, F. (2004). Exact training of a neural syntactic language model. In Proceedings of ICASSP 2004, pages 245-248, Montreal, Canada.
9. Federico, M., Bentivogli, L., Paul, M., and Stuker, S. (2011). Overview of the iwslt 2011 evaluation campaign. In Proceedings of IWSLT 2011, pages 11-27, San Francisco, USA.
10. Goodman, J. (2001). Classes for fast maximum entropy training. In Proceedings of ICASSP 2001, Utah, USA.
11. Khudanpur, S. and Wu, J. (2000). Maximum entropy techniques for exploiting syntactic, semantic and collocational dependencies in language modeling. Computer Speech and Language, pages 355-372.
12. Kuo, H.-K. J., Mangu, L., Emami, A., Zitouni, I., and Lee, Y.-S. (2009). Syntactic features for arabic speech recognition. In Proceedings of Automatic Speech Recognition & Understanding (ASRU) 2009, pages 327-332, Merano, Italy.
13. Kuo, H.-K. J., Arisoy, E., Emami, A., and Vozila, P. (2012). Large scale hierarchical neural network language models. In Proceedings of Interspeech 2012.
14. Mikolov, T., Anoop, D., Stefan, K., Burget, L., and Cernocky, J. (2011a). Empirical evaluation and combination of advanced language modeling techniques. In Proceedings of INTERSPEECH 2011, pages 605-608, Florence, Italy.
15. Mikolov, T., Karafiat, M., Burget, L., Cernocky, J. H., and Khudanpur, S. (2010). Recurrent neural network based language model. In Proceedings of INTERSPEECH 2010, pages 1045-1048, Makuhari, Japan.
16. Mikolov, T., Kombrink, S., Burget, L., Cernocky, J., and Khudanpur, S. (2011b). Extensions of recurrent neural network language model. In Proceedings of ICASSP 2011, pages 5528-5531, Prague, Czech Republic.
17. Moore, C., Lewis, W. (2010). Intelligent Selection of Language Model Training Data. In Proceedings of the ACL 2010 Conference Short Papers, pages 220C224, Hawaii, USA.
18. Povey, D., Ghoshal, A., Boulianne, G., Burget, L., Glembek, O., Goel, N., Hannemann, M., Motlicek, P., Qian, Y., Schwarz, P., Silovsky, J., Stemmer, G., and Vesely, K. (2011). The kaldi speech recognition toolkit. In IEEE 2011 Workshop on Automatic Speech Recognition and Understanding. IEEE Signal Processing Society. IEEE Catalog No.: CFP11SRW-USB.
19. Schwenk, H. (2007). Continuous space language models. Computer Speech and Language, 21(3):492-518.
20. Schwenk, H., Rousseau A., and Attik, M. (2012) Large, Pruned or Continuous Space Language Models on a GPU for Statistical Machine Translation. In Proceedings of NAACL workshop on the Future of Language Modeling, pages 11-19, Canada.
21. Stolcke, A. (2002). Srilm - an extensible language modeling toolkit. In Proceedings of INTERSPEECH 2002, pages 901-904, Colorado, USA.
22. Xu, P., Chelba, C., and Jelinek, F. (2002). A study on richer syntactic dependencies for structured language modeling. In Proceedings of ACL 2002, pages 191-198, Philadelphia, USA.
23. Xu, P. and Jelinek, F. (2004). Random forests in language modeling. In Proceedings of EMNLP 2004, pages 325-332, Barcelona, Spain.