Factored language model based on recurrent neural network

24th International Conference on Computational Linguistics - Proceedings of COLING 2012: Technical Papers

Volumn , Issue , 2012, Pages 2835-2850

  Wu, Youzheng ^a   Lu, Xugang ^a   Yamamoto, Hitoshi ^a   Matsuda, Shigeki ^a   Hori, Chiori ^a   Kashioka, Hideki ^a  

^a NATIONAL INSTITUTE OF INFORMATION AND COMMUNICATIONS TECHNOLOGY   (Japan)

Author keywords

Factored languagemodel; Large vocabulary continuous speech recognition; Recurrent neural network

Indexed keywords

BETTER PERFORMANCE; FACTORED LANGUAGEMODEL; LANGUAGE MODEL; LARGE VOCABULARY CONTINUOUS SPEECH RECOGNITION; LINGUISTIC INFORMATION; NETWORK-BASED; TREEBANKS; WORD ERROR RATE;

CONTINUOUS SPEECH RECOGNITION; RECURRENT NEURAL NETWORKS; SEMANTICS;

COMPUTATIONAL LINGUISTICS;

EID: 84876802979     PISSN: None     EISSN: None     Source Type: Conference Proceeding    
DOI: None     Document Type: Conference Paper

References (32)

1. Alexandrescu, A. and Kirchhoff, K. (2006). Factored neural language models. In Proceedings of the NAACL 2006, pages 1-4, New York, USA.
2. Arisoy, E., Sainath, T. N., Kingsbury, B., and Ramabhadran, B. (2012). Deep neural network language models. In Proceedings of NAACL-HLT 2012 Workshop: Will We Ever Really Replace the N-gram Model? On the Future of Language Modeling for HLT, pages 20-28, Montreal, Canada.
3. Bengio, Y., Ducharme, R., Vincent, P., and Jauvin, C. (2003). A neural probabilistic language model. Journal of Machine Learning Research, pages 1137-1155.
4. Boden, M. (2002). A guide to recurrent neural networks and backpropagation. In The Dallas Project, Sics Technical Report.
5. Brown, P. F., deSouza, P. V., Mercer, R. L., Pietra, V. J. D., and Lai, J. C. (1992). Class-based n-gram models of natural language. Computational Linguistics, 18(4):467-470.
6. Chelba, C. and Jelinek, F. (1998). Exploiting syntactic structure for language modeling. In Proceedings of the 36th Annual Meeting of the Association for Computational Linguistics and 17th International Conference on Computational Linguistics, pages 225-231, Montreal, Canada.
7. Chelba, C. and Jelinek, F. (2000). Structured language modeling. Computer Speech and Language, 13(4):283-332.
8. Chen, S. F. and Goodman, J. T. (1996). An empirical study of smoothing techniques for language modeling. In Proceedings of ACL 1996, pages 310-318, California, USA.
9. Collins, M., Roark, B., and Saraclar, M. (2005). Discriminative syntactic language modeling for speech recognition. In Proceedings of ACL 2005, pages 507-514, Michigan, USA.
10. Duh, K. and Kirchhoff, K. (2004). Automatic learning of language model structure. In Proceedings of the International Conference on Computational Linguistics (COLING) 2004, pages 148-154, Geneva, Switzerland.
11. Emami, A. and Jelinek, F. (2004). Exact training of a neural syntactic language model. In Proceedings of ICASSP 2004, pages 245-248, Montreal, Canada.
12. Federico, M., Bentivogli, L., Paul, M., and Stuker, S. (2011). Overview of the iwslt 2011 evaluation campaign. In Proceedings of IWSLT 2011, pages 11-27, San Francisco, USA.
13. Gildea, D. and Hofmann, T. (1999). Topic-based language models using em. In Proceedings of Eurospeech 1999, pages 2167-2170.
14. Goodman, J. (2001). Classes for fast maximum entropy training. In Proceedings of ICASSP 2001, Utah, USA.
15. Hsu, B.-J. P. and Glass, J. (2006). Style and topic language model adaptation using hmm-lda. In Proceedings of EMNLP 2006, pages 373-381, Sydney, Australia.
16. Khudanpur, S. and Wu, J. (2000). Maximumentropy techniques for exploiting syntactic, semantic and collocational dependencies in language modeling. Computer Speech and Language, pages 355-372.
17. Koehn, P. and Hoang, H. (2007). Factored translation models. In Proceedings of EMNLP 2007, pages 868-876, Prague, Czech Republic.
18. Kuo, H.-K. J., Arisoy, E., Emami, A., and Vozila, P. (2012). Large scale hierarchical neural network language models. In Proceedings of Interspeech 2012.
19. Kuo, H.-K. J., Mangu, L., Emami, A., Zitouni, I., and Lee, Y.-S. (2009). Syntactic features for arabic speech recognition. In Proceedings of Automatic Speech Recognition & Understanding (ASRU) 2009, pages 327-332, Merano, Italy.
20. Mikolov, T., Anoop, D., Stefan, K., Burget, L., and Cernocky, J. (2011a). Empirical evaluation and combination of advanced language modeling techniques. In Proceedings of INTERSPEECH 2011, pages 605-608, Florence, Italy.
21. Mikolov, T., Karafiat, M., Burget, L., Cernocky, J. H., and Khudanpur, S. (2010). Recurrent neural network based language model. In Proceedings of INTERSPEECH 2010, pages 1045- 1048, Makuhari, Japan.
22. Mikolov, T., Kombrink, S., Burget, L., Cernocky, J., and Khudanpur, S. (2011b). Extensions of recurrent neural network language model. In Proceedings of ICASSP 2011, pages 5528-5531, Prague, Czech Republic.
23. Povey, D., Ghoshal, A., Boulianne, G., Burget, L., Glembek, O., Goel, N., Hannemann, M., Motlicek, P., Qian, Y., Schwarz, P., Silovsky, J., Stemmer, G., and Vesely, K. (2011). The kaldi speech recognition toolkit. In IEEE 2011 Workshop on Automatic Speech Recognition and Understanding. IEEE Signal Processing Society. IEEE Catalog No.: CFP11SRW-USB.
24. Rastrow, A., Dredze, M., and Khudanpur, S. (2012). Fast syntactic analysis for statistical language modeling via substructure sharing and uptraining. In Proceedings of the 50th Annual Meeting of the Association for Computational Linguistics, pages 175-183, Jeju, Korea.
25. Rosenfeld, R. (1996). A maximum entropy approach to adaptive statistical language modeling. Computational Linguistics, 10(3):187-228.
26. Schwenk, H. (2007). Continuous space language models. Computer Speech and Language, 21(3):492-518.
27. Schwenk, H., Rousseau, A., and Attik, M. (2012). Large, pruned or continuous space language models on a gpu for statistical machine translation. In Proceedings of NAACL-HLT 2012 Workshop: Will We Ever Really Replace the N-gram Model? On the Future of Language Modeling for HLT, pages 11-19, Montreal, Canada.
28. Seide, F., Li, G., Chen, X., and Yu, D. (2011). Feature engineering in context-dependentdeep neural networks for conversational speech transcription. In Proceedings of ASRU 2011, Hawaii, USA.
29. Shi, Y., Wiggers, P., and Catholijn, M. J. (2012). Towards recurrent neural networks language models with linguistic and contextual features. In Proceedings of Interspeech 2012.
30. Stolcke, A. (2002). Srilm - an extensible language modeling toolkit. In Proceedings of INTERSPEECH 2002, pages 901-904, Colorado, USA.
31. Xu, P., Chelba, C., and Jelinek, F. (2002). A study on richer syntactic dependencies for structured language modeling. In Proceedings of the 40th Annual Meeting of the Association for Computational Linguistics, pages 191-198, Philadelphia, USA.
32. Xu, P. and Jelinek, F. (2004). Random forests in language modeling. In Proceedings of EMNLP 2004, pages 325-332, Barcelona, Spain.