Large, pruned or continuous space language models on a GPU for statistical machine translation

WLM 2012 - Will We Ever Really Replace the N-gram Model? On the Future of Language Modeling for HLT at the 2012 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, NAACL-HLT 2012

Volumn , Issue , 2012, Pages 11-19

  Schwenk, Holger ^a   Rousseau, Anthony ^a   Attik, Mohammed ^a  

^a UNIVERSITÉ DU MAINE   (France)

Author keywords

[No Author keywords available]

Indexed keywords

COMPUTATIONAL LINGUISTICS; GRAPHICS PROCESSING UNIT; MACHINE TRANSLATION; PROGRAM PROCESSORS; SPEECH RECOGNITION; SPEECH TRANSMISSION;

BACKOFFS; CONTINUOUS SPACES; DATA SELECTION; EXPERIMENTAL COMPARISON; LANGUAGE MODEL; LARGE SPACES; LARGE VOCABULARY SPEECH RECOGNITION; OPEN SOURCE IMPLEMENTATION; STATISTICAL MACHINE TRANSLATION; STATISTICAL MACHINE TRANSLATION SYSTEM;

COMPUTER AIDED LANGUAGE TRANSLATION;

EID: 85045980083     PISSN: None     EISSN: None     Source Type: Conference Proceeding    
DOI: None     Document Type: Conference Paper

References (27)

1. Yoshua Bengio and Rejean Ducharme. 2001. A neural probabilistic language model. In NIPS, volume 13, pages 932-938.
2. Yoshua Bengio, Rejean Ducharme, Pascal Vincent, and Christian Jauvin. 2003. A neural probabilistic language model. JMLR, 3(2):1137-1155.
3. Yoshua Bengio. 2007. learning deep architectures for AI. Technical report, University of Montréal.
4. Thorsten Brants, Ashok C. Popat, Peng Xu, Franz J. Och, and Jeffrey Dean. 2007. Large language models in machine translation. In EMNLP, pages 858-867.
5. Stanley F. Chen and Joshua T. Goodman. 1999. An empirical study of smoothing techniques for language modeling. Computer Speech & Language, 13(4):359-394.
6. Marcello Federico and Maura Cettolo. 2007. Efficient handling of n-gram language models for statistical machine translation. In Second Workshop on SMT, pages 88-95.
7. Kenneth Heafield. 2011. KenLM: Faster and smaller language model queries. In Sixth Workshop on SMT, pages 187-197.
8. Philipp Koehn, Hieu Hoang, Alexandra Birch, Chris Callison-Burch, Marcello Federico, Nicola Bertoldi, Brooke Cowan, Wade Shen, Christine Moran, Richard Zens, Chris Dyer, Ondrej Bojar, Alexandra Constantin, and Evan Herbst. 2007. Moses: Open source toolkit for statistical machine translation. In ACL, demonstration session.
9. L. Lamel, J.-L. Gauvain, V.-B. Le, I. Oparin, and S. Meng. 2011. Improved models for mandarin speech-to-text transcription. In ICASSP, pages 4660-4663.
10. H.S. Le, A. Allauzen, G.Wisniewski, and F. Yvon. 2010. Training continuous space language models: Some practical issues. In EMNLP, pages 778-788.
11. H.S. Le, I. Oparin, A. Allauzen, J-L. Gauvain, and F. Yvon. 2011a. Structured output layer neural network language model. In ICASSP, pages 5524-5527.
12. H.S. Le, I. Oparin, A. Messaoudi, A. Allauzen, J-L. Gauvain, and F. Yvon. 2011b. Large vocabulary SOUL neural network language models. In Interspeech.
13. X. Liu, M. J. F. Gales, and P. C. Woodland. 2011. Improving LVCSR system combination using neural network language model cross adaptation. In Interspeech, pages 2857-2860.
14. Tomáš Mikolov, Martin Karafiát, Lukáš Burget, Jan Černocký, and Sanjeev Khudanpur. 2010. Recurrent neural network based language model. In Interspeech, pages 1045-1048.
15. T. Mikolov, S. Kombrink, L. Burget, J.H. Cernocky, and S. Khudanpur. 2011. Extensions of recurrent neural network language model. In ICASSP, pages 5528-5531.
16. Andriy Mnih and Geoffrey Hinton. 2008. A scalable hierarchical distributed language model. In NIPS.
17. Robert C. Moore and William Lewis. 2010. Intelligent selection of language model training data. In ACL, pages 220-224.
18. Frederic Morin and Yoshua Bengio. 2005. Hierarchical probabilistic neural network language model. In Proceedings of the Tenth International Workshop on Artificial Intelligence and Statistics.
19. Junho Park, Xunying Liu, Mark J. F. Gales, and Phil C. Woodland. 2010. Improved neural network based language modelling and adaptation. In Interspeech, pages 1041-1044.
20. Holger Schwenk and Yannick Estève. 2008. Data selection and smoothing in an open-source system for the 2008 NIST machine translation evaluation. In Interspeech, pages 2727-2730.
21. Holger Schwenk and Jean-Luc Gauvain. 2002. Connectionist language modeling for large vocabulary continuous speech recognition. In ICASSP, pages I: 765-768.
22. Holger Schwenk, Daniel Déchelotte, and Jean-Luc Gauvain. 2006. Continuous space language models for statistical machine translation. In Proceedings of the COLING/ACL 2006 Main Conference Poster Sessions, pages 723-730.
23. Holger Schwenk. 2004. Efficient training of large neural networks for language modeling. In IJCNN, pages 3059-3062.
24. Holger Schwenk. 2007. Continuous space language models. Computer Speech and Language, 21:492-518.
25. Holger Schwenk. 2010. Continuous space language models for statistical machine translation. The Prague Bulletin of Mathematical Linguistics, (93):137-146.
26. David Talbot and Miles Osborne. 2007. Smoothed bloom filter language models: Tera-scale lms on the cheap. In EMNLP, pages 468-476.
27. Puyang Xu, Asela Gunawardana, and Sanjeev Khudanpur. 2011. Efficient subsampling for training complex language models. In EMNLP, pages 1128-1136.