Pragmatic neural language modelling in machine translation

NAACL HLT 2015 - 2015 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Proceedings of the Conference

Volumn , Issue , 2015, Pages 820-829

  Baltescu, Paul ^a   Blunsom, Phil ^a  

^a UNIVERSITY OF OXFORD   (United Kingdom)

Author keywords

[No Author keywords available]

Indexed keywords

COMPUTATIONAL LINGUISTICS; MODELING LANGUAGES; NATURAL LANGUAGE PROCESSING SYSTEMS;

LANGUAGE MODELLING; MACHINE TRANSLATIONS; NATURAL LANGUAGE APPLICATIONS; SPEED IMPROVEMENT; TRADITIONAL MODELS; TRAINING ALGORITHMS; TRANSLATION QUALITY; TRANSLATION SYSTEMS;

TRANSLATION (LANGUAGES);

EID: 84944029121     PISSN: None     EISSN: None     Source Type: Conference Proceeding    
DOI: 10.3115/v1/n15-1083     Document Type: Conference Paper

References (27)

1. Michael Auli and Jianfeng Gao. Decoder integration and expected bleu training for recurrent neural network language models. In Proceedings of the 52nd Annual Meeting of the Association for Computational Linguistics (ACL '14), pages 136-142, Baltimore, Maryland, June 2014. Association for Computational Linguistics.
2. Michael Auli, Michel Galley, Chris Quirk, and Geoffrey Zweig. Joint language and translation modeling with recurrent neural networks. In Proceedings of the 2013 Conference on Empirical Methods in Natural Language Processing, pages 1044-1054, Seattle, Washington, USA, October 2013. Association for Computational Linguistics.
3. Paul Baltescu, Phil Blunsom, and Hieu Hoang. Oxlm: A neural language modelling framework for machine translation. The Prague Bulletin of Mathematical Linguistics, 102(1):81-92, October 2014.
4. Yoshua Bengio and Jean-Sbastien Senecal. Adaptive importance sampling to accelerate training of a neural probabilistic language model. IEEE Transactions on Neural Networks, 19(4):713-722, 2008.
5. Yoshua Bengio, Réjean Ducharme, Pascal Vincent, and Christian Janvin. A neural probabilistic language model. Journal of Machine Learning Research, 3:1137-1155, 2003.
6. Jan A. Botha and Phil Blunsom. Compositional morphology for word representations and language modelling. In Proceedings of the 31st International Conference on Machine Learning (ICML '14), Beijing, China, 2014.
7. Peter F. Brown, Peter V. deSouza, Robert L. Mercer, Vincent J. Della Pietra, and Jenifer C. Lai. Class-based n-gram models of natural language. Computational Linguistics, 18(4):467-479, 1992.
8. Ciprian Chelba, Tomas Mikolov, Mike Schuster, Qi Ge, Thorsten Brants, and Phillipp Koehn. One billion word benchmark for measuring progress in statistical language modeling. CoRR, 2013.
9. Stanley F. Chen and Joshua Goodman. An empirical study of smoothing techniques for language modeling. Computer Speech & Language, 13(4): 359-393, 1999.
10. KyungHyun Cho, Bart van Merrienboer, Dzmitry Bahdanau, and Yoshua Bengio. On the properties of neural machine translation: Encoder-decoder approaches. CoRR, 2014.
11. Jacob Devlin, Rabih Zbib, Zhongqiang Huang, Thomas Lamar, Richard M. Schwartz, and John Makhoul. Fast and robust neural network joint models for statistical machine translation. In Proceedings of the 52nd Annual Meeting of the Association for Computational Linguistics (ACL'14), Baltimore, MD, USA, June 2014.
12. Joshua Goodman. Classes for fast maximum entropy training. CoRR, 2001.
13. Kenneth Heafield. Kenlm: Faster and smaller language model queries. In Proceedings of the Sixth Workshop on Statistical Machine Translation (WMT '11), pages 187-197, Edinburgh, Scotland, July 2011. Association for Computational Linguistics.
14. David A. Huffman. A method for the construction of minimum-redundancy codes. Proceedings of the Institute of Radio Engineers, 40(9):1098-1101, September 1952.
15. Philipp Koehn, Hieu Hoang, Alexandra Birch, Chris Callison-Burch, Marcello Federico, Nicola Bertoldi, Brooke Cowan, Wade Shen, Christine Moran, Richard Zens, Chris Dyer, Ondrej Bojar, Alexandra Constantin, and Evan Herbst. Moses: Open source toolkit for statistical machine translation. In Proceedings of the 45th Annual Meeting of the Association for Computational Linguistics (ACL '07), pages 177-180, Prague, Czech Republic, June 2007. Association for Computational Linguistics.
16. P. Liang. Semi-supervised learning for natural language. Master's thesis, Massachusetts Institute of Technology, 2005.
17. Tomas Mikolov, Anoop Deoras, Daniel Povey, Lukas Burget, and Jan Cernocky. Strategies for training large scale neural network language models. In Proceedings of the 2011 Automatic Speech Recognition and Understanding Workshop, pages 196-201. IEEE Signal Processing Society, 2011a.
18. Tom Mikolov, Stefan Kombrink, Luk Burget, Jan ernock, and Sanjeev Khudanpur. Extensions of recurrent neural network language model. In Proceedings of the 2011 IEEE International Conference on Acoustics, Speech, and Signal Processing, ICASSP 2011, pages 5528-5531. IEEE Signal Processing Society, 2011b.
19. Andriy Mnih and Geoffrey Hinton. A scalable hierarchical distributed language model. In Advances in Neural Information Processing Systems, volume 21, pages 1081-1088, 2009.
20. Andriy Mnih and Yee Whye Teh. A fast and simple algorithm for training neural probabilistic language models. In Proceedings of the 29th International Conference on Machine Learning (ICML '12), pages 1751-1758, Edinburgh, Scotland, 2012.
21. Frederic Morin and Yoshua Bengio. Hierarchical probabilistic neural network language model. In Proceedings of the 10th International Workshop on Artificial Intelligence and Statistics (AISTATS '05), pages 246-252. Society for Artificial Intelligence and Statistics, 2005.
22. Franz Josef Och. Minimum error rate training in statistical machine translation. In Proceedings of the 41st Annual Meeting on Association for Computational Linguistics (ACL'03), pages 160-167. Association for Computational Linguistics, 2003.
23. Holger Schwenk. Continuous space language models. Computer Speech & Language, 21(3):492-518, 2007.
24. Holger Schwenk. Continuous-space language models for statistical machine translation. Prague Bulletin of Mathematical Linguistics, 93:137-146, 2010.
25. Ilya Sutskever, Oriol Vinyals, and Quoc V. Le. Sequence to sequence learning with neural networks. CoRR, 2014.
26. Ashish Vaswani, Yinggong Zhao, Victoria Fossum, and David Chiang. Decoding with large-scale neural language models improves translation. In Proceedings of the 2013 Conference on Empirical Methods in Natural Language Processing, pages 1387-1392, Seattle, Washington, USA, October 2013. Association for Computational Linguistics.
27. Yinggong Zhao, Shujian Huang, Huadong Chen, and Jiajun Chen. An investigation on statistical machine translation with neural language models. In Chinese Computational Linguistics and Natural Language Processing Based on Naturally Annotated Big Data-13th China National Conference, CCL 2014, and Second International Symposium, NLP-NABD, pages 175-186,Wuhan, China, October 2014.