Decoder integration and expected BLEU training for recurrent neural network language models

52nd Annual Meeting of the Association for Computational Linguistics, ACL 2014 - Proceedings of the Conference

Volumn 2, Issue , 2014, Pages 136-142

  Auli, Michael ^a   Gao, Jianfeng ^a  

^a MICROSOFT RESEARCH   (United States)

Author keywords

[No Author keywords available]

Indexed keywords

COMPUTATIONAL LINGUISTICS; COMPUTER AIDED LANGUAGE TRANSLATION; DECODING;

CROSS ENTROPY; FRENCH-ENGLISH; IMPROVE-A; MACHINE TRANSLATIONS; NETWORK LANGUAGE; PHRASE-BASED STATISTICAL MACHINE TRANSLATION; RECURRENT NETWORKS;

RECURRENT NEURAL NETWORKS;

EID: 84906923072     PISSN: None     EISSN: None     Source Type: Conference Proceeding    
DOI: 10.3115/v1/p14-2023     Document Type: Conference Paper

References (29)

1. Alexandre Allauzen, Hélene Bonneau-Maynard, Hai-Son Le, Aurélien Max, Guillaume Wisniewski, François Yvon, Gilles Adda, Josep Maria Crego, Adrien Lardilleux, Thomas Lavergne, and Artem Sokolov. 2011. LIMSI @ WMT11. In Proc. of WMT, pages 309-315, Edinburgh, Scotland, July. Association for Computational Linguistics.
2. Ebru Arisoy, Tara N. Sainath, Brian Kingsbury, and Bhuvana Ramabhadran. 2012. Deep Neural Network Language Models. In NAACL-HLT Workshop on the Future of Language Modeling for HLT, pages 20-28, Stroudsburg, PA, USA. Association for Computational Linguistics.
3. Michael Auli and Adam Lopez. 2011. Training a Log-Linear Parser with Loss Functions via Softmax- Margin. In Proc. of EMNLP, pages 333-343. Association for Computational Linguistics, July.
4. Michael Auli, Michel Galley, Chris Quirk, and Geoffrey Zweig. 2013. Joint Language and Translation Modeling with Recurrent Neural Networks. In Proc. of EMNLP, October.
5. Yoshua Bengio, Réjean Ducharme, Pascal Vincent, and Christian Jauvin. 2003. A Neural Probabilistic Language Model. Journal of Machine Learning Research, 3:1137-1155.
6. Michel Galley and Christopher D. Manning. 2008. A Simple and Effective Hierarchical Phrase Reordering Model. In Proc. of EMNLP, pages 848-856.
7. Jianfeng Gao and Xiaodong He. 2013. Training MRFBased Phrase Translation Models using Gradient Ascent. In Proc. of NAACL-HLT, pages 450-459. Association for Computational Linguistics, June.
8. Jianfeng Gao, Xiaodong He, Scott Wen tau Yih, and Li Deng. 2014. Learning Continuous Phrase Representations for Translation Modeling. In Proc. of ACL. Association for Computational Linguistics, June.
9. Joshua Goodman. 1996. Parsing Algorithms and Metrics. In Proc. of ACL, pages 177-183, Santa Cruz, CA, USA, June.
10. Xiaodong He and Li Deng. 2012. Maximum Expected BLEU Training of Phrase and Lexicon Translation Models. In Proc. of ACL, pages 8-14. Association for Computational Linguistics, July.
11. Yuening Hu, Michael Auli, Qin Gao, and Jianfeng Gao. 2014. Minimum Translation Modeling with Recurrent Neural Networks. In Proc. of EACL. Association for Computational Linguistics, April.
12. Nal Kalchbrenner and Phil Blunsom. 2013. Recurrent Continuous Translation Models. In Proc. of EMNLP, pages 1700-1709, Seattle, Washington, USA, October. Association for Computational Linguistics.
13. Philipp Koehn, Hieu Hoang, Alexandra Birch, Chris Callison-Burch, Marcello Federico, Nicola Bertoldi, Brooke Cowan, Wade Shen, Christine Moran, Richard Zens, Chris Dyer, Ondrej Bojar, Alexandra Constantin, and Evan Herbst. 2007. Moses: Open Source Toolkit for Statistical Machine Translation. In Proc. of ACL Demo and Poster Sessions, pages 177-180, Prague, Czech Republic, Jun.
14. Hai-Son Le, Alexandre Allauzen, and François Yvon. 2012a. Continuous Space Translation Models with Neural Networks. In Proc. of HLT-NAACL, pages 39-48, Montréal, Canada. Association for Computational Linguistics.
15. Hai-Son Le, Thomas Lavergne, Alexandre Allauzen, Marianna Apidianaki, Li Gong, Aurélien Max, Artem Sokolov, Guillaume Wisniewski, and François Yvon. 2012b. LIMSI@WMT12. In Proc. of WMT, pages 330-337, Montréal, Canada, June. Association for Computational Linguistics.
16. Percy Liang, Alexandre Bouchard-Côté, Ben Taskar, and Dan Klein. 2006. An end-to-end discriminative approach to machine translation. In Proc. of ACLCOLING, pages 761-768, Jul.
17. Tomáš Mikolov, Karafiát Martin, Lukáš Burget, Jan Cernocký, and Sanjeev Khudanpur. 2010. Recurrent Neural Network based Language Model. In Proc. of INTERSPEECH, pages 1045-1048.
18. Tomáš Mikolov, Anoop Deoras, Daniel Povey, Lukáš Burget, and Jan Černocký. 2011. Strategies for Training Large Scale Neural Network Language Models. In Proc. of ASRU, pages 196-201.
19. Tomáš Mikolov. 2012. Statistical Language Models based on Neural Networks. Ph.D. thesis, Brno University of Technology.
20. Preslav Nakov, Francisco Guzman, and Stephan Vogel. 2012. Optimizing for Sentence-Level BLEU+1 Yields Short Translations. In Proc. of COLING. Association for Computational Linguistics.
21. Franz Josef Och. 2003. Minimum Error Rate Training in Statistical Machine Translation. In Proc. of ACL, pages 160-167, Sapporo, Japan, July.
22. Antti-Veikko I Rosti, Bing Zhang, Spyros Matsoukas, and Richard Schwartz. 2010. BBN System Description for WMT10 System Combination Task. In Proc. of WMT, pages 321-326. Association for Computational Linguistics, July.
23. Antti-Veikko I Rosti, Bing Zhang, Spyros Matsoukas, and Richard Schwartz. 2011. Expected BLEU Training for Graphs: BBN System Description for WMT11 System Combination Task. In Proc. of WMT, pages 159-165. Association for Computational Linguistics, July.
24. David E. Rumelhart, Geoffrey E. Hinton, and Ronald J. Williams. 1986. Learning Internal Representations by Error Propagation. In Symposium on Parallel and Distributed Processing.
25. Holger Schwenk, Anthony Rousseau, and Mohammed Attik. 2012. Large, Pruned or Continuous Space Language Models on a GPU for Statistical Machine Translation. In NAACL-HLT Workshop on the Future of Language Modeling for HLT, pages 11-19. Association for Computational Linguistics.
26. Martin Sundermeyer, Ilya Oparin, Jean-Luc Gauvain, Ben Freiberg, Ralf Schlüter, and Hermann Ney. 2013. Comparison of Feedforward and Recurrent Neural Network Language Models. In IEEE International Conference on Acoustics, Speech, and Signal Processing, pages 8430-8434, May.
27. Roy W. Tromble, Shankar Kumar, Franz Och, and Wolfgang Macherey. 2008. Lattice Minimum Bayes-Risk Decoding for Statistical Machine Translation. In Proc. of EMNLP, pages 620-629. Association for Computational Linguistics, October.
28. Ashish Vaswani, Yinggong Zhao, Victoria Fossum, and David Chiang. 2013. Decoding with Large-scale Neural Language Models improves Translation. In Proc. of EMNLP. Association for Computational Linguistics, October.
29. Heng Yu, Liang Huang, Haitao Mi, and Kai Zhao. 2013. Max-Violation Perceptron and Forced Decoding for Scalable MT Training. In Proc. of EMNLP, pages 1112-1123. Association for Computational Linguistics, October.