A comparison of update strategies for large-scale maximum expected BLEU training

NAACL HLT 2015 - 2015 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Proceedings of the Conference

Volumn , Issue , 2015, Pages 1516-1526

  Wuebker, Joern ^a   Muehr, Sebastian ^a   Lehnen, Patrick ^a   Peitz, Stephan ^a   Ney, Hermann ^a  

^a RWTH AACHEN UNIVERSITY   (Germany)

Author keywords

[No Author keywords available]

Indexed keywords

COMPUTER AIDED LANGUAGE TRANSLATION; GRADIENT METHODS; STOCHASTIC SYSTEMS;

DISCRIMINATIVE TRAINING; GROWTH TRANSFORMATION; RPROP ALGORITHMS; STATISTICAL MACHINE TRANSLATION; STOCHASTIC GRADIENT DESCENT; TRAINING DATA;

COMPUTATIONAL LINGUISTICS;

EID: 84960157557     PISSN: None     EISSN: None     Source Type: Conference Proceeding    
DOI: 10.3115/v1/n15-1175     Document Type: Conference Paper

References (43)

1. Michael Auli, Michel Galley, and Jianfeng Gao. 2014. Large-scale Expected BLEU Training of Phrase-based Reordering ModelsI. In Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing (EMNLP), pages 1250-1260, Doha, Qatar, October.
2. Phil Blunsom, Trevor Cohn, and Miles Osborne. 2008. A discriminative latent variable model for statistical machine translation. In Proc. of ACL-HLT.
3. Peter F. Brown, Stephan A. Della Pietra, Vincent J. Della Pietra, and Robert L. Mercer. 1993. The Mathematics of Statistical Machine Translation: Parameter Estimation. Computational Linguistics, 19(2):263-311, June.
4. Francisco Casacuberta and Enrique Vidal. 2004. Machine translation with inferred stochastic finite-state transducers. Computational Linguistics, 30(2):205-225.
5. Stanley F. Chen and Joshuo Goodman. 1998. An Empirical Study of Smoothing Techniques for Language Modeling. Technical Report TR-10-98, Computer Science Group, Harvard University, Cambridge, MA, August.
6. D. Chiang, Y. Marton, and P. Resnik. 2008. Online largemargin training of syntactic and structural translation features. In Conference on Empirical Methods in Natural Language Processing, pages 224-233, Honolulu, Hawaii.
7. David Chiang. 2005. A Hierarchical Phrase-Based Model for Statistical Machine Translation. In Proceedings of the 43rd Annual Meeting on Association for Computational Linguistics, pages 263-270, Ann Arbor, Michigan, USA, June.
8. Jonathan H. Clark, Chris Dyer, Alon Lavie, and Noah A. Smith. 2011. Better hypothesis testing for statistical machine translation: Controlling for optimizer instability. In 49th Annual Meeting of the Association for Computational Linguistics:shortpapers, pages 176-181, Portland, Oregon, June.
9. John Duchi and Yoram Singer. 2009. Efficient online and batch learning using forward backward splitting. Journal of Machine Learning Research, 10:2899-2934, December.
10. John Duchi, Elad Hazan, and Yoram Singer. 2011. Adaptive subgradient methods for online learning and stochastic optimization. Journal of Machine Learning Research, 12:2121-2159, July.
11. Michel Galley and Christopher D. Manning. 2008. A simple and effective hierarchical phrase reordering model. In Proceedings of the Conference on Empirical Methods in Natural Language Processing, EMNLP '08, pages 848-856, Stroudsburg, PA, USA. Association for Computational Linguistics.
12. Jianfeng Gao and Xiaodong He. 2013. Training MRF Based Phrase Translation Models using Gradient Ascent. In Proceedings of the North American Chapter of the Association for Computational Linguistics- Human Language Technologies conference (NAACL HLT), pages 450-459, Atlanta, Georgia, USA, Jun.
13. Spence Green, Sida Wang, Daniel Cer, and Christopher D. Manning. 2013. Fast and adaptive online training of feature-rich translation models. In 51st Annual Meeting of the Association for Computational Linguistics, pages 311-321, Sofia, Bulgaria, August.
14. Spence Green, Daniel Cer, and Christopher D. Manning. 2014. An empirical comparison of features and tunign for phrase-based machine translation. In Proceedings of the Ninth Workshop on Statistical Machine Translation, pages 466-476, Baltimore, Maryland, USA, June.
15. Stefan Hahn, Marco Dinarelli, Christian Raymond, Fabrice Lefevre, Patrick Lehnen, Renato De Mori, Alessandro Moschitti, Hermann Ney, and Giuseppe Riccardi. 2011. Comparing Stochastic Approaches to Spoken Language Understanding in Multiple Languages. IEEE Transactions on Audio, Speech, and Language Processing, 19(6):1569-1583, August.
16. Saša Hasan, Juri Ganitkevitch, Hermann Ney, and Jesús Andrés-Ferrer. 2008. Triplet lexicon models for statistical machine translation. In Conference on Empirical Methods in Natural Language Processing, pages 372-381, Honolulu, Hawaii, October. Association for Computational Linguistics.
17. Xiaodong He and Li Deng. 2012. Maximum Expected BLEU Training of Phrase and Lexicon Translation Models. In Proceedings of the 50th Annual Meeting of the Association for Computational Linguistics (ACL), pages 292-301, Jeju, Republic of Korea, Jul.
18. Kenneth Heafield, Ivan Pouzyrevsky, Jonathan H. Clark, and Philipp Koehn. 2013. Scalable modified Kneser-Ney language model estimation. In Proceedings of the 51st Annual Meeting of the Association for Computational Linguistics, pages 690-696, Sofia, Bulgaria, August.
19. Georg Heigold, Stefan Hahn, Patrick Lehnen, and Hermann Ney. 2011. EM-Style Optimization of Hidden Conditional Random Fields for Grapheme-to-Phoneme Conversion. In IEEE International Conference on Acoustics, Speech, and Signal Processing, pages 4920-4923, Prague, Czech Republic, May.
20. Mark Hopkins and Jonathan May. 2011. Tuning as ranking. In In Proceedings of the 2011 Conference on Empirical Methods in Natural Language Processing, pages 1352-1362, Edinburgh, Scotland, July.
21. Matthias Huck, Joern Wuebker, Felix Rietig, and Hermann Ney. 2013. A phrase orientation model for hierarchical machine translation. In Workshop on Statistical Machine Translation, pages 452-463, Sofia, Bulgaria, August.
22. Abraham Ittycheriah and Salim Roukos. 2007. Direct Translation Model 2. In Proceedings of the North American Chapter of the Association for Computational Linguistics - Human Language Technologies conference (NAACL HLT), pages 57-64, Rochester, NY, USA, Apr.
23. Reinerd Kneser and Hermann Ney. 1995. Improved backing-off for M-gram language modeling. In Proceedings of the International Conference on Acoustics, Speech, and Signal Processingw, volume 1, pages 181-184, May.
24. P. Koehn, F. J. Och, and D. Marcu. 2003. Statistical Phrase-Based Translation. In Proceedings of the 2003 Meeting of the North American chapter of the Association for Computational Linguistics (NAACL-03), pages 127-133, Edmonton, Alberta.
25. Thomas Lavergne, Alexandre Allauzen, Josep Maria Crego, and François Yvon. 2011. From n-grambased to crf-based translation models. In Proceedings of the Sixth Workshop on Statistical Machine Translation, pages 542-553, Edinburgh, Scotland, July. Association for Computational Linguistics.
26. Percy Liang, Alexandre Buchard-Côté, Dan Klein, and Ben Taskar. 2006. An End-to-End Discriminative Approach to Machine Translation. In Proceedings of the 21st International Conference on Computational Linguistics and the 44th annual meeting of the Association for Computational Linguistics, pages 761-768, Sydney, Australia.
27. José B Mariño, Rafael E Banchs, Josep M Crego, Adrià de Gispert, Patrik Lambert, José A R Fonollosa, and Marta R Costa-jussà. 2006. N-gram-based Machine Translation. Comput. Linguist., 32(4):527-549, December.
28. R.C. Moore and W. Lewis. 2010. Intelligent Selection of Language Model Training Data. In ACL (Short Papers), pages 220-224, Uppsala, Sweden, July.
29. Franz Josef Och and Hermann Ney. 2004. The alignment template approach to statistical machine translation. Computational Linguistics, 30(4):417-449, December.
30. Franz Josef Och. 2003. Minimum Error Rate Training in Statistical Machine Translation. In Proc. of the 41th Annual Meeting of the Association for Computational Linguistics (ACL), pages 160-167, Sapporo, Japan, July.
31. Kishore Papineni, Salim Roukos, Todd Ward, and Wei-Jing Zhu. 2002. Bleu: a Method for Automatic Evaluation of Machine Translation. In Proceedings of the 41st Annual Meeting of the Association for Computational Linguistics, pages 311-318, Philadelphia, Pennsylvania, USA, July.
32. Martin Riedmiller and Heinrich Braun. 1993. A direct adaptive method for faster backpropagation learning: The rprop algorithm. In IEEE International Conference on Neural Networks, pages 586-591.
33. Hendra Setiawan and Bowen Zhou. 2013. Discriminative training of 150 million translation parameters and its application to pruning. In NAACL-HLT 2013, pages 335-341, Atlanta, Georgia, June.
34. Patrick Simianer, Stefan Riezler, and Chris Dyer. 2012. Joint Feature Selection in Distributed Stochastic Learning for Large-Scale Discriminative Training in SMT. In Proceedings of the 50th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 11-21, Jeju Island, Korea, July. Association for Computational Linguistics.
35. Andreas Stolcke. 2002. SRILM - An Extensible Language Modeling Toolkit. In Proc. of the Int. Conf. on Speech and Language Processing (ICSLP), volume 2, pages 901-904, Denver, CO, September.
36. Martin Sundermeyer, Ralf Schlüter, and Hermann Ney. 2012. Lstm neural networks for language modeling. In Interspeech, Portland, OR, USA, September.
37. David Vilar, Daniel Stein, Matthias Huck, and Hermann Ney. 2010. Jane: Open source hierarchical translation, extended with reordering and lexicon models. In ACL 2010 Joint Fifth Workshop on Statistical Machine Translation and Metrics MATR, pages 262-270, Uppsala, Sweden, July.
38. Taro Watanabe, Jun Suzuki, Hajime Tsukada, and Hideki Isozaki. 2007. Online large-margin training for statistical machine translation. In Proceedings of the 2007 Conference on Empirical Methods in Natural Language Processing, pages 764-773, Prague, Czech Republic, June.
39. Simon Wiesler, Alexander Richard, Ralf Schlüter, and Hermann Ney. 2013. A Critical Evaluation of Stochastic Algorithms for Convex Optimization. In IEEE International Conference on Acoustics, Speech, and Signal Processing, pages 6955-6959, Vancouver, Canada, May.
40. Joern Wuebker, Arne Mauser, and Hermann Ney. 2010. Training phrase translation models with leaving-oneout. In Proceedings of the 48th Annual Meeting of the Assoc. for Computational Linguistics, pages 475-484, Uppsala, Sweden, July.
41. Joern Wuebker, Matthias Huck, Stephan Peitz, Malte Nuhn, Markus Freitag, Jan-Thorsten Peter, Saab Mansour, and Hermann Ney. 2012. Jane 2: Open source phrase-based and hierarchical statistical machine translation. In International Conference on Computational Linguistics, pages 483-491, Mumbai, India, December.
42. Joern Wuebker, Stephan Peitz, Felix Rietig, and Hermann Ney. 2013. Improving statistical machine translation with word class models. In Conference on Empirical Methods in Natural Language Processing, pages 1377-1381, Seattle, USA, October.
43. Heng Yu, Liang Huang, Haitao Mi, and Kai Zhao. 2013. Max-violation perceptron and forced decoding for scalable mt training. In Conference on Empirical Methods in Natural Language Processing, pages 1112-1123, Seattle, USA, October.