Translation modeling with bidirectional recurrent neural networks

EMNLP 2014 - 2014 Conference on Empirical Methods in Natural Language Processing, Proceedings of the Conference

Volumn , Issue , 2014, Pages 14-25

  Sundermeyer, Martin ^a   Alkhouli, Tamer ^a   Wuebker, Joern ^a   Ney, Hermann ^a,b,c  

^a RWTH AACHEN UNIVERSITY   (Germany)

^b UNIVERSITÉ PARIS SACLAY   (France)

^c CNRS   (France)

Author keywords

[No Author keywords available]

Indexed keywords

COMPUTATIONAL LINGUISTICS; NATURAL LANGUAGE PROCESSING SYSTEMS; TRANSLATION (LANGUAGES);

BIDIRECTIONAL RECURRENT NEURAL NETWORKS; LANGUAGE MODEL; MACHINE TRANSLATIONS; NEURAL MODELS; TRANSLATION MODELS; WORD ALIGNMENT;

RECURRENT NEURAL NETWORKS;

EID: 84961291354     PISSN: None     EISSN: None     Source Type: Conference Proceeding    
DOI: 10.3115/v1/d14-1003     Document Type: Conference Paper

References (43)

1. Ebru Arisoy, Tara N. Sainath, Brian Kingsbury, and Bhuvana Ramabhadran. 2012. Deep neural network language models. In Proceedings of the NAACLHLT 2012 Workshop: Will We Ever Really Replace the N-gram Model? On the Future of Language Modeling for HLT, pages 20-28. Association for Computational Linguistics.
2. Michael Auli, Michel Galley, Chris Quirk, and Geoffrey Zweig. 2013. Joint Language and Translation Modeling with Recurrent Neural Networks. In Conference on Empirical Methods in Natural Language Processing, pages 1044-1054, Seattle, USA, October.
3. Yoshua Bengio, Patrice Simard, and Paolo Frasconi. 1994. Learning long-term dependencies with gra- dient descent is difficult. Neural Networks, IEEE Transactions on, 5(2):157-166.
4. Maria Asunción Castaño and Francisco Casacuberta. 1997. A connectionist approach to machine translation. In 5th International Conference on Speech Communication and Technology (EUROSPEECH- 97), Rhodes, Greece.
5. Maria Asunción Castaño and Francisco Casacuberta. 1999. Text-to-text machine translation using the RECONTRA connectionist model. In Lecture Notes in Computer Science (IWANN 99), volume 1607, pages 683-692, Alicante, Spain.
6. Maria Asunción Castaño, Francisco Casacuberta, and Enrique Vidal. 1997. Machine translation using neural networks and finite-state models. In 7th International Conference on Theoretical and Methodological Issues in Machine Translation. TMI'97, pages 160-167, Santa Fe, USA.
7. Stanley F. Chen and Joshua Goodman. 1998. An Empirical Study of Smoothing Techniques for Language Modeling. Technical Report TR-10-98, Computer Science Group, Harvard University, Cambridge, MA, August.
8. Jacob Devlin, Rabih Zbib, Zhongqiang Huang, Thomas Lamar, Richard Schwartz, and John Makhoul. 2014. Fast and Robust Neural Network Joint Models for Statistical Machine Translation. In 52nd Annual Meeting of the Association for Computational Linguistics, page to appear, Baltimore, MD, USA, June.
9. Michel Galley and Christopher D. Manning. 2008. A simple and effective hierarchical phrase reordering model. In Proceedings of the Conference on Empirical Methods in Natural Language Processing, EMNLP '08, pages 848-856, Stroudsburg, PA, USA. Association for Computational Linguistics.
10. Felix A. Gers, Jürgen Schmidhuber, and Fred Cummins. 2000. Learning to forget: Continual prediction with LSTM. Neural computation, 12(10):2451-2471.
11. Felix A. Gers, Nicol N. Schraudolph, and Jürgen Schmidhuber. 2003. Learning precise timing with lstm recurrent networks. The Journal of Machine Learning Research, 3:115-143.
12. Joshua Goodman. 2001. Classes for fast maximum entropy training. In Acoustics, Speech, and Signal Processing, 2001. Proceedings.(ICASSP'01). 2001 IEEE International Conference on, volume 1, pages 561-564. IEEE.
13. Alex Graves and Jürgen Schmidhuber. 2005. Framewise phoneme classification with bidirectional LSTM and other neural network architectures. Neural Networks, 18(5):602-610.
14. James Henderson. 2004. Discriminative training of a neural network statistical parser. In Proceedings of the 42nd Annual Meeting on Association for Computational Linguistics, page 95. Association for Computational Linguistics.
15. Sepp Hochreiter and Jürgen Schmidhuber. 1997. Long short-term memory. Neural computation, 9(8):1735-1780.
16. Yuening Hu, Michael Auli, Qin Gao, and Jianfeng Gao. 2014. Minimum translation modeling with recurrent neural networks. In Proceedings of the 14th Conference of the European Chapter of the Association for Computational Linguistics, pages 20-29, Gothenburg, Sweden, April. Association for Computational Linguistics.
17. Matthias Huck, Joern Wuebker, Felix Rietig, and Hermann Ney. 2013. A phrase orientation model for hierarchical machine translation. In ACL 2013 Eighth Workshop on Statistical Machine Translation, pages 452-463, Sofia, Bulgaria, August.
18. Nal Kalchbrenner and Phil Blunsom. 2013. Recurrent continuous translation models. In Proceedings of the 2013 Conference on Empirical Methods in Natural Language Processing, pages 1700-1709, Seattle, Washington, USA, October. Association for Computational Linguistics.
19. Reinhard Kneser and Hermann Ney. 1995. Improved backing-off for M-gram language modeling. In Proceedings of the International Conference on Acoustics, Speech, and Signal Processingw, volume 1, pages 181-184, May.
20. Philipp Koehn, Franz Josef Och, and Daniel Marcu. 2003. Statistical Phrase-Based Translation. In Proceedings of the 2003 Meeting of the North American chapter of the Association for Computational Linguistics (NAACL-03), pages 127-133, Edmonton, Alberta.
21. Hai Son Le, Alexandre Allauzen, and François Yvon. 2012. Continuous Space Translation Models with Neural Networks. In Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, pages 39-48, Montreal, Canada, June.
22. Xunying Liu, Yongqiang Wang, Xie Chen, Mark J. F. Gales, and Phil C.Woodland. 2014. Efficient lattice rescoring using recurrent neural network language models. In Acoustics, Speech and Signal Processing (ICASSP), 2014 IEEE International Conference on, pages 4941-4945. IEEE.
23. Grégoire Mesnil, Xiaodong He, Li Deng, and Yoshua Bengio. 2013. Investigation of recurrent-neuralnetwork architectures and learning methods for spoken language understanding. In Interspeech, pages 3771-3775.
24. Tomas Mikolov, Stefan Kombrink, Lukas Burget, JH Cernocky, and Sanjeev Khudanpur. 2011. Extensions of recurrent neural network language model. In Acoustics, Speech and Signal Processing (ICASSP), 2011 IEEE International Conference on, pages 5528-5531. IEEE.
25. Robert C. Moore andWilliam Lewis. 2010. Intelligent Selection of Language Model Training Data. In ACL (Short Papers), pages 220-224, Uppsala, Sweden, July.
26. Frederic Morin and Yoshua Bengio. 2005. Hierarchical probabilistic neural network language model. In Proceedings of the international workshop on artificial intelligence and statistics, pages 246-252.
27. Franz Josef Och. 2003. Minimum Error Rate Training in Statistical Machine Translation. In Proc. of the 41th Annual Meeting of the Association for Computational Linguistics (ACL), pages 160-167, Sapporo, Japan, July.
28. Kishore Papineni, Salim Roukos, ToddWard, andWei- Jing Zhu. 2002. Bleu: A Method for Automatic Evaluation of Machine Translation. In Proceedings of the 41st Annual Meeting of the Association for Computational Linguistics, pages 311-318, Philadelphia, Pennsylvania, USA, July.
29. David E. Rumelhart, Geoffrey E. Hinton, and Ronald J. Williams. 1986. Learning Internal Representations by Error Propagation. In: J. L. McClelland, D. E. Rumelhart, and The PDP Research Group: "Parallel Distributed Processing, Volume 1: Foundations". The MIT Press.
30. Mike Schuster and Kuldip K. Paliwal. 1997. Bidirectional recurrent neural networks. IEEE Transactions on Signal Processing, 45(11):2673-2681.
31. Holger Schwenk, Daniel Déchelotte, and Jean-Luc Gauvain. 2006. Continuous Space Language Models for Statistical Machine Translation. In Proceedings of the COLING/ACL 2006 Main Conference Poster Sessions, pages 723-730, Sydney, Australia, July.
32. Holger Schwenk. 2012. Continuous Space Translation Models for Phrase-Based Statistical Machine Translation. In 25th International Conference on Computational Linguistics (COLING), pages 1071-1080, Mumbai, India, December.
33. Matthew Snover, Bonnie Dorr, Richard Schwartz, Linnea Micciulla, and John Makhoul. 2006. A Study of Translation Edit Rate with Targeted Human Annotation. In Proceedings of the 7th Conference of the Association for Machine Translation in the Americas, pages 223-231, Cambridge, Massachusetts, USA, August.
34. Andreas Stolcke. 2002. SRILM - An Extensible Language Modeling Toolkit. In Proc. of the Int. Conf. on Speech and Language Processing (ICSLP), volume 2, pages 901-904, Denver, CO, September.
35. Martin Sundermeyer, Ralf Schlüter, and Hermann Ney. 2012. LSTM neural networks for language modeling. In Interspeech, Portland, OR, USA, September.
36. Martin Sundermeyer, Ilya Oparin, Jean-Luc Gauvain, Ben Freiberg, Ralf Schlüter, and Hermann Ney. 2013. Comparison of feedforward and recurrent neural network language models. In IEEE International Conference on Acoustics, Speech, and Signal Processing, pages 8430-8434, Vancouver, Canada, May.
37. Ashish Vaswani, Yinggong Zhao, Victoria Fossum, and David Chiang. 2013. Decoding with largescale neural language models improves translation. In Proceedings of the 2013 Conference on Empirical Methods in Natural Language Processing, pages 1387-1392, Seattle, Washington, USA, October. Association for Computational Linguistics.
38. David Vilar, Daniel Stein, Matthias Huck, and Hermann Ney. 2010. Jane: Open source hierarchical translation, extended with reordering and lexicon models. In ACL 2010 Joint Fifth Workshop on Statistical Machine Translation and Metrics MATR, pages 262-270, Uppsala, Sweden, July.
39. Paul J. Werbos. 1990. Backpropagation through time: what it does and how to do it. Proceedings of the IEEE, 78(10):1550-1560.
40. Ronald J. Williams and David Zipser. 1995. Gradient- Based Learning Algorithms for Recurrent Networks and Their Computational Complexity. In: Yves Chauvain and David E. Rumelhart: "Back- Propagation: Theory, Architectures and Applications". Lawrence Erlbaum Publishers.
41. Joern Wuebker, Arne Mauser, and Hermann Ney. 2010. Training phrase translation models with leaving-one-out. In Proceedings of the 48th Annual Meeting of the Assoc. for Computational Linguistics, pages 475-484, Uppsala, Sweden, July.
42. Joern Wuebker, Matthias Huck, Stephan Peitz, Malte Nuhn, Markus Freitag, Jan-Thorsten Peter, Saab Mansour, and Hermann Ney. 2012. Jane 2: Open source phrase-based and hierarchical statistical machine translation. In International Conference on Computational Linguistics, pages 483-491, Mumbai, India, December.
43. Joern Wuebker, Stephan Peitz, Felix Rietig, and Hermann Ney. 2013. Improving statistical machine translation with word class models. In Conference on Empirical Methods in Natural Language Processing, pages 1377-1381, Seattle, USA, October.