ReVal: A simple and effective machine translation evaluation metric based on recurrent neural networks

Conference Proceedings - EMNLP 2015: Conference on Empirical Methods in Natural Language Processing

Volumn , Issue , 2015, Pages 1066-1072

  Gupta, Rohit ^a   Orəsan, Constantin ^a   Van Genabith, Josef ^b  

^a UNIVERSITY OF WOLVERHAMPTON   (United Kingdom)

^b SAARLAND UNIVERSITY   (Germany)

Author keywords

[No Author keywords available]

Indexed keywords

COMPUTATIONAL LINGUISTICS; COMPUTER AIDED LANGUAGE TRANSLATION; CORRELATION METHODS; NATURAL LANGUAGE PROCESSING SYSTEMS; VECTOR SPACES;

EVALUATION METRICS; EXTERNAL RESOURCES; LANGUAGE PAIRS; MACHINE TRANSLATION EVALUATIONS; MACHINE TRANSLATIONS; PEARSON CORRELATION; RECURRENT NEURAL NETWORK (RNNS); STATE OF THE ART;

RECURRENT NEURAL NETWORKS;

EID: 84959892731     PISSN: None     EISSN: None     Source Type: Conference Proceeding    
DOI: 10.18653/v1/d15-1124     Document Type: Conference Paper

References (28)

1. Yoshua Bengio, Patrice Simard, and Paolo Frasconi. 1994. Learning long-term dependencies with gradient descent is difficult. IEEE Transactions on Neural Networks, 5(2):157-166.
2. Alexandra Birch and Miles Osborne. 2011. Reordering metrics for MT. In Proceedings of the 49th Annual Meeting of the Association for Computational Linguistics: Human Language Technologies-Volume 1, pages 1027-1035. Association for Computational Linguistics.
3. Ondfej Bojar, Christian Buck, Chris Callison-Burch, Christian Federmann, Barry Haddow, Philipp Koehn, Christof Monz, Matt Post, Radu Soricut, and Lucia Specia. 2013. Findings of the 2013 Workshop on Statistical Machine Translation. In Proceedings of the Eighth Workshop on Statistical Machine Translation, pages 1-44, Sofia, Bulgaria, August. Association for Computational Linguistics.
4. Michael Collins and Nigel Duffy. 2001. Convolution kernels for natural language. In Advances in Neural Information Processing Systems, pages 625-632.
5. Marie-Catherine De Marneffe, Bill MacCartney, Christopher D Manning, et al. 2006. Generating typed dependency parses from phrase structure parses. In Proceedings of LREC, volume 6, pages 449-454.
6. Michael Denkowski and Alon Lavie. 2014. Meteor universal: Language specific translation evaluation for any target language. In Proceedings of the EACL 2014 Workshop on Statistical Machine Translation.
7. George Doddington. 2002. Automatic evaluation of machine translation quality using n-gram cooccurrence statistics. In Proceedings of the second international conference on Human Language Technology Research, pages 138-145. Morgan Kaufmann Publishers Inc.
8. Shubham Gautam and Pushpak Bhattacharyya. 2014. Layered: Metric for machine translation evaluation. In Proceedings of the Ninth Workshop on Statistical Machine Translation.
9. Jesus Gimenez and Lluis Marquez. 2010. Linguistic measures for automatic machine translation evaluation. Machine Translation, 24(3-4):209-240.
10. Francisco Guzman, Shafiq Joty, Lluis Marquez, and Preslav Nakov. 2015. Pairwise neural machine translation evaluation. In Proceedings of the 53rd Annual Meeting of the Association for Computational Linguistics and the 7th International Joint Conference on Natural Language Processing (Volume 1: Long Papers), pages 805-814, Beijing, China. Association for Computational Linguistics.
11. Sepp Hochreiter and Jürgen Schmidhuber. 1997. Long short-term memory. Neural computation, 9(8):1735-1780.
12. Mark Hopkins and Jonathan May. 2013. Models of translation competitions. In Proceedings of the 51st Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 1416-1424, Sofia, Bulgaria.
13. Shafiq Joty, Francisco Guzman, Lluis Marquez, and Preslav Nakov. 2014. DiscoTK: Using Discourse Structure for Machine Translation Evaluation. In Proceedings of the Ninth Workshop on Statistical Machine Translation.
14. Philipp Koehn. 2012. Simulating human judgment in machine translation evaluation campaigns. In Proceedings of the Ninth International Workshop on Spoken Language Translation, pages 179-184, Hong Kong.
15. Quoc Le and Tomas Mikolov. 2014. Distributed representations of sentences and documents. In Proceedings of The 31st International Conference on Machine Learning, pages 1188-1196.
16. Matous Machacek and Ondrej Bojar. 2014. Results of the WMT-14 metrics shared task. In Proceedings of the Ninth Workshop on Statistical Machine Translation.
17. Marco Marelli, Stefano Menini, Marco Baroni, Luisa Bentivogli, Raffaella Bernardi, and Roberto Zamparelli. 2014. A sick cure for the evaluation of compositional distributional semantic models. In Proceedings of LREC 2014.
18. Tomas Mikolov, Quoc V Le, and Ilya Sutskever. 2013a. Exploiting Similarities among Languages for Machine Translation. CoRR, pages 1-10.
19. Tomas Mikolov, Ilya Sutskever, Kai Chen, Greg S Corrado, and Jeff Dean. 2013b. Distributed representations of words and phrases and their compositionality. In Advances in Neural Information Processing Systems, pages 3111-3119.
20. Kishore Papineni, Salim Roukos, Todd Ward, and Wei-Jing Zhu. 2002. BLEU: a method for automatic evaluation of machine translation. In Proceedings of the ACL, pages 311-318.
21. Jeffrey Pennington, Richard Socher, and Christopher D Manning. 2014. Glove: Global vectors for word representation. In Proceedings of the Empiricial Methods in Natural Language Processing (EMNLP 2014), pages 1532-1543.
22. Keisuke Sakaguchi, Matt Post, and Benjamin Van Durme. 2014. Efficient elicitation of annotations for human evaluation of machine translation. In In Proceedings of the Ninth Workshop on Statistical Machine Translation, Baltimore, Maryland.
23. Matthew Snover, Nitin Madnani, Bonnie Dorr, and Richard Schwartz. 2008. TERp system description. In MetricsMATR workshop at AMTA. Citeseer.
24. Richard Socher, Eh Huang, and Jeffrey Pennington. 2011. Dynamic Pooling and Unfolding Recursive Autoencoders for Paraphrase Detection. In Advances in Neural Information Processing Systems, pages 801-809.
25. Richard Socher, John Bauer, Christopher D. Manning, and Andrew Y. Ng. 2013a. Parsing With Compositional Vector Grammars. In Proceedings of the ACL, pages 455-465.
26. Richard Socher, Alex Perelygin, and Jy Wu. 2013b. Recursive deep models for semantic compositionality over a sentiment treebank. In Proceedings of EMNLP, pages 1631-1642.
27. Kai Sheng Tai, Richard Socher, and Christopher D. Manning. 2015. Improved semantic representations from tree-structured long short-term memory networks. In Proceedings of the 53rd Annual Meeting of the Association for Computational Linguistics and the 7th International Joint Conference on Natural Language Processing (Volume 1: Long Papers), pages 1556-1566, Beijing, China. Association for Computational Linguistics.
28. Wojciech Zaremba and Ilya Sutskever. 2014. Learning to execute. arXiv preprint arXiv: 1410.4615.