RED: A reference dependency based MT evaluation metric

COLING 2014 - 25th International Conference on Computational Linguistics, Proceedings of COLING 2014: Technical Papers

Volumn , Issue , 2014, Pages 2042-2051

  Yu, Hui ^a,b   Wu, Xiaofeng ^c   Xie, Jun ^a   Jiang, Wenbin ^a   Liu, Qun ^a,c   Lin, Shouxun ^a  

^a INSTITUTE OF GEOLOGY AND GEOPHYSICS   (China)

^b UNIVERSITY OF CHINESE ACADEMY OF SCIENCES   (China)

^c DUBLIN CITY UNIVERSITY   (Ireland)

Author keywords

[No Author keywords available]

Indexed keywords

LINGUISTICS; SYNTACTICS; TRANSLATION (LANGUAGES);

AUTOMATIC EVALUATION; DEPENDENCY INFORMATIONS; DEPENDENCY STRUCTURES; FLOATING STRUCTURES; LINGUISTIC RESOURCES; LONG-DISTANCE DEPENDENCIES; MACHINE TRANSLATIONS; STATE-OF-THE-ART PERFORMANCE;

COMPUTATIONAL LINGUISTICS;

EID: 84959869171     PISSN: None     EISSN: None     Source Type: Conference Proceeding    
DOI: None     Document Type: Conference Paper

References (19)

1. Boxing Chen and Roland Kuhn. 2011. Amber: A modified bleu, enhanced ranking metric. In Proceedings of the Sixth Workshop on Statistical Machine Translation, pages 71-77, Edinburgh, Scotland, July. Association for Computational Linguistics.
2. Boxing Chen, Roland Kuhn, and George Foster. 2012. Improving amber, an mt evaluation metric. In Proceedings of the Seventh Workshop on Statistical Machine Translation, WMT '12, pages 59-63, Stroudsburg, PA, USA. Association for Computational Linguistics.
3. Michael Denkowski and Alon Lavie. 2011. Meteor 1.3: Automatic metric for reliable optimization and evaluation of machine translation systems. In Proceedings of the Sixth Workshop on Statistical Machine Translation, pages 85-91. Association for Computational Linguistics.
4. Jes̈us Gimenez and LlúisMarquez. 2007. Linguistic features for automatic evaluation of heterogenous mt systems. In Proceedings of the Second Workshop on Statistical Machine Translation, pages 256-264. Association for Computational Linguistics.
5. Maurice G Kendall. 1938. A new measure of rank correlation. Biometrika, 30(1/2):81-93.
6. Alon Lavie and Abhaya Agarwal. 2007. Meteor: An automatic metric for mt evaluation with high levels of correlation with human judgments. In Proceedings of the Second Workshop on Statistical Machine Translation, StatMT '07, pages 228-231, Stroudsburg, PA, USA. Association for Computational Linguistics.
7. Ding Liu and Daniel Gildea. 2005. Syntactic features for evaluation of machine translation. In Proceedings of the ACL Workshop on Intrinsic and Extrinsic Evaluation Measures for Machine Translation and/or Summarization, pages 25-32.
8. Chi-kiu Lo and Dekai Wu. 2013. MEANT at WMT 2013: A tunable, accurate yet inexpensive semantic frame based MT evaluation metric. In Proceedings of the Eighth Workshop on Statistical Machine Translation, pages 422-428, Sofia, Bulgaria, August. Association for Computational Linguistics.
9. Chi-kiu Lo, Anand Karthik Tumuluru, and Dekai Wu. 2012. Fully automatic semantic mt evaluation. In Proceedings of the Seventh Workshop on Statistical Machine Translation, pages 243-252, Montŕeal, Canada, June. Association for Computational Linguistics.
10. Matous Machəcek and Onɱrej Bojar. 2011. Approximating a deep-syntactic metric for mt evaluation and tuning. In Proceedings of the Sixth Workshop on Statistical Machine Translation, pages 92-98. Association for Computational Linguistics.
11. Dennis Mehay and Chris Brew. 2007. BLEUTRE: Flattening Syntactic Dependencies for MT Evaluation. In Proceedings of the 11th Conference on Theoretical and Methodological Issues in Machine Translation (TMI).
12. F.J. Och. 2003. Minimum error rate training in statistical machine translation. In Proceedings of the 41st Annual Meeting on Association for Computational Linguistics-Volume 1, pages 160-167. Association for Computational Linguistics.
13. Karolina Owczarzak, Josef van Genabith, and Andy Way. 2007. Dependency-based automatic evaluation for machine translation. In Proceedings of the NAACL-HLT 2007/AMTA Workshop on Syntax and Structure in Statistical Translation, SSST '07, pages 80-87, Stroudsburg, PA, USA. Association for Computational Linguistics.
14. K. Papineni, S. Roukos, T. Ward, and W.J. Zhu. 2002. BLEU: A method for automatic evaluation of machine translation. In Proceedings of the 40th annual meeting on association for computational linguistics, pages 311-318. Association for Computational Linguistics.
15. W Pirie. 1988. Spearman rank correlation coefficient. Encyclopedia of statistical sciences.
16. Martin F Porter. 2001. Snowball: A language for stemming algorithms.
17. Libin Shen, Jinxi Xu, and Ralph Weischedel. 2010. String-to-dependency statistical machine translation. Computational Linguistics, 36(4):649-671.
18. Matthew Snover, Bonnie Dorr, Richard Schwartz, Linnea Micciulla, and John Makhoul. 2006. A study of translation edit rate with targeted human annotation. In Proceedings of Association for Machine Translation in the Americas, pages 223-231.
19. Matthew Snover, Nitin Madnani, Bonnie J Dorr, and Richard Schwartz. 2009. Fluency, adequacy, or hter?: exploring different human judgments with a tunable mt metric. In Proceedings of the Fourth Workshop on Statistical Machine Translation, pages 259-268. Association for Computational Linguistics.