Meteor, m-bleu and m-ter: Evaluation metrics for high-correlation with human rankings of machine translation output

3rd Workshop on Statistical Machine Translation, WMT 2008 at the Annual Meeting of the Association for Computational Linguistics, ACL 2008

Volumn , Issue , 2008, Pages 115-118

  Agarwal, Abhaya ^a   Lavie, Alon ^a  

^a Carnegie Mellon University   (United States)

Author keywords

[No Author keywords available]

Indexed keywords

COMPUTER AIDED LANGUAGE TRANSLATION; MACHINE TRANSLATION; NATURAL LANGUAGE PROCESSING SYSTEMS;

EVALUATION METRICS; FLEXIBLE MATCHING; MACHINE TRANSLATION EVALUATIONS; MACHINE TRANSLATIONS; WORD MATCHING; WORDNET;

COMPUTATIONAL LINGUISTICS;

EID: 84858395027     PISSN: None     EISSN: None     Source Type: Conference Proceeding    
DOI: None     Document Type: Conference Paper

References (13)

1. Satanjeev Banerjee and Alon Lavie. 2005. METEOR: An Automatic Metric for MT Evaluation with Improved Correlation with Human Judgments. In Proceedings of the ACL Workshop on Intrinsic and Extrinsic Evaluation Measures for Machine Translation and/or Summarization, pages 65-72, Ann Arbor, Michigan, June.
2. Chris Callison-Burch, Cameron Fordyce, Philipp Koehn, Christof Monz, and Josh Schroeder. 2007. (meta-) evaluation of machine translation. In Proceedings of the Second Workshop on Statistical Machine Translation, pages 136-158, Prague, Czech Republic, June. Association for Computational Linguistics.
3. Alon Lavie and Abhaya Agarwal. 2007. METEOR: An Automatic Metric for MT Evaluation with High Levels of Correlation with Human Judgments. In Proceedings of the Second ACL Workshop on Statistical Machine Translation, pages 228-231, Prague, Czech Republic, June.
4. Alon Lavie, Kenji Sagae, and Shyamsundar Jayaraman. 2004. The Significance of Recall in Automatic Metrics for MT Evaluation. In Proceedings of the 6th Conference of the Association for Machine Translation in the Americas (AMTA-2004), pages 134-143, Washington, DC, September.
5. Gregor Leusch, Nicola Ueffing, and Hermann Ney. 2006. CDER: Efficient MT Evaluation Using Block Movements. In Proceedings of the Thirteenth Conference of the European Chapter of the Association for Computational Linguistics.
6. Chin-Yew Lin and Franz Josef Och. 2004. Orange: a method for evaluating automatic evaluation metrics for machine translation. In COLING'04: Proceedings of the 20th international conference on Computational Linguistics, page 501, Morristown, NJ, USA. Association for Computational Linguistics.
7. Ding Liu and Daniel Gildea. 2006. Stochastic iterative alignment for machine translation evaluation. In Proceedings of the COLING/ACL on Main conference poster sessions, pages 539-546, Morristown, NJ, USA. Association for Computational Linguistics.
8. I. Dan Melamed, Ryan Green, and Joseph Turian. 2003. Precision and Recall of Machine Translation. In Proceedings of the HLT-NAACL 2003 Conference: Short Papers, pages 61-63, Edmonton, Alberta.
9. Franz Josef Och. 2003. Minimum Error Rate Training for Statistical Machine Translation. In Proceedings of the 41st Annual Meeting of the Association for Computational Linguistics.
10. Kishore Papineni, Salim Roukos, Todd Ward, and Wei-Jing Zhu. 2002. BLEU: a Method for Automatic Evaluation of Machine Translation. In Proceedings of 40th Annual Meeting of the Association for Computational Linguistics (ACL), pages 311-318, Philadelphia, PA, July.
11. Matthew Snover, Bonnie Dorr, Richard Schwartz, Linnea Micciulla, and John Makhoul. 2006. A Study of Translation Edit Rate with Targeted Human Annotation. In Proceedings of the 7th Conference of the Association for Machine Translation in the Americas (AMTA-2006), pages 223-231, Cambridge, MA, August.
12. C. van Rijsbergen, 1979. Information Retrieval. Butterworths, London, UK, 2nd edition.
13. Yang Ye, Ming Zhou, and Chin-Yew Lin. 2007. Sentence level machine translation evaluation as a ranking. In Proceedings of the Second Workshop on Statistical Machine Translation, pages 240-247, Prague, Czech Republic, June. Association for Computational Linguistics.