The METEOR metric for automatic evaluation of Machine Translation

Machine Translation

Volumn 23, Issue 2-3, 2009, Pages 105-115

  Lavie, Alon ^a   Denkowski, Michael J ^a  

^a Carnegie Mellon University   (United States)

Author keywords

Automatic metrics; Machine Translation; MT evaluation

Indexed keywords

AUTOMATIC EVALUATION; FREE PARAMETERS; INITIAL DESIGN; MACHINE TRANSLATIONS; MORPHOLOGICAL VARIANTS; OPTIMAL CORRELATION; OPTIMAL PARAMETER; PARAMETERIZED; TRANSLATION QUALITY; WORD MATCHING;

INFORMATION THEORY; OPTIMIZATION; SPEECH RECOGNITION; SPEECH TRANSMISSION;

TRANSLATION (LANGUAGES);

EID: 77954763029     PISSN: 09226567     EISSN: None     Source Type: Journal    
DOI: 10.1007/s10590-009-9059-4     Document Type: Article

References (19)

1. Agarwal A, Lavie A (2008) Meteor, m-bleu and m-ter: evaluation metrics for high-correlation with human rankings of machine translation output. In: Proceedings of the third ACL workshop on statistical machine translation. Columbus, OH, pp 115-118
2. Banerjee S, Lavie A (2005) METEOR: an automatic metric for MT Evaluation with improved correlation with human judgments. In: Proceedings of the ACL workshop on intrinsic and extrinsic evaluation measures for machine translation and/or summarization. Ann Arbor, MI, pp 65-72
3. Blatz J, Fitzgerald E, Foster G, Gandrabur S, Goutte C, Kulesza A, Sanchis A, Ueffing N (2003) Confidence estimation for machine translation. Technical report natural language engineering workshop final report, Johns Hopkins University
4. Callison-Burch C, Fordyce C, Koehn P, Monz C, Schroeder J (2007) (Meta-) evaluation of machine translation. In: Proceedings of the second workshop on statistical machine translation. Prague, Czech Republic, pp 136-158
5. Callison-Burch C, Fordyce C, Koehn P, Monz C, Schroeder J (2008) Further meta-evaluation of machine translation. In: Proceedings of the third workshop on statistical machine translation. Columbus, OH, pp 70-106
6. Callison-Burch C, Koehn P, Monz C, Schroeder J (2009) Findings of the 2009 workshop on statistical machine translation. In: Proceedings of the fourth workshop on statistical machine translation. Athens, Greece, pp 1-28
7. Heafield K, Hanneman G, Lavie A (2009) Machine translation system combination with flexible word ordering. In: Proceedings of the fourth workshop on statistical machine translation. Athens, Greece, pp 56-60
8. Lavie A, Agarwal A (2007) METEOR: an automatic metric for MT evaluation with high levels of correlation with human judgments. In: Proceedings of the second ACL workshop on statistical machine translation. Prague, Czech Republic, pp 228-231
9. Lavie A, Sagae K, Jayaraman S (2004) The significance of recall in automatic metrics for MT evaluation. In: Proceedings of the 6th conference of the association for machine translation in the Americas (AMTA-2004). Washington, DC, pp 134-143
10. Leusch G, Ueffing N, Ney H (2006) CDER: efficient MT evaluation using block movements. In: Proceedings of the thirteenth conference of the European chapter of the association for computational linguistics. pp 241-248
11. Melamed ID, Green R, Turian J (2003) Precision and recall of machine translation. In: Proceedings of the HLT-NAACL 2003 conference: short papers. Edmonton, Alberta, pp 61-63
12. Miller G, Fellbaum C (2007) WordNet. http://wordnet.princeton.edu/
13. Och FJ (2003) Minimum error rate training for statistical machine translation. In: Proceedings of the 41st annual meeting of the association for computational linguistics. pp 160-167
14. Papineni K, Roukos S, Ward T, Zhu W-J (2002) BLEU: a method for automatic evaluation of machine translation. In: Proceedings of 40th annual meeting of the association for computational linguistics (ACL). Philadelphia, PA, pp 311-318
15. Porter M (2001) Snowball: a language for stemming algorithms. http://snowball.tartarus.org/texts/introduction.html
16. Przybocki M, Peterson K, Bronsart S (2008) Official results of the NIST 2008 "Metrics for MAchine TRanslation" Challenge (MetricsMATR08). http://nist.gov/speech/tests/metricsmatr/2008/results/
17. Snover M, Dorr B, Schwartz R, Micciulla L, Makhoul J (2006) A study of translation edit rate with targeted human annotation. In: Proceedings of the 7th conference of the association for machine translation in the Americas (AMTA-2006). Cambridge, MA, pp 223-231
18. van Rijsbergen C (1979) Information retrieval, Chap. 7, 2nd edn. Butterworths, London, UK
19. Ye Y, Zhou M, Lin C-Y (2007) Sentence level machine translation evaluation as a ranking. In: Proceedings of the second workshop on statistical machine translation. Prague, Czech Republic, pp 240-247