Meteor 1.3: Automatic Metric for Reliable Optimization and Evaluation of Machine Translation Systems

WMT 2011 - 6thWorkshop on Statistical Machine Translation, Proceedings of the Workshop

Volumn , Issue , 2011, Pages 85-91

  Denkowski, Michael ^a   Lavie, Alon ^a  

^a Carnegie Mellon University   (United States)

Author keywords

[No Author keywords available]

Indexed keywords

COMPUTATIONAL LINGUISTICS; MACHINE TRANSLATION; NATURAL LANGUAGE PROCESSING SYSTEMS;

AUTOMATIC EVALUATION; AUTOMATIC METRICS; EVALUATION METRICS; FUNCTION WORDS; HIGH-PRECISION; MACHINE TRANSLATION SYSTEMS; MATCHINGS; OPTIMISATIONS; STATISTICAL MACHINE TRANSLATION; TEXT NORMALISATION;

COMPUTER AIDED LANGUAGE TRANSLATION;

EID: 85120305515     PISSN: None     EISSN: None     Source Type: Conference Proceeding    
DOI: None     Document Type: Conference Paper

References (20)

1. Satanjeev Banerjee and Alon Lavie. 2005. METEOR: An Automatic Metric for MT Evaluation with Improved Correlation with Human Judgments. In Proc. of ACL WIEEMMTS 2005.
2. Colin Bannard and Chris Callison-Burch. 2005. Paraphrasing with bilingual parallel corpora. In Proc. of ACL2005.
3. Chris Callison-Burch, Philipp Koehn, Christof Monz, Kay Peterson, Mark Przybocki, and Omar Zaidan. 2010. Findings of the 2010 Joint Workshop on Statistical Machine Translation and Metrics for Machine Translation. In Proc. of ACL WMT/MetricsMATR 2010.
4. Michael Denkowski and Alon Lavie. 2010a. Choosing the Right Evaluation for Machine Translation: an Examination of Annotator and Automatic Metric Performance on Human Judgment Tasks. In Proc. of AMTA 2010.
5. Michael Denkowski and Alon Lavie. 2010b. METEOR-NEXT and the METEOR Paraphrase Tables: Improve Evaluation Support for Five Target Languages. In Proc. of ACL WMT/MetricsMATR 2010.
6. Qin Gao and Stephan Vogel. 2008. Parallel Implementations of Word Alignment Tool. In Proc. of ACL WSETQANLP 2008.
7. Hieu Hoang, Alexandra Birch, Chris Callison-burch, Richard Zens, Rwth Aachen, Alexandra Constantin, Marcello Federico, Nicola Bertoldi, Chris Dyer, Brooke Cowan, Wade Shen, Christine Moran, and Ondej Bojar. 2007. Moses: Open Source Toolkit for Statistical Machine Translation. In Proc. of ACL 2007.
8. Philipp Koehn, Franz Josef Och, and Daniel Marcu. 2003. Statistical Phrase-Based Translation. In Proc. of NAACL/HLT 2003.
9. Philipp Koehn. 2005. Europarl: A Parallel Corpus for Statistical Machine Translation. In Proc. of MT Summit 2005.
10. Zhifei Li, Chris Callison-Burch, Chris Dyer, Juri Ganitkevitch, Sanjeev Khudanpur, Lane Schwartz, Wren Thornton, Jonathan Weese, and Omar Zaidan. 2009. Joshua: An Open Source Toolkit for Parsing-based Machine Translation. In Proc. of WMT 2009.
11. George Miller and Christiane Fellbaum. 2007. WordNet. http://wordnet.princeton.edu/.
12. Joseph Olive. 2005. Global Autonomous Language Exploitation (GALE). DARPA/IPTO Proposer Information Pamphlet.
13. Kishore Papineni, Salim Roukos, Todd Ward, and Wei-Jing Zhu. 2002. BLEU: a Method for Automatic Evaluation of Machine Translation. In Proc. of ACL 2002.
14. Robert Parker, David Graff, Junbo Kong, Ke Chen, and Kazuaki Maeda. 2009. English Gigaword Fourth Edition. Linguistic Data Consortium, LDC2009T13.
15. Martin Porter. 2001. Snowball: A language for stemming algorithms. http://snowball.tartarus.org/texts/.
16. Mark Przybocki. 2009. NIST Open Machine Translation 2009 Evaluation. http://www.itl.nist.gov/iad/mig/tests/mt/2009/.
17. Matthew Snover, Bonnie Dorr, Richard Schwartz, Linnea Micciulla, and John Makhoul. 2006. A Study of Translation Edit Rate with Targeted Human Annotation. In Proc. of AMTA 2006.
18. Andreas Stolke. 2002. SRILM - an Extensible Language Modeling Toolkit. In Proc. of ICSLP 2002.
19. C. van Rijsbergen, 1979. Information Retrieval, chapter 7. 2nd edition.
20. Omar F. Zaidan. 2009. Z-MERT: A Fully Configurable Open Source Tool for Minimum Error Rate Training of Machine Translation Systems. The Prague Bulletin of Mathematical Linguistics.