Fluency, Adequacy, or HTER? Exploring Different Human Judgments with a Tunable MT Metric

EACL 2009 - 4th Workshop on Statistical Machine Translation, Proceedings of theWorkshop

Volumn , Issue , 2009, Pages 259-268

  Snover, Matthew ^a   Madnani, Nitin ^b   Dorr, Bonnie J ^c,d   Schwartz, Richard ^d,e  

^a Laboratory For Computational Linguistics And Information Processing

^b Institute For Advanced Computer Studies

^c UNIVERSITY OF MARYLAND   (United States)

^d Human Language Technology Center of Excellence

^e BBN TECHNOLOGIES   (United States)

Author keywords

[No Author keywords available]

Indexed keywords

COMPUTATIONAL LINGUISTICS;

AUTOMATIC MACHINES; EVALUATION METRICS; HUMAN JUDGMENTS; MACHINE TRANSLATION EVALUATIONS; MACHINE TRANSLATIONS; PEARSON CORRELATION; PERFORMANCE; SPEARMAN CORRELATION; TUNABLE PARAMETER; TUNABLES;

MACHINE TRANSLATION;

EID: 84858395031     PISSN: None     EISSN: None     Source Type: Conference Proceeding    
DOI: None     Document Type: Conference Paper

References (18)

1. Satanjeev Banerjee and Alon Lavie. 2005. METEOR: An Automatic Metric for MT Evaluation with Improved Correlation with Human Judgments. In Proceedings of the ACL 2005Workshop on Intrinsic and Extrinsic Evaulation Measures for MT and/or Summarization.
2. Colin Bannard and Chris Callison-Burch. 2005. Paraphrasing with Bilingual Parallel Corpora. In Proceedings of the 43rd Annual Meeting of the Association for Computational Linguistics (ACL 2005), pages 597-604, Ann Arbor, Michigan, June.
3. Chris Callison-Burch. 2008. Syntactic constraints on paraphrases extracted from parallel corpora. In Proceedings of the 2008 Conference on Empirical Methods in Natural Language Processing, pages 196- 205, Honolulu, Hawaii, October. Association for Computational Linguistics.
4. Christiane Fellbaum. 1998. WordNet: An Electronic Lexical Database. MIT Press. http://www.cogsci.princeton.edu/wn [2000, September 7].
5. David Kauchak and Regina Barzilay. 2006. Paraphrasing for Automatic Evaluation. In Proceedings of the Human Language Technology Conference of the North American Chapter of the ACL, pages 455- 462.
6. Gregor Leusch, Nicola Ueffing, and Hermann Ney. 2006. CDER: Efficient MT Evaluation Using Block Movements. In Proceedings of the 11th Conferenceof the European Chapter of the Association for Computational Linguistics (EACL 2006).
7. V. I. Levenshtein. 1966. Binary Codes Capable of Correcting Deletions, Insertions, and Reversals. Soviet Physics Doklady, 10:707-710.
8. Daniel Lopresti and Andrew Tomkins. 1997. Block edit models for approximate string matching. Theoretical Computer Science, 181(1):159-179, July.
9. Nitin Madnani, Necip Fazil Ayan, Philip Resnik, and Bonnie J. Dorr. 2007. Using paraphrases for parameter tuning in statistical machine translation. In Proceedings of the Workshop on Statistical Machine Translation, Prague, Czech Republic, June. Association for Computational Linguistics.
10. Nitin Madnani, Philip Resnik, Bonnie J. Dorr, and Richard Schwartz. 2008. Are Multiple Reference Translations Necessary? Investigating the Value of Paraphrased Reference Translations in Parameter Optimization. In Proceedings of the Eighth Conference of the Association for Machine Translation in the Americas, October.
11. S. Niessen, F.J. Och, G. Leusch, and H. Ney. 2000. An evaluation tool for machine translation: Fast evaluation for MT research. In Proceedings of the 2nd International Conference on Language Resources and Evaluation (LREC-2000), pages 39-45.
12. Joseph Olive. 2005. Global Autonomous Language Exploitation (GALE). DARPA/IPTO Proposer Information Pamphlet.
13. Kishore Papineni, Salim Roukos, ToddWard, andWei- Jing Zhu. 2002. Bleu: a Method for Automatic Evaluation of Machine Traslation. In Proceedings of the 40th Annual Meeting of the Association for Computational Linguistics.
14. Martin F. Porter. 1980. An algorithm for suffic stripping. Program, 14(3):130-137.
15. Mark Przybocki, Kay Peterson, and Sebastian Bronsart. 2008. Official results of the NIST 2008 "Metrics for MAchine TRanslation" Challenge (MetricsMATR08). http://nist.gov/speech/tests/metricsmatr/2008/results/, October.
16. Antti-Veikko Rosti, Spyros Matsoukas, and Richard Schwartz. 2007. Improved word-level system combination for machine translation. In Proceedings of the 45th Annual Meeting of the Association of Computational Linguistics, pages 312-319, Prague, Czech Republic, June. Association for Computational Linguistics.
17. Matthew Snover, Bonnie Dorr, Richard Schwartz, Linnea Micciulla, and John Makhoul. 2006. A Study of Translation Edit Rate with Targeted Human Annotation. In Proceedings of Association for Machine Translation in the Americas.
18. Liang Zhou, Chon-Yew Lin, and Eduard Hovy. 2006. Re-evaluating Machine Translation Results with Paraphrase Support. In Proceedings of the 2006 Conference on Empirical Methods in Natural Language Processing (EMNLP 2006), pages 77-84.