Appraise: An open-source toolkit for manual phrase-based evaluation of translations

Proceedings of the 7th International Conference on Language Resources and Evaluation, LREC 2010

Volumn , Issue , 2010, Pages 1731-1734

  Federmann, Christian ^a  

^a GERMAN RESEARCH CENTER FOR ARTIFICIAL INTELLIGENCE DFKI   (Germany)

Author keywords

[No Author keywords available]

Indexed keywords

COMPUTATIONAL LINGUISTICS; COMPUTER AIDED LANGUAGE TRANSLATION;

FINE GRAINED; HUMAN EVALUATION; KAPPA VALUES; MACHINE TRANSLATIONS; OPEN SOURCES; SCORING MODELS; SCORING SCHEMES;

TRANSLATION (LANGUAGES);

EID: 84961317988     PISSN: None     EISSN: None     Source Type: Conference Proceeding    
DOI: None     Document Type: Conference Paper

References (12)

1. Chris Callison-Burch, Cameron Fordyce, Philipp Koehn, Christof Monz, and Josh Schroeder. 2007. (Meta-) evaluation of machine translation. In Proceedings of the Second Workshop on Statistical Machine Translation, pages 136-158, Prague, Czech Republic, June. Association for Computational Linguistics.
2. Chris Callison-Burch, Cameron Fordyce, Philipp Koehn, Christof Monz, and Josh Schroeder. 2008. Further metaevaluation of machine translation. In Proceedings of the Third Workshop on Statistical Machine Translation, pages 70-106, Columbus, Ohio, June. Association for Computational Linguistics.
3. Chris Callison-Burch, Philipp Koehn, Christof Monz, and Josh Schroeder. 2009. Findings of the 2009 Workshop on Statistical Machine Translation. In Proceedings of the Fourth Workshop on Statistical Machine Translation, pages 1-28, Athens, Greece, March. Association for Computational Linguistics.
4. Jean Carletta. 1996. Assessing agreement on classification tasks: the kappa statistic. Comput. Linguist., 22(2):249-254.
5. Deborah Coughlin. 2003. Correlating Automated and Human Assessments of Machine Translation Quality. In Proceedings of MT Summit IX, pages 63-70, New Orleans, LA.
6. J. R. Landis and G. G. Koch. 1977. The measurement of observer agreement for categorical data. Biometrics, 33(1):159-174, March.
7. Alon Lavie and Abhaya Agarwal. 2007. METEOR: An automatic metric for MT evaluation with high levels of correlation with human judgments. In Proceedings of the Second Workshop on Statistical Machine Translation, pages 228-231, Prague, Czech Republic, June. Association for Computational Linguistics.
8. Chin-Yew Lin and Eduard Hovy. 2002. Manual and automatic evaluation of summaries. In Proceedings of the ACL-02 Workshop on Automatic Summarization, pages 45-51, Phildadelphia, Pennsylvania, USA, July. Association for Computational Linguistics.
9. Franz Josef Och and Hermann Ney. 2003. A systematic comparison of various statistical alignment models. Computational Linguistics, 29(1):19-51.
10. Franz Josef Och. 2003. Minimum error rate training in statistical machine translation. In ACL '03: Proceedings of the 41st Annual Meeting on Association for Computational Linguistics, pages 160-167, Morristown, NJ, USA. Association for Computational Linguistics.
11. Kishore Papineni, Salim Roukos, Todd Ward, and Wei-Jing Zhu. 2001. Bleu: a method for automatic evaluation of machine translation. IBM Research Report RC22176(W0109-022), IBM.
12. Omar F. Zaidan and Chris Callison-Burch. 2009. Feasibility of human-in-the-loop minimum error rate training. In Proceedings of the 2009 Conference on Empirical Methods in Natural Language Processing, pages 52-61, Singapore, August. Association for Computational Linguistics.