Efficient elicitation of annotations for human evaluation of machine translation

Proceedings of the Annual Meeting of the Association for Computational Linguistics

Volumn , Issue , 2014, Pages 1-11

  Sakaguchi, Keisuke ^a   Post, Matt ^b   Van Durme, Benjamin ^b  

^a Center for Language and Speech Processing   (United States)

^b Johns Hopkins University   (United States)

Author keywords

[No Author keywords available]

Indexed keywords

COMPUTATIONAL LINGUISTICS; COMPUTER AIDED LANGUAGE TRANSLATION;

HUMAN EVALUATION; MACHINE TRANSLATIONS; MICROSOFT; MODEL SELECTION; STATISTICAL MACHINE TRANSLATION; SYSTEMS COMPETITION; TRUESKILL;

MACHINE TRANSLATION;

EID: 85122625439     PISSN: 0736587X     EISSN: None     Source Type: Conference Proceeding    
DOI: None     Document Type: Conference Paper

References (10)

1. Ondrej Bojar, Milos Ercegovcevíc, Martin Popel, and Omar Zaidan. 2011. A Grain of Salt for the WMT Manual Evaluation. In Proceedings of the Sixth Workshop on Statistical Machine Translation, pages 1?11, Edinburgh, Scotland, July. Association for Computational Linguistics.
2. Ondrej Bojar, Christian Buck, Chris Callison-Burch, Christian Federmann, Barry Haddow, Philipp Koehn, Christof Monz, Matt Post, Radu Soricut, and Lucia Specia. 2013. Findings of the 2013 Workshop on Statistical Machine Translation. In Proceedings of the Eighth Workshop on Statistical Machine Translation, pages 1?44, Sofia, Bulgaria, August. Association for Computational Linguistics.
3. Chris Callison-Burch, Philipp Koehn, Christof Monz, and Omar Zaidan. 2011. Findings of the 2011 Workshop on Statistical Machine Translation. In Proceedings of the Sixth Workshop on Statistical Machine Translation, pages 22?64, Edinburgh, Scotland, July. Association for Computational Linguistics.
4. Chris Callison-Burch, Philipp Koehn, Christof Monz, Matt Post, Radu Soricut, and Lucia Specia. 2012. Findings of the 2012 Workshop on Statistical Machine Translation. In Proceedings of the Seventh Workshop on Statistical Machine Translation, pages 10?51, Montŕeal, Canada, June. Association for Computational Linguistics.
5. Ralf Herbrich, Tom Minka, and Thore Graepel. 2006. TrueSkillTM: A Bayesian Skill Rating System. In Proceedings of the Twentieth Annual Conference on Neural Information Processing Systems, pages 569? 576, Vancouver, British Columbia, Canada, December. MIT Press.
6. Mark Hopkins and Jonathan May. 2013. Models of translation competitions. In Proceedings of the 51st Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 1416?1424, Sofia, Bulgaria, August. Association for Computational Linguistics.
7. Philipp Koehn. 2012. Simulating Human Judgment in Machine Translation Evaluation Campaigns. In Proceedings of the 9th International Workshop on Spoken Language Translation (IWSLT), pages 179? 184, Hong Kong, China, December. International Speech Communication Association.
8. Adam Lopez. 2012. Putting Human Assessments of Machine Translation Systems in Order. In Proceedings of the Seventh Workshop on Statistical Machine Translation, pages 1?9, Montŕeal, Canada, June. Association for Computational Linguistics.
9. Kishore Papineni, Salim Roukos, ToddWard, andWei-Jing Zhu. 2002. BLEU: A Method for Automatic Evaluation of Machine Translation. In Proceedings of 40th Annual Meeting of the Association for Computational Linguistics, pages 311?318, Philadelphia, Pennsylvania, July. Association for Computational Linguistics.
10. Xingyi Song and Trevor Cohn. 2011. Regression and Ranking based Optimisation for Sentence Level MT Evaluation. In Proceedings of the Sixth Workshop on Statistical Machine Translation, pages 123?129, Edinburgh, Scotland, July. Association for Computational Linguistics.