Automatic annotation of machine translation datasets with binary quality judgements

Proceedings of the 9th International Conference on Language Resources and Evaluation, LREC 2014

Volumn , Issue , 2014, Pages 1788-1792

  Turchi, Marco ^a   Negri, Matteo ^a  

^a FONDAZIONE BRUNO KESSLER   (Italy)

Author keywords

Corpus Annotation; Machine Translation; Quality Estimation

Indexed keywords

BINS; COMPUTATIONAL LINGUISTICS; COMPUTER AIDED LANGUAGE TRANSLATION;

APPLICATION SCENARIO; AUTOMATIC ANNOTATION; AUTOMATIC ESTIMATION; CORPUS ANNOTATIONS; MACHINE TRANSLATIONS; QUALITY ESTIMATION; REGRESSION PROBLEM; SUPERVISED LEARNING METHODS;

TRANSLATION (LANGUAGES);

EID: 84906924092     PISSN: None     EISSN: None     Source Type: Conference Proceeding    
DOI: None     Document Type: Conference Paper

References (25)

1. Nguyen Bach, Fei Huang, and Yaser Al-Onaizan. 2011. Goodness: a Method for Measuring Machine Translation Confidence. In The 49th Annual Meeting of the Association for Computational Linguistics: Human Language Technologies, Proceedings of the Conference, 19-24 June, 2011, Portland, Oregon, USA, pages 211-219. The Association for Computer Linguistics.
2. Frédéric Blain, Holger Schwenk, and Jean Senellart. 2012. Incremental Adaptation Using Translation Information and Post-Editing Analysis. In International Workshop on Spoken Language Translation, pages 234-241, HongKong (China).
3. Ondrej Bojar, Christian Buck, Chris Callison-Burch, Christian Federmann, Barry Haddow, Philipp Koehn, Christof Monz, Matt Post, Radu Soricut, and Lucia Specia. 2013. Findings of the 2013 Workshop on Statistical Machine Translation. In Eighth Workshop on Statistical Machine Translation, WMT-2013, pages 1-44, Sofia, Bulgaria.
4. Chris Callison-Burch, Philipp Koehn, Christof Monz, Matt Post, Radu Soricut, and Lucia Specia. 2012. Findings of the 2012 Workshop on Statistical Machine Translation. In Proceedings of the Seventh Workshop on Statistical Machine Translation (WMT'12), pages 10-51, Montréal, Canada.
5. Chih-Chung Chang and Chih-Jen Lin. 2011. LIBSVM: a Library for Support Vector Machines. ACM Trans. Intell. Syst. Technol., 2(3):27:1-27:27, May.
6. Chien-Ying Chen, Jen-Yuan Yeh, and Hao-Ren Ke. 2010. Plagiarism Detection using ROUGE and WordNet. Journal of Computing, 2(3).
7. Philipp Koehn, Hieu Hoang, Alexandra Birch, Chris Callison-Burch, Marcello Federico, Nicola Bertoldi, Brooke Cowan, Wade Shen, Christine Moran, Richard Zens, et al. 2007. Moses: Open Source Toolkit for Statistical Machine Translation. In Proceedings of the 45th Annual Meeting of the ACL on Interactive Poster and Demonstration Sessions, pages 177-180. Association for Computational Linguistics.
8. Philip Koehn. 2005. Europarl: a Parallel Corpus for Statistical Machine Translation. In Proceedings of Machine Translation Summit X, pages 79-86, Phuket, Thailand.
9. Maarit Koponen. 2012. Comparing Human Perceptions of Post-editing Effort with Post-editing Operations. In Proceedings of the Seventh Workshop on Statistical Machine Translation, pages 181-190. Association for Computational Linguistics.
10. Michael Lesk. 1986. Automated Sense Disambiguation Using Machine-readable Dictionaries: How to Tell a Pine Cone from an Ice Cream Cone. In Proceedings of the 5th annual international conference on Systems documentation (SIGDOC86).
11. Chin-Yew Lin. 2004. ROUGE: A Package for Automatic Evaluation of Summaries. In Proceedings of the ACL workshop on Text Summarization Branches Out., pages 74-81, Barcelona, Spain.
12. Alessia Mammone, Marco Turchi, and Nello Cristianini. 2009. Support vector machines. Wiley Interdisciplinary Reviews: Computational Statistics, 1(3):283-289.
13. Yashar Mehdad, Matteo Negri, and Marcello Federico. 2012. Match without a Referee: Evaluating MT Adequacy without Reference Translations. In Proceedings of the Seventh Workshop on Statistical Machine Translation, WMT'12, pages 171-180, Stroudsburg, PA, USA. Association for Computational Linguistics.
14. Kishore Papineni, Salim Roukos, Todd Ward, and Wei-Jing Zhu. 2002. BLEU: a Method for Automatic Evaluation of Machine Translation. In Proceedings of the 40th Annual Meeting on Association for Computational Linguistics, ACL'02, pages 311-318, Stroudsburg, PA, USA. Association for Computational Linguistics.
15. M. Porter. 2001. Snowball: A language for stemming algorithms.
16. Marion Potet, Laurent Besacier, and Hervé Blanchon. 2010. The LIG Machine Translation System for WMT 2010. In Proceedings of the Joint Fifth Workshop on Statistical Machine Translation and MetricsMATR, pages 161-166, Uppsala, Sweden, July. Association for Computational Linguistics.
17. Marion Potet, Emmanuelle Esperana-Rodier, Laurent Be-sacier, and Herv Blanchon. 2012. Collection of a Large Database of French-English SMT Output Corrections. In Nicoletta Calzolari (Conference Chair), Khalid Choukri, Thierry Declerck, Mehmet Uur Doan, Bente Maegaard, Joseph Mariani, Jan Odijk, and Stelios Piperidis, editors editors, Proceedings of the Eight International Conference on Language Resources and Evaluation (LREC'12), Istanbul, Turkey, may. European Language Resources Association (ELRA).
18. Martin Potthast, Alberto Barrón-Cedeño, Andreas Eiselt, Benno Stein, and Paolo Rosso. 2010. Overview of the 2nd International Competition on Plagiarism Detection. Notebook Papers of CLEF, 10.
19. Matthew Snover, Nitin Madnani, Bonnie J. Dorr, and Richard Schwartz. 2009. Fluency, Adequacy, or HTER?: Exploring Different Human Judgments with a Tunable MT Metric. In Proceedings of the Fourth Workshop on Statistical Machine Translation, StatMT'09, pages 259-268.
20. Radu Soricut and Abdessamad Echihabi. 2010. TrustRank: Inducing Trust in Automatic Translations via Ranking. In Proceedings of the 48th Annual Meeting of the Association for Computational Linguistics, ACL'10, pages 612-621, Stroudsburg, PA, USA. Association for Computational Linguistics.
21. Lucia Specia, Nicola Cancedda, Marc Dymetman, Marco Turchi, and Nello Cristianini. 2009. Estimating the sentence-level quality of machine translation systems. In Proceedings of the 13th Annual Conference of the European Association for Machine Translation (EAMT'09), pages 28-35, Barcelona, Spain.
22. Lucia Specia. 2011. Exploiting Objective Annotations for Measuring Translation Post-editing Effort. pages 73-80.
23. Ralf Steinberger, Bruno Pouliquen, Anna Widiger, Camelia Ignat, Tomaz Erjavec, Dan Tufis, and Dániel Varga. 2006. The jrc-acquis: A multilingual aligned parallel corpus with 20+ languages. arXiv preprint cs/0609058.
24. Marco Turchi, Matteo Negri, and Marcello Federico. 2013. Coping with the Subjectivity of Human Judgements in MT Quality Estimation. In Proceedings of the 8th Workshop on Statistical Machine Translation (WMT'13), Sofia, Bulgaria.
25. Marco Turchi, Antonios Anastasopoulos, José G.C. de Souza, and Matteo Negri. 2014. Adaptive Quality Estimation for Machine Translation. In Proceedings of the 52nd Annual Meeting of the Association for Computational Linguistics (ACL '14). Association for Computational Linguistics.