Results of the wmt15 metrics shared task

10th Workshop on Statistical Machine Translation, WMT 2015 at the 2015 Conference on Empirical Methods in Natural Language Processing, EMNLP 2015 - Proceedings

Volumn , Issue , 2015, Pages 256-273

  Stanojević, Miloš ^a   Kamran, Amir ^a   Koehn, Philipp ^b   Bojar, Ondřej ^c  

^a UNIVERSITY OF AMSTERDAM   (Netherlands)

^b JOHNS HOPKINS UNIVERSITY   (United States)

^c CHARLES UNIVERSITY   (Czech Republic)

Author keywords

[No Author keywords available]

Indexed keywords

TRANSLATION (LANGUAGES);

LEVEL CORRELATION; METRIC SCORES; RESEARCH GROUPS; STANDARD METRICS; SYSTEM LEVELS;

NATURAL LANGUAGE PROCESSING SYSTEMS;

EID: 85119988145     PISSN: None     EISSN: None     Source Type: Conference Proceeding    
DOI: None     Document Type: Conference Paper

References (24)

1. Ondřej Bojar, Miloš Ercegovčević, Martin Popel, and Omar Zaidan. 2011. A Grain of Salt for the WMT Manual Evaluation. In Proceedings of the Sixth Workshop on Statistical Machine Translation, pages 1-11, Edinburgh, Scotland, July. Association for Computational Linguistics.
2. Ondřej Bojar, Matouš Macháček, Aleš Tamchyna, and Daniel Zeman. 2013. Scratching the Surface of Possible Translations. In Proc. of TSD 2013, Lecture Notes in Artificial Intelligence, Berlin/Heidelberg. Západočeská univerzita v Plzni, Springer Verlag.
3. Ondřej Bojar, Rajen Chatterjee, Christian Federmann, Barry Haddow, Matthias Huck, Chris Hokamp, Philipp Koehn, Varvara Logacheva, Christof Monz, Matteo Negri, Matt Post, Carolina Scarton, Lucia Specia, and Marco Turchi. 2015. Findings of the 2015 Workshop on Statistical Machine Translation. In Proceedings of the Tenth Workshop on Statistical Machine Translation, Lisboa, Portugal, September. Association for Computational Linguistics.
4. Boxing Chen, Hongyu Guo, and Roland Kuhn. 2015. Multi-level Evaluation for Machine Translation. In Proceedings of the Tenth Workshop on Statistical Machine Translation, Lisboa, Portugal, September. Association for Computational Linguistics.
5. Elisabet Comelles and Jordi Atserias. 2015. VERTa: A Linguistically-motivated Metric at the WMT15 Metrics Task. In Proceedings of the Tenth Workshop on Statistical Machine Translation, Lisboa, Portugal, September. Association for Computational Linguistics.
6. George Doddington. 2002. Automatic evaluation of machine translation quality using n-gram cooccurrence statistics. In Proceedings of the Second International Conference on Human Language Technology Research, HLT '02, pages 138-145, San Francisco, CA, USA. Morgan Kaufmann Publishers Inc.
7. Marina Fomicheva, Núria Bel, Iria da Cunha, and Anton Malinovskiy. 2015. UPF-Cobalt Submission to WMT15 Metrics Task. In Proceedings of the Tenth Workshop on Statistical Machine Translation, Lisboa, Portugal, September. Association for Computational Linguistics.
8. Rohit Gupta, Constantin Orasan, and Josef van Genabith. 2015a. Machine Translation Evaluation using Recurrent Neural Networks. In Proceedings of the Tenth Workshop on Statistical Machine Translation, Lisboa, Portugal, September. Association for Computational Linguistics.
9. Rohit Gupta, Constantin Orasan, and Josef van Genabith. 2015b. ReVal: A Simple and Effective Machine Translation Evaluation Metric Based on Recurrent Neural Networks. In Proceedings of the Conference on Empirical Methods in Natural Language Processing, EMNLP '15, Lisbon, Portugal.
10. Philipp Koehn and Christof Monz. 2006. Manual and automatic evaluation of machine translation between european languages. In Proceedings of the Workshop on Statistical Machine Translation, StatMT '06, pages 102-121, Stroudsburg, PA, USA. Association for Computational Linguistics.
11. Gregor Leusch, Nicola Ueffing, and Hermann Ney. 2006. Cder: Efficient mt evaluation using block movements. In In Proceedings of EACL, pages 241-248.
12. Matouš Macháček and Ondřej Bojar. 2013. Results of the WMT13 Metrics Shared Task. In Proceedings of the Eighth Workshop on Statistical Machine Translation, pages 45-51, Sofia, Bulgaria, August. Association for Computational Linguistics.
13. Matouš Macháček and Ondřej Bojar. 2014. Results of the WMT14 Metrics Shared Task. In Proceedings of the Ninth Workshop on Statistical Machine Translation, pages 293-301, Baltimore, Maryland, USA, June. Association for Computational Linguistics.
14. Benjamin Marie and Marianna Apidianaki. 2015. Alignment-based sense selection in METEOR and the RATATOUILLE recipe. In Proceedings of the Tenth Workshop on Statistical Machine Translation, Lisboa, Portugal, September. Association for Computational Linguistics.
15. Kishore Papineni, Salim Roukos, ToddWard, andWei-Jing Zhu. 2002. Bleu: A method for automatic evaluation of machine translation. In Proceedings of the 40th Annual Meeting on Association for Computational Linguistics, ACL '02, pages 311-318, Stroudsburg, PA, USA. Association for Computational Linguistics.
16. Maja Popović. 2015. chrF: Character n-gram F-score for automatic MT evaluation. In Proceedings of the Tenth Workshop on Statistical Machine Translation, Lisboa, Portugal, September. Association for Computational Linguistics.
17. Raphael Rubino, Tommi Pirinen, Miquel Esplà-Gomis, Nikola Ljubešić, Sergio Ortiz Rojas, Vassilis Papavassiliou, Prokopis Prokopidis, and Antonio Toral. 2015. Abu-MaTran at WMT 2015 Translation Task: Morphological Segmentation and Web Crawling. In Proceedings of the Tenth Workshop on Statistical Machine Translation, Lisboa, Portugal, September. Association for Computational Linguistics.
18. Matthew Snover, Bonnie Dorr, Richard Schwartz, Linnea Micciulla, and John Makhoul. 2006. A study of translation edit rate with targeted human annotation. In In Proceedings of Association for Machine Translation in the Americas, pages 223-231.
19. Miloš Stanojević and Khalil Sima'an. 2015. BEER 1.1: ILLC UvA submission to metrics and tuning task. In Proceedings of the Tenth Workshop on Statistical Machine Translation, Lisboa, Portugal, September. Association for Computational Linguistics.
20. Miloš Stanojević, Amir Kamran, and Ondřej Bojar. 2015. Results of the WMT15 Tuning Shared Task. In Proceedings of the Tenth Workshop on Statistical Machine Translation, Lisboa, Portugal, September. Association for Computational Linguistics.
21. Yolanda Vazquez-Alvarez and Mark Huckvale. 2002. The reliability of the ITU-t p.85 standard for the evaluation of text-to-speech systems. In Proc. of ICSLP - INTERSPEECH.
22. Mihaela Vela and Liling Tan. 2015. Predicting Machine Translation Adequacy with Document Embeddings. In Proceedings of the Tenth Workshop on Statistical Machine Translation, Lisboa, Portugal, September. Association for Computational Linguistics.
23. Sami Virpioja and Stig-Arne Grönroos. 2015. LeBLEU: N-gram-based Translation Evaluation Score for Morphologically Complex Languages. In Proceedings of the Tenth Workshop on Statistical Machine Translation, Lisboa, Portugal, September. Association for Computational Linguistics.
24. Hui Yu, Qingsong Ma, Xiaofeng Wu, and Qun Liu. 2015. CASICT-DCU Participation in WMT2015 Metrics Task. In Proceedings of the Tenth Workshop on Statistical Machine Translation, Lisboa, Portugal, September. Association for Computational Linguistics.