Lebleu: N-gram-based translation evaluation score for morphologically complex languages

10th Workshop on Statistical Machine Translation, WMT 2015 at the 2015 Conference on Empirical Methods in Natural Language Processing, EMNLP 2015 - Proceedings

Volumn , Issue , 2015, Pages 411-416

  Virpioja, Sami ^a   Grönroos, Stig Arne ^b  

^a Ling soft Inc Inc   (Finland)

^b AALTO UNIVERSITY   (Finland)

Author keywords

[No Author keywords available]

Indexed keywords

MACHINE TRANSLATION; NATURAL LANGUAGE PROCESSING SYSTEMS;

DATA SET; DISTANCE MEASURE; FUZZY MATCH; HUMAN EVALUATION; MACHINE TRANSLATIONS; MATCHINGS; MORPHOLOGICAL PREPROCESSING; N-GRAMS; WORD N-GRAMS;

COMPUTATIONAL LINGUISTICS;

EID: 85072819122     PISSN: None     EISSN: None     Source Type: Conference Proceeding    
DOI: None     Document Type: Conference Paper

References (19)

1. Satanjeev Banerjee and Alon Lavie. 2005. METEOR: An automatic metric for MT evaluation with improved correlation with human judgments. In Proceedings of the ACL Workshop on Intrinsic and Extrinsic Evaluation Measures for Machine Translation and/or Summarization, pages 65-72, Ann Arbor, Michigan, June. Association for Computational Linguistics.
2. Ilaria Bartolini, Paolo Ciaccia, and Marco Patella. 2002. String matching with metric trees using an approximate distance. In String processing and information retrieval, pages 271-283. Springer.
3. Boxing Chen and Roland Kuhn. 2011. AMBER: A modified BLEU, enhanced ranking metric. In Proceedings of the Sixth Workshop on Statistical Machine Translation, pages 71-77, Edinburgh, Scotland, July. Association for Computational Linguistics.
4. Mathias Creutz and Krista Lagus. 2005. Unsupervised morpheme segmentation and morphology induction from text corpora using Morfessor 1.0. Technical Report A81, Publications in Computer and Information Science, Helsinki University of Technology.
5. Mathias Creutz, Teemu Hirsimäki, Mikko Kurimo, Antti Puurula, Janne Pylkkönen, Vesa Siivola, Matti Varjokallio, Ebru Arisoy, Murat Saraçlar, and Andreas Stolcke. 2007. Morph-based speech recognition and modeling of out-of-vocabulary words across languages. ACM Transactions on Speech and Language Processing, 5(1):3:1-3:29, December.
6. Michael Denkowski and Alon Lavie. 2011. Meteor 1.3: Automatic metric for reliable optimization and evaluation of machine translation systems. In Proceedings of the Sixth Workshop on Statistical Machine Translation, pages 85-91, Edinburgh, Scotland, July. Association for Computational Linguistics.
7. Etienne Denoual and Yves Lepage. 2005. BLEU in characters: Towards automatic MT evaluation in languages without word delimiters. In Companion Volume to the Proceedings of the Second International Joint Conference on Natural Language Processing, pages 81-86.
8. Robert Frederking and Sergei Nirenburg. 1994. Three heads are better than one. In Proceedings of the Fourth Conference on Applied Natural Language Processing, pages 95-100, Stuttgart, Germany, October. Association for Computational Linguistics.
9. Petr Homola, Vladislav Kuboň, and Pavel Pecina. 2009. A simple automatic MT evaluation metric. In Proceedings of the Fourth Workshop on Statistical Machine Translation, pages 33-36, Athens, Greece, March. Association for Computational Linguistics.
10. Vladimir I. Levenshtein. 1966. Binary codes capable of correcting deletions, insertions and reversals. Soviet Physics Doklady, 10(8):707-710.
11. Jindřich Libovický and Pavel Pecina. 2014. Tolerant BLEU: A submission to the WMT14 metrics task. In Proceedings of the Ninth Workshop on Statistical Machine Translation, pages 409-413, Baltimore, Maryland, USA, June. Association for Computational Linguistics.
12. Matouš Macháček and Ondřej Bojar. 2013. Results of the WMT13 metrics shared task. In Proceedings of the Eighth Workshop on Statistical Machine Translation, pages 45-51, Sofia, Bulgaria, August. Association for Computational Linguistics.
13. Matouš Macháček and Ondřej Bojar. 2014. Results of the WMT14 metrics shared task. In Proceedings of the Ninth Workshop on Statistical Machine Translation, pages 293-301, Baltimore, Maryland, USA, June. Association for Computational Linguistics.
14. Kishore Papineni, Salim Roukos, ToddWard, andWei-Jing Zhu. 2002. BLEU: A method for automatic evaluation of machine translation. In Proceedings of 40th Annual Meeting of the Association for Computational Linguistics, pages 311-318, Philadelphia, PA, USA, July. Association for Computational Linguistics.
15. Maja Popović. 2011. Morphemes and POS tags for n-gram based evaluation metrics. In Proceedings of the Sixth Workshop on Statistical Machine Translation, pages 104-107, Edinburgh, Scotland, July. Association for Computational Linguistics.
16. Dana Shapira and James A. Storer. 2002. Edit distance with move operations. In Proceedings of the 13th Annual Symposium on Computational Pattern Matching, pages 85-98.
17. Matthew Snover, Bonnie Dorr, Richard Schwartz, Linnea Micciulla, and John Makhoul. 2006. A study of translation edit rate with targeted human annotation. In Proceedings of Association for Machine Translation in the Americas, pages 223-231.
18. Xingyi Song, Trevor Cohn, and Lucia Specia. 2013. BLEU deconstructed: Designing a better MT evaluation metric, March. On-line: http://staffwww.dcs.shef.ac.uk/people/X.Song/song13deconstructed.pdf. Accessed Oct 2013.
19. Mengqiu Wang and Christopher Manning. 2012. SPEDE: Probabilistic edit distance metrics for MT evaluation. In Proceedings of the Seventh Workshop on Statistical Machine Translation, pages 76-83, Montréal, Canada, June. Association for Computational Linguistics.