HyTER: Meaning-equivalent semantics for translation evaluation

NAACL HLT 2012 - 2012 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Proceedings of the Conference

Volumn , Issue , 2012, Pages 162-171

  Dreyer, Markus ^a   Marcu, Daniel ^a  

^a SDL Language Weaver   (United States)

Author keywords

[No Author keywords available]

Indexed keywords

TRANSLATION (LANGUAGES);

ANNOTATION TOOL; COMMON KNOWLEDGE; EMPIRICAL ESTIMATE; EVALUATION METRICS; EXPONENTIAL NUMBERS; MAPPING PROCESS; NATURALLY OCCURRING;

SEMANTICS;

EID: 84884925008     PISSN: None     EISSN: None     Source Type: Conference Proceeding    
DOI: None     Document Type: Conference Paper

References (19)

1. C. Allauzen, M. Riley, J. Schalkwyk, W. Skut, and M. Mohri. 2007. OpenFst: a general and efficient weighted finite-state transducer library. In Proceedings of the 12th international conference on Implementation and application of automata, page 1123.
2. O. Bojar, M. Ercegovčević, M. Popel, and O. Zaidan. 2011. A grain of salt for the wmt manual evaluation. In Proceedings of the Sixth Workshop on Statistical Machine Translation, pages 1-11, Edinburgh, Scotland, July. Association for Computational Linguistics.
3. C. Callison-Burch, Ph. Koehn, Ch. Monz, and O. Zaidan. 2011. Findings of the 2011 workshop on statistical machine translation. In Proceedings of the SixthWorkshop on Statistical Machine Translation, pages 22-64, Edinburgh, Scotland, July. Association for Computational Linguistics.
4. G. Cormode and S. Muthukrishnan. 2007. The string edit distance matching problem with moves. ACM Transactions on Algorithms (TALG), 3(1):1-19.
5. M. Denkowski and A. Lavie. 2011. Meteor 1.3: Automatic Metric for Reliable Optimization and Evaluation of Machine Translation Systems. In Proceedings of the EMNLP 2011 Workshop on Statistical Machine Translation.
6. Q. Gao, N. Bach, S. Vogel, B. Chen, G. Foster, R. Kuhn, C. Callison-Burch, P. Koehn, C. Monz, K. Peterson, et al., 2010. Proceedings of the Joint Fifth Workshop on Statistical Machine Translation and Metrics (MATR), pages 121-126.
7. P.E. Hart, N.J. Nilsson, and B. Raphael. 1972. Correction to "A formal basis for the heuristic determination of minimum cost paths". ACM SIGART Bulletin, pages 28-29, December. ACM ID: 1056779.
8. S. Kanthak, D. Vilar, E. Matusov, R. Zens, and H. Ney. 2005. Novel reordering approaches in phrase-based statistical machine translation. In Proceedings of the ACL Workshop on Building and Using Parallel Texts, pages 167-174.
9. D. Karakos, J. Eisner, S. Khudanpur, and M. Dreyer. 2008. Machine translation system combination using ITG-based alignments. Proc. ACL-08: HLT, Short Papers (Companion Volume), page 8184.
10. K. Krammer and Y. Singer. 2001. On the algorithmic implementation of multi-class SVMs. In Proc. of JMLR.
11. A. Lavie and M. Denkowski. 2009. The meteor metric for automatic evaluation of machine translation. Machine Translation, 23(2-3):105-115.
12. L. I. Levenshtein. 1966. Binary codes capable of correcting deletions, insertions and reversals. Soviet Physics Doklady, 10:707-710.
13. C.Y. Lin and F.J. Och. 2004. Orange: a method for evaluating automatic evaluation metrics for machine translation. In Proceedings of the 20th international conference on Computational Linguistics, page 501. Association for Computational Linguistics.
14. M. Mohri. 2002. Semiring frameworks and algorithms for shortest-distance problems. Journal of Automata, Languages and Combinatorics, 7(3):321-350.
15. J. Olive, C. Christianson, and J. McCary, editors. 2011. Handbook of Natural Language Processing and Machine Translation. DARPA Global Autonomous Language Exploitation. Springer.
16. K. Papineni, S. Roukos, T. Ward, and W.-J. Zhu. 2002. BLEU: a method for automatic evaluation of machine translation. In Proceedings of 40th Annual Meeting of the Association for Computational Linguistics, pages 311-318, Philadelphia, Pennsylvania, USA, July. Association for Computational Linguistics.
17. M. Przybocki, A. Le, G. Sanders, S. Bronsart, S. Strassel, and M. Glenn, 2011. GALE Machine Translation Metrology: Definition, Implementation, and Calculation, chapter 5.4, pages 583-811. Springer.
18. M. Snover, B. Dorr, R. Schwartz, L. Micciulla, and J. Makhoul. 2006. A study of translation edit rate with targeted human annotation. In Proceedings of the Association for Machine Translation in the Americas, pages 223-231.
19. M. Snover, N. Madnani, B. Dorr, and R. Schwartz. 2009. Fluency, adequacy, or HTER? Exploring different human judgments with a tunable MT metric. In Proceedings of the Fourth Workshop on Statistical Machine Translation at the 12th Meeting of the EACL.