SPEDE: Probabilistic Edit Distance Metrics for MT Evaluation

WMT 2012 - 7th Workshop on Statistical Machine Translation, Proceedings of the Workshop

Volumn , Issue , 2012, Pages 76-83

  Wang, Mengqiu ^a   Manning, Christopher D ^a  

^a Stanford University   (United States)

Author keywords

[No Author keywords available]

Indexed keywords

DISTANCE METRICS; EDIT DISTANCE; FINITE STATE MACHINE MODEL; LEARN+; MT EVALUATIONS; PROBABILISTIC FINITE STATE MACHINES; PROBABILISTICS; STANFORD UNIVERSITY; TRANSLATION QUALITY; WEIGHTED EDIT DISTANCE;

COMPUTATIONAL LINGUISTICS;

EID: 85041240376     PISSN: None     EISSN: None     Source Type: Conference Proceeding    
DOI: None     Document Type: Conference Paper

References (32)

1. J. Albrecht and R. Hwa. 2007a. A re-examination of machine learning approaches for sentence-level MT evaluation. In Proceedings of ACL.
2. J. Albrecht and R. Hwa. 2007b. Regression for sentencelevel MT evaluation with pseudo references. In Proceedings of ACL.
3. S. Banerjee and A. Lavie. 2005. Meteor: An automatic metric for MT evaluation with improved correlation with human judgments. In Proceedings of ACL Workshop on Intrinsic and Extrinsic Evaluation Measures.
4. A. Birch, P. Blunsom, and M. Osborne. 2009. A quantitative analysis of reordering phenomena. In Proceedings of WMT 09.
5. C. Callison-Burch, P. Koehn, C. Monz, and J. Schroeder. 2009. Findings of the 2009 Workshop on Statistical Machine Translation. In Proceedings of the Fourth Workshop on Statistical Machine Translation.
6. C. Callison-Burch, P. Koehn, C. Monz, K. Peterson, M. Przybocki, and O. Zaidan. 2010. Findings of the 2010 joint workshop on Statistical Machine Translation and metrics for Machine Translation. In Proceedings of Joint WMT 10 and MetricsMatr Workshop at ACL.
7. C. Callison-Burch, P. Koehn, C. Monz, and O. Zaidan. 2011. Findings of the 2011 workshop on statistical machine translation. In Proceedings of the Sixth Workshop on Statistical Machine Translation.
8. T. H. Cormen, C. E. Leiserson, R. L. Rivest, and C. Stein. 2001. Introduction to Algorithms, Second Edition. MIT Press.
9. M. Denkowski and A. Lavie. 2010. Extending the METEOR machine translation evaluation metric to the phrase level. In Proceedings of HLT/NAACL.
10. G. Doddington. 2002. Automatic evaluation of machine translation quality using n-gram cooccurrence statistics. In Proceedings of HLT.
11. Y. Dombb, O. Lipsky, B. Porat, E. Porat, and A. Tsur. 2010. The approximate swap and mismatch edit distance. Theoretical Computer Science, 411(43).
12. J. Eisner. 2002. Parameter estimation for probabilistic finite-state transducers. In Proceedings of ACL.
13. J. Esparza and A. Kucera. 2005. Quantitative analysis of probabilistic pushdown automata: Expectations and variances. In Proceedings of the 20th Annual IEEE Symposium on Logic in Computer Science.
14. Y. He, J. Du, A. Way, and J. van Genabith. 2010. The DCU dependency-based metric inWMT-MetricsMATR 2010. In Proceedings of Joint WMT 10 and Metrics- Matr Workshop at ACL.
15. K. Knight and Y. Al-Onaizan. 1998. Translation with finite-state devices. In Proceedings of AMTA.
16. P. Koehn. 2004. Statistical significance tests for machine translation evaluation. In Proceedings of EMNLP.
17. S. Kumar and W. Byrne. 2003. A weighted finite state transducer implementation of the alignment template model for statistical machine translation. In Proceedings of HLT/NAACL.
18. D. C. Liu and J. Nocedal. 1989. On the limited memory BFGS method for large scale optimization. Math. Programming, 45:503-528.
19. D. Liu,, and D. Gildea. 2005. Syntactic features for evaluation of machine translation. In Proceedings of the ACL Workshop on Intrinsic and Extrinsic Evaluation Measures.
20. M. P. Marcus, M. A. Marcinkiewicz, and B. Santorini. 1993. Building a large annotated corpus of english: the Penn Treebank. Computational Linguistics, 19(2):313- 330.
21. G. A. Miller, R. Beckwith, C. Fellbaum, D. Gross, and K. J. Miller. 1990. WordNet: an on-line lexical database. International Journal of Lexicography, 3(4).
22. F. Och. 2003. Minimum error rate training in statistical machine translation. In Proceedings of ACL.
23. K. Owczarzak, J. van Genabith, and A. Way. 2008. Evaluating machine translation with LFG dependencies. Machine Translation, 21(2):95-119.
24. S. Pado, M. Galley, D. Jurafsky, and C. D. Manning. 2009. Robust machine translation evaluation with entailment features. In Proceedings of ACL.
25. K. Papineni, S. Roukos, T. Ward, and W. Zhu. 2002. BLEU: a method for automatic evaluation of machine translation. In Proceedings of ACL.
26. M.F. Porter. 1980. An algorithm for suffix stripping. Program, 14(3):130-137.
27. M. Snover, B. Dorr, R. Schwartz, L. Micciulla, and J. Makhoul. 2006. A study of translation edit rate with targeted human annotation. In Proceedings of AMTA.
28. M. Snover,, N. Madnani, B. Dorr, and R. Schwartz. 2009. Fluency, adequacy, or HTER? exploring different human judgments with a tunable MT metric. In Proceedings of WMT09 Workshop.
29. S. Sun, Y. Chen, and J. Li. 2008. A re-examination on features in regression based approach to automatic MT evaluation. In Proceedings of ACL.
30. E. Vidal, F. Thollard, C. de la Higuera, F. Casacuberta, and R. C. Carrasco. 2005. Probabilistic finite-state machines part I. IEEE Transactions on Pattern Analysis and Machine Intelligence, 27(7):1013-1025.
31. D. Wu, 2010. CRC Handbook of Natural Language Processing, chapter How to Select an Answer String?, pages 367-408. CRC Press.
32. L. Zhou, C.Y. Lin, and E. Hovy. 2006. Re-evaluating machine translation results with paraphrase support. In Proceedings of EMNLP. 83.