Robust machine translation evaluation with entailment features

ACL-IJCNLP 2009 - Joint Conf. of the 47th Annual Meeting of the Association for Computational Linguistics and 4th Int. Joint Conf. on Natural Language Processing of the AFNLP, Proceedings of the Conf.

Volumn , Issue , 2009, Pages 297-305

  Padó, Sebastian ^a   Galley, Michel ^b   Jurafsky, Dan ^b   Manning, Chris ^b  

^a UNIVERSITY OF STUTTGART   (Germany)

^b STANFORD UNIVERSITY   (United States)

Author keywords

[No Author keywords available]

Indexed keywords

COMPUTATIONAL LINGUISTICS; COMPUTER AIDED LANGUAGE TRANSLATION; NATURAL LANGUAGE PROCESSING SYSTEMS; SEMANTICS;

ARGUMENT STRUCTURES; EVALUATION METRICS; INDIVIDUAL SCORES; LEXICAL SEMANTICS; MACHINE TRANSLATION EVALUATIONS; MACHINE TRANSLATIONS; STATE OF THE ART; TEXTUAL ENTAILMENT;

QUALITY CONTROL;

EID: 84859900464     PISSN: None     EISSN: None     Source Type: Conference Proceeding    
DOI: 10.3115/1687878.1687922     Document Type: Conference Paper

References (25)

1. Enrique Amigó, Jesús Giménez, Julio Gonzalo, and Lluís Màrquez. 2006. MT Evaluation: Humanlike vs. human acceptable. In Proceedings of COLING/ACL 2006, pages 17-24, Sydney, Australia.
2. Satanjeev Banerjee and Alon Lavie. 2005. METEOR: An automatic metric for MT evaluation with improved correlation with human judgments. In Proceedings of the ACL Workshop on Intrinsic and Extrinsic Evaluation Measures, pages 65-72, Ann Arbor, MI.
3. Chris Callison-Burch, Miles Osborne, and Philipp Koehn. 2006. Re-evaluating the role of BLEU in machine translation research. In Proceedings of EACL, pages 249-256, Trento, Italy.
4. Chris Callison-Burch, Cameron Fordyce, Philipp Koehn, Christof Monz, and Josh Schroeder. 2008. Further meta-evaluation of machine translation. In Proceedings of the ACL Workshop on Statistical Machine Translation, pages 70-106, Columbus, OH.
5. Yee Seng Chan and Hwee Tou Ng. 2008. MAXSIM: A maximum similarity metric for machine translation evaluation. In Proceedings of ACL-08: HLT, pages 55-62, Columbus, Ohio, June.
6. Ido Dagan, Oren Glickman, and Bernardo Magnini. 2005. The PASCAL recognising textual entailment challenge. In Proceedings of the PASCAL ChallengesWorkshop on Recognising Textual Entailment, Southampton, UK.
7. Marie-Catherine de Marneffe, Trond Grenager, Bill MacCartney, Daniel Cer, Daniel Ramage, Chloé Kiddon, and Christopher D. Manning. 2007. Aligning semantic graphs for textual inference and machine reading. In Proceedings of the AAAI Spring Symposium, Stanford, CA.
8. George Doddington. 2002. Automatic evaluation of machine translation quality using n-gram cooccurrence statistics. In Proceedings of HLT, pages 128-132, San Diego, CA.
9. Jesús Giménez and Lluís Márquez. 2008. Heterogeneous automatic MT evaluation through nonparametric metric combinations. In Proceedings of IJCNLP, pages 319-326, Hyderabad, India.
10. Sanda Harabagiu and Andrew Hickl. 2006. Methods for using textual entailment in open-domain question answering. In Proceedings of ACL, pages 905-912, Sydney, Australia.
11. Eduard Hovy, Chin-Yew Lin, Liang Zhou, and Junichi Fukumoto. 2006. Automated summarization evaluation with basic elements. In Proceedings of LREC, Genoa, Italy.
12. David Kauchak and Regina Barzilay. 2006. Paraphrasing for automatic evaluation. In Proceedings of HLTNAACL, pages 455-462.
13. Phillip Koehn. 2005. Europarl: A parallel corpus for statistical machine translation. In Proceedings of the MT Summit X, Phuket, Thailand.
14. Chin-Yew Lin and Franz Josef Och. 2004. ORANGE: a method for evaluating automatic evaluation metrics for machine translation. In Proceedings of COLING, pages 501-507, Geneva, Switzerland.
15. Ding Liu and Daniel Gildea. 2005. Syntactic features for evaluation of machine translation. In Proceedings of the ACL Workshop on Intrinsic and Extrinsic Evaluation Measures, pages 25-32, Ann Arbor, MI.
16. Bill MacCartney and Christopher D. Manning. 2008. Modeling semantic containment and exclusion in natural language inference. In Proceedings of COLING, pages 521-528, Manchester, UK.
17. Bill MacCartney, Trond Grenager, Marie-Catherine de Marneffe, Daniel Cer, and Christopher D. Manning. 2006. Learning to recognize features of valid textual entailments. In Proceedings of NAACL, pages 41-48, New York City, NY.
18. Franz Josef Och. 2003. Minimum error rate training in statistical machine translation. In Proceedings of ACL, pages 160-167, Sapporo, Japan.
19. Karolina Owczarzak, Josef van Genabith, and Andy Way. 2008. Evaluating machine translation with LFG dependencies. Machine Translation, 21(2):95-119.
20. Sebastian Padó, Michel Galley, Dan Jurafsky, and Christopher D. Manning. 2009. Textual entailment features for machine translation evaluation. In Proceedings of the EACL Workshop on Statistical Machine Translation, pages 37-41, Athens, Greece.
21. Kishore Papineni, Salim Roukos, Todd Ward, and Wei-Jing Zhu. 2002. BLEU: a method for automatic evaluation of machine translation. In Proceedings of ACL, pages 311-318, Philadelphia, PA.
22. Michael Paul, Andrew Finch, and Eiichiro Sumita. 2007. Reducing human assessment of machine translation quality to binary classifiers. In Proceedings of TMI, pages 154-162, Skövde, Sweden.
23. Matthew Snover, Bonnie Dorr, Richard Schwartz, Linnea Micciulla, and John Makhoul. 2006. A study of translation edit rate with targeted human annotation. In Proceedings of AMTA, pages 223-231, Cambridge, MA.
24. Matthew Snover, Nitin Madnani, Bonnie J. Dorr, and Richard Schwartz. 2009. Fluency, adequacy, or HTER? Exploring different human judgments with a tunable MT metric. In Proceedings of the EACL Workshop on Statistical Machine Translation, pages 259-268, Athens, Greece.
25. Liang Zhou, Chin-Yew Lin, and Eduard Hovy. 2006. Re-evaluating machine translation results with paraphrase support. In Proceedings of EMNLP, pages 77-84, Sydney, Australia.