Fully Automatic Semantic MT Evaluation

WMT 2012 - 7th Workshop on Statistical Machine Translation, Proceedings of the Workshop

Volumn , Issue , 2012, Pages 243-252

  Lo, Chi Kiu ^a   Tumuluru, Anand Karthik ^a   Wu, Dekai ^a  

^a HONG KONG UNIVERSITY OF SCIENCE AND TECHNOLOGY   (Hong Kong)

Author keywords

[No Author keywords available]

Indexed keywords

AUTOMATION; COMPUTATIONAL LINGUISTICS; LEARNING ALGORITHMS; MACHINE TRANSLATION; NATURAL LANGUAGE PROCESSING SYSTEMS; SEMANTICS; SYNTACTICS;

AUTOMATIC METRICS; EVALUATION METRICS; FRAME-BASED; FULLY AUTOMATED; HUMAN JUDGMENTS; MACHINE TRANSLATIONS; MT EVALUATIONS; OCCAM'S RAZOR; SEMANTIC FRAMES; SIMPLE++;

COMPUTER AIDED LANGUAGE TRANSLATION;

EID: 85122945200     PISSN: None     EISSN: None     Source Type: Conference Proceeding    
DOI: None     Document Type: Conference Paper

References (21)

1. Satanjeev Banerjee and Alon Lavie. METEOR: An Automatic Metric for MT Evaluation with Improved Correlation with Human Judgments. In Proceedings of the 43th Annual Meeting of the Association of Computational Linguistics (ACL- 05), pages 65-72, 2005.
2. Chris Callison-Burch, Miles Osborne, and Philipp Koehn. Reevaluating the role of BLEU in Machine Translation Research. In Proceedings of the 13th Conference of the European Chapter of the Association for Computational Linguistics (EACL-06), pages 249-256, 2006.
3. Chris Callison-Burch, Cameron Fordyce, Philipp Koehn, Christof Monz, and Josh Schroeder. (Meta-) evaluation of Machine Translation. In Proceedings of the 2nd Workshop on Statistical Machine Translation, pages 136-158, 2007.
4. Chris Callison-Burch, Cameron Fordyce, Philipp Koehn, Christof Monz, and Josh Schroeder. Further Meta-evaluation of Machine Translation. In Proceedings of the 3rd Workshop on Statistical Machine Translation, pages 70-106, 2008.
5. Chris Callison-Burch, Philipp Koehn, Christof Monz, Kay Peterson, Mark Pryzbocki, and Omar Zaidan. Findings of the 2010 Joint Workshop on Statistical Machine Translation and Metrics for Machine Translation. In Proceedings of the Joint 5th Workshop on Statistical Machine Translation and MetricsMATR, pages 17-53, Uppsala, Sweden, 15-16 July 2010.
6. Ido Dagan. Contextual word similarity. In Robert Dale, Herman Moisl, and Harold Somers, editors, Handbook of Natural Language Processing, pages 459-476. Marcel Dekker, New York, 2000.
7. G. Doddington. Automatic Evaluation of Machine Translation Quality using N-gram Co-occurrence Statistics. In Proceedings of the 2nd International Conference on Human Language Technology Research (HLT-02), pages 138-145, San Francisco, CA, USA, 2002. Morgan Kaufmann Publishers Inc.
8. Jeśus Giḿenez and Llúis M`arquez. Linguistic Features for Automatic Evaluation of Heterogenous MT Systems. In Proceedings of the 2nd Workshop on Statistical Machine Translation, pages 256-264, Prague, Czech Republic, June 2007. Association for Computational Linguistics.
9. Jeśus Giḿenez and Llúis M`arquez. A Smorgasbord of Features for Automatic MT Evaluation. In Proceedings of the 3rd Workshop on Statistical Machine Translation, pages 195- 198, Columbus, OH, June 2008. Association for Computational Linguistics.
10. Philipp Koehn and Christof Monz. Manual and Automatic Evaluation of Machine Translation between European Languages. In Proceedings of the Workshop on Statistical Machine Translation, pages 102-121, 2006.
11. Gregor Leusch, Nicola Ueffing, and Hermann Ney. CDer: Efficient MT Evaluation Using Block Movements. In Proceedings of the 13th Conference of the European Chapter of the Association for Computational Linguistics (EACL-06), 2006.
12. Chi-kiu Lo and Dekai Wu. A Radically Simple, Effective Annotation and Alignment Methodology for Semantic Frame Based SMT and MT Evaluation. In Proceedings of International Workshop on Using Linguistic Information for Hybrid Machine Translation (LiHMT 2011), organized by OpenMT- 2., 2011.
13. Chi-kiu Lo and Dekai Wu. MEANT: An Inexpensive, High- Accuracy, Semi-Automatic Metric for Evaluating Translation Utility based on Semantic Roles. In Proceedings of the Joint conference of the 49th Annual Meeting of the Association for Computational Linguistics : Human Language Technologies (ACL-HLT-11), 2011.
14. Chi-kiu Lo and Dekai Wu. SMT vs. AI redux: How semantic frames evaluate MT more accurately. In Proceedings of the 22nd International Joint Conference on Artificial Intelligence (IJCAI-11), 2011.
15. Chi-kiu Lo and Dekai Wu. Structured vs. Flat Semantic Role Representations for Machine Translation Evaluation. In Proceedings of the 5th Workshop on Syntax and Structure in Statistical Translation (SSST-5), 2011.
16. I. Dan Melamed. Automatic construction of clean broadcoverage translation lexicons. In Proceedings of the 2nd Conference of the Association for Machine Translation in the Americas (AMTA-1996), 1996.
17. Sonja Nießen, Franz Josef Och, Gregor Leusch, and Hermann Ney. A Evaluation Tool for Machine Translation: Fast Evaluation for MT Research. In Proceedings of the 2nd International Conference on Language Resources and Evaluation (LREC-2000), 2000.
18. Kishore Papineni, Salim Roukos, Todd Ward, and Wei-Jing Zhu. BLEU: A Method for Automatic Evaluation of Machine Translation. In Proceedings of the 40th Annual Meeting of the Association for Computational Linguistics (ACL- 02), pages 311-318, 2002.
19. Sameer Pradhan, Wayne Ward, Kadri Hacioglu, James H. Martin, and Dan Jurafsky. Shallow Semantic Parsing Using Support Vector Machines. In Proceedings of the 2004 Conference on Human Language Technology and the North American Chapter of the Association for Computational Linguistics (HLT-NAACL-04), 2004.
20. Matthew Snover, Bonnie J. Dorr, Richard Schwartz, Linnea Micciulla, and John Makhoul. A Study of Translation Edit Rate with Targeted Human Annotation. In Proceedings of the 7th Conference of the Association for Machine Translation in the Americas (AMTA-06), pages 223-231, 2006.
21. Christoph Tillmann, Stephan Vogel, Hermann Ney, Arkaitz Zubiaga, and Hassan Sawaf. Accelerated DP Based Search For Statistical Translation. In Proceedings of the 5th European Conference on Speech Communication and Technology (EUROSPEECH-97), 1997.