Linguistic features for automatic evaluation of heterogenous MT systems

Proceedings of the Annual Meeting of the Association for Computational Linguistics

Volumn , Issue , 2007, Pages 256-264

  Giménez, Jesús ^a   Màrquez, Lluís ^a  

^a UNIVERSITAT POLITÈCNICA DE CATALUNYA   (Spain)

Author keywords

[No Author keywords available]

Indexed keywords

ABSTRACT LEVELS; AUTOMATIC EVALUATION; EVALUATION RESULTS; LEXICAL MATCHING; LINGUISTIC FEATURES; LINGUISTIC INFORMATION; QUALITY ASPECTS; QUALITY INDICATORS; RELIABLE SYSTEMS; SYSTEM RANKINGS;

EID: 80053429932     PISSN: 0736587X     EISSN: None     Source Type: Conference Proceeding    
DOI: None     Document Type: Conference Paper

References (25)

1. Enrique Amigó, Julio Gonzalo, Anselmo Peñas, and Felisa Verdejo. 2005. QARLA: a Framework for the Evaluation of Automatic Sumarization. In Proceedings of the 43th Annual Meeting of the Association for Computational Linguistics.
2. Enrique Amigó, Jesús Giménez, Julio Gonzalo, and Lluís Màrquez. 2006. MT Evaluation: Human-Like vs. Human Acceptable. In Proceedings of COLING-ACL06.
3. Satanjeev Banerjee and Alon Lavie. 2005. METEOR: An Automatic Metric for MT Evaluation with Improved Correlation with Human Judgments. In Proceedings of ACL Workshop on Intrinsic and Extrinsic Evaluation Measures for Machine Translation and/or Summarization.
4. Chris Callison-Burch, Miles Osborne, and Philipp Koehn. 2006. Re-evaluating the Role of BLEU in Machine Translation Research. In Proceedings of EACL.
5. Chris Callison-Burch. 2005. Linear B system description for the 2005 NIST MT evaluation exercise. In Proceedings of the NIST 2005 Machine Translation Evaluation Workshop.
6. Xavier Carreras, Lluís Márquez, and Jorge Castro. 2005. Filtering-ranking perceptron learning for partial parsing. Machine Learning, 59:1-31.
7. Eugene Charniak and Mark Johnson. 2005. Coarse-to-fine n-best parsing and MaxEnt discriminative reranking. In Proceedings of ACL.
8. George Doddington. 2002. Automatic evaluation of machine translation quality using n-gram co-occurrence statistics. In Proceedings of the 2nd IHLT.
9. C. Fellbaum, editor. 1998. WordNet. An Electronic Lexical Database. The MIT Press.
10. Jesús Giménez and Enrique Amigó. 2006. IQMT: A Framework for Automatic Machine Translation Evaluation. In Proceedings of the 5th LREC.
11. Jesús Giménez and Lluís Màrquez. 2004. SVMTool: A general POS tagger generator based on Support Vector Machines. In Proceedings of 4th LREC.
12. Philipp Koehn and Christof Monz. 2006. Manual and Automatic Evaluation of Machine Translation between European Languages. In Proceedings of the Workshop on Statistical Machine Translation, pages 102-121.
13. Audrey Le and Mark Przybocki. 2005. NIST 2005 machine translation evaluation official results. Technical report, NIST, August.
14. Chin-Yew Lin and Franz Josef Och. 2004a. Automatic Evaluation of Machine Translation Quality Using Longest Common Subsequence and Skip-Bigram Statics. In Proceedings of ACL.
15. Chin-Yew Lin and Franz Josef Och. 2004b. ORANGE: a Method for Evaluating Automatic Evaluation Metrics for Machine Translation. In Proceedings of COLING.
16. Dekang Lin. 1998. Dependency-based Evaluation of MINIPAR. In Proceedings of the Workshop on the Evaluation of Parsing Systems.
17. Ding Liu and Daniel Gildea. 2005. Syntactic Features for Evaluation of Machine Translation. In Proceedings of ACL Workshop on Intrinsic and Extrinsic Evaluation Measures for Machine Translation and/or Summarization.
18. Mitchell P. Marcus, Beatrice Santorini, and Mary Ann Marcinkiewicz. 1993. Building a large annotated corpus of english: The penn treebank. Computational Linguistics, 19(2):313-330.
19. I. Dan Melamed, Ryan Green, and Joseph P. Turian. 2003. Precision and Recall of Machine Translation. In Proceedings of HLT/NAACL.
20. Lluís Màrquez, Mihai Surdeanu, Pere Comas, and Jordi Turmo. 2005. Robust Combination Strategy for Semantic Role Labeling. In Proceedings of HLT/EMNLP.
21. S. Nießen, F.J. Och, G. Leusch, and H. Ney. 2000. Evaluation Tool for Machine Translation: Fast Evaluation for MT Research. In Proceedings of the 2nd LREC.
22. Kishore Papineni, Salim Roukos, Todd Ward, and Wei-Jing Zhu. 2001. Bleu: a method for automatic evaluation of machine translation, rc22176, ibm. Technical report, IBM T.J. Watson Research Center.
23. Florence Reeder, Keith Miller, Jennifer Doyon, and John White. 2001. The Naming of Things and the Confusion of Tongues: an MT Metric. In Proceedings of the Workshop on MT Evaluation”Who did what to whom?” at MT Summit VIII, pages 55-59.
24. Mihai Surdeanu, Jordi Turmo, and Eli Comelles. 2005. Named Entity Recognition from Spontaneous Open-Domain Speech. In Proceedings of the 9th International Conference on Speech Communication and Technology (Interspeech).
25. C. Tillmann, S. Vogel, H. Ney, A. Zubiaga, and H. Sawaf. 1997. Accelerated DP based Search for Statistical Translation. In Proceedings of European Conference on Speech Communication and Technology.