The contribution of linguistic features to Automatic Machine Translation Evaluation

ACL-IJCNLP 2009 - Joint Conf. of the 47th Annual Meeting of the Association for Computational Linguistics and 4th Int. Joint Conf. on Natural Language Processing of the AFNLP, Proceedings of the Conf.

Volumn , Issue , 2009, Pages 306-314

  Amigó, Enrique ^a   Giménez, Jesús ^b   Gonzalo, Julio ^a   Verdejo, Felisa ^a  

^a UNIVERSIDAD NACIONAL DE EDUCACIÓN A DISTANCIA   (Spain)

^b UNIVERSITAT POLITÈCNICA DE CATALUNYA   (Spain)

Author keywords

[No Author keywords available]

Indexed keywords

COMPUTATIONAL LINGUISTICS;

AUTOMATIC MACHINES; EVALUATION METRICS; EVALUATION PERFORMANCE; LINGUISTIC FEATURES; LINGUISTIC INFORMATION; LINGUISTIC KNOWLEDGE; MT EVALUATIONS; N-GRAMS;

NATURAL LANGUAGE PROCESSING SYSTEMS;

EID: 80053288311     PISSN: None     EISSN: None     Source Type: Conference Proceeding    
DOI: 10.3115/1687878.1687923     Document Type: Conference Paper

References (26)

1. Joshua Albrecht and Rebecca Hwa. 2007. Regression for Sentence-Level MT Evaluation with Pseudo References. In Proceedings of the 45th Annual Meeting of the Association for Computational Linguistics (ACL), pages 296-303.
2. Enrique Amigó, Julio Gonzalo, Anselmo Pe nas, and Felisa Verdejo. 2005. QARLA: a Framework for the Evaluation of Automatic Summarization. In Proceedings of the 43rd Annual Meeting of the Association for Computational Linguistics (ACL), pages 280-289.
3. Enrique Amigó, Jesús Giménez, Julio Gonzalo, and Lluís Màrquez. 2006. MT Evaluation: Human-Like vs. Human Acceptable. In Proceedings of the Joint 21st International Conference on Computational Linguistics and the 44th Annual Meeting of the Association for Computational Linguistics (COLING-ACL), pages 17-24.
4. Satanjeev Banerjee and Alon Lavie. 2005. METEOR: An Automatic Metric for MT Evaluation with Improved Correlation with Human Judgments. In Proceedings of ACL Workshop on Intrinsic and Extrinsic Evaluation Measures for MT and/or Summarization.
5. Chris Callison-Burch, Miles Osborne, and Philipp Koehn. 2006. Re-evaluating the Role of BLEU in Machine Translation Research. In Proceedings of 11th Conference of the European Chapter of the Association for Computational Linguistics (EACL).
6. Chris Callison-Burch. 2005. Linear B system description for the 2005 NIST MT evaluation exercise. In Proceedings of the NIST 2005 Machine Translation Evaluation Workshop.
7. Christopher Culy and Susanne Z. Riehemann. 2003. The Limits of N-gram Translation Evaluation Metrics. In Proceedings of MT-SUMMIT IX, pages 1-8.
8. George Doddington. 2002. Automatic Evaluation of Machine Translation Quality Using N-gram Co-Occurrence Statistics. In Proceedings of the 2nd International Conference on Human Language Technology, pages 138-145.
9. Jesús Giménez and Lluís Màrquez. 2007. Linguistic Features for Automatic Evaluation of Heterogeneous MT Systems. In Proceedings of the ACL Workshop on Statistical Machine Translation, pages 256-264.
10. Jesús Giménez and Lluís Màrquez. 2008a. Heterogeneous Automatic MT Evaluation Through Non-Parametric Metric Combinations. In Proceedings of the Third International Joint Conference on Natural Language Processing (IJCNLP), pages 319-326.
11. Jesús Giménez and Lluís Màrquez. 2008b. On the Robustness of Linguistic Features for Automatic MT Evaluation. (Under submission).
12. Jesús Giménez and Lluís Màrquez. 2009. On the Robustness of Syntactic and Semantic Features for Automatic MT Evaluation. In Proceedings of the 4th Workshop on Statistical Machine Translation (EACL 2009).
13. Jesús Giménez. 2007. IQMT v 2.0. Technical Manual (LSI-07-29-R). Technical report, TALP Research Center. LSI Department. http://www.lsi.upc.edu/n~lp/IQMT/IQMT.v2.1.pdf.
14. Alex Kulesza and Stuart M. Shieber. 2004. A learning approach to improving sentence-level MT evaluation. In Proceedings of the 10th International Conference on Theoretical and Methodological Issues in Machine Translation (TMI), pages 75-84.
15. Audrey Le and Mark Przybocki. 2005. NIST 2005 machine translation evaluation official results. In Official release of automatic evaluation scores for all submissions, August.
16. Chin-Yew Lin and Franz Josef Och. 2004. Automatic Evaluation of Machine Translation Quality Using Longest Common Subsequence and Skip-Bigram Statics. In Proceedings of the 42nd Annual Meeting of the Association for Computational Linguistics (ACL).
17. Ding Liu and Daniel Gildea. 2005. Syntactic Features for Evaluation of Machine Translation. In Proceedings of ACL Workshop on Intrinsic and Extrinsic Evaluation Measures for MT and/or Summarization, pages 25-32.
18. Dennis Mehay and Chris Brew. 2007. BLEUATRE: Flattening Syntactic Dependencies for MT Evaluation. In Proceedings of the 11th Conference on Theoretical and Methodological Issues in Machine Translation (TMI).
19. I. Dan Melamed, Ryan Green, and Joseph P. Turian. 2003. Precision and Recall of Machine Translation. In Proceedings of the Joint Conference on Human Language Technology and the North American Chapter of the Association for Computational Linguistics (HLT-NAACL).
20. Sonja Nießen, Franz Josef Och, Gregor Leusch, and Hermann Ney. 2000. An Evaluation Tool for Machine Translation: Fast Evaluation for MT Research. In Proceedings of the 2nd International Conference on Language Resources and Evaluation (LREC).
21. Karolina Owczarzak, Declan Groves, Josef Van Genabith, and Andy Way. 2006. Contextual Bitext-Derived Paraphrases in Automatic MT Evaluation. In Proceedings of the 7th Conference of the Association for Machine Translation in the Americas (AMTA), pages 148-155.
22. Karolina Owczarzak, Josef van Genabith, and Andy Way. 2007. Labelled Dependencies in Machine Translation Evaluation. In Proceedings of the ACL Workshop on Statistical Machine Translation, pages 104-111.
23. Kishore Papineni, Salim Roukos, ToddWard, and Wei-Jing Zhu. 2001. Bleu: a method for automatic evaluation of machine translation, RC22176. Technical report, IBM T.J. Watson Research Center.
24. Maja Popovic and Hermann Ney. 2007. Word Error Rates: Decomposition over POS classes and Applications for Error Analysis. In Proceedings of the Second Workshop on Statistical Machine Translation, pages 48-55, Prague, Czech Republic, June. Association for Computational Linguistics.
25. Florence Reeder, Keith Miller, Jennifer Doyon, and John White. 2001. The Naming of Things and the Confusion of Tongues: an MT Metric. In Proceedings of the Workshop on MT Evaluation "Who did what to whom?" at Machine Translation Summit VIII, pages 55-59.
26. Christoph Tillmann, Stefan Vogel, Hermann Ney, A. Zubiaga, and H. Sawaf. 1997. Accelerated DP based Search for Statistical Translation. In Proceedings of European Conference on Speech Communication and Technology.