Towards heterogeneous automatic MT error analysis

Proceedings of the 6th International Conference on Language Resources and Evaluation, LREC 2008

Volumn , Issue , 2008, Pages 1894-1901

  Giménez, Jesús ^a   Màrquez, Lluís ^a  

^a UNIVERSITAT POLITÈCNICA DE CATALUNYA   (Spain)

Author keywords

[No Author keywords available]

Indexed keywords

ERRORS;

ANALYSIS PROCESS; AUTOMATIC EVALUATION; DEVELOPMENT CYCLE; HUMAN LABOR; SPEED UP; TRANSLATION QUALITY; WORK STUDY;

ERROR ANALYSIS;

EID: 80052759026     PISSN: None     EISSN: None     Source Type: Conference Proceeding    
DOI: None     Document Type: Conference Paper

References (27)

1. Enrique Amigó, Julio Gonzalo, Anselmo Peñas, and Felisa Verdejo. 2005. QARLA: a Framework for the Evaluation of Automatic Sumarization. In Proceedings of the 43th Annual Meeting of the Association for Computational Linguistics.
2. Enrique Amigó, Jesús Giménez, Julio Gonzalo, and Llúis Màrquez. 2006. MT Evaluation: Human-Like vs. Human Acceptable. In Proceedings of the Joint 21st International Conference on Computational Linguistics and the 44th Annual Meeting of the Association for Computational Linguistics (COLING-ACL).
3. Satanjeev Banerjee and Alon Lavie. 2005. METEOR: An Automatic Metric for MT Evaluation with Improved Correlation with Human Judgments. In Proceedings of ACL Workshop on Intrinsic and Extrinsic Evaluation Measures for MT and/or Summarization.
4. Chris Callison-Burch, Miles Osborne, and Philipp Koehn. 2006. Re-evaluating the Role of BLEU in Machine Translation Research. In Proceedings of EACL.
5. Simon Corston-Oliver, Michael Gamon, and Chris Brockett. 2001. A Machine Learning Approach to the Automatic Evaluation of Machine Translation. In Proceedings of the 39th Annual Meeting of the Association for Computational Linguistics (ACL), pages 140-147.
6. George Doddington. 2002. Automatic evaluation of machine translation quality using n-gram co-occurrence statistics. In Proceedings of the 2nd Internation Conference on Human Language Technology, pages 138-145.
7. Michael Gamon, Anthony Aue, and Martine Smets. 2005. Sentence-Level MT evaluation without reference translations: beyond language modeling. In Proceedings of EAMT, pages 103-111.
8. Jesús Giménez and Lluis Màrquez. 2008. On the Robustness of Linguistic Features for Automatic MT Evaluation. (Under submission).
9. Jesús Giménez and Llúis Màrquez. 2007. Linguistic Features for Automatic Evaluation of Heterogeneous MT Systems. In Proceedings of the ACL Workshop on Statistical Machine Translation, pages 256-264.
10. Jesús Giménez. 2007. IQMT v2.1. Technical Manual (LSI-07-29-R). Technical report, TALP Research Center. LSI Department. http://www.lsi.upc.edu/~nlp/-IQMT/IQMT.v2.1.pdf.
11. Hans Kamp. 1981. A Theory of Truth and Semantic Representation. In J. A. G. Groenendijk, T. M. V. Janssen, and M. B. J. Stokhof, editors, Formal Methods in the Study of Language, pages 277-322. Mathematisch Centrum, address = Amsterdam.
12. David Kauchak and Regina Barzilay. 2006. Paraphrasing for Automatic Evaluation. In Proceedings of the Joint Conference on Human Language Technology and the North American Chapter of the Association for Computational Linguistics (HLT-NAACL), pages 455-462.
13. Alex Kulesza and Stuart M. Shieber. 2004. A learning approach to improving sentence-level MT evaluation. In Proceedings of the 10th International Conference on Theoretical and Methodological Issues in Machine Translation.
14. Audrey Le and Mark Przybocki. 2005. NIST 2005 machine translation evaluation official results. In Official release of automatic evaluation scores for all submissions, August.
15. Chin-Yew Lin and Franz Josef Och. 2004a. Automatic Evaluation of Machine Translation Quality Using Longest Common Subsequence and Skip-Bigram Statics. In Proceedings of the 42nd Annual Meeting of the Association for Computational Linguistics (ACL).
16. Chin-Yew Lin and Franz Josef Och. 2004b. ORANGE: a Method for Evaluating Automatic Evaluation Metrics for Machine Translation. In Proceedings of the 20th International Conference on Computational Linguistics (COLING).
17. Ding Liu and Daniel Gildea. 2005. Syntactic Features for Evaluation of Machine Translation. In Proceedings of ACL Workshop on Intrinsic and Extrinsic Evaluation Measures for MT and/or Summarization.
18. Dennis Mehay and Chris Brew. 2007. BLEUATRE: Flattening Syntactic Dependencies for MT Evaluation. In Proceedings of the 11th Conference on Theoretical and Methodological Issues in Machine Translation (TMI).
19. I. Dan Melamed, Ryan Green, and Joseph P. Turian. 2003. Precision and Recall of Machine Translation. In Proceedings of the Joint Conference on Human Language Technology and the North American Chapter of the Association for Computational Linguistics (HLT-NAACL).
20. Sonja Nießen, Franz Josef Och, Gregor Leusch, and Hermann Ney. 2000. An Evaluation Tool for Machine Translation: Fast Evaluation for MT Research. In Proceedings of the 2nd International Conference on Language Resources and Evaluation.
21. Karolina Owczarzak, Declan Groves, Josef Van Genabith, and Andy Way. 2006. Contextual Bitext-Derived Paraphrases in Automatic MT Evaluation. In Proceedings of the 7th Conference of the Association for Machine Translation in the Americas (AMTA), pages 148-155.
22. Karolina Owczarzak, Josef van Genabith, and Andy Way. 2007. Dependency-Based Automatic Evaluation for Machine Translation. In Proceedings of SSST, NAACL-HLT/AMTA Workshop on Syntax and Structure in Statistical Translation, pages 80-87.
23. Kishore Papineni, Salim Roukos, Todd Ward, and Wei-Jing Zhu. 2001. Bleu: a method for automatic evaluation of machine translation, rc22176. Technical report, IBM T. J. Watson Research Center.
24. Grazia Russo-Lassner, Jimmy Lin, and Philip Resnik. 2005. A Paraphrase-Based Approach to Machine Translation Evaluation. Technical report, University of Maryland, College Park.
25. Matthew Snover, Bonnie Dorr, Richard Schwartz, Linnea Micciulla, and John Makhoul. 2006. A Study of Translation Edit Rate with Targeted Human Annotation. In Proceedings of AMTA, pages 223-231.
26. C. Tillmann, S. Vogel, H. Ney, A. Zubiaga, and H. Sawaf. 1997. Accelerated DP based Search for Statistical Translation. In Proceedings of European Conference on Speech Communication and Technology.
27. Liang Zhou, Chin-Yew Lin, and Eduard Hovy. 2006. Reevaluating Machine Translation Results with Paraphrase Support. In Proceedings of the Conference on Empirical Methods in Natural Language Processing (EMNLP), pages 77-84.