Heterogeneous Automatic MT Evaluation through Non-Parametric Metric Combinations

IJCNLP 2008 - 3rd International Joint Conference on Natural Language Processing, Proceedings of the Conference

Volumn 1, Issue , 2008, Pages 319-326

  Giḿenez, Jes us ^a  

^a UNIVERSITAT POLITÈCNICA DE CATALUNYA   (Spain)

Author keywords

[No Author keywords available]

Indexed keywords

NATURAL LANGUAGE PROCESSING SYSTEMS; PARAMETER ESTIMATION; QUALITY CONTROL;

MT EVALUATIONS; NONPARAMETRIC METHODS; NONPARAMETRICS; QUALITY DIMENSION;

SEMANTICS;

EID: 77954761860     PISSN: None     EISSN: None     Source Type: Conference Proceeding    
DOI: None     Document Type: Conference Paper

References (29)

1. Joshua Albrecht and Rebecca Hwa. 2007a. A Reexamination of Machine Learning Approaches for Sentence-Level MT Evaluation. In Proceedings of ACL, pages 880-887.
2. Joshua Albrecht and Rebecca Hwa. 2007b. Regression for Sentence-LevelMT Evaluation with Pseudo References. In Proceedings of ACL, pages 296-303.
3. Enrique Amiǵo, Julio Gonzalo, Anselmo Pẽnas, and Felisa Verdejo. 2005. QARLA: a Framework for the Evaluation of Automatic Sumarization. In Proceedings of the 43th Annual Meeting of the Association for Computational Linguistics.
4. Enrique Amiǵo, Jeśus Giḿenez, Julio Gonzalo, and Llúis M`arquez. 2006. MT Evaluation: Human-Like vs. Human Acceptable. In Proceedings of COLING-ACL06.
5. Satanjeev Banerjee and Alon Lavie. 2005. METEOR: An Automatic Metric for MT Evaluation with Improved Correlation with Human Judgments. In Proceedings of ACL Workshop on Intrinsic and Extrinsic Evaluation Measures for Machine Translation and/or Summarization.
6. Chris Callison-Burch, Miles Osborne, and Philipp Koehn. 2006. Re-evaluating the Role of BLEU inMachine Translation Research. In Proceedings of EACL.
7. Simon Corston-Oliver, Michael Gamon, and Chris Brockett. 2001. A Machine Learning Approach to the Automatic Evaluation of Machine Translation. In Proceedings of ACL, pages 140-147.
8. Christopher Culy and Susanne Z. Riehemann. 2003. The Limits of N-gram Translation Evaluation Metrics. In Proceedings of MT-SUMMIT IX, pages 1-8.
9. George Doddington. 2002. Automatic Evaluation of Machine Translation Quality Using N-gram Co- Occurrence Statistics. In Proceedings of the 2nd IHLT.
10. Michael Gamon, Anthony Aue, and Martine Smets. 2005. Sentence-Level MT evaluation without reference translations: beyond language modeling. In Proceedings of EAMT.
11. Jeśus Giḿenez and Llúis M`arquez. 2007. Linguistic Features for Automatic Evaluation of Heterogeneous MT Systems. In Proceedings of the ACL Workshop on Statistical Machine Translation.
12. Jeśus Giḿenez. 2007. IQMT v 2.1. Technical Manual. Technical report, TALP Research Center. LSI Department. http://www.lsi.upc.edu/nlp/IQMT/- IQMT.v2.1.pdf.
13. David Kauchak and Regina Barzilay. 2006. Paraphrasing for Automatic Evaluation. In Proceedings of NLHNAACL.
14. Philipp Koehn and Christof Monz. 2006. Manual and Automatic Evaluation ofMachine Translation between European Languages. In Proceedings of the Workshop on Statistical Machine Translation, pages 102-121.
15. Alex Kulesza and Stuart M. Shieber. 2004. A learning approach to improving sentence-level MT evaluation. In Proceedings of the 10th International Conference on Theoretical and Methodological Issues in Machine Translation.
16. Audrey Le and Mark Przybocki. 2005. NIST 2005 machine translation evaluation official results. Technical report, NIST, August.
17. Chin-Yew Lin and Franz Josef Och. 2004a. Automatic Evaluation of Machine Translation Quality Using Longest Common Subsequence and Skip-Bigram Statics. In Proceedings of ACL.
18. Chin-Yew Lin and Franz Josef Och. 2004b. ORANGE: a Method for Evaluating Automatic Evaluation Metrics for Machine Translation. In Proceedings of COLING.
19. Ding Liu and Daniel Gildea. 2005. Syntactic Features for Evaluation of Machine Translation. In Proceedings of ACL Workshop on Intrinsic and Extrinsic EvaluationMeasures forMachine Translation and/or Summarization.
20. Ding Liu and Daniel Gildea. 2007. Source-Language Features and Maximum Correlation Training for Machine Translation Evaluation. In Proceedings of the 2007Meeting of the North American chapter of the Association for Computational Linguistics (NAACL-07).
21. Dennis Mehay and Chris Brew. 2007. BLEUATRE: Flattening Syntactic Dependencies forMT Evaluation. In Proceedings of the 11th Conference on Theoretical andMethodological Issues in Machine Translation (TMI).
22. I. Dan Melamed, Ryan Green, and Joseph P. Turian. 2003. Precision and Recall of Machine Translation. In Proceedings of HLT/NAACL.
23. Sonja Nießen, Franz Josef Och, Gregor Leusch, and Hermann Ney. 2000. Evaluation Tool for Machine Translation: Fast Evaluation for MT Research. In Proceedings of the 2nd LREC.
24. Karolina Owczarzak, Declan Groves, Josef Van Genabith, and Andy Way. 2006. Contextual Bitext- Derived Paraphrases in Automatic MT Evaluation. In Proceedings of the 7th Conference of the Association for Machine Translation in the Americas (AMTA), pages 148-155.
25. Karolina Owczarzak, Josef van Genabith, and Andy Way. 2007. Dependency-Based Automatic Evaluation for Machine Translation. In Proceedings of SSST, NAACL-HLT/AMTA Workshop on Syntax and Structure in Statistical Translation.
26. Kishore Papineni, Salim Roukos, Todd Ward, and Wei- Jing Zhu. 2001. Bleu: a method for automatic evaluation of machine translation, RC22176, IBM. Technical report, IBM T.J. Watson Research Center.
27. Chris Quirk. 2004. Training a Sentence-Level Machine Translation Confidence Metric. In Proceedings of LREC.
28. Matthew Snover, Bonnie Dorr, Richard Schwartz, Linnea Micciulla,, and John Makhoul. 2006. A Study of Translation Edit Rate with Targeted Human Annotation. In Proceedings of AMTA, pages 223-231.
29. Liang Zhou, Chin-Yew Lin, and Eduard Hovy. 2006. Re-evaluating Machine Translation Results with Paraphrase Support. In Proceedings of EMNLP.