On the Robustness of Syntactic and Semantic Features for Automatic MT Evaluation

EACL 2009 - 4th Workshop on Statistical Machine Translation, Proceedings of theWorkshop

Volumn , Issue , 2009, Pages 250-258

  Giḿenez, Jesus ^a  

^a UNIVERSITAT POLITÈCNICA DE CATALUNYA   (Spain)

Author keywords

[No Author keywords available]

Indexed keywords

COMPUTATIONAL LINGUISTICS; SEMANTICS; TRANSLATION (LANGUAGES);

EMPIRICAL STUDIES; LINGUISTIC ANALYSIS; LOW QUALITIES; MT EVALUATIONS; PERFORMANCE; SEMANTIC FEATURES; SEMANTICS INFORMATION; SPEECH TRANSLATION; SYNTACTIC FEATURES; SYNTACTIC INFORMATION;

SYNTACTICS;

EID: 84906928138     PISSN: None     EISSN: None     Source Type: Conference Proceeding    
DOI: None     Document Type: Conference Paper

References (22)

1. Enrique Amiǵo, Julio Gonzalo, Anselmo Pe nas, and Felisa Verdejo. 2005. QARLA: a Framework for the Evaluation of Automatic Sumarization. In Pro-ceedings of the 43rd AnnualMeeting of the Association for Computational Linguistics (ACL).
2. Enrique Amiǵo, Jeśus Giḿenez, Julio Gonzalo, and Llúis M`arquez. 2006. MT Evaluation: Human- Like vs. Human Acceptable. In Proceedings of the Joint 21st International Conference on Computational Linguistics and the 44th Annual Meeting of the Association for Computational Linguistics (COLING-ACL).
3. Satanjeev Banerjee and Alon Lavie. 2005. METEOR: An Automatic Metric for MT Evaluation with Improved Correlation with Human Judgments. In Proceedings of ACL Workshop on Intrinsic and Extrinsic EvaluationMeasures for MT and/or Summarization.
4. Johan Bos, Stephen Clark, Mark Steedman, James R. Curran, and Julia Hockenmaier. 2004. Wide- Coverage Semantic Representations from a CCG Parser. In Proceedings of the 20th International Conference on Computational Linguistics (COLING), pages 1240-1246.
5. Johan Bos. 2005. Towards Wide-Coverage Semantic Interpretation. In Proceedings of the Sixth International Workshop on Computational Semantics, pages 42-53.
6. Stephen Clark and James R. Curran. 2004. Parsing the WSJ using CCG and Log-Linear Models. In Proceedings of the 42nd Annual Meeting of the Association for Computational Linguistics (ACL), pages 104-111.
7. George Doddington. 2002. Automatic Evaluation of Machine Translation Quality Using N-gram Co- Occurrence Statistics. In Proceedings of the 2nd Internation Conference on Human Language Technology, pages 138-145.
8. Jeśus Giḿenez and Llúis M`arquez. 2007. Linguistic Features for Automatic Evaluation of Heterogeneous MT Systems. In Proceedings of the ACL Workshop on Statistical Machine Translation, pages 256-264.
9. Hans Kamp. 1981. A Theory of Truth and Semantic Representation. In J.A.G. Groenendijk, T.M.V. Janssen,, andM.B.J. Stokhof, editors, FormalMethods in the Study of Language, pages 277-322, Amsterdam. Mathematisch Centrum.
10. LDC. 2005. Linguistic Data Annotation Specification: Assessment of Adequacy and Fluency in Translations. Revision 1.5. Technical report, Linguistic Data Consortium. http://www.ldc.upenn.edu/- Projects/TIDES/Translation/TransAssess04.pdf.
11. Chin-Yew Lin and Franz Josef Och. 2004a. Automatic Evaluation of Machine Translation Quality Using Longest Common Subsequence and Skip- Bigram Statics. In Proceedings of the 42nd Annual Meeting of the Association for Computational Linguistics (ACL).
12. Chin-Yew Lin and Franz Josef Och. 2004b. ORANGE: a Method for Evaluating Automatic Evaluation Metrics for Machine Translation. In Proceedings of the 20th International Conference on Computational Linguistics (COLING).
13. Ding Liu and Daniel Gildea. 2005. Syntactic Features for Evaluation of Machine Translation. In Proceedings of ACL Workshop on Intrinsic and Extrinsic EvaluationMeasures for MT and/or Summarization.
14. Dennis Mehay and Chris Brew. 2007. BLEUATRE: Flattening Syntactic Dependencies for MT Evaluation. In Proceedings of the 11th Conference on Theoretical and Methodological Issues in Machine Translation (TMI).
15. I. Dan Melamed, Ryan Green, and Joseph P. Turian. 2003. Precision and Recall of Machine Translation. In Proceedings of the Joint Conference on Human Language Technology and the North American Chapter of the Association for Computational Linguistics (HLT-NAACL).
16. Sonja Nießen, Franz Josef Och, Gregor Leusch, and Hermann Ney. 2000. An Evaluation Tool for Machine Translation: Fast Evaluation forMT Research. In Proceedings of the 2nd International Conference on Language Resources and Evaluation.
17. Karolina Owczarzak, Josef van Genabith, and Andy Way. 2007. Dependency-Based Automatic Evaluation for Machine Translation. In Proceedings of SSST, NAACL-HLT/AMTA Workshop on Syntax and Structure in Statistical Translation, pages 80-87.
18. Kishore Papineni, Salim Roukos, ToddWard, andWei- Jing Zhu. 2001. Bleu: a method for automatic evaluation of machine translation, RC22176. Technical report, IBM T.J. Watson Research Center.
19. Michael Paul. 2006. Overview of the IWSLT 2006 Evaluation Campaign. In Proceedings of the International Workshop on Spoken Language Translation, pages 1-15.
20. Maja Popovic and Hermann Ney. 2007. Word Error Rates: Decomposition over POS classes and Applications for Error Analysis. In Proceedings of the Second Workshop on Statistical Machine Translation, pages 48-55, Prague, Czech Republic, June. Association for Computational Linguistics.
21. Matthew Snover, Bonnie Dorr, Richard Schwartz, Linnea Micciulla, and John Makhoul. 2006. A Study of Translation Edit Rate with Targeted Human Annotation. In Proceedings of AMTA, pages 223-231.
22. C. Tillmann, S. Vogel, H. Ney, A. Zubiaga, and H. Sawaf. 1997. Accelerated DP based Search for Statistical Translation. In Proceedings of European Conference on Speech Communication and Technology.