Machine Translation Evaluation Inside QARLA

2005 International Workshop on Spoken Language Translation, IWSLT 2005

Volumn , Issue , 2005, Pages 189-196

  Giménez, Jesús ^a   Amigó, Enrique ^b   Hori, Chiori ^c  

^a UNIVERSITAT POLITÈCNICA DE CATALUNYA   (Spain)

^b UNIVERSIDAD NACIONAL DE EDUCACIÓN A DISTANCIA   (Spain)

^c CARNEGIE MELLON UNIVERSITY   (United States)

Author keywords

[No Author keywords available]

Indexed keywords

MACHINE TRANSLATION; SPEECH RECOGNITION;

EVALUATION METRICS; HUMAN ASSESSMENT; MACHINE TRANSLATION EVALUATIONS; MT EVALUATIONS; SCALING PROBLEM; SYSTEM LEVELS;

COMPUTATIONAL LINGUISTICS;

EID: 84863381342     PISSN: None     EISSN: None     Source Type: Conference Proceeding    
DOI: None     Document Type: Conference Paper

References (21)

1. Enrique Amigó, Julio Gonzalo, Anselmo Peñas, and Felisa Verdejo, “QARLA: a Framework for the Evaluation of Automatic Sumarization”, Proceedings of the 43th Annual Meeting of the Association for Computational Linguistics, 2005.
2. Franz Josef Och, Daniel Gildea, Sanjeev Khudanpur, Anoop Sarkar, Kenji Yamada, Alex Fraser, Shankar Kumar, Libin Shen, David Smith, Katherine Eng, Viren Jain, Zhen Jin and Dragomir Radev, “Final Report of the Summer Workshop on Syntax for Statistical Machine Translation”, Johns Hopkins University, 2003.
3. Enrique Amigó, Julio Gonzalo, Anselmo Peñas, and Felisa Verdejo, “Evaluating DUC 2004 with QARLA Framework”, Proceedings of the ACL’05 Workshop on Intrinsic and Extrinsic Evaluation Measures for MT and/or Summarization, 2005.
4. Yasuhiro Akiba, Marcello Federico, Noriko Kando, Hiromi Nakaiwa, Michael Paul and Jun’ichi Tsujii, “Overview of the IWSLT04 Evaluation Campaign”, Proceedings of the International Workshop on Spoken Language Translation, 2004.
5. Eugene Charniak, Kevin Knight and Kenji Yamada, “Syntax-based Language Models for Machine Translation”, Proceedings of MT SUMMIT IX, 2003.
6. Eugene Charniak, “Immediate-Head Parsing for Language Models”, Proceedings of ACL, 2001.
7. Kenji Yamada and Kevin Knight, “A Syntax-based Statistical Translation Model”, Proceedings of ACL, 2001.
8. Kishore Papineni, Salim Roukos, Todd Ward and Wei-Jing Zhu, “Bleu: a method for automatic evaluation of machine translation, IBM Research Report, RC22176”, IBM T.J. Watson Research Center, 2001.
9. George Doddington, “Automatic Evaluation of Machine Translation Quality Using N-gram Co-Occurrence Statistics”, Proc. of the 2nd Internation Conference on Human Language Technology, 2002.
10. C. Tillmann, S. Vogel, H. Ney, A. Zubiaga and H. Sawaf, “Accelerated DP based Search for Statistical Translation”, Proceedings of European Conference on Speech Communication and Technology, 1997.
11. I. Dan Melamed, Ryan Green and Joseph P. Turian, “Precision and Recall of Machine Translation”, Proceedings of HLT/NAACL, 2003.
12. Satanjeev Banerjee and Alon Lavie, “METEOR: An Automatic Metric for MT Evaluation with Improved Correlation with Human Judgments”, Proceedings of ACL Workshop on Intrinsic and Extrinsic Evaluation Measures for MT and/or Summarization, 2005.
13. Chin-Yew Lin and Franz Josef Och, “Automatic Evaluation of Machine Translation Quality Using Longest Common Subsequence and Skip-Bigram Statics”, Proceedings of ACL, 2004.
14. Bogdan Babych and Tony Hartley, “Extending the BLEU MT Evaluation Method with Frequency Weightings”, Proceedings of ACL, 2004.
15. C. Fellbaum, “WordNet. An Electronic Lexical Database”, The MIT Press, 1998.
16. Ding Liu and Daniel Gildea, “Syntactic Features for Evaluation of Machine Translation”, Proceedings of ACL Workshop on Intrinsic and Extrinsic Evaluation Measures for MT and/or Summarization, 2005.
17. Alex Kulesza and Stuart M. Shieber, “A learning approach to improving sentence-level MT evaluation”, Proceedings of the 10th International Conference on Theoretical and Methodological Issues in Machine Translation, 2004.
18. LDC, “Linguistic Data Annotation Specification: Assessment of Fluency and Adequacy in Chinese-English Translations Revision 1.0”, Linguistic Data Consortium. http://www.ldc.upenn.edu/Projects/TIDES/Translation/TransAssess02.pdf, 2002.
19. J. B. MacQueen, “Some Methods for classification and Analysis of Multivariate Observations”, Proceedings of 5th Berkeley Symposium on Mathematical Statistics and Probability, Berkeley, University of California Press, 1:281-297, 1967.
20. Matthias Eck and Chiori Hori, “Overview of the IWSLT 2005 Evaluation Campaign”, Proceedings of the International Workshop on Spoken Language Translation, 2005.
21. Michael Gamon, Anthony Aue and Martine Smets, “Sentence-Level MT evaluation without reference translations: beyond language modeling”, Proceedings of EAMT, 2005.