Sensitivity of automated MT evaluation metrics on higher quality MT output: BLEU vs task-based evaluation methods

Proceedings of the 6th International Conference on Language Resources and Evaluation, LREC 2008

Volumn , Issue , 2008, Pages 2133-2136

  Babych, Bogdan ^a   Hartley, Anthony ^a  

^a UNIVERSITY OF LEEDS   (United Kingdom)

Author keywords

[No Author keywords available]

Indexed keywords

AUTOMATION;

AUTOMATED PROCESS; EVALUATION METHODS; LANGUAGE LEVELS; LEXICAL LEVEL; MT EVALUATIONS; NAMED ENTITY RECOGNITION; PERFORMANCE BASED; TASK-BASED;

QUALITY CONTROL;

EID: 85037541783     PISSN: None     EISSN: None     Source Type: Conference Proceeding    
DOI: None     Document Type: Conference Paper

References (13)

1. Babych, Bogdan and Hartley, Anthony. (2004). Comparative Evaluation of Automatic Named Entity Recognition from Machine Translation Output. In IJCNLP Workshop on Named Entity Recognition for Natural Language Processing Applications.
2. th International Conference on Computational Linguistics.
3. Babych, Bogdan and Hartley, Anthony and Elliott, Debbie. (2005) Estimating the predictive power of n-gram MT evaluation metrics across language and text types. In MT Summit X.
4. Babych, Bogdan and Hartley, Anthony and Sharoff, Serge. (2007b). Translating from under-resourced languages: comparing direct transfer against pivot translation. In MT Summit XI, Copenhagen, Denmark.
5. th Conference of the European Chapter of the Association for Computational Linguistics) (EACL06)
6. Cunningham, H. and Maynard, D and Bontcheva, K. and Tablan V. (2002). GATE: A Framework and Graphical Development Environment for Robust NLP Tools and Applications. In Proceedings of the 40th Anniversary Meeting of the Association for Computational Linguistics (ACL'02). Philadelphia. 2002.
7. Hutchins, W. John and Somers, Harold. (1992). An introduction to machine translation. London: Academic Press.
8. Marcu, Daniel and Wang, Wei and Echihabi, Abdessamad and Knight, Kevin. (2006). SPMT: Statistical Machine Translation with Syntactified Target Language Phrases. In Proceedings of the 2006 Conference on Empirical Methods in Natural Language Processing.
9. Papineni Kishore and Roukos Salim and Ward Todd and Zhu WeiJing. (2002). Bleu: a method for automatic evaluation of machine translation. In ACL 2002: the 40th Annual Meeting of the Association for Computational Linguistics.
10. Popescu-Belis, Andrei. (2007). The place of automatic evaluation metrics in external quality models for machine translation. In MT Summit XI Workshop: Automatic procedures in MT evaluation, Copenhagen.
11. Rajman Martin and Hartley Anthony. (2001). Automatically predicting MT systems ranking compatible with fluency adequacy and Informativeness scores. In: 4th ISLE Workshop on MT Evaluation, MT Summit VIII
12. Thurmair, Gregor. (2007). Automatic evaluation in MT system production. In MT Summit XI Workshop: Automatic procedures in MT evaluation, Copenhagen.
13. White J. and O'Connell T. and O'Mara F. (1994). The ARPA MT evaluation methodologies: evolution lessons and future approaches. In 1st Conference of the Association for Machine Translation in the Americas.