Machine translation evaluation versus quality estimation

Machine Translation

Volumn 24, Issue 1, 2010, Pages 39-50

  Specia, Lucia ^a   Raj, Dhwaj ^b   Turchi, Marco ^c  

^a UNIVERSITY OF WOLVERHAMPTON   (United Kingdom)

^b INDIAN INSTITUTE OF INFORMATION TECHNOLOGY   (India)

^c JOINT RESEARCH CENTRE   (Italy)

Author keywords

Confidence estimation; Machine translation evaluation; Quality estimation

Indexed keywords

COMMONLY USED; CONFIDENCE ESTIMATION; DE FACTO; EVALUATION METRICS; GOOD CORRELATIONS; HUMAN EVALUATION; MACHINE TRANSLATION EVALUATIONS; MACHINE TRANSLATIONS; PREDICTION TASKS; QUALITY ESTIMATION;

ESTIMATION; INFORMATION THEORY;

QUALITY CONTROL;

EID: 78650047783     PISSN: 09226567     EISSN: None     Source Type: Journal    
DOI: 10.1007/s10590-010-9077-2     Document Type: Article

References (27)

1. Albrecht J, Hwa R (2007a) A re-examination of machine learning approaches for sentence-level MT evaluation. In: 45th meeting of the association for computational linguistics, Prague, pp 880-887
2. Albrecht J, Hwa R (2007b) Regression for sentence-level MT evaluation with pseudo references. In: 45th meeting of the association for computational linguistics, Prague, pp 296-303
3. Blatz J, Fitzgerald E, Foster G, Gandrabur S, Goutte C, Kulesza A, Sanchis A, Ueffing N (2003) Confidence estimation for machine translation. Technical report. Johns Hopkins University, Baltimore
4. Blatz J, Fitzgerald E, Foster G, Gandrabur S, Goutte C, Kulesza A, Sanchis A, Ueffing N (2004) Confidence estimation for machine translation. In: 20th coling, Geneva, pp 315-321
5. Callison-Burch C, Fordyce C, Koehn P, Monz C, Schroeder J (2008) Further meta-evaluation of machine translation. In: 3rd workshop on statistical machine translation, Columbus, pp 70-106
6. Callison-Burch C, Koehn P, Monz C, Schroeder J (2009) Findings of the 2009 workshop on statistical machine translation. In: 4th workshop on statistical machine translation, Athens, pp 1-28
7. Chang C, Lin C (2001) LIBSVM: a library for support vector machines. Software available at http://www.csie.ntu.edu.tw/cjlin/libsvm
8. Cohen J (1960) A coefficient of agreement for nominal scales. Educ Psychol Meas 20(1):37-46
9. Doddington G (2002) Automatic evaluation of machine translation quality using n-gram co-occurrence statistics. In: Conference on human language technology, San Diego, pp 138-145
10. Gamon M, Aue A, Smets M (2005) Sentence-level MT evaluation without reference translations: beyond language modeling. In: 10th meeting of the European association for machine translation, Budapest
11. Gandrabur S, Foster G(2003) Confidence estimation for translation prediction. In: 7th conference on natural language learning, Edmonton, pp 95-102
12. Gimenez J, Marquez L (2008) A smorgasbord of features for automatic MT evaluation. In: 3rd workshop on statistical machine translation, Columbus, OH, pp 195-198
13. Joachims T (1999) Making large-scale SVM learning practical. In: Schoelkopf B, Burges CJC, Smola AJ (eds) Advances in Kernel methods - support vector learning. MIT Press, Cambridge
14. Johnson H, Sadat F, Foster G, Kuhn R, Simard M, Joanis E, Larkin S (2006) Portage: with smoothed phrase tables and segment choice models. In:Workshop on statistical machine translation, New York, pp 134-137
15. Kääriäinen M(2009) Sinuhe - statistical machine translation using a globally trained conditional exponential family translation model. In: Conference on empirical methods in natural language processing, Singapore, pp 1027-1036
16. Kadri Y, Nie JY (2006) Improving query translation with confidence estimation for cross language information retrieval. In: 15th ACM international conference on information and knowledge management, Arlington, pp 818-819
17. Koehn P (2004) Statistical significance tests formachine translation evaluation. In: Conference on empirical methods in natural language processing, Barcelona
18. Lavie A, Agarwal A (2007) METEOR: an automatic metric for MT evaluation with high levels of correlation with human judgments. In: 2nd workshop on statistical machine translation, Prague, Czech Republic, pp 228-231
19. Lin CY, Och FJ (2004) ORANGE: a method for evaluating automatic evaluation metrics for machine translation. In: Coling-2004, Geneva, pp 501-507
20. Pado S, Galley M, Jurafsky D, Manning CD (2009) Textual entailment features for machine translation evaluation. In: 4th workshop on statistical machine translation, Athens, pp 37-41
21. Papineni K, Roukos S, Ward T, Zhu W (2002) Bleu: a method for automatic evaluation of machine translation. In: 40th meeting of the association for computational linguistics, Morristown, pp 311-318
22. Quirk CB (2004) Training a sentence-level machine translation confidence measure. In: 4th language resources and evaluation conference, Lisbon, pp 825-828
23. Saunders C (2008) Application of Markov approaches to SMT. Technical report. SMART Project Deliverable 2.2
24. Simard M, Cancedda N, Cavestro B, Dymetman M, Gaussier E, Goutte C, Yamada K (2005) Translating with non-contiguous phrases. In: Conference on empirical methods in natural language processing, Vancouver, pp 755-762
25. Snover M, Dorr B, Schwartz R, Micciulla L, Makhoul J (2006) A study of translation edit rate with targeted human annotation. In: Conference of the 7th association for machine translation in the Americas, Cambridge, MA, pp 223-231
26. Specia L, Turchi M, Cancedda N, Dymetman M, Cristianini N (2009) Estimating the sentence-level quality of machine translation systems. In: 13th meeting of the European association for machine translation, Barcelona
27. Ueffing N, Ney H (2005) Application of word-level confidence measures in interactive statistical machine translation. In: 10th meeting of the European association for machine translation, Budapest, pp 262-270