Improving evaluation of machine translation quality estimation

ACL-IJCNLP 2015 - 53rd Annual Meeting of the Association for Computational Linguistics and the 7th International Joint Conference on Natural Language Processing of the Asian Federation of Natural Language Processing, Proceedings of the Conference

Volumn 1, Issue , 2015, Pages 1804-1813

  Graham, Yvette ^a  

^a TRINITY COLLEGE DUBLIN   (Ireland)

Author keywords

[No Author keywords available]

Indexed keywords

COMPUTATIONAL LINGUISTICS; CORRELATION METHODS; FORECASTING; NATURAL LANGUAGE PROCESSING SYSTEMS; TRANSLATION (LANGUAGES);

AREAS OF CONCERNS; GOLD STANDARDS; LABEL DISTRIBUTION; MACHINE TRANSLATIONS; PEARSON CORRELATION; QUALITY ESTIMATION; SIGNIFICANCE TESTING; SYSTEM RANKINGS;

QUALITY CONTROL;

EID: 84943798555     PISSN: None     EISSN: None     Source Type: Conference Proceeding    
DOI: 10.3115/v1/p15-1174     Document Type: Conference Paper

References (11)

1. J. Blatz, E. Fitzgerald, G. Foster, S. Gandrabur, C. Goutte, A. Kulesza, A. Sanchis, and N. Ueffing. 2004. Confidence estimation for machine translation. In Proceedings of the 20th international conference on Computational Linguistics, pages 315-321. Association for Computational Linguistics.
2. O. Bojar, C. Buck, C. Federmann, B. Haddow, P. Koehn, J. Leveling, C. Monz, P. Pecina, M. Post, H. Saint-Amand, R. Soricut, L. Specia, and A. Tamchyna. 2014. Findings of the 2014 Workshop on Statistical Machine Translation. In Proc. 9th Wkshp. Statistical Machine Translation, Baltimore, MA. Association for Computational Linguistics.
3. Y. Graham and T. Baldwin. 2014. Testing for significance of increased correlation with human judgment. In Proceedings of the Conference on Empirical Methods in Natural Language Processing, pages 172-176, Doha, Qatar. Association for Computational Linguistics.
4. Y. Graham, N. Mathur, and T. Baldwin. 2014. Randomized significance tests in machine translation. In Proceedings of the ACL 2014 Ninth Workshop on Statistical Machine Translation, pages 266-274. Association for Computational Linguistics.
5. Yvette Graham, Nitika Mathur, and Timothy Baldwin. 2015. Accurate evaluation of segment-level machine translation metrics. In Proceedings of the 2015 Conference of the North American Chapter of the Association for Computational Linguistics Human Language Technologies, Denver, Colorado.
6. P. Koehn. 2004. Statistical significance tests for machine translation evaluation. In Proc. of Empirical Methods in Natural Language Processing, pages 388-395, Barcelona, Spain. Association for Computational Linguistics.
7. E. Moreau and C. Vogel. 2014. Limitations of mt quality estimation supervised systems: The tails prediction problem. In Proceedings of COLING 2014, the 25th International Conference on Computational Linguistics, pages 2205-2216.
8. M. Snover, N. Madnani, B.J. Dorr, and R. Schwartz. 2009. Fluency, adequacy, or hter?: exploring different human judgments with a tunable mt metric. In Proceedings of the Fourth Workshop on Statistical Machine Translation, pages 259-268. Association for Computational Linguistics.
9. L. Specia, M. Turchi, N. Cancedda, M. Dymetman, and N. Cristianini. 2009. Estimating the sentence-level quality of machine translation systems. In 13th Conference of the European Association for Machine Translation, pages 28-37.
10. J.H. Steiger. 1980. Tests for comparing elements of a correlation matrix. Psychological Bulletin, 87(2):245.
11. E.J. Williams. 1959. Regression analysis, volume 14. Wiley New York.