Exploring normalization techniques for human judgments of machine translation adequacy collected using Amazon Mechanical Turk

Workshop on Creating Speech and Language Data with Amazon's Mechanical Turk, Mturk 2010 at the 2010 Annual Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, NAACL-HLT 2010 - Proceedings

Volumn , Issue , 2010, Pages 57-61

  Denkowski, Michael ^a   Lavie, Alon ^a  

^a Carnegie Mellon University   (United States)

Author keywords

[No Author keywords available]

Indexed keywords

COMPUTATIONAL LINGUISTICS; MACHINE TRANSLATION;

AMAZON'S MECHANICAL TURKS; ASSESSMENT TASKS; HUMAN JUDGMENTS; MACHINE TRANSLATION EVALUATIONS; MACHINE TRANSLATIONS; NORMALISATION; PERFORMANCE;

COMPUTER AIDED LANGUAGE TRANSLATION;

EID: 84890495909     PISSN: None     EISSN: None     Source Type: Conference Proceeding    
DOI: None     Document Type: Conference Paper

References (5)

1. Satanjeev Banerjee and Alon Lavie. 2005. METEOR: An Automatic Metric for MT Evaluation with Improved Correlation with Human Judgments. In Proc. ACL WIEEMMTS.
2. Chris Callison-Burch, Philipp Koehn, Christof Monz, and Josh Schroeder. 2009. Findings of WMT09. In Proc. WMT09.
3. Chris Callison-Burch. 2009. Fast, cheap, and creative: Evaluating translation quality using Amazon's Mechanical Turk. In Proc. EMNLP09.
4. John Garofolo. 2001. NIST Open Machine Translation Evaluation. http://www.itl.nist.gov/iad/mig/tests/mt/.
5. N. C. Smeeton. 1985. Early History of the Kappa Statistic. In Biometrics, volume 41.