Preliminary experience with Amazon's Mechanical Turk for annotating medical named entities

Workshop on Creating Speech and Language Data with Amazon's Mechanical Turk, Mturk 2010 at the 2010 Annual Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, NAACL-HLT 2010 - Proceedings

Volumn , Issue , 2010, Pages 180-183

  Yetisgen Yildiz, Meliha ^a   Solti, Imre ^a   Xia, Fei ^a   Halgrim, Scott Russell ^a  

^a UNIVERSITY OF WASHINGTON   (United States)

Author keywords

[No Author keywords available]

Indexed keywords

AMAZON'S MECHANICAL TURKS; CLINICAL TRIAL; DOMAIN EXPERTS; ENTITY-TYPES; GOLD STANDARDS; LABORATORY TEST; MECHANICAL TURKS; MEDICAL CONDITIONS; MEDICAL LABORATORIES; NAMED ENTITIES;

NATURAL LANGUAGE PROCESSING SYSTEMS;

EID: 84877354135     PISSN: None     EISSN: None     Source Type: Conference Proceeding    
DOI: None     Document Type: Conference Paper

References (7)

1. Chris Callison-Burch. 2009. Fast, Cheap, and Creative: Evaluating Translation Quality Using Amazon's Mechanical Turk. In Proceedings of EMNLP'09.
2. Melonie Heron, Donna L. Hoyert, Sherry L. Murphy, Jiaquan Xu, Kenneth D. Kochanek, and Betzaida Tejada-Vera. 2009. Deaths: Final data for 2006. National Vital Statistics Reports, 57:14.
3. Aniket Kittur, Ed H. Chi, and Bongwon Suh. 2008. Crowdsourcing User Studies with Mechanical Turk. In Proceedings of CHI'08.
4. Preslav Nakov. 2008. Noun compound interpretation using paraphrasing verbs: Feasibility study. In Proceedings of the 13th international conference on Artificial Intelligence: Methodology, Systems and Applications (AIMSA 2008), 103-117.
5. Philip V. Ogren. 2006. Knowtator: a protégé plug-in for annotated corpus construction. In Proceedings NAACL HLT'06, 273-275.
6. Rion Snow, Brendan O'Connor, Daniel Jurafsky and Andrew Y. Ng. 2008. Cheap and Fast - But is it Good? Evaluating Non-Expert Annotations for Natural Language Tasks. In Proceedings of EMNLP'08, 254-263.
7. Alexander Sorokin and David Forsyth. Utility data annotation with Amazon Mechanical Turk. In Proceedings of Computer Vision and Pattern Recognition Workshop at CVPR'08.