Annotating large email datasets for named entity recognition with Mechanical Turk

Workshop on Creating Speech and Language Data with Amazon's Mechanical Turk, Mturk 2010 at the 2010 Annual Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, NAACL-HLT 2010 - Proceedings

Volumn , Issue , 2010, Pages 71-79

  Lawson, Nolan ^a   Eustice, Kevin ^a   Perkowitz, Mike ^a   Yetisgen Yildiz, Meliha ^b  

^a Kiha Inc   (United States)

^b UNIVERSITY OF WASHINGTON   (United States)

Author keywords

[No Author keywords available]

Indexed keywords

LARGE DATASET; NATURAL LANGUAGE PROCESSING SYSTEMS;

AMAZON'S MECHANICAL TURKS; EXPERT ANNOTATIONS; MECHANICAL TURKS; NAMED ENTITIES; NAMED ENTITY RECOGNITION; RECOGNITION MODELS; SIMILAR MODELS;

ELECTRONIC MAIL;

EID: 84877318482     PISSN: None     EISSN: None     Source Type: Conference Proceeding    
DOI: None     Document Type: Conference Paper

References (14)

1. Michele Banko and Eric Brill. 2001. Scaling to very very large corpora for natural language disambiguation. In ACL '01:2633.
2. Chris CallisonBurch. 2009. Fast, cheap, and creative: evaluating translation quality using Amazon's Mechanical Turk. In EMNLP '09:286295.
3. Doug Downey, Matthew Broadhead, and Oren Etzioni. 2007. Locating complex named entities in web text. In IJCAI '07.
4. Ralph Grishman and Beth Sundheim. 1996. Message Understanding Conference6: a brief history. In Proceedings of the 16th conference on Computational Linguistics:466471.
5. Aniket Kittur, Ed H. Chi, and Bongwon Suh. 2008. Crowdsourcing user studies with Mechanical Turk. In Proceedings of CHI 2008.
6. Zornitsa Kozareva. 2006. Bootstrapping named entity recognition with automatically generated gazetteer lists. In Proceedings of the Eleventh Conference of the European Chapter of the Association for Computational Linguistics:1521.
7. Andrei Mikheev, Marc Moens, and Claire Grover. 1999. Named entity recognition without gazetteers. In Proceedings of the Ninth Conference of the European chapter of the Association for Computational Linguistics:18.
8. Einat Minkov, Richard C. Wang, and William W. Cohen. 2005. Extracting personal names from email: applying named entity recognition to informal text. In HTL'05:443450.
9. David Nadeau and Satoshi Sekine. 2007. A survey of named entity recognition and classification. Linguisticae Investigationes, 30(1):326.
10. Preslav Nakov. 2008. Noun compound interpretation using paraphrasing verbs: Feasibility study. In Proceedings of the 13th international conference on Artificial Intelligence: Methodology, Systems and Applications (AIMSA 2008):103-117.
11. Jason Pontin. 2007. Artificial Intelligence, With Help From the Humans. In New York Times (March 25, 2007).
12. Burr Settles. 2004. Biomedical named entity recognition using conditional random fields and rich feature sets. In Proceedings of the COLING 2004 International Joint Workshop on Natural Language Processing in Biomedicine and its Applications.
13. Rion Snow, Brendan O'Connor, Daniel Jurafsky, and Andrew Y. Ng. 2008. Cheap and fast - but is it good?: evaluating nonexpert annotations for natural language tasks. In EMNLP '08:254263.
14. Alexander Sorokin and David Forsyth. Utility data annotation with Amazon MTurk. In Proceedings of Computer Vision and Pattern Recognition Workshop at CVPR'08.