Quality management on Amazon Mechanical Turk

Workshop Proceedings - Human Computation Workshop 2010, HCOMP2010

Volumn , Issue , 2010, Pages 64-67

  Ipeirotis, Panagiotis G ^a   Provost, Foster ^a   Wang, Jing ^a  

^a NEW YORK UNIVERSITY   (United States)

Author keywords

[No Author keywords available]

Indexed keywords

COST-SENSITIVE CLASSIFICATION; CROWDSOURCING; ERROR RATE; LOW QUALITIES; MECHANICAL TURKS; SPAMMERS;

QUALITY ASSURANCE; REDUNDANCY;

ALGORITHMS;

EID: 77956245055     PISSN: None     EISSN: None     Source Type: Conference Proceeding    
DOI: 10.1145/1837885.1837906     Document Type: Conference Paper

References (4)

1. Carpenter, B. Multilevel bayesian models of categorical data annotation. Available at http://lingpipe-blog.com/lingpipe-white-papers/, 2008.
2. Dawid, A. P., and Skene, A. M. Maximum likelihood estimation of observer error-rates using the EM algorithm. Applied Statistics 28, 1 (Sept. 1979), 20-28.
3. Malone, T. W., Laubacher, R., and Dellarocas, C. Harnessing crowds: Mapping the genome of collective intelligence. Available at http://ssrn.com/ abstract=1381502, 2010.
4. Raykar, V. C., Yu, S., Zhao, L. H., Valadez, G. H., Florin, C., Bogoni, L., and Moy, L. Learning from crowds. Journal of Machine Learning Research 11 (Apr. 2010), 1297-1322.