Quality control for comparison microtasks

Proceedings of the ACM SIGKDD International Conference on Knowledge Discovery and Data Mining

Volumn , Issue , 2012, Pages 15-21

  Venetis, Petros ^a   Garcia Molina, Hector ^a  

^a STANFORD UNIVERSITY   (United States)

Author keywords

[No Author keywords available]

Indexed keywords

CONTROL MECHANISM; CROWDSOURCING; ERROR MASKING; MECHANICAL TURKS; MICROTASKS; STUDY QUALITIES; TASK DIFFICULTY;

DATA MINING; QUALITY CONTROL;

QUALITY ASSURANCE;

EID: 84875158877     PISSN: None     EISSN: None     Source Type: Conference Proceeding    
DOI: 10.1145/2442657.2442660     Document Type: Conference Paper

References (10)

1. Amazon's Mechanical Turk. www.mturk.com, June 2012.
2. Paul Heymann and Hector Garcia-Molina. Turkalytics: Analytics for Human Computation. WWW, pages 477-486, 2011.
3. John Joseph Horton and Lydia B. Chilton. The Labor Economics of Paid Crowdsourcing. EC, pages 209-218, 2010.
4. David R. Karger, Sewoong Oh, and Devavrat Shah. Iterative Learning for Reliable Crowdsourcing Systems. In NIPS, pages 1953-1961, 2011.
5. oDesk. www.odesk.com, June 2012.
6. Drazen Prelec. A Bayesian Truth Serum for Subjective Data. Science, 306(5695):462-466, 2004.
7. Aaron D. Shaw, John J. Horton, and Daniel L. Chen. Designing Incentives for Inexpert Human Raters. CSCW, pages 275-284. ACM, 2011.
8. James Surowiecki. The Wisdom of Crowds. Anchor, 2005.
9. Petros Venetis, Hector Garcia-Molina, Kerui Huang, and Neoklis Polyzotis. Max Algorithms in Crowdsourcing Environments. In WWW, pages 989-998, 2012.
10. Jeroen Vuurens, Arjen P. de Vries, and Carsten Eickhoff. How Much Spam Can You Take? An Analysis of Crowdsourcing Results to Increase Accuracy. CIR, 2011.