Cost-optimal validation mechanisms and cheat-detection for crowdsourcing platforms

Proceedings - 2011 5th International Conference on Innovative Mobile and Internet Services in Ubiquitous Computing, IMIS 2011

Volumn , Issue , 2011, Pages 316-321

  Hirth, Matthias ^a   Hoßfeld, Tobias ^a   Tran Gia, Phuoc ^a  

^a UNIVERSITY OF WÜRZBURG   (Germany)

Author keywords

cheat detection mechanism; crowdsourcing

Indexed keywords

CHEAT DETECTION; CHEAT-DETECTION MECHANISM; CROWDSOURCING; KNOWLEDGE BASE; WORK FORCE;

HUMAN COMPUTER INTERACTION; INTERNET; KNOWLEDGE BASED SYSTEMS; TELECOMMUNICATION NETWORKS; UBIQUITOUS COMPUTING;

WEB SERVICES;

EID: 80052379038     PISSN: None     EISSN: None     Source Type: Conference Proceeding    
DOI: 10.1109/IMIS.2011.91     Document Type: Conference Paper

References (12)

1. J. Howe. (2006, Jun.) The Rise of Crowdsourcing. [Online]. Available: http://www.wired.com/wired/archive/14.06/crowds.html
2. O. Alonso, D. E. Rose, and B. Stewart, "Crowdsourcing for Relevance Evaluation," SIGIR Forum, vol. 42, no. 2, 2008.
3. P. Hsueh, P. Melville, and V. Sindhwani, "Data Quality from Crowd-sourcing: A Study of Annotation Selection Criteria," in Proceedings of the NAACL HLT Workshop on Active Learning for Natural Language Processing, Boulder, Colorado, USA, May 2009.
4. InnoCentive, Inc., "InnoCentive," www.innocentive.com [Accessed Mar. 07, 2011].
5. A. Kittur, E. H. Chi, and B. Suh, "Crowdsourcing User Studies with Mechanical Turk," in Proceeding of the ACM SIGCHI Conference on Human Factors in Computing Systems, Florence, Italy, Apr. 2008.
6. A. Kittur and R. E. Kraut, "Harnessing the Wisdom of Crowds in Wikipedia: Quality through Coordination," in Proceedings of the ACM Conference on Computer Supported Cooperative Work, San Diego, California, USA, Nov. 2008.
7. C. Eickhoff and A. de Vries, "How Crowdsourcable is Your Task?" in Proceedings of the ACM WSDM Workshop on Crowdsourcing for Search and Data Mining, Hong Kong, China, Feb. 2011.
8. K. Chen, C. Chang, C. Wu, Y. Chang, C. Lei, and C. Sinica, "Quadrant of Euphoria: A Crowdsourcing Platform for QoE Assessment," IEEE Network, vol. 24, no. 2, 2010.
9. L. Von Ahn and L. Dabbish, "Labeling Images with a Computer Game," in Proceedings of the ACM SIGCHI Conference on Human Factors in Computing Systems, Vienna, Austria, Apr. 2004.
10. T. Moore and R. Clayton, "Evaluating the Wisdom of Crowds in assessing Phishing Websites," Financial Cryptography and Data Security, 2009.
11. P. G. Ipeirotis, F. Provost, and J. Wang, "Quality Management on Amazon Mechanical Turk," in Proceedings of the ACM SIGKDD Workshop on Human Computation, Washington, DC, USA, Jul. 2010.
12. M. Hirth, T. Hoßfeld, and P. Tran-Gia, "Cheat-Detection Mechanisms for Crowdsourcing," University of Würzburg, Tech. Rep. 474, 2010.