Competitive game designs for improving the cost effectiveness of crowdsourcing

CIKM 2014 - Proceedings of the 2014 ACM International Conference on Information and Knowledge Management

Volumn , Issue , 2014, Pages 1469-1478

  Rokicki, Markus ^a   Chelaru, Sergiu ^a   Zerr, Sergej ^a   Siersdorfer, Stefan ^a  

^a L3S RESEARCH CENTER   (Germany)

Author keywords

Competitions; Crowdsourcing; Lotteries; Reward schemes

Indexed keywords

BUDGET CONTROL; COMPETITION; COST EFFECTIVENESS; KNOWLEDGE MANAGEMENT; PETROLEUM RESERVOIR EVALUATION; SEMANTICS;

AMAZON MECHANICAL TURKS; COMPETITIVE GAMES; EXPERIMENTAL EVALUATION; LARGE AMOUNTS OF DATA; LOTTERIES; MACHINE LEARNING MODELS; REWARD SCHEMES; SEMANTIC ANNOTATIONS;

CROWDSOURCING;

EID: 84937598773     PISSN: None     EISSN: None     Source Type: Conference Proceeding    
DOI: 10.1145/2661829.2661946     Document Type: Conference Paper

References (29)

1. GamifIR '14: 1st International Workshop on Gamification for Information Retrieval, NY, USA.
2. TREC Crowdsourcing Task 2013. https://sites.google.com/site/treccrowd/home.
3. O. Alonso and R. Baeza-Yates. Design and implementation of relevance assessments using crowdsourcing. In ECIR'11.
4. O. Alonso and S. Mizzaro. Using crowdsourcing for TREC relevance assessment. Information Processing and Management, 2012.
5. N. Archak. Money, glory and cheap talk: Analyzing strategic behavior of contestants in simultaneous crowdsourcing contests on topcoder.com. In WWW'10.
6. N. Archak and A. Sundararajan. Optimal design of crowdsourcing contests. In ICIS'09.
7. R. Cavallo and S. Jain. Efficient crowdsourcing contests. In AAMAS'12.
8. R. Cavallo and S. Jain. Winner-take-all crowdsourcing contests with stochastic production. In HCOMP'13.
9. L. E. Celis, S. Roy, and V. Mishra. Lottery-based payment mechanism for microtasks. In HCOMP'13.
10. D. DiPalantino and M. Vojnovic. Crowdsourcing and all-pay auctions. In EC'09.
11. C. Eickhoff, C. G. Harris, A. P. de Vries, and P. Srinivasan. Quality through flow and immersion: Gamifying crowdsourced relevance assessments. In SIGIR'12.
12. J. He, M. Bron, L. Azzopardi, and A. de Vries. Studying user browsing behavior through gamified search tasks. In GamifIR '14.
13. H. Jiang and S. Matsubara. Improving crowdsourcing efficiency based on division strategy. WI'12.
14. J. Joukhador, A. Blaszczynski, and F. Maccallum. Superstitious beliefs in gambling among problem and non-problem gamblers: Preliminary data. Journal of Gambling Studies, 2004.
15. G. Kazai. In search of quality in crowdsourcing for search engine evaluation. In ECIR'11.
16. G. Kazai, J. Kamps, and N. Milic-Frayling. Worker types and personality traits in crowdsourcing relevance labels. In CIKM'11.
17. N. Kumar, A. C. Berg, P. N. Belhumeur, and S. K. Nayar. Attribute and Simile Classifiers for Face Verification. In ICCV'09.
18. D. Lam. An exploratory study of gambling motivations and their impact on the purchase frequencies of various gambling products. Psychology and Marketing, 2007.
19. W. Mason and D. J. Watts. Financial incentives and the "performance of crowds". SIGKDD'09.
20. D. Pothineni, P. Mishra, A. Rasheed, and D. Sundararajan. Incentive design to mould online behavior: A game mechanics perspective. In GamifIR'14.
21. J. P. Rula, V. Navda, F. E. Bustamante, R. Bhagwan, and S. Guha. No "one-size fits all": Towards a principled approach for incentives in mobile crowdsourcing. In HotMobile '14.
22. N. Savage. Gaining wisdom from crowds. Magazine Communications of the ACM, 2012.
23. K. Stoddart. Behind the scenes of crowdsourcing: Who are crowd workers?, 2012. Available at http://www.crowdsource.com/blog/2012/11/behind-the-scenes-of-crowdsourcing-who-are-crowd-workers/.
24. J. Surowiecki. The Wisdom of Crowds. Anchor, 2005.
25. L. von Ahn and L. Dabbish. Designing games with a purpose. Magazine Communications of the ACM, 2008.
26. L. von Ahn and L. Dabbish. Labeling images with a computer game. In CHI'04.
27. P. Welinder and P. Perona. Online crowdsourcing: Rating annotators and obtaining cost-effective labels. In CVPRW'10.
28. J. Yang, L. A. Adamic, and M. S. Ackerman. Crowdsourcing and knowledge sharing: Strategic user behavior on taskcn. In EC'08.
29. M.-C. Yuen, I. King, and K.-S. Leung. A survey of crowdsourcing systems. In Passat Workshop at socialcom'11.