Pay it backward: Per-task payments on crowdsourcing platforms reduce productivity

Conference on Human Factors in Computing Systems - Proceedings

Volumn , Issue , 2016, Pages 4111-4121

  Ikeda, Kazushi ^a   Bernstein, Michael S ^b  

^a KDDI R AND D LABORATORIES INC   (Japan)

^b STANFORD UNIVERSITY   (United States)

Author keywords

Crowd work; Crowdsourcing; Incentives; Motivation

Indexed keywords

CROWDSOURCING; HUMAN COMPUTER INTERACTION; HUMAN ENGINEERING; MOTIVATION;

BEHAVIORAL ECONOMICS; CROWD WORK; CROWDSOURCING PLATFORMS; FIELD EXPERIMENT; INCENTIVES; MOBILE CROWDSOURCING;

BEHAVIORAL RESEARCH;

EID: 85014757422     PISSN: None     EISSN: None     Source Type: Conference Proceeding    
DOI: 10.1145/2858036.2858327     Document Type: Conference Paper

References (42)

1. Amazon Mechanical Turk, https://www.mturk.com/
2. Help Find Jim. http://www.helpfindjim.com
3. Aakar, G., Thies, W., Cutrell, E. and Balakrishnan, R. mClerk: enabling mobile crowdsourcing in developing regions. In Proc. CHI 2012, 1843-1852.
4. Antin, J. and Shaw, A. Social desirability bias in reports of motivation for US and India workers on Mechanical Turk. In Proc. CSCW, 2011.
5. Anderson, A., Huttenlocher, D., Kleinberg, J. and Leskovec, J. Steering user behavior with badges. In Proc. of WWW, 2013, 95-106.
6. Arkes, H. R. and Blumer, C. The psychology of sunk cost. Organizational behavior and human decision processes, 35, 1, 1985, 124-140.
7. Beenen, G., Ling, K., Wang, X., Chang, K., Frankowski, D., Resnick, P. and Kraut, R. Using social psychology to motivate contributions to online communities. In Proc. CSCW, 2004, 212-221.
8. Church, A. H. Estimating the effect of incentives on mail survey response rates: A meta-analysis. Public Opinion Quarterly, 57, 1, 1993, 62-79.
9. Cooper, S., Khatib, F., Treuille, A., Barbero, J., Lee, J., Beenen, M., Fay, A. L., Baker, D. and Popovic, Z. Predicting protein structures with a multiplayer online game. Nature, 466, 7307, 2010, 756-760.
10. Deren, S., Stephens, R., Davis, W. R., Feucht, T. E. and Tortu, S. The impact of providing incentives for attendance at AIDS prevention sessions. Public health reports, 109, 4, 1994, 548.
11. Difallah, D. E., Catasta, M., Demartini, G. and Cudré-Mauroux, P. Scaling-up the crowd: Micro-task pricing schemes for worker retention and latency improvement. In Proc. HCOMP, 2014.
12. Dow, S., Kulkarni, A., Klemmer, S. and Hartmann, B. Shepherding the crowd yields better work. In Proc. CSCW, 2012, 1013-1022.
13. Edwards, P., Roberts, I., Clarke, M., DiGuiseppi, C., Pratap, S., Wentz, R. and Kwan, I. Increasing response rates to postal questionnaires: systematic review. 324, 7347, 2002, 1183.
14. Frey, B. S. and Jegen, R. Motivation crowding theory. Economic Surveys, 15, 5, 2001, 589-611.
15. Harris, C. G. The effects of pay-to-quit incentives on crowdworker task quality. In Proc. CSCW 2015, 1801-1812.
16. Horton, J. J., and Chilton, L. B. The labor economics of paid crowdsourcing. In Proc. of the 11th ACM conference on Electronic commerce, 2010, pp. 209-218.
17. Ipeirotis, P. G. Analyzing the amazon mechanical turk marketplace. XRDS: Crossroads, The ACM Magazine for Students, 17, 2, 2010, 16-21.
18. Kane, R. L., Johnson, P. E., Town, R. J. and Butler, M. A structured review of the effect of economic incentives on consumers' preventive behavior. American journal of preventive medicine, 27, 4, 2004, 327-352.
19. Kobayashi, M., Arita, S., Itoko, T., Saito, S. and Takagi, H. Motivating multi-generational crowd workers in social-purpose work. In Proc. CSCW, 2015, 1813-1824.
20. Kosfeld, M. and Neckermann, S. Getting more work for nothing? Symbolic awards and worker performance. American Economic Journal, 3, 3, 2011, 86-99.
21. Latham, G. P. and Baldes, J. J. The practical significance of Locke's theory of goal setting. Applied Psychology, 60, 1, 1975, 122-124.
22. Le, J., Edmonds, A., Hester, V. and Biewald, L. Ensuring quality in crowdsourced search relevance evaluation: The effects of training question distribution. In Proc. CSE, 2010, 21-26.
23. Locke, E. A. Toward a theory of task motivation and incentives. Organizational Behavior and Human Performance, 3, 2, 1968, 157-189.
24. Locke, E. A. and Latham, G. P. Building a practically useful theory of goal setting and task motivation: A 35-year odyssey. American psychologist, 57, 9, 2002, 705-714.
25. Lutge, E. E., Wiysonge, C. S., Knight, S. E. and Volmink, J. Material incentives and enablers in the management of tuberculosis. Cochrane Database System Review, 1, 2012.
26. Mason, W. and Watts, D. J. Financial incentives and the "performance of crowds". In Proc. HCOMP, 2009, 77-85.
27. Massung, E., Coyle, D., Cater, K. F., Jay, M. and Preist, C. Using crowdsourcing to support proenvironmental community activism. In Proc. CHI, 2013, 371-380.
28. Preist, C., Massung, E. and Coyle, D. Competing or aiming to be average?: Normification as a means of engaging digital volunteers. In Proc. CSCW, 2013, 1222-1233.
29. Rogstadius, J., Kostakos, V., Kittur, A., Smus, B., Laredo, J. and Vukovic, M. An Assessment of Intrinsic and Extrinsic Motivation on Task Performance in Crowdsourcing Markets. In Proc. ICWSM, 2011.
30. Rula, J. P., Navda, V., Bustamante, F. E., Bhagwan, R. and Guha, S. No one-size fits all: Towards a principled approach for incentives in mobile crowdsourcing. In Proc. HotMobile, 2014.
31. Ryu, E., Couper, M. P. and Marans, R. W. Survey incentives: Cash vs. in-kind; face-to-face vs. mail; response rate vs. nonresponse error. Public Opinion Research, 18, 1, 2006, 89-106.
32. Rzeszotarski, J. M., Chi, E., Paritosh, P. and Dai, P. Inserting micro-breaks into crowdsourcing workflows. In Proc. HCOMP 2013.
33. Salehi, N., Irani, L. C. and Bernstein, M. S. We are dynamo: Overcoming stalling and friction in collective action for crowd workers. In Proc. CHI 2015, 1621-1630.
34. Shaw, A. D., Horton, J. J. and Chen, D. L. Designing incentives for inexpert human raters. In Proc. CSCW 2011, 275-284.
35. Tomasic, A., Zimmerman, J., Steinfeld, A. and Huang, Y. Motivating contribution in a participatory sensing system via quid-pro-quo. Proc. CSCW, 2014, 979-988.
36. von Ahn L. and Dabbish L. Labeling images with a computer game. In Proc CHI, 2004, 319-326.
37. von Ahn, L., Maurer, B., Mcmillen, C., Abraham D. and Blum M. reCAPTCHA: Human-based character recognition via Web security measures. Science, 321, 5895, 2008, 1465-1468.
38. Weber, E. U. and Hsee, C. Cross-cultural differences in risk perception, but cross-cultural similarities in attitudes towards perceived risk. Management Science, 44, 9, 1998, 1205-1217.
39. Yin, M., Chen, Y. and Sun, Y. A. Monetary Interventions in Crowdsourcing Task Switching. In Proc. HCOMP 2014.
40. Yu, L., André, P., Kittur, A. and Kraut, R. A comparison of social, learning, and financial strategies on crowd engagement and output quality. In Proc. CSCW 2014, 967-978.
41. Zhang, Y. and Schaar, M. Reputation-based incentive protocols in crowdsourcing applications. In Proc. INFOCOM 2012, 2140-2148.
42. Zyskowski, K., Morris, M. R., Bigham, J. P., Gray, M. L. and Kane, S. K. Accessible crowdwork? Understanding the value in and challenge of microtask employment for people with disabilities. In Proc. CSCW 2015, 1682-1693.