Keep it simple: Reward and task design in crowdsourcing

ACM International Conference Proceeding Series

Volumn , Issue , 2013, Pages

  Finnerty, Ailbhe ^a   Kucherbaev, Pavel ^b   Tranquillini, Stefano ^b   Convertino, Gregorio ^c  

^a ITC IRST   (Italy)

^b DISI UniTN   (Italy)

^c XEROX RESEARCH CENTRE EUROPE   (France)

Author keywords

Cognition; Crowdsourcing; Motivation; User interface

Indexed keywords

EXPERIMENTS; MOTIVATION; USER INTERFACES;

COGNITION; CROWDSOURCING; HUMAN ABILITIES; LONG-TERM GOALS; ONE-FACTOR; QUALITY OF RESULTS; QUALITY OF WORK; TASK DESIGN;

DESIGN;

EID: 84886404918     PISSN: None     EISSN: None     Source Type: Conference Proceeding    
DOI: 10.1145/2499149.2499168     Document Type: Conference Paper

References (14)

1. Chipman, S. F., and Schraagen., J. M., & Shalin, V. L. Introduction to cognitive task analysis. In J. M. Schraagen et.al. (Eds.), Cognitive task analysis (pp. 3-23). Mahwah, NJ: Lawrence Erlbaum Associates (2000).
2. Crump MJC, McDonnell JV, Gureckis TM Evaluating Amazon's Mechanical Turk as a Tool for Experimental Behavioral Research. PLoS ONE 8 (3). (2013).
3. Dow, S., and Kulkarni., A. P., Bunge, B., Nguyen, T., Klemmer, S. R., and Hartmann, B. Shepherding the crowd: managing and providing feedback to crowd workers. In CHI Extend. Abstracts (2011), 1669-1674.
4. Heer, J. and Bostock, M. Crowdsourcing AMT graphical perception: using mechanical turk to assess visualization design. In CHI (2010), 203-212.
5. Howe, J. The rise of crowdsourcing Wired Magazine 14 (6). (2006)
6. Kaufmann, N., and Schulze, T. Worker motivation in crowdsourcing and human computation. Education 17, (2011).
7. Kieras, D. and Polson P.G., An approach to the formal analysis of user complexity. International Journal of Man-Machine Studies, 22 (1985), 365-394.
8. Kittur, A. Crowdsourcing, Collaboration and Creativity. XRDS, 17 (2010), 22-26.
9. Kittur, A. Nickerson, J V. Bernstein, Ml Gerber, E. Shaw, A. Zimmerman, J. Lease, M. and Horton, J. The future of crowd work. In CSCW (2013)., 1301-1318.
10. Komarov, S. Reinecke, K. and Gajos K. Z. Crowdsourcing Performance Evaluations of User Interfaces. In CHI (2013), 207-216.
11. Mason, W. A., and Watts, D. J. Financial incentives and the "performance of crowds,". In KDD Workshop on Human Computation (2009), 77-85.
12. Rogstadius, J., Kostakos, V., Kittur, A., Smus, B., Laredo, J., and Vukovic, M. An assessment of intrinsic and extrinsic motivation on task performance in crowdsourcing markets. In ICWSM (2011).
13. Shaw, A. D., and Horton., J. J., and Chen, D. L. Designing incentives for inexpert human raters. In CSCW (2011), 275-284.
14. Sweller, J. Cognitive Load During Problem Solving: Effects on Learning. Cognitive Science, 12 (1988) 257-85.