Shepherding the crowd: Managing and providing feedback to crowd workers

Conference on Human Factors in Computing Systems - Proceedings

Volumn , Issue , 2011, Pages 1669-1674

  Dow, Steven P ^a   Bunge, Brie ^a   Nguyen, Truc ^a   Klemmer, Scott R ^a   Kulkarni, Anand ^b   Hartmann, Björn ^c  

^a HCI Group   (United States)

^b DEPARTMENT OF INDUSTRIAL ENGINEERING AND OPERATIONS RESEARCH   (United States)

^c UNIVERSITY OF CALIFORNIA   (United States)

Author keywords

Crowdsourcing; Feedback system; Human computation

Indexed keywords

CONTENT CREATION; DESIGN SPACES; FEEDBACK SYSTEMS; HIGH QUALITY; HUMAN COMPUTATION; LEARN+; MICROTASKS; PROTOTYPE SYSTEM; TASK PERFORMANCE; WORKERS';

CROWDSOURCING;

EID: 79957940849     PISSN: None     EISSN: None     Source Type: Conference Proceeding    
DOI: 10.1145/1979742.1979826     Document Type: Conference Paper

References (15)

1. Annett, J. Feedback and human behaviour: the effects of knowledge of results, incentives, and reinforcement on learning and performance. Penguin Books, 1969.
2. Benkler, Y. Coase's Penguin, or, Linux and "The Nature of the Firm." The Yale Law Journal 112, 3 (2002), 369-446. (Pubitemid 41018134)
3. Bernstein, M.S., Little, G., Miller, R.C., et al. Soylent: a word processor with a crowd inside. Proc. 23nd ACM Symp. on UIST, ACM (2010), 313-322.
4. Cheshire, C. and Antin, J. The Social Psychological Effects of Feedback on the Production of Internet Information Pools. Journal of Computer-Mediated Communication 13, 3 (2008), 705-727.
5. Dow, S.P. Using Crowds to Study Creativity. CrowdConf, (2010).
6. Ericsson, K.A. and Smith, J. Toward a General Theory of Expertise: Prospects and Limits. Cambridge University Press, 1991.
7. Hinds, P. The Curse of Expertise: The Effects of Expertise and Debiasing Methods on Predictions of Novice Performance. Journal of Experimental Applied Psychology 5, (1999), 205-221.
8. Horton, J.J. Employer Expectations, Peer Effects and Productivity: Evidence from a Series of Field Experiments. SSRN eLibrary, (2010).
9. Ipeirotis, P.G., Provost, F., and Wang, J. Quality management on Amazon Mechanical Turk. Proc. ACM SIGKDD HCOMP, ACM (2010), 64-67.
10. Kittur, A., Chi, E.H., and Suh, B. Crowdsourcing user studies with Mechanical Turk. Proc. 26th ACM SIGCHI Human Factors (2008), 453-456.
11. Lampe, C. and Resnick, P. Slash(dot) and burn: distributed moderation in a large online conversation space. ACM SIGCHI Hum. factors (2004), 543-550.
12. Lave, J. and Wenger, E. Situated Learning: Legitimate Peripheral Participation. Cambridge, 1991.
13. Little, G., Chilton, L.B., Goldman, M., and Miller, R.C. TurKit: tools for iterative tasks on mechanical Turk. Proc. ACM SIGKDD Workshop on HCOMP, (2009), 29-30.
14. Musico, C. There's No Place Like Home. destinationCRM.com, 2008.
15. Viégas, F., Wattenberg, M., and Mckeon, M. The Hidden Order of Wikipedia. In Online Communities and Social Computing. 2007, 445-454.