TurKit: Human computation algorithms on Mechanical Turk

UIST 2010 - 23rd ACM Symposium on User Interface Software and Technology

Volumn , Issue , 2010, Pages 57-66

  Little, Greg ^a   Chilton, Lydia B ^b   Goldman, Max ^a   Miller, Robert C ^a  

^a MASSACHUSETTS INSTITUTE OF TECHNOLOGY   (United States)

^b UNIVERSITY OF WASHINGTON   (United States)

Author keywords

Human computation; MTurk; Toolkit

Indexed keywords

FUNCTION CALLS; HUMAN COMPUTATION; IMPERATIVE PROGRAMMING; INDEPENDENT TASKS; MECHANICAL TURKS; MTURK; PROGRAMMING MODELS; PROTOTYPING; TOOLKIT;

ALGORITHMS; USER INTERFACES;

WEB SERVICES;

EID: 78649569877     PISSN: None     EISSN: None     Source Type: Conference Proceeding    
DOI: 10.1145/1866029.1866040     Document Type: Conference Paper

References (20)

1. von Ahn, L. Games With A Purpose. IEEE Computer Magazine, June 2006. Pages 96-98.
2. von Ahn, L. and Dabbish, L. Labeling Images with a Computer Game. ACM Conference on Human Factors in Computing Systems, CHI 2004.
3. von Ahn, L., Maurer, B., McMillen, C., Abraham, D. and Blum, M. reCAPTCHA: Human-Based Character Recognition via Web Security Measures. Science, September 12, 2008. pp 1465-1468.
4. Bernstein, M.S., Little, G., Miller, R.C., Hartmann, B., Ackerman, M.S., Karger, D.R., Crowell, D., Panovich, K. "Soylent: A Word Processor with a Crowd Inside". UIST 2010.
5. Bryant, S.L., et al. Becoming Wikipedian: transformation of participation in a collaborative online encyclopedia. GROUP 2005.
6. Cornsweet, T.N. The Staircase-Method in Psychophysics. The American Journal of Psychology, Vol. 75, No. 3 (Sep., 1962), pp. 485-491.
7. Dai, P., Mausam, Weld, D.S. Decision-Theoretic Control of Crowd-Sourced Workflows. AAAI 2010.
8. Dia, M.A. "On Decision Making in Tandem Networks". M.Eng. Thesis. Massachusetts Institute of Technology. 2009.
9. Feldman, S. I. and Brown, C. B. IGOR: a system for program debugging via reversible execution. Proc. ACM SIGPLAN Workshop on Parallel and Distributed Debugging. 1988.
10. Heer, J., Bostock, M. Crowdsourcing Graphical Perception: Using MTurk to Assess Visualization Design. CHI 2010.
11. Kittur, A., Chi, E. H., and Suh, B. Crowdsourcing user studies with MTurk. CHI 2008.
12. Kittur, A. and Kraut, R. E. Harnessing the wisdom of crowds in wikipedia: quality through coordination. CSCW 2008.
13. Ko, A. J. and Myers, B. A. Finding causes of program output with the Java Whyline. CHI 2009.
14. Kosorukoff A. Human based genetic algorithm. IlliGAL report no. 2001004. UIUC, 2001.
15. Little, G., Chilton, L.B., Goldman, M. and Miller, R.C. Exploring Iterative and Parallel Human Computation Processes. KDD-HCOMP 2010.
16. Mason, W. and Watts, D. J. Financial incentives and the "performance of crowds". KDD-HCOMP 2009.
17. Quinn, A. J., Bederson, B. B. A Taxonomy of Distributed Human Computation. Technical Report HCIL-2009-23. University of Maryland. 2009.
18. Russell, B., Torralba, A., Murphy, K., Freeman, W. LabelMe: a database and web-based tool for image annotation. International Journal of Computer Vision, Vol. 77, No. 1 (1 May 2008), pp. 157-173.
19. Snow, R., O'Connor, B., Jurafsky, D., and Ng, A. Y. Cheap and fast-but is it good?: evaluating non-expert annotations for natural language tasks. EMNLP 2008.
20. Sorokin, A. and D. Forsyth, "Utility data annotation with Amazon MTurk". CVPR 2008.