AUTOMAN: A platform for integrating human-based and digital computation

ACM SIGPLAN Notices

Volumn 47, Issue 10, 2012, Pages 639-654

  Barowy, Daniel W ^a   Curtsinger, Charlie ^a   Berger, Emery D ^a   McGregor, Andrew ^a  

^a Biologically Inspired Neural and Dynamical Systems Laboratory   (United States)

Author keywords

Crowdsourcing; Programming Languages; Quality Control

Indexed keywords

AMAZON'S MECHANICAL TURKS; COMPUTATIONAL PLATFORMS; COMPUTATIONAL POWER; CONFIDENCE LEVELS; CROWDSOURCING; DIGITAL COMPUTATION; FINANCIAL INCENTIVES; FUNCTION CALLS; HUMAN SKILLS; HUMAN TASKS; LATENCY COSTS; PROGRAMMING LOGIC; RUNTIME SYSTEMS; STANDARD PROGRAMMING LANGUAGE;

COMPUTER PROGRAMMING LANGUAGES; QUALITY CONTROL; SCHEDULING;

BUDGET CONTROL;

EID: 84870420224     PISSN: 15232867     EISSN: None     Source Type: Journal    
DOI: 10.1145/2398857.2384663     Document Type: Conference Paper

References (31)

1. S. Ahmad, A. Battle, Z. Malkani, and S. Kamvar. The Jabberwocky Programming Environment for Structured Social Computing. In UIST, pp. 53-64, 2011.
2. Amazon. Mechanical Turk. http://www.mturk.com.
3. C. N. Anagnostopoulos. MediaLab LPR Database. http://www.medialab.ntua. gr/research/LPRdatabase.html.
4. M. S. Bernstein, G. Little, R. C. Miller, B. Hartmann, M. S. Ackerman, D. R. Karger, D. Crowell, and K. Panovich. Soylent: A Word Processor with a Crowd Inside. In UIST, pp. 313-322, 2010.
5. H. Chafi, Z. DeVito, A. Moors, T. Rompf, A. K. Sujeeth, P. Hanrahan, M. Odersky, and K. Olukotun. Language Virtualization for Heterogeneous Parallel Computing. In Onward!, 2010.
6. S. Cooper, F. Khatib, A. Treuille, J. Barbero, J. Lee, M. Beenen, A. Leaver-Fay, D. Baker, Z. Popovi, and F. Players. Predicting Protein Structures With a Multiplayer Online Game. NatureNatureNature, 466(7307): 756-760, 2010.
7. A. DasGupta. Probability for Statistics and Machine Learning: Fundamentals and Advanced Topics. Springer, 1st edition, 2011.
8. J. Dean and S. Ghemawat. MapReduce: Simplified Data Processing on Large Clusters. In OSDI, pp. 137-150, 2004.
9. J. R. Douceur. The Sybil Attack. In IPTPS, pp. 251-260, 2001.
10. S. Dow, A. Kulkarni, B. Bunge, T. Nguyen, S. Klemmer, and B. Hartmann. Shepherding the Crowd: Managing and Providing Feedback to Crowd Workers. In CHI, pp. 1669-1674, 2011.
11. S. Due, M. Ibrahim, M. Shehata, and W. Badawy. Automatic License Plate Recognition (ALPR): A State of the Art Review. IEEE Transactions on Circuits and Systems for Video TechnologyIEEE Transactions on Circuits and Systems for Video TechnologyIEEE Transactions on Circuits and Systems for Video Technology, 2012.
12. W. Feller. An Introduction to Probability Theory and Applications, volume 1. John Wiley & Sons Publishers, 3rd edition, 1968.
13. M. Fogus. BAYSICK-a DSL for Scala implementing a subset of BASIC. https://github.com/fogus/baysick, March 2009.
14. M. J. Franklin, D. Kossmann, T. Kraska, S. Ramesh, and R. Xin. CrowdDB: Answering Queries with Crowdsourcing. In SIGMOD, pp. 61-72, 2011.
15. M. Gentile. UI widget images come from the "Vector UI Kit" package. http://www.icondeposit.com/design: 9, Mar 2012. Creative Commons c2012 Matt Gentile.
16. J. Howe. The Rise of Crowdsourcing. Wired Magazine, 14(6): 176-178, 2006.
17. P. G. Ipeirotis. Demographics of Mechanical Turk. Tech. Rep. Working Paper CeDER-10-01, NYU Center for Digital Economy Research, 2010.
18. P. G. Ipeirotis, F. Provost, and J. Wang. Quality Management on Amazon Mechanical Turk. In HCOMP, pp. 64-67, 2010.
19. A. Kittur, B. Smus, and R. E. Kraut. CrowdForge: Crowdsourcing Complex Work. Tech. Rep. CMU-HCII-11-100, Human-Computer Interaction Institute, School of Computer Science, Carnegie Mellon University, February 2011.
20. A. P. Kulkarni, M. Can, and B. Hartmann. Turkomatic: Automatic Recursive Task and Workflow Design for. Mechanical Turk. In CHI, pp. 2053-2058, 2011.
21. K. Land, A. Slosar, C. Lintott, D. Andreescu, S. Bamford, P. Murray, R. Nichol, M. J. Raddick, K. Schawinski, A. Szalay, D. Thomas, and J. Vandenberg. Galaxy Zoo: the Large-Scale Spin Statistics of Spiral Galaxies in the Sloan Digital Sky Survey. Monthly Notices of the Royal Astronomical SocietyMonthly Notices of the Royal Astronomical SocietyMonthly Notices of the Royal Astronomical Society, 388: 1686-1692, Aug. 2008.
22. G. Little, L. B. Chilton, M. Goldman, and R. C. Miller. TurKit: Human Computation Algorithms on Mechanical Turk. In UIST, pp. 57-66, 2010.
23. M. Marge, S. Banerjee, and A. Rudnicky. Using the Amazon Mechanical Turk for Transcription of Spoken Language. In ICASSP, pp. 5270-5273, Mar 2010.
24. A. McCallum, K. Schultz, and S. Singh. FACTORIE: Probabilistic Programming via Imperatively Defined Factor Graphs. In NIPS, 2009.
25. M. Odersky and M. Zenger. Scalable Component Abstractions. In OOPSLA, pp. 41-57, 2005.
26. D. Oleson, V. Hester, A. Sorokin, G. Laughlin, J. Le, and L. Biewald. Programmatic Gold: Targeted and Scalable Quality Assurance in Crowdsourcing. In HCOMP, pp. 43-48, 2011.
27. D. Parikh and L. Zitnick. Human-Debugging of Machines. In NIPS CSS, 2011.
28. D. Shahaf and E. Amir. Towards a Theory of AI Completeness. In Commonsense, 2007.
29. D. Tamir, P. Kanth, and P. Ipeirotis. Mechanical Turk: Now With 40.92% Spam., Dec 2010.
30. L. von Ahn, B. Maurer, C. Mcmillen, D. Abraham, and M. Blum. reCAPTCHA: Human-Based Character Recognition via Web Security Measures. Science, 321(5895): 1465-1468, Aug 2008.
31. T. Yan, V. Kumar, and D. Ganesan. CrowdSearch: Exploiting Crowds for Accurate Real-Time Image Search on Mobile Phones. In MobiSys, pp. 77-90, 2010.