Toward automatic task design: A progress report

Workshop Proceedings - Human Computation Workshop 2010, HCOMP2010

Volumn , Issue , 2010, Pages 77-85

  Huang, Eric ^a   Zhang, Haoqi ^a   Parkes, David C ^a   Gajos, Krzysztof Z ^a   Chen, Yiling ^a  

^a HARVARD UNIVERSITY   (United States)

Author keywords

Human computation; Mechanical Turk; Peer production

Indexed keywords

BREAK DOWN; BUDGET CONSTRAINT; COMMON PROBLEMS; CONSTRUCT MODELS; DESIGN DECISIONS; DESIGN TASKS; HUMAN COMPUTATION; IMAGE LABELING; MECHANICAL TURKS; OPTIMIZED DESIGNS; PEER PRODUCTION; PER UNIT; PROBLEM SOLVING PROCESS; PROGRESS REPORT; QUALITY METRICES; QUALITY OF WORK; TASK DESIGN; WORK QUALITY;

ARCHITECTURAL DESIGN; FORECASTING; SCHEDULING;

PROBLEM SOLVING;

EID: 77956242658     PISSN: None     EISSN: None     Source Type: Conference Proceeding    
DOI: 10.1145/1837885.1837908     Document Type: Conference Paper

References (16)

1. P. Dai, Mausam, and D. S. Weld. Decision-theoretic control of crowd-sourced workflows. In the 24th AAAI Conference on Artificial Intelligence (AAAI'10), Atlanta, Georgia, 2010.
2. J. Horton and L. Chilton. The labor economics of paid crowdsourcing. CoRR, abs/1001.0627, 2010.
3. J. Horton, D. G. Rand, and R. J. Zeckhauser. The Online Laboratory: Conducting Experiments in a Real Labor Market. SSRN eLibrary, 2010.
4. P. G. Ipeirotis. Demographics of mechanical turk. CeDER Working Papers, 2010.
5. G. Little, L. B. Chilton, M. Goldman, and R. C. Miller. Turkit: tools for iterative tasks on mechanical turk. In KDD-HCOMP '09, Paris, France, June 2009.
6. W. Mason and D. J. Watts. Financial incentives and the "Performance of Crowds". In KDD-HCOMP '09, Paris, France, June 2009.
7. J. Pontin. Artificial intelligence, with help from the humans. The New York Times, March 2007.
8. M. F. Porter. An algorithm for suffix stripping. Morgan Kaufmann Publishers Inc., San Francisco, CA, USA, 1997.
9. H. A. Simon. The sciences of the artificial. Cambridge, Ma: MIT Press, 1969.
10. R. Snow, B. O'Connor, D. Jurafsky, and A. Y. Ng. Cheap and fast - but is it good? evaluating non-expert annotations for natural language tasks. In the 2008 Conference on Empirical Methods in Natural Language Processing, pages 254-263, Honolulu, October 2008.
11. Q. Su, D. Pavlov, J.-H. Chou, and W. C. Baker. Internet-scale collection of human-reviewed data. In the 2007 International World Wide Web Conference, Banff, Alberta, Canada, May 2007.
12. L. von Ahn and L. Dabbish. Designing games with a purpose. Commun. ACM, 51(8):58-67, 2008.
13. Wikipedia statistics, January 2010. http://stats.wikimedia.org/EN/ TablesWikipediaEN.htm.
14. H. Zhang, Y. Chen, and D. C. Parkes. A general approach to environment design with one agent. In the 21st International Joint Conference on Artificial Intelligence (IJCAI-09), Pasadena, CA, 2009.
15. H. Zhang and D. C. Parkes. Value-based policy teaching with active indirect elicitation. In the 23rd AAAI Conference on Artificial Intelligence (AAAI'08), Chicago, IL, 2008.
16. H. Zhang, D. C. Parkes, and Y. Chen. Policy teaching through reward function learning. In the 10th ACM Electronic Commerce Conference (EC'09), Stanford, CA, 2009.