Crowdsourcing control: Moving beyond multiple choice

Uncertainty in Artificial Intelligence - Proceedings of the 28th Conference, UAI 2012

Volumn , Issue , 2012, Pages 491-500

  Lin, Christopher H ^a   Daniel, Mausam ^a   Weld, S ^a  

^a University of Washington   (United States)

Author keywords

[No Author keywords available]

Indexed keywords

AMAZON MECHANICAL TURKS; DECISION-THEORETIC; DESIGN AND IMPLEMENTS; MULTIPLE CHOICE; MULTIPLE TASKS; MULTIPLE-CHOICE QUESTIONS; PRIOR KNOWLEDGE; PROBABILISTIC GRAPHICAL MODELS;

ALGORITHMS; ARTIFICIAL INTELLIGENCE; EXPERIMENTS;

DESIGN;

EID: 84882621746     PISSN: None     EISSN: None     Source Type: Conference Proceeding    
DOI: None     Document Type: Conference Paper

References (19)

1. Aldous, 1985] Aldous, D. J. (1985). Exchangeability and related topics. In Ecole d' Ete de Probabilites de Saint-Flour XIII 1983, volume 1117 of Lecture Notes in Mathematics, pages 1-198. Springer Berlin / Heidelberg. 10.1007/BFb0099421.
2. Bernstein et al., 2010] Bernstein, M. S., Little, G., Miller, R. C., Hartmann, B., Ackerman, M. S., Karger, D. R., Crowell, D., and Panovich, K. (2010). Soylent: A word processor with a crowd inside. In UIST.
3. Dai et al., 2010] Dai, P., Mausam, and Weld, D. S. (2010). Decision-theoretic control of crowd-sourced workflows. In AAAI.
4. Dai et al., 2011] Dai, P., Mausam, and Weld, D. S. (2011). Artificial intelligence for artificial intelligence. In AAAI.
5. Grier, 2011] Grier, D. A. (2011). Error identification and correction in human computation: Lessons from the WPA. In HCOMP.
6. Kamar et al., 2012] Kamar, E., Hacker, S., and Horvitz, E. (2012). Combining human and machine intelligence in large-scale crowdsourcing. In AAMAS.
7. Karger et al., 2011] Karger, D. R., Oh, S., , and Shah, D. (2011). Budget-optimal crowdsourcing using lowrank matrix approximations. In Allerton.
8. Lin et al., 2012] Lin, C. H., Mausam, and Weld, D. S. (2012). Dynamically switching between synergistic workflows for crowdsourcing. In AAAI.
9. Little et al., 2009] Little, G., Chilton, L. B., Goldman, M., and Miller, R. C. (2009). Turkit: tools for iterative tasks on mechanical turk. In KDDHCOMP, pages 29-30.
10. Noronha et al., 2011] Noronha, J., Hysen, E., Zhang, H., and Gajos, K. Z. (2011). Platemate: Crowdsourcing nutrition analysis from food photographs. In UIST.
11. Parameswaran et al., 2010] Parameswaran, A., Garcia-Molina, H., Park, H., Polyzotis, N., Ramesh, A., and Widom, J. (2010). Crowdscreen: Algorithms for filtering data with humans. In VLDB.
12. Raykar et al., 2010] Raykar, V. C., Yu, S., Zhao, L. H., and Valadez, G. (2010). Learning from crowds. Journal of Machine Learning Research, 11:1297-1322.
13. Romney et al., 1986] Romney, A. K., Weller, S. C., and Batchelder, W. H. (1986). Culture as consensus: A theory of culture and informant accuracy. American Anthropologist, 88(2):313-338.
14. Shahaf and Horvitz, 2010] Shahaf, D. and Horvitz, E. (2010). Generlized markets for human and machine computation. In AAAI.
15. Sheng et al., 2008] Sheng, V. S., Provost, F., and Ipeirotis, P. G. (2008). Get another label? improving data quality and data mining using multiple, noisy labelers. In Proceedings of the Fourteenth ACM SIGKDD International Conference on Knowledge Discovery and Data Mining.
16. Snow et al., 2008] Snow, R., O'Connor, B., Jurafsky, D., and Ng, A. Y. (2008). Cheap and fast-but is it good? evaluating non-expert annotations for natural language tasks. In EMNLP, pages 254-263.
17. Weld et al., 2011] Weld, D. S., Mausam, and Dai, P. (2011). Human intelligence needs artificial intelligence. In HCOMP.
18. Welinder et al., 2010] Welinder, P., Branson, S., Belongie, S., and Perona, P. (2010). The multidimensional wisdom of crowds. In NIPS.
19. Whitehill et al., 2009] Whitehill, J., Ruvolo, P., Bergsma, J., Wu, T., and Movellan, J. (2009). Whose vote should count more: Optimal integration of labels from labelers of unknown expertise. In NIPS.