Better human computation through principled voting

Proceedings of the 27th AAAI Conference on Artificial Intelligence, AAAI 2013

Volumn , Issue , 2013, Pages 1142-1148

  Mao, Andrew ^a   Procaccia, Ariel D ^b   Chen, Yiling ^a  

^a HARVARD UNIVERSITY   (United States)

^b CARNEGIE MELLON UNIVERSITY   (United States)

Author keywords

[No Author keywords available]

Indexed keywords

AMAZON MECHANICAL TURKS; HUMAN COMPUTATION; LONG-TERM GOALS; MULTIPLE PEOPLE; SOCIAL CHOICE; VOTING METHOD; VOTING RULES;

ARTIFICIAL INTELLIGENCE;

EID: 84879739984     PISSN: None     EISSN: None     Source Type: Conference Proceeding    
DOI: None     Document Type: Conference Paper

References (29)

1. Azari Soufiani, H.; Parkes, D. C.; and Xia, L. 2012. Random utility theory for social choice: Theory and algorithms. In Proc. of 26th NIPS.
2. Brandt, F.; Fischer, F.; Harrenstein, P.; and Mair, M. 2008. A computational analysis of the Tournament Equilibrium Set. In Proc. of 23rd AAAI, 38-43.
3. Conitzer, V., and Sandholm, T. 2005. Common voting rules as maximum likelihood estimators. In Proc. of 21st UAI, 145-152.
4. Conitzer, V.; Davenport, A.; and Kalagnanam, H. 2006. Improved bounds for computing Kemeny rankings. In Proc. of 21st AAAI, 620-626.
5. Conitzer, V.; Rognlie, M.; and Xia, L. 2009. Preference functions that score rankings and maximum likelihood estimation. In Proc. of 21st IJCAI, 109-115.
6. Conitzer, V.; Sandholm, T.; and Lang, J. 2007. When are elections with few candidates hard to manipulate? Journal of the ACM 54(3):1-33.
7. Conitzer, V. 2006. Computing Slater rankings using similarities among candidates. In Proc. of 21st AAAI, 613-619.
8. Cooper, S.; Khatib, F.; Treuille, A.; Barbero, J.; Lee, J.; Beenen, M.; Leaver-Fay, A.; Baker, D.; and Popović, Z. 2010a. Predicting protein structures with a multiplayer online game. Nature 466:756-760.
9. Cooper, S.; Treuille, A.; Barbero, J.; Leaver-Fay, A.; Tuite, K.; Khatib, F.; Snyder, A. C.; Beenen, M.; Salesin, D.; Baker, D.; and Popović, Z. 2010b. The challenge of designing scientific discovery games. In Proceedings of the Fifth International Conference on the Foundations of Digital Games, FDG '10, 40-47. New York, NY, USA: ACM.
10. Dai, P.; Mausam; and Weld, D. S. 2010. Decision-theoretic control of crowd-sourced workflows. In Proc. of 24th AAAI, 1168-1174.
11. Dwork, C.; Kumar, R.; Naor, M.; and Sivakumar, D. 2001. Rank aggregation methods for the web. In Proc. of 10th WWW, 613-622.
12. Faliszewski, P., and Procaccia, A. D. 2010. AI's war on manipulation: Are we winning? AI Magazine 31(4):53-64.
13. Forsythe, R.; Rietz, T.; Myerson, R.; and Weber, R. 1996. An experimental study of voting rules and polls in threecandidate elections. International Journal of Game Theory 25(3):355-383.
14. Goel, A., and Lee, D. 2012. Triadic consensus - a randomized algorithm for voting in a crowd. In Proc. of 8th WINE, 434-447.
15. Hemaspaandra, E.; Hemaspaandra, L. A.; and Rothe, J. 1997. Exact analysis of Dodgson elections: Lewis Carroll's 1876 voting system is complete for parallel access to NP. Journal of the ACM 44(6):806-825.
16. Horton, J. J. 2010. The dot-guessing game: A fruit fly for human computation research. Available at SSRN: http://ssrn.com/abstract=1600372 or http://dx.doi.org/10.2139/ssrn.1600372.
17. Kamar, E.; Hacker, S.; and Horvitz, E. 2012. Combining human and machine intelligence in large-scale crowdsourcing. In Proc. of 11th AAMAS.
18. Little, G.; Chilton, L. B.; Goldman, M.; and Miller, R. C. 2010a. Exploring iterative and parallel human computation processes. In Proc. of 2nd HCOMP.
19. Little, G.; Chilton, L. B.; Goldman, M.; and Miller, R. C. 2010b. Turkit: Human computation algorithms on Mechanical Turk. In Proc. of 23rd UIST, 57-66.
20. McFadden, D. 1974. Conditional Logit Analysis of Qualitative Choice Behavior. In Zarembka, P., ed., Frontiers in econometrics. New York: Academic Press. 105-142.
21. Mosteller, F. 1951. Remarks on the method of paired comparisons: I. the least squares solution assuming equal standard deviations and equal correlations. Psychometrika 16(1):3-9.
22. Palfrey, T. R. 2009. Laboratory experiments in political economy. Annual Review of Political Science 12:379-388.
23. Pfeiffer, T.; Gao, X. A.; Mao, A.; Chen, Y.; and Rand, D. G. 2012. Adaptive polling and information aggregation. In Proc. of 26th AAAI.
24. Procaccia, A. D., and Rosenschein, J. S. 2007. Junta distributions and the average-case complexity of manipulating elections. Journal of Artificial Intelligence Research 28:157-181.
25. Procaccia, A. D.; Reddi, S. J.; and Shah, N. 2012. A maximum likelihood approach for selecting sets of alternatives. In Proc. of 28th UAI, 695-704.
26. Shahaf, D., and Horvitz, E. 2010. Generalized task markets for human and machine computation. In Proc. of 24th AAAI, 986-993.
27. Thurstone, L. L. 1927. A law of comparative judgement. Psychological Review 34:273-286.
28. von Ahn, L., and Dabbish, L. 2008. Designing games with a purpose. Communications of the ACM 51(8):58-67.
29. Young, H. P. 1988. Condorcet's theory of voting. The American Political Science Review 82(4):1231-1244.