Crowdsourced top-k algorithms: An experimental evaluation

Proceedings of the VLDB Endowment

Volumn 9, Issue 8, 2016, Pages 612-623

  Zhang, Xiaohang ^a   Li, Guoliang ^a   Feng, Jianhua ^a  

^a TSINGHUA UNIVERSITY   (China)

Author keywords

[No Author keywords available]

Indexed keywords

ARTIFICIAL INTELLIGENCE; CROWDSOURCING; LEARNING ALGORITHMS; LEARNING SYSTEMS; MAXIMUM LIKELIHOOD; MAXIMUM LIKELIHOOD ESTIMATION;

AMAZON MECHANICAL TURKS; COMPREHENSIVE COMPARISONS; CROWDSOURCING PLATFORMS; EXPERIMENTAL EVALUATION; K ALGORITHM; MACHINE LEARNING COMMUNITIES; OBJECTS-BASED; REAL DATA SETS;

ALGORITHMS;

EID: 84975797359     PISSN: None     EISSN: 21508097     Source Type: Conference Proceeding    
DOI: 10.14778/2921558.2921559     Document Type: Conference Paper

References (20)

1. R. M. Adelsman and A. B. Whinston. Sophisticated voting with information for two voting functions. Journal of Economic Theory, 15(1):145-159, 1977.
2. R. A. Bradley and M. E. Terry. Rank analysis of incomplete block designs: I. the method of paired comparisons. Biometrika, pages 324-345, 1952.
3. R. Busa-Fekete, B. Szorenyi, W. Cheng, P. Weng, and E. Hullermeier. Top-k selection based on adaptive sampling of noisy preferences. In ICML, pages 1094-1102, 2013.
4. X. Chen, P. N. Bennett, K. Collins-Thompson, and E. Horvitz. Pairwise ranking aggregation in a crowdsourced setting. In WSDM, pages 193-202, 2013.
5. S. B. Davidson, S. Khanna, T. Milo, and S. Roy. Using the crowd for top-k and group-by queries. In ICDT, pages 225-236, 2013.
6. A. E. Elo. The rating of chessplayers, past and present, volume 3. Batsford London, 1978.
7. B. Eriksson. Learning to top-k search using pairwise comparisons. In AISTATS, pages 265-273, 2013.
8. U. Feige, P. Raghavan, D. Peleg, and E. Upfal. Computing with noisy information. SIAM J. Comput., 23(5):1001-1018, 1994.
9. S. Guo, A. G. Parameswaran, and H. Garcia-Molina. So who won?: dynamic max discovery with the crowd. In SIGMOD, pages 385-396, 2012.
10. R. Herbrich, T. Minka, and T. Graepel. Trueskill: A bayesian skill rating system. In NIPS, pages 569-576, 2006.
11. X. Jiang, L. Lim, Y. Yao, and Y. Ye. Statistical ranking and combinatorial hodge theory. Math. Program., 127(1):203-244, 2011.
12. A. R. Khan and H. Garcia-Molina. Hybrid strategies for finding the max with the crowd. Technical report, 2014.
13. G. Li, J. Wang, Y. Zheng, and M. Franklin. Crowdsourced data management: A survey. IEEE Trans. Knowl. Data Eng., 2016.
14. A. Marcus, E. Wu, D. R. Karger, S. Madden, and R. C. Miller. Human-powered sorts and joins. PVLDB, 5(1):13-24, 2011.
15. S. Negahban, S. Oh, and D. Shah. Iterative ranking from pair-wise comparisons. In NIPS, pages 2483-2491, 2012.
16. T. Pfeiffer, X. A. Gao, Y. Chen, A. Mao, and D. G. Rand. Adaptive polling for information aggregation. In AAAI, 2012.
17. J.-C. Pomerol and S. Barba-Romero. Multicriterion decision in management: principles and practice, volume 25. Springer Science & Business Media, 2012.
18. P. Venetis, H. Garcia-Molina, K. Huang, and N. Polyzotis. Max algorithms in crowdsourcing environments. In WWW, pages 989-998, 2012.
19. F. L. Wauthier, M. I. Jordan, and N. Jojic. Efficient ranking from pairwise comparisons. In ICML, pages 109-117, 2013.
20. P. Ye and D. Doermann. Combining preference and absolute judgements in a crowd-sourced setting. In Proc. of Intl. Conf. on Machine Learning, pages 1-7, 2013.