Reducing crowdsourcing to graphon estimation, statistically

International Conference on Artificial Intelligence and Statistics, AISTATS 2018

Volumn , Issue , 2018, Pages 1741-1750

  Shah, Devavrat ^a   Yu, Christina Lee ^b  

^a MASSACHUSETTS INSTITUTE OF TECHNOLOGY   (United States)

^b MICROSOFT RESEARCH   (United Kingdom)

Author keywords

[No Author keywords available]

Indexed keywords

ARTIFICIAL INTELLIGENCE;

EDGE INTENSITIES; ESTIMATION ALGORITHM; INFORMATION LIMITS; LOWER BOUNDS; MICROTASKS; SINGLE SNAPSHOTS; USE CONCEPT; VOTING SCHEMES;

CROWDSOURCING;

EID: 85048780031     PISSN: None     EISSN: None     Source Type: Conference Proceeding    
DOI: None     Document Type: Conference Paper

References (34)

1. Emmanuel Abbe and Colin Sandon. Community detection in general stochastic block models: fundamental limits and efficient recovery algorithms. FOCS, 2015.
2. Emmanuel Abbe and Colin Sandon. Recovering communities in the general stochastic block model without knowing the parameters. In Advances in Neural Information Processing Systems, pages 676–684, 2015.
3. Animashree Anandkumar, Rong Ge, Daniel Hsu, and Sham Kakade. A tensor spectral approach to learning mixed membership community models. In Conference on Learning Theory, pages 867–881, 2013.
4. Thomas Bonald and Richard Combes. Crowdsourcing: Low complexity, minimax optimal algorithms. arXiv preprint arXiv:1606.00226, 2016.
5. Christian Borgs, Jennifer Chayes, Christina E. Lee, and Devavrat Shah. Thy friend is my friend: Iterative collaborative filtering for sparse matrix estimation. Proceedings of Neural Information Processing Systems, 2017.
6. Christian Borgs, Jennifer T Chayes, Henry Cohn, and Shirshendu Ganguly. Consistent nonparametric estimation for heavy-tailed sparse graphs. arXiv preprint arXiv:1508.06675, 2015.
7. Stanley H Chan and Edoardo Airoldi. A consistent histogram estimator for exchangeable graph models. In ICML, pages 208–216, 2014.
8. Sourav Chatterjee. Matrix estimation by universal singular value thresholding. The Annals of Statistics, 43(1):177–214, 2015.
9. Yudong Chen and Martin J Wainwright. Fast low-rank estimation by projected gradient descent: General statistical and algorithmic guarantees. arXiv preprint arXiv:1509.03025, 2015.
10. Peter Chin, Anup Rao, and Van Vu. Stochastic block model and community detection in the sparse graphs: A spectral algorithm with optimal rate of recovery. arXiv preprint arXiv:1501.05021, 2(4), 2015.
11. Nilesh Dalvi, Anirban Dasgupta, Ravi Kumar, and Vibhor Rastogi. Aggregating crowdsourced binary ratings. In Proceedings of the 22nd international conference on World Wide Web, pages 285–294. ACM, 2013.
12. Mark A Davenport, Yaniv Plan, Ewout van den Berg, and Mary Wootters. 1-bit matrix completion. Information and Inference, 3(3):189–223, 2014.
13. Alexander Philip Dawid and Allan M Skene. Maximum likelihood estimation of observer error-rates using the em algorithm. Applied statistics, pages 20–28, 1979.
14. Chao Gao and Dengyong Zhou. Minimax optimal convergence rates for estimating ground truth from crowdsourced labels. arXiv preprint arXiv:1310.5764, 2013.
15. Arpita Ghosh, Satyen Kale, and Preston McAfee. Who moderates the moderators?: crowdsourcing abuse detection in user-generated content. In Proceedings of the 12th ACM conference on Electronic commerce, pages 167–176. ACM, 2011.
16. Chien-Ju Ho, Shahin Jabbari, and Jennifer Wortman Vaughan. Adaptive task assignment for crowdsourced classification. 2013.
17. Chien-Ju Ho and Jennifer Wortman Vaughan. Online task assignment in crowdsourcing markets. In Proceedings of the 26th Conference on Artificial Intelligence, volume 12, pages 45–51, 2012.
18. David R Karger, Sewoong Oh, and Devavrat Shah. Iterative learning for reliable crowdsourcing systems. In Advances in neural information processing systems, pages 1953–1961, 2011.
19. David R Karger, Sewoong Oh, and Devavrat Shah. Efficient crowdsourcing for multi-class labeling. ACM SIGMETRICS Performance Evaluation Review, 41(1):81–92, 2013.
20. David R Karger, Sewoong Oh, and Devavrat Shah. Budget-optimal task allocation for reliable crowdsourcing systems. Operations Research, 62(1):1–24, 2014.
21. Ashish Khetan and Sewoong Oh. Achieving budget-optimality with adaptive schemes in crowdsourcing. In Advances in Neural Information Processing Systems, pages 4844–4852, 2016.
22. Olga Klopp, Alexandre B Tsybakov, and Nicolas Verzelen. Oracle inequalities for network models and sparse graphon estimation. To appear in Annals of Statistics, 2015.
23. Christina E Lee, Yihua Li, Devavrat Shah, and Dogyoon Song. Blind regression: Nonparametric regression for latent variable models via collaborative filtering. In Advances in Neural Information Processing Systems, pages 2155–2163, 2016.
24. Hongwei Li and Bin Yu. Error rate bounds and iterative weighted majority voting for crowdsourcing. arXiv preprint arXiv:1411.4086, 2014.
25. Qiang Liu, Jian Peng, and Alexander T Ihler. Variational inference for crowdsourcing. In Advances in Neural Information Processing Systems, pages 692–700, 2012.
26. Jungseul Ok, Sewoong Oh, Jinwoo Shin, and Yung Yi. Optimality of belief propagation for crowdsourced classification. arXiv preprint arXiv:1602.03619, 2016.
27. Nihar B Shah, Sivaraman Balakrishnan, and Martin J Wainwright. A permutation-based model for crowd labeling: Optimal estimation and robustness. arXiv preprint arXiv:1606.09632, 2016.
28. Jacob Steinhardt, Gregory Valiant, and Moses Charikar. Avoiding imposters and delinquents: Adversarial crowdsourcing and peer prediction. In Advances in Neural Information Processing Systems, pages 4439–4447, 2016.
29. David Steurer and Sam Hopkins. Efficient estimation from few samples: Stochastic block models and related problems. 2017.
30. Peter Welinder, Steve Branson, Serge J Belongie, and Pietro Perona. The multidimensional wisdom of crowds. In Advances in neural information processing systems, pages 2424–2432, 2010.
31. Jiaming Xu. Rates of Convergence of Spectral Methods for Graphon Estimation. ArXiv e-prints, September 2017.
32. Justin Yang, Christina Han, and Edoardo Airoldi. Nonparametric estimation and testing of exchangeable graph models. In AISTATS, pages 1060–1067, 2014.
33. Yuan Zhang, Elizaveta Levina, and Ji Zhu. Estimating network edge probabilities by neighborhood smoothing. arXiv preprint arXiv:1509.08588, 2015.
34. Yuchen Zhang, Xi Chen, Denny Zhou, and Michael I Jordan. Spectral methods meet em: A provably optimal algorithm for crowdsourcing. In Advances in neural information processing systems, pages 1260–1268, 2014.