Spectral methods meet EM: A provably optimal algorithm for crowdsourcing

Advances in Neural Information Processing Systems

Volumn 2, Issue January, 2014, Pages 1260-1268

  Zhang, Yuchen ^a   Chen, Xi ^b   Zhou, Dengyong ^c   Jordan, Michael I ^a  

^a UNIVERSITY OF CALIFORNIA   (United States)

^b NEW YORK UNIVERSITY   (United States)

^c MICROSOFT RESEARCH   (United States)

Author keywords

[No Author keywords available]

Indexed keywords

INFORMATION SCIENCE; SPECTROSCOPY;

EMPIRICAL APPROACH; INITIAL ESTIMATE; LOG-LIKELIHOOD FUNCTIONS; OBJECTIVE FUNCTIONS; OPTIMAL ALGORITHM; OPTIMAL CONVERGENCE; REAL DATA SETS; SPECTRAL METHODS;

ALGORITHMS;

EID: 84937883035     PISSN: 10495258     EISSN: None     Source Type: Conference Proceeding    
DOI: None     Document Type: Conference Paper

References (27)

1. A. Anandkumar, D. P. Foster, D. Hsu, S. M. Kakade, and Y.-K. Liu. A spectral algorithm for latent Dirichlet allocation. arXiv preprint: 1204.6703, 2012.
2. A. Anandkumar, R. Ge, D. Hsu, and S. M. Kakade. A tensor spectral approach to learning mixed membership community models. In Annual Conference on Learning Theory, 2013.
3. A. Anandkumar, R. Ge, D. Hsu, S. M. Kakade, and M. Telgarsky. Tensor decompositions for learning latent variable models. arXiv preprint: 1210.7559, 2012.
4. A. Anandkumar, D. Hsu, and S. M. Kakade. A method of moments for mixture models and hidden Markov models. In Annual Conference on Learning Theory, 2012.
5. A. T. Chaganty and P. Liang. Spectral experts for estimating mixtures of linear regressions. arXiv preprint: 1306.3729, 2013.
6. X. Chen, Q. Lin, and D. Zhou. Optimistic knowledge gradient policy for optimal budget allocation in crowdsourcing. In Proceedings of ICML, 2013.
7. N. Dalvi, A. Dasgupta, R. Kumar, and V. Rastogi. Aggregating crowdsourced binary ratings. In Proceedings of World Wide Web Conference, 2013.
8. A. P. Dawid and A. M. Skene. Maximum likelihood estimation of observer error-rates using the EM algorithm. Journal of the Royal Statistical Society, Series C, pages 20-28, 1979.
9. J. Deng, W. Dong, R. Socher, L.-J. Li, K. Li, and L. Fei-Fei. Imagenet: A large-scale hierarchical image database. In IEEE CVPR, 2009.
10. C. Gao and D. Zhou. Minimax optimal convergence rates for estimating ground truth from crowdsourced labels. arXiv preprint arXiv: 1310.5764, 2014.
11. A. Ghosh, S. Kale, and P. McAfee. Who moderates the moderators? crowdsourcing abuse detection in user-generated content. In Proceedings of the ACM Conference on Electronic Commerce, 2011.
12. D. Hsu, S. M. Kakade, and T. Zhang. A spectral algorithm for learning hidden Markov models. Journal of Computer and System Sciences, 78(5): 1460-1480, 2012.
13. P. Jain and S. Oh. Learning mixtures of discrete product distributions using spectral decompositions. arXiv preprint: 1311.2972, 2013.
14. D. R. Karger, S. Oh, and D. Shah. Efficient crowdsourcing for multi-class labeling. In ACM SIGMETRICS, 2013.
15. D. R. Karger, S. Oh, and D. Shah. Budget-optimal task allocation for reliable crowdsourcing systems. Operations Research, 62(1): 1-24, 2014.
16. M. Lease and G. Kazai. Overview of the TREC 2011 crowdsourcing track. In Proceedings of TREC 2011, 2011.
17. E. Lehmann and G. Casella. Theory of Point Estimation. Springer, 2nd edition, 2003.
18. P. Liang. Partial information from spectral methods. NIPS Spectral Learning Workshop, 2013.
19. Q. Liu, J. Peng, and A. T. Ihler. Variational inference for crowdsourcing. In NIPS, 2012.
20. V. C. Raykar, S. Yu, L. H. Zhao, G. H. Valadez, C. Florin, L. Bogoni, and L. Moy. Learning from crowds. Journal of Machine Learning Research, 11: 1297-1322, 2010.
21. R. Snow, B. O'Connor, D. Jurafsky, and A. Y. Ng. Cheap and fast - but is it good? evaluating non-expert annotations for natural language tasks. In Proceedings of EMNLP, 2008.
22. P. Welinder, S. Branson, S. Belongie, and P. Perona. The multidimensional wisdom of crowds. In NIPS, 2010.
23. J. Whitehill, P. Ruvolo, T. Wu, J. Bergsma, and J. R. Movellan. Whose vote should count more: Optimal integration of labels from labelers of unknown expertise. In NIPS, 2009.
24. Y. Zhang, X. Chen, D. Zhou, and M. I. Jordan. Spectral methods meet EM: A provably optimal algorithm for crowdsourcing. arXiv preprint arXiv: 1406.3824, 2014.
25. D. Zhou, Q. Liu, J. C. Platt, and C. Meek. Aggregating ordinal labels from crowds by minimax conditional entropy. In Proceedings of ICML, 2014.
26. D. Zhou, J. C. Platt, S. Basu, and Y. Mao. Learning from the wisdom of crowds by minimax entropy. In NIPS, 2012.
27. J. Zou, D. Hsu, D. Parkes, and R. Adams. Contrastive learning using spectral methods. In NIPS, 2013.