Learning from the wisdom of crowds by minimax entropy

Advances in Neural Information Processing Systems

Volumn 3, Issue , 2012, Pages 2195-2203

  Zhou, Dengyong ^a   Platt, John C ^a   Basu, Sumit ^a   Mao, Yi ^a  

^a MICROSOFT RESEARCH   (United States)

Author keywords

[No Author keywords available]

Indexed keywords

COORDINATE DESCENT; ENTROPY PRINCIPLE; GROUND TRUTH; KULLBACK-LEIBLER DIVERGENCE; MINIMAX; NOISY LABELS; TRAINING SETS; WISDOM OF CROWDS;

PROBABILITY DISTRIBUTIONS;

ENTROPY;

EID: 84877729010     PISSN: 10495258     EISSN: None     Source Type: Conference Proceeding    
DOI: None     Document Type: Conference Paper

References (22)

1. Y. Altun and A. Smola. Unifying divergence minimization and statistical inference via convex duality. In Proceedings of the 19th Annual Conference on Learning Theory, 2006.
2. Amazon Mechanical Turk. https://www.mturk.com/mturk.
3. S. Boyd and L. Vandenberghe. Convex Optimization. Cambridge University Press, 2004.
4. CrowdFlower. http://crowdflower.com/.
5. A. P. Dawid and A. M. Skene. Maximum likeihood estimation of observer error-rates using the EM algorithm. Journal of the Royal Statistical Society, 28(1):20-28, 1979.
6. O. Dekel and O. Shamir. Vox populi: Collecting high-quality labels from a crowd. In Proceedings of the 22nd Annual Conference on Learning Theory, 2009.
7. A. P. Dempster, N. M. Laird, and D. B. Rubin. Maximum likelihood from incomplete data via the EM algorithm. Journal of the Royal Statistical Society, 39(1):1-38, 1977.
8. J. Deng, W. Dong, R. Socher, L.-J. Li, K. Li, and L. Fei-Fei. ImageNet: A large-scale hierarchical image database. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pages 248-255, 2009.
9. M. Dudik, S. J. Phillips, and R. E. Schapire. Maximum entropy density estimation with generalized regularization and an application to species distribution modeling. Journal of Machine Learning Research, 8:1217-1260, 2007.
10. S. Ertekin, H. Hirsh, and C. Rudin. Approximating the wisdom of the crowd. In Proceedings of the Workshop on Computational Social Science and the Wisdom of Crowds, 2011.
11. P. G. Ipeirotis, F. Provost, and J. Wang. Quality management on Amazon Mechanical Turk. In Proceedings of the ACM SIGKDD Workshop on Human Computation, pages 64-67, 2010.
12. E. Kamar, S. Hacker, and E. Horvitz. Combining human and machine intelligence in large-scale crowdsourcing. In Proceedings of the 11th International Conference on Autonomous Agents and Multiagent Systems, pages 467-474, 2012.
13. D. R. Karger, S. Oh, and D. Shah. Iterative learning for reliable crowdsourcing systems. In Advances in Neural Information Processing Systems 24, pages 1953-1961, 2011.
14. G. Lebanon and J. Lafferty. Boosting and maximum likelihood for exponential models. In Advances in Neural Information Processing Systems 14, pages 447-454, 2001.
15. R. M. Neal and G. E. Hinton. A view of the EM algorithm that justifies incremental, sparse, and other variants. In M. I. Jordan, editor, Learning in Graphical Models, pages 355-368. Kluwer Academic, Dordrecht, MA, 1998.
16. G. Rasch. On general laws and the meaning of measurement in psychology. In Proceedings of the 4th Berkeley Symposium on Mathematical Statistics and Probability, volume 4, pages 321-333, Berkeley, CA, 1961.
17. V. C. Raykar, S. Yu, L. H. Zhao, G. H. Valadez, C. Florin, L. Bogoni, and L. Moy. Learning from crowds. Journal of Machine Learning Research, 11:1297-1322, 2010.
18. P. Smyth, U. Fayyad, M. Burl, P. Perona, and P. Baldi. Inferring ground truth from subjective labelling of venus images. In Advances in neural information processing systems, pages 1085- 1092, 1995.
19. R. Snow, B. O'Connor, D. Jurafsky, and A. Y. Ng. Cheap and fast-but is it good? Evaluating non-expert annotations for natural language tasks. In Proceedings of the Conference on Empirical Methods in Natural Language Processing, pages 254-263, 2008.
20. P.Welinder, S. Branson, S. Belongie, and P. Perona. The multidimensional wisdom of crowds. In Advances in Neural Information Processing Systems 23, pages 2424-2432, 2010.
21. J. Whitehill, P. Ruvolo, T. Wu, J. Bergsma, and J. Movellan. Whose vote should count more: optimal integration of labels from labelers of unknown expertise. In Advances in Neural Information Processing Systems 22, pages 2035-2043, 2009.
22. S. C. Zhu, Y. N. Wu, and D. B. Mumford. Minimax entropy principle and its applications to texture modeling. Neural Computation, 9:1627-1660, 1997.