A Convex Formulation for Learning from Crowds

Proceedings of the 26th AAAI Conference on Artificial Intelligence, AAAI 2012

Volumn , Issue , 2012, Pages 73-79

  Kajino, Hiroshi ^a   Tsuboi, Yuta ^b   Kashima, Hisashi ^a  

^a UNIVERSITY OF TOKYO   (Japan)

^b IBM RESEARCH TOKYO   (Japan)

Author keywords

[No Author keywords available]

Indexed keywords

ARTIFICIAL INTELLIGENCE; CONVEX OPTIMIZATION; ITERATIVE METHODS; LEARNING SYSTEMS;

CONVEX OPTIMIZATION PROBLEMS; LABELED DATA; LARGE AMOUNTS; LEARNING CLASSIFIERS; LOW-COSTS; MACHINE-LEARNING; NOISY DATA; QUALITY PROBLEMS; SHORT PERIODS; WORKERS';

CROWDSOURCING;

EID: 84880366208     PISSN: None     EISSN: None     Source Type: Conference Proceeding    
DOI: None     Document Type: Conference Paper

References (21)

1. Caruana, R. 1997. Multitask Learning. Machine Learning.
2. Cortez, P.; Cerdeira, A.; Almeida, F.; Matos, T.; and Reis, J. 2009. Modeling wine preferences by data mining from physicochemical properties. Decision Support Systems 47(4):547-553.
3. Dawid, A. P., and Skene, A. M. 1979. Maximum Likelihood Estimation of Observer Error-Rates Using the EM Algorithm. Journal of the Royal Statistical Society. Series C (Applied Statics) 28(1):20-28.
4. Dekel, O., and Shamir, O. 2009. Vox Populi: Collecting High-Quality Labels from a Crowd. In Proceedings of the 22nd Annual Conference on Learning Theory.
5. Donmez, P.; Carbonell, J. G.; and Schneider, J. 2009. Efficiently Learning the Accuracy of Labeling Sources for Selective Sampling. In Proceedings of the 15th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, 259-268.
6. Evgeniou, T., and Pontil, M. 2004. Regularized multi-task learning. In Proceedings of the 10th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, 109-117.
7. Finin, T.; Murnane, W.; Karandikar, A.; Keller, N.; Martineau, J.; and Dredze, M. 2010. Annotating Named Entities in Twitter Data with Crowdsourcing. In Proceedings of the NAACL HLT 2010 Workshop on Creating Speech and Language Data with Amazon's Mechanical Turk, 80-88.
8. Frank, A., and Asuncion, A. 2010. UCI Machine Learning Repository.
9. Gomes, R.; Welinder, P.; Krause, A.; and Perona, P. 2011. Crowdclustering. In Advances in Neural Information Processing 24, 558-566.
10. Kajino, H., and Kashima, H. 2012. Convex Formulations of Learning from Crowds. Journal of Japanese Society of Artificial Intelligence 27(3):133-142.
11. Raykar, V. C.; Yu, S.; Zhao, L. H.; Florin, C.; Bogoni, L.; and Moy, L. 2010. Learning From Crowds. Journal of Machine Learning Research 11:1297-1322.
12. Ritter, A.; Clark, S.; and Etzioni, O. 2011. Named Entity Recognition in Tweets: An Experimental Study. In Proceedings of the 2011 Conference on Empirical Methods in Natural Language Processing, 1524-1534.
13. Sheng, V. S.; Provost, F.; and Ipeirotis, P. G. 2008. Get Another Label? Improving Data Quality and Data Mining Using Multiple, Noisy Labelers. In Proceedings of the 14th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, 614-622.
14. Smyth, P.; Fayyad, U.; Burl, M.; Perona, P.; and Baldi, P. 1995. Inferring Ground Truth from Subjective Labelling of Venus Images. In Advances in Neural Information Processing Systems 7, 1085-1092.
15. Snow, R.; O'Connor, B.; Jurafsky, D.; and Ng, A. Y. 2008. Cheap and Fast - But is it Good? Evaluating Non-Expert Annotations for Natural Language Tasks. In Proceedings of the Conference on Empirical Methods in Natural Language Processing, 254-263.
16. Tamuz, O.; Liu, C.; Belongie, S.; Shamir, O.; and Kalai, A. T. 2011. Adaptively Learning the Crowd Kernel. In Proceedings of the 28th International Conference on Machine Learning, 673-680.
17. Welinder, P., and Perona, P. 2010. Online crowdsourcing: Rating annotators and obtaining cost-effective labels. In Workshop on Advancing Computer Vision with Humans in the Loop, IEEE Conference on Computer Vision and Pattern Recognition, 25-32.
18. Welinder, P.; Branson, S.; Belongie, S.; and Perona, P. 2010. The Multidimensional Wisdom of Crowds. In Advances in Neural Information Processing Systems 23, 2424-2432.
19. Whitehill, J.; Ruvolo, P.; Wu, T.; Bergsma, J.; and Movellan, J. 2009. Whose Vote Should Count More: Optimal Integration of Labels from Labelers of Unknown Expertise. In Advances in Neural Information Processing Systems 22, 2035-2043.
20. Yan, Y.; Rosales, R.; Fung, G.; and Dy, J. 2010. Modeling Multiple Annotator Expertise in the Semi-Supervised Learning Scenario. In Proceedings of Conference on Uncertainty in Artificial Intelligence 2010, 674-682.
21. Yan, Y.; Rosales, R.; Fung, G.; and Dy, J. G. 2011. Active Learning from Crowds. In Proceedings of the 28th International Conference on Machine Learning, 1161-1168.