Cost-saving effect of crowd sourcing learning

IJCAI International Joint Conference on Artificial Intelligence

Volumn 2016-January, Issue , 2016, Pages 2111-2117

  Wang, Lu ^a   Zhou, Zhi Hua ^a  

^a NANJING UNIVERSITY   (China)

Author keywords

[No Author keywords available]

Indexed keywords

ARTIFICIAL INTELLIGENCE; COSTS; LEARNING SYSTEMS;

COST SAVING; MACHINE LEARNING COMMUNITIES; THEORETICAL FOUNDATIONS; TRAINING DATA; UPPER BOUND;

CROWDSOURCING;

EID: 85006109890     PISSN: 10450823     EISSN: None     Source Type: Conference Proceeding    
DOI: None     Document Type: Conference Paper

References (28)

1. Allan Afuah and Christopher L Tucci. Crowdsourcing as a solution to distant search. Academy of Management Review, 37 (3): 355-375, 2012.
2. Dana Angluin and Philip D. Laird. Learning from noisy examples. Machine Learning, 2 (4): 343-370, 1987.
3. Anselm Blumer, Andrzej Ehrenfeucht, David Haussler, and Manfred K. Warmuth. Classifying learnable geometric concepts with the Vapnik-Chervonenkis dimension. In STOC, pages 273-282, 1986.
4. Daren C Brabham. Crowdsourcing as a model for problem solving: An introduction and cases. Convergence, 14 (1): 75-90, 2008.
5. Alexander Philip Dawid and Allan M Skene. Maximum likelihood estimation of observer error-rates using the EM algorithm. Applied Statistics, pages 20-28, 1979.
6. Ofer Dekel and Ohad Shamir. Good learners for evil teachers. In ICML, pages 233-240, 2009.
7. Benôt Frénay and Michel Verleysen. Classification in the presence of label noise: A survey. IEEE Trans. on Neural Networks and Learning Systems, 25 (5): 845-869, 2014.
8. Panagiotis G. Ipeirotis, Foster J. Provost, Victor S. Sheng, and Jing Wang. Repeated labeling using multiple noisy labelers. Data Mining and Knowledge Discovery, 28 (2): 402-441, 2014.
9. David R. Karger, Sewoong Oh, and Devavrat Shah. Iterative learning for reliable crowdsourcing systems. In NIPS, pages 1953-1961, 2011.
10. Michael J. Kearns. Efficient noise-tolerant learning from statistical queries. Journal of the ACM, 45 (6): 983-1006, 1998.
11. Ronan LeBras, Richard Bernstein, Carla P. Gomes, Bart Selman, and R. Bruce van Dover. Crowdsourcing backdoor identification for combinatorial optimization. In IJCAI, pages 2840-2847, 2013.
12. Boyang Li, Stephen Lee-Urban, George Johnston, and Mark Riedl. Story generation with crowdsourced plot graphs. In AAAI, pages 598-604, 2013.
13. Tongliang Liu and Dacheng Tao. Classification with noisy labels by importance reweighting. IEEE Trans. on Pattern Analysis and Machine Intelligence, 38 (3): 447-461, 2016.
14. Aditya Krishna Menon, Brendan van Rooyen, Cheng Soon Ong, and BobWilliamson. Learning from corrupted binary labels via class-probability estimation. In ICML, pages 125-134, 2015.
15. Satoshi Oyama, Yukino Baba, Yuko Sakurai, and Hisashi Kashima. Accurate integration of crowdsourced labels using workers' self-reported confidence scores. In IJCAI, pages 2554-2560, 2013.
16. Vikas C. Raykar, Shipeng Yu, Linda H. Zhao, Gerardo Hermosillo Valadez, Charles Florin, Luca Bogoni, and Linda Moy. Learning from crowds. Journal of Machine Learning Research, 11: 1297-1322, 2010.
17. Victor S. Sheng, Foster J. Provost, and Panagiotis G. Ipeirotis. Get another label Improving data quality and data mining using multiple, noisy labelers. In KDD, pages 614-622, 2008.
18. Tian Tian and Jun Zhu. Max-margin majority voting for learning from crowds. In NIPS, pages 1612-1620, 2015.
19. Ruth Urner, Shai Ben-David, and Ohad Shamir. Learning from weak teachers. In AISTATS, pages 1252-1260, 2012.
20. Wei Wang and Zhi-Hua Zhou. Crowdsourcing label quality: A theoretical analysis. Science China Information Sciences, 58 (11): 1-12, 2015.
21. Jacob Whitehill, Paul Ruvolo, Tingfan Wu, Jacob Bergsma, and Javier R. Movellan. Whose vote should count more: Optimal integration of labels from labelers of unknown expertise. In Advances in Neural Information Processing Systems 22, pages 2035-2043, 2009.
22. Ian H. Witten and Eibe Frank. Data Mining: Practical Machine Learning Tools and Techniques with Java Implementations. Morgan Kaufmann, San Francisco, CA, 1999.
23. Yan Yan, Rómer Rosales, Glenn Fung, and Jennifer G. Dy. Active learning from crowds. In ICML, pages 1161-1168, 2011.
24. Dejun Yang, Guoliang Xue, Xi Fang, and Jian Tang. Crowdsourcing to smartphones: Incentive mechanism design for mobile phone sensing. In Proceedings of the 18th Annual International Conference on Mobile Computing and Networking, pages 173-184, 2012.
25. Yuchen Zhang, Xi Chen, Dengyong Zhou, and Michael I. Jordan. Spectral methods meet EM: A provably optimal algorithm for crowdsourcing. In NIPS, pages 1260-1268, 2014.
26. Jinhong Zhong, Ke Tang, and Zhi-Hua Zhou. Active learning from crowds with unsure option. In IJCAI, pages 1061-1068, 2015.
27. Dengyong Zhou, John C. Platt, Sumit Basu, and Yi Mao. Learning from the wisdom of crowds by minimax entropy. In NIPS, pages 2204-2212, 2012.
28. Zhi-Hua Zhou. Ensemble Methods: Foundations and Algorithms. CRC Press, Boca Raton, FL, 2012.