A brief introduction to weakly supervised learning

National Science Review

Volumn 5, Issue 1, 2018, Pages 44-53

  Zhou, Zhi Hua ^a  

^a NANJING UNIVERSITY   (China)

Author keywords

Machine learning; Supervised learning; Weakly supervised learning

Indexed keywords

EID: 85042566177     PISSN: 20955138     EISSN: 2053714X     Source Type: Journal    
DOI: 10.1093/nsr/nwx106     Document Type: Review

References (103)

1. Goodfellow I, Bengio Y and Courville A. Deep Learning. Cambridge: MIT Press, 2016.
2. Settles B. Active learning literature survey. Technical Report Department of Computer Sciences, University of Wisconsin at Madison, Wisconsin, WI, 2010 [ http://pages. cs.wisc.edu/~bsettles/pub/settles.activelearning.pdf].
3. Chapelle O, Schölkopf B and Zien A (eds). Semi-Supervised Learning. Cambridge: MIT Press, 2006.
4. Zhu X. Semi-supervised learning literature survey. Technical Report 1530. Department of Computer Sciences, University of Wisconsin at Madison, Madison, WI, 2008 [ http://www.cs. wisc.edu/~jerryzhu/pub/ssl?survey.pdf].
5. Zhou Z-H and Li M. Semi-supervised learning by disagreement. Knowl Inform Syst 2010; 24: 415-39.
6. Huang SJ, Jin R and Zhou ZH. Active learning by querying informative and representative examples. IEEE Trans Pattern Anal Mach Intell 2014; 36: 1936-49.
7. Lewis D and Gale W. A sequential algorithm for training text classifiers. In 17th Annual International ACMSIGIR Conference on Research and Development in Information Retrieval, Dublin, Ireland, 1994; 3-12.
8. Seung H, Opper M and Sompolinsky H. Query by committee. In 5th ACM Workshop on Computational Learning Theory, Pittsburgh, PA, 1992; 287-94.
9. Abe N and Mamitsuka H. Query learning strategies using boosting and bagging. In 15th International Conference onMachine Learning, Madison, WI, 1998; 1-9.
10. Nguyen HT and Smeulders AWM. Active learning using preclustering. In 21st International Conference onMachine Learning, Banff, Canada, 2004; 623-30.
11. Dasgupta S and Hsu D. Hierarchical sampling for active learning. In 25th International Conference on Machine Learning, Helsinki, Finland, 2008; 208-15.
12. Wang Z and Ye J. Querying discriminative and representative samples for batch mode active learning. In 19th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, Chicago, IL, 2013; 158-66.
13. Dasgupta S, Kalai AT and Monteleoni C. Analysis of perceptron-based active learning. In 28th Conference on Learning Theory, Paris, France, 2005; 249-63.
14. Dasgupta S. Analysis of a greedy active learning strategy. In Advances in Neural Information Processing Systems 17, Cambridge, MA: MIT Press, 2005; 337-44.
15. Kääriäinen M. Active learning in the non-realizable case. In 21st International Conference on Computational Linguistics and 44th Annual Meeting of the Association for Computational Linguistics, Sydney, Australia, 2006; 63-77.
16. Balcan MF, Broder AZ and Zhang T. Margin based active learning. In 20th Annual Conference on Learning Theory, San Diego, CA, 2007; 35-50.
17. Hanneke S. Adaptive rates of convergence in active learning. In 22nd Conference on Learning Theory,Montreal, Canada, 2009.
18. Wang W and Zhou ZH. Multi-view active learning in the non-realizable case. In Advances in Neural Information Processing Systems 23, Cambridge, MA: MIT Press, 2010; 2388-96.
19. Miller DJ and Uyar HS. A mixture of experts classifier with learning based on both labelled and unlabelled data. In Advances in Neural Information Processing Systems 9, Cambridge, MA: MIT Press, 1997; 571-7.
20. Nigam K, McCallum AK and Thrun S et al. Text classification from labeled and unlabeled documents using EM. Mach Learn 2000; 39: 103-34.
21. Dempster AP, Laird NM and Rubin DB. Maximum likelihood from incomplete data via the EM algorithm. J Roy Stat Soc B Stat Meth 1977; 39: 1-38.
22. Fujino A, Ueda N and Saito K. A hybrid generative/ discriminative approach to semi-supervised classifier design. In 20th National Conference on Artificial Intelligence, Pittsburgh, PA, 2005; 764-9.
23. Blum A and Chawla S. Learning from labeled and unlabeled data using graph mincuts. In ICML, 2001; 19-26.
24. Zhu X, Ghahramani Z and Lafferty J. Semi-supervised learning using Gaussian fields and harmonic functions. In 20th International Conference on Machine Learning, Washington, DC, 2003; 912-9.
25. Zhou D, Bousquet O and Lal TN et al. Learning with local and global consistency. In Advances in Neural Information Processing Systems 16, Cambridge, MA: MIT Press, 2004; 321-8.
26. Carreira-Perpinan MA and Zemel RS. Proximity graphs for clustering and manifold learning. In Advances in Neural Information Processing Systems 17, Cambridge, MA: MIT Press, 2005; 225-32.
27. Wang F and Zhang C. Label propagation through linear neighborhoods. In 23rd International Conference onMachine Learning, Pittsburgh, PA, 2006; 985-92.
28. Hein M and Maier M. Manifold denoising. In Advances in Neural Information Processing Systems 19, Cambridge, MA: MIT Press, 2007; pp. 561-8.
29. Joachims T. Transductive inference for text classification using support vector machines. In 16th International Conference on Machine Learning, Bled, Slovenia, 1999; 200-9.
30. Chapelle O and Zien A. Semi-supervised learning by low density separation. In 10th International Workshop on Artificial Intelligence and Statistics, Barbados, 2005; 57-64.
31. Li YF, Tsang IW and Kwok JT et al. Convex and scalable weakly labeled SVMs. J Mach Learn Res 2013; 14: 2151-88.
32. Blum A and Mitchell T. Combining labeled and unlabeled data with cotraining. In 11th Conference on Computational Learning Theory,Madison, WI, 1998; 92-100.
33. Zhou Z-H and Li M. Tri-training: exploiting unlabeled data using three classifiers. IEEE Trans Knowl Data Eng 2005; 17: 1529-41.
34. Zhou Z-H. When semi-supervised learning meets ensemble learning. In 8th International Workshop on Multiple Classifier Systems, Reykjavik, Iceland, 2009; 529-38.
35. Zhou Z-H. Ensemble Methods: Foundations and Algorithms. Boca Raton: CRC Press, 2012.
36. Cozman FG and Cohen I. Unlabeled data can degrade classification performance of generative classifiers. In 15th International Conference of the Florida Artificial Intelligence Research Society, Pensacola, FL, 2002; 327-31.
37. Li YF and Zhou ZH. Towards making unlabeled data never hurt. IEEE Trans Pattern Anal Mach Intell 2015; 37: 175-88.
38. Castelli V and Cover TM. On the exponential value of labeled samples. Pattern Recogn Lett 1995; 16: 105-11.
39. Wang W and Zhou ZH. Theoretical foundation of co-training and disagreement-based algorithms. arXiv:1708.04403, 2017.
40. Dietterich TG, Lathrop RH and Lozano-Pérez T. Solving the multiple-instance problem with axis-parallel rectangles. Artif Intell 1997; 89: 31-71.
41. Foulds J and Frank E. A review of multi-instance learning assumptions. Knowl Eng Rev 2010; 25: 1-25.
42. Zhou Z-H.Multi-instance learning from supervised view. J Comput Sci Technol 2006; 21: 800-9.
43. Zhou Z-H and Zhang M-L. Solving multi-instance problems with classifier ensemble based on constructive clustering. Knowl Inform Syst 2007; 11: 155-70.
44. Wei X-S,Wu J and Zhou Z-H Scalable algorithms for multi-instance learning. IEEE Trans Neural Network Learn Syst 2017; 28:975-87.
45. Amores J. Multiple instance classification: review, taxonomy and comparative study. Artif Intell 2013; 201: 81-105.
46. Zhou Z-H and Xu J-M. On the relation between multi-instance learning and semi-supervised learning. In 24th International Conference onMachine Learning, Corvallis, OR, 2007; 1167-74.
47. Zhou Z-H, Sun Y-Y and Li Y-F. Multi-instance learning by treating instances as non-i.i.d. samples. In 26th International Conference on Machine Learning, Montreal, Canada, 2009; 1249-56.
48. Chen Y and Wang JZ. Image categorization by learning and reasoning with regions. J Mach Learn Res 2004; 5: 913-39.
49. Zhang Q, Yu W and Goldman SA et al. Content-based image retrieval using multiple-instance learning. In 19th International Conference on Machine Learning, Sydney, Australia, 2002; 682-9.
50. Tang JH, Li HJ and Qi GJ et al. Image annotation by graph-based inference with integrated multiple/single instance representations. IEEE Trans Multimed 2010; 12: 131-41.
51. Andrews S, Tsochantaridis I and Hofmann T. Support vector machines for multiple-instance learning. In Advances in Neural Information Processing Systems 15, Cambridge, MA: MIT Press, 2003; 561-8.
52. Settles B, CravenMand Ray S.Multiple-instance active learning. In Advances in Neural Information Processing Systems 20, Cambridge, MA: MIT Press, 2008; 1289-96.
53. Jorgensen Z, Zhou Y and Inge M. A multiple instance learning strategy for combating good word attacks on spam filters. JMach Learn Res 2008; 8: 993-1019.
54. Fung G, Dundar M and Krishnappuram B et al. Multiple instance learning for computer aided diagnosis. In Advances in Neural Information Processing Systems 19, Cambridge, MA: MIT Press, 2007; 425-32.
55. Viola P, Platt J and Zhang C. Multiple instance boosting for object detection. In Advances in Neural Information Processing Systems 18, Cambridge, MA: MIT Press, 2006; 1419-26.
56. Felzenszwalb PF, Girshick RB and McAllester D et al. Object detection with discriminatively trained part-based models. IEEE Trans Pattern Anal Mach Intell 2010; 32: 1627-45.
57. Zhu J-Y,Wu J and Xu Y et al. Unsupervised object class discovery via saliencyguided multiple class learning. IEEE Trans Pattern Anal Mach Intell 2015; 37: 862-75.
58. Babenko B, YangMHand Belongie S. Robust object tracking with onlinemultiple instance learning. IEEE Trans Pattern Anal Mach Intell 2011; 33: 1619-32.
59. Wei X-S and Zhou Z-H. An empirical study on image bag generators for multiinstance learning. Mach Learn 2016; 105: 155-98.
60. Liu G, Wu J and Zhou ZH. Key instance detection in multi-instance learning. In 4th Asian Conference on Machine Learning, Singapore, 2012; 253-68.
61. Xu X and Frank E. Logistic regression and boosting for labeled bags of instances. In 8th Pacific-Asia Conference on Knowledge Discovery and Data Mining, Sydney, Australia, 2004; 272-81.
62. Chen Y, Bi J and Wang JZ. MILES: multiple-instance learning via embedded instance selection. IEEE Trans Pattern Anal Mach Intell 2006; 28: 1931-47.
63. Weidmann N, Frank E and Pfahringer B. A two-level learning method for generalized multi-instance problem. In 14th European Conference on Machine Learning, Cavtat-Dubrovnik, Croatia, 2003; 468-79.
64. Long PM and Tan L. PAC learning axis-aligned rectangles with respect to product distributions from multiple-instance examples. Mach Learn 1998; 30: 7-21.
65. Auer P, Long PM and Srinivasan A. Approximating hyper-rectangles: learning and pseudo-random sets. J Comput Syst Sci 1998; 57: 376-88.
66. Blum A and Kalai A. A note on learning from multiple-instance examples. Mach Learn 1998; 30: 23-9.
67. Sabato S and Tishby N. Homogenous multi-instance learning with arbitrary dependence. In 22nd Conference on Learning Theory,Montreal, Canada, 2009.
68. Frénay B and Verleysen M. Classification in the presence of label noise: a survey. IEEE Trans Neural Network Learn Syst 2014; 25: 845-69.
69. Angluin D and Laird P. Learning from noisy examples. Mach Learn 1988; 2: 343-70.
70. Blum A, Kalai A and Wasserman H. Noise-tolerant learning, the parity problem, and the statistical query model. J ACM2003; 50: 506-19.
71. Gao W, Wang L and Li YF et al. Risk minimization in the presence of label noise. In 30th AAAI Conference on Artificial Intelligence, Phoenix, AZ, 2016; 1575-81.
72. Brodley CE and Friedl MA. Identifying mislabeled training data. J Artif Intell Res 1999; 11: 131-67.
73. Muhlenbach F, Lallich S and Zighed DA. Identifying and handling mislabelled instances. J Intell Inform Syst 2004; 22: 89-109.
74. Brabham DC. Crowdsourcing as a model for problem solving: an introduction and cases. Convergence 2008; 14: 75-90.
75. Sheng VS, Provost FJ and Ipeirotis PG. Get another label? Improving data quality and data mining using multiple, noisy labelers. In 14th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, Las Vegas, NV, 2008; 614-22.
76. Snow R, O'Connor B and Jurafsky D et al. Cheap and fast -but is it good? Evaluating non-expert annotations for natural language tasks. In 2008 Conference on Empirical Methods in Natural Language Processing, Honolulu, HI, 2008; 254-63.
77. Raykar VC, Yu S and Zhao LH et al. Learning from crowds. J Mach Learn Res 2010; 11: 1297-322.
78. Whitehill J, Ruvolo P and Wu T et al. Whose vote should count more: optimal integration of labels from labelers of unknown expertise. In Advances in Neural Information Processing Systems 22, Cambridge, MA:MIT Press, 2009; 2035-43.
79. Raykar VC and Yu S. Eliminating spammers and ranking annotators for crowdsourced labeling tasks. J Mach Learn Res 2012; 13: 491-518.
80. Wang W and Zhou ZH. Crowdsourcing label quality: a theoretical analysis. Sci China Inform Sci 2015; 58: 1-12.
81. Dekel O and Shamir O. Good learners for evil teachers. In 26th International Conference on Machine Learning, Montreal, Canada, 2009; 233-40.
82. Urner R, Ben-David S and Shamir O. Learning from weak teachers. In 15th International Conference on Artificial Intelligence and Statistics, La Palma, Canary Islands, 2012; 1252-60.
83. Wang L and Zhou ZH. Cost-saving effect of crowdsourcing learning. In 25th International Joint Conference on Artificial Intelligence, New York, NY, 2016; 2111-7.
84. Karger DR, Sewoong O and Devavrat S. Iterative learning for reliable crowdsourcing systems. In Advances in Neural Information Processing Systems 24, Cambridge, MA: MIT Press, 2011; 1953-61.
85. Tran-Thanh L, Venanzi M and Rogers A et al. Efficient budget allocation with accuracy guarantees for crowdsourcing classification tasks. In 12th International conference on Autonomous Agents and Multi-Agent Systems, Saint Paul, MN, 2013; 901-8.
86. Ho CJ, Jabbari S and Vaughan JW. Adaptive task assignment for crowdsourced classification. In 30th International Conference onMachine Learning, Atlanta, GA, 2013; 534-42.
87. Chen X, Lin Q and Zhou D. Optimistic knowledge gradient policy for optimal budget allocation in crowdsourcing. In 30th International Conference on Machine Learning, Atlanta, GA, 2013; 64-72.
88. Dawid AP and Skene AM. Maximum likelihood estimation of observer errorrates using the EM algorithm. J Roy Stat Soc C Appl Stat 1979; 28: 20-8.
89. Zhong J, Tang K and Zhou Z-H. Active learning from crowds with unsure option. In 24th International Joint Conference on Artificial Intelligence, Buenos Aires, Argentina, 2015; 1061-7.
90. Ding YX and Zhou ZH. Crowdsourcing with unsure opinion. arXiv:1609.00292, 2016.
91. Shah NB and Zhou D. Double or nothing: multiplicative incentive mechanisms for crowdsourcing. In Advances in Neural Information Processing Systems 28, Cambridge, MA: MIT Press, 2015; 1-9.
92. Rahmani R and Goldman SA. MISSL:multiple-instance semi-supervised learning. In 23rd International Conference on Machine Learning, Pittsburgh, PA, 2006; 705-12.
93. Yan Y, Rosales R and Fung G et al. Active learning from crowds. In 28th International Conference on Machine Learning, Bellevue, WA, 2011; 1161-8.
94. Sutton RS and Barto AG. Reinforcement Learning: An Introduction. Cambridge: MIT Press, 1998.
95. Schwenker F and Trentin E. Partially supervised learning for pattern recognition. Pattern Recogn Lett 2014; 37: 1-3.
96. Garcia-Garcia D and Williamson RC. Degrees of supervision. In Advances in Neural Information Processing Systems 17, Cambridge, MA:MIT PressWorkshops, 2011.
97. Hernández-González J, Inza I and Lozano JA.Weak supervision and other nonstandard classification problems: a taxonomy. Pattern Recogn Lett 2016; 69: 49-55.
98. Kuncheva LI, Rodíguez JJ and Jackson AS. Restricted set classification: who is there? Pattern Recogn 2017; 63:158-70.
99. Zhang M-L and Zhou Z-H. A review on multi-label learning algorithms. IEEE Trans Knowl Data Eng 2014; 26: 1819-37.
100. Sun YY, Zhang Y and Zhou ZH. Multi-label learning with weak label. In 24th AAAI Conference on Artificial Intelligence, Atlanta, GA, 2010; 593-8.
101. Li X and Guo Y. Active learning with multi-label SVM classification. In 23rd International Joint Conference on Artificial Intelligence, Beijing, China, 2013; 1479-85.
102. Qi GJ, Hua XS and Rui Y et al. Two-dimensional active learning for image classification. In IEEE Computer Society Conference on Computer Vision and Pattern Recognition, Anchorage, AK, 2008.
103. Huang SJ, Chen S and Zhou ZH. Multi-label active learning: query type matters. In 24th International Joint Conference on Artificial Intelligence, Buenos Aires, Argentina, 2015; 946-52.