Pattern classification and clustering: A review of partially supervised learning approaches

Pattern Recognition Letters

Volumn 37, Issue 1, 2014, Pages 4-14

  Schwenker, Friedhelm ^a   Trentin, Edmondo ^b  

^a UNIVERSITY OF ULM   (Germany)

^b UNIVERSITY OF SIENA   (Italy)

Author keywords

Active learning; Multi view learning; Neural network; Partially supervised learning; Semi supervised learning; Transductive learning

Indexed keywords

CLASSIFICATION (OF INFORMATION); CLUSTER ANALYSIS; CLUSTERING ALGORITHMS; FUZZY NEURAL NETWORKS; NEURAL NETWORKS; PATTERN RECOGNITION; SUPERVISED LEARNING;

ACTIVE LEARNING; MULTI-VIEW LEARNING; PARTIALLY SUPERVISED LEARNING; SEMI- SUPERVISED LEARNING; TRANSDUCTIVE LEARNING;

LEARNING ALGORITHMS;

EID: 84891629945     PISSN: 01678655     EISSN: None     Source Type: Journal    
DOI: 10.1016/j.patrec.2013.10.017     Document Type: Article

References (118)

1. M. Adankon, and M. Cheriet Genetic algorithm based training for semi-supervised SVM Neural Computing and Applications 19 2010 1197 1206
2. E. Alpaydin Introduction to Machine Learning second ed. 2010 MIT Press
3. R. Artstein, and M. Poesio Inter-coder agreement for computational linguistics Computational Linguistics 34 4 2008 555 596
4. Basu, S., 2005. Semi-supervised clustering: probabilistic models, algorithms and experiments. Ph.D. Thesis, University of Texas at Austin. .
5. Basu, S., Banerjee, A., Mooney, R., 2002. Semi-supervised clustering by seeding. In: Proc. of the 19th International Conference on, Machine Learning (ICML'02), pp. 19-26.
6. Basu, S., Bilenko, M., Mooney, R., 2004. A probabilistic framework for semi-supervised clustering. In: Proc. of the 10th ACM SIGKDD Conference on Knowledge Discovery and Data Mining (KDD'04), pp. 59-68. (Pubitemid 40114916)
7. M. Belkin, P. Niyogi, and V. Sindhwani Manifold regularization: a geometric framework for learning from labeled and unlabeled examples Journal of Machine Learning Research 7 2006 2399 2434 (Pubitemid 44708005)
8. A. Bertoni, M. Frasca, and G. Valentini COSNet: a cost sensitive neural network for semi-supervised learning in graphs Proc. of the 2011 European Conference on Machine Learning and Knowledge Discovery in Databases (ECML/PKDD'11) 2011 Springer Berlin, Heidelberg 219 234
9. T.D. Bie, and N. Cristianini Semi-supervised learning using semi-definite programming Chapelle O., B. Schölkopf, A. Zien, Semi-Supervised Learning 2006 MIT Press 119 135
10. C.M. Bishop Pattern Recognition and Machine Learning 2006 Springer
11. Blum, A., Chawla, S., 2001. Learning from labeled and unlabeled data using graph mincuts. In: Proc. of the 18th International Conference on, Machine Learning (ICML'01), pp. 19-26.
12. Blum, A., Mitchell, T., 1998. Combining labeled and unlabeled data with co-training. In: Proc. of the 11th Annual Conference on Computational Learning Theory (COLT 1998), pp. 92-100.
13. Bouchachia, A., 2005. RBF networks for learning from partially labeled data. In: Proc. of the 22nd ICML Workshop on Learning with Partially Classified Training Data, pp. 10-19.
14. Chapelle, O., Zien, A., 2005. Semi-supervised learning by low density separation. In: Proc. of the 10th International Workshop on Artificial Intelligence and, Statistics, pp. 57-64.
15. Chapelle, O., Chi, M., Zien, A., A continuation method for semi-supervised SVMs. In: International Conference on, Machine Learning (ICML'06), pp. 185-192.
16. Chapelle, O., Sindhwani, V., Keerthi, S., Branch and bound for semi-supervised support vector machines. In: Advances in Neural Information Processing Systems (NIPS'06), pp. 217-224.
17. O. Chapelle, V. Sindhwani, and S.S. Keerthi Optimization techniques for semi-supervised support vector machines The Journal of Machine Learning Research 9 2008 203 233 (Pubitemid 351469022)
18. O. Chapelle, B. Schölkopf, and A. Zien Semi-Supervised Learning 2010 MIT Press
19. S.M. Chu, H. Tang, and T.S. Huang Fisher-voice and semi-supervised speaker clustering IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP'F09) 2009 IEEE 4089 4092
20. C. Constantinopoulos, and A. Likas Semi-supervised and active learning with the probabilistic RBF classifier Neurocomputing 71 13-15 2008 2489 2498
21. Cozman, F.G., Cohen, I., 2002. Unlabeled data can degrade classification performance of generative classifiers. In: Proc. of the 15th International Conference of the Florida Artificial Intelligence Research Society (FLAIRS'02), pp. 327-331.
22. I. Dagan, and S.P. Engelson Committee-based sampling for training probabilistic classifiers Proc. of the 12th International Conference on Machine Learning (ICML'95) 1995 Morgan Kaufmann 150 157
23. A. Dempster, N. Laird, and D. Rubin Maximum likelihood from incomplete data via the EM algorithm Journal of the Royal Statistical Society. Series B Methodological 39 1 1977 1 38
24. S. Faußer, and F. Schwenker Semi-supervised kernel clustering with sample-to-cluster weights F. Schwenker, E. Trentin, Partially Supervised Learning (PSL'11) LNAI vol. 7081 2012 Springer 72 81
25. M. Frasca, A. Bertoni, M. Re, and G. Valentini A neural network algorithm for semi-supervised node label learning from unbalanced data Neural Networks 43 2013 84 98
26. Y. Freund, H. Seung, E. Shamir, and N. Tishby Selective sampling using the query by committee algorithm Machine Learning 28 2-3 1997 133 168 (Pubitemid 127506338)
27. V. Frinken, and H. Bunke Evaluating retraining rules for semi-supervised learning in neural network based cursive word recognition Proc. of the 10th International Conference on Document Analysis and Recognition (ICDAR'09) 2009 IEEE Computer Society Washington, DC, USA 31 35
28. V. Frinken, and H. Bunke Self-training strategies for handwriting word recognition Advances in Data Mining. Applications and Theoretical Aspects 2009 Springer 291 300
29. G. Fung, and O. Mangasarian Semi-supervised support vector machines for unlabeled data classification Optimization Methods and Software 15 2001 29 44 (Pubitemid 33817502)
30. Furao, S., Sakurai, K., Kamiya, Y., Hasegawa, O., 2007. An online semi-supervised active learning algorithm with self-organizing incremental neural network. In: International Joint Conference on Neural Networks (IJCNN 2007), pp. 1139-1144.
31. N.E. Gayar, F. Schwenker, and G. Palm A study of the robustness of KNN classifiers trained using soft labels F. Schwenker, S. Marinai, Artificial Neural Networks in Pattern Recognition (ANNPR'06) LNAI vol. 4087 2006 Springer 67 80 (Pubitemid 44502872)
32. Goldman, S., Zhou, Y., 2000. Enhancing supervised learning with unlabeled data. In: Proc. of the 17th International Conference on, Machine Learning (ICML'00), pp. 327-334.
33. Y. Grandvalet, and Y. Bengio Semi-supervised learning by entropy minimization Advances in Neural Information Processing Systems (NIPS'05) 17 2005 529 536
34. D. Guan, A. Gavrilov, W. Yuan, Y.-K. Lee, and S. Lee A novel hybrid neural network for data clustering H.R. Arabnia, M. Dehmer, F. Emmert-Streib, M.Q. Yang, MLMTA 2007 CSREA Press 284 288
35. Guillaumin, M., Verbeek, J., Schmid, C., 2010. Multimodal semi-supervised learning for image classification. In: IEEE Conference on Computer Vision & Pattern Recognition, pp. 902-909.
36. M. Abdel Hady, and F. Schwenker Combining committee-based semi-supervised learning and active learning Journal of Computer Science and Technology (JCST): Special Issue on Advances in Machine Learning and Applications 25 4 2010 681 698
37. M.F.A. Hady, and F. Schwenker Semi-supervised learning Handbook on Neural Information Processing 2013 Springer 215 239
38. M.F.A. Hady, F. Schwenker, and G. Palm Semi-supervised learning for tree-structured ensembles of RBF networks with co-training Neural Networks 23 4 2010 497 509
39. D. Hakkani-Tür, G. Riccardi, and G. Tur An active approach to spoken language processing ACM Transaction on Speech Language Processing 3 3 2006 1 31 (Pubitemid 44728698)
40. M. Haque, L. Holder, M. Skinner, and D. Cook Generalized query based active learning to identify differentially methylated regions in DNA IEEE/ACM Transactions on Computational Biology and Bioinformatics 2013 1
41. Herrmann, L., Ultsch, A., 2007. Label propagation for semi-supervised learning in self-organizing maps. In: Proc. of the 6th International Workshop on Self-Organizing Maps (WSOM'07), Bielefeld Germany.
42. J. Hertz, A. Krogh, and R. Palmer Introduction to the Theory of Neural Computation 1991 Addison Wesley
43. M. Inoue, and N. Ueda Exploitation of unlabeled sequences in hidden Markov models IEEE Transactions on Pattern Analysis and Machine Intelligence 25 12 2003 1570 1581
44. A.K. Jain Data clustering: 50 years beyond k-means Pattern Recognition Letters 31 8 2010 651 666
45. A.K. Jain, R.P.W. Duin, and J. Mao Statistical pattern recognition: a review IEEE Transactions on Pattern Analysis and Machine Intelligence 22 1 2000 4 37 (Pubitemid 32132953)
46. E.P. Jiang Semi-supervised text classification using RBF networks N.M. Adams, C. Robardet, A. Siebes, J.-F. Boulicaut, IDA vol. 5772 2009 Springer 95 106
47. Joachims, T., 1999. Transductive inference for text classification using support vector machines. In: Proc.of the 16th International Conference on, Machine Learning (ICML'99), pp. 200-209.
48. M. Kalakech, P. Biela, L. Macaire, and D. Hamad Constraint scores for semi-supervised feature selection: a comparative study Pattern Recognition Letters 32 5 2011 656 665
49. Kamiya, Y., Ishii, T., Furao, S., Hasegawa, O., 2007. An online semi-supervised clustering algorithm based on a self-organizing incremental neural network, In: International Joint Conference on, Neural Networks (IJCNN'07), pp. 1061-1066.
50. N. Karayiannis, and J. Bezdek An integrated approach to fuzzy learning vector quantization and fuzzy c-means clustering IEEE Transactions on Fuzzy Systems 5 4 1997 622 628 (Pubitemid 127768540)
51. Kiritchenko, S., Matwin, S., 2001. E-mail classification with co-training. In: Proc. of the 2001 Conference of the Centre for Advanced Studies on Collaborative Research (CASCON'01), pp. 8-19.
52. B. Kulis, S. Basu, I. Dhillon, and R. Mooney Semi-supervised graph clustering: a kernel approach Machine Learning 74 1 2009 1 22
53. H. Lee, J. Yoo, and S. Choi Semi-supervised nonnegative matrix factorization IEEE Signal Processing Letters 17 1 2010 4 7
54. Leite, D., Costa, P., Gomide, F., 2010. Evolving granular neural network for semi-supervised data stream classification. In: International Joint Conference on Neural Networks (IJCNN), pp. 1877-1884.
55. Lewis, D., Catlett, J., 1994. Heterogeneous uncertainty sampling for supervised learning. In: Proc. of the 11th International Conference on, Machine Learning (ICML'94), pp. 148-156.
56. M. Li, and Z.-H. Zhou Improve computer-aided diagnosis with machine learning techniques using undiagnosed samples IEEE Transactions on Systems, Man and Cybernetics Part A: Systems and Humans 37 6 2007 1088 1098 (Pubitemid 350130846)
57. Y. Li, C. Guan, H. Li, and Z. Chin A self-training semi-supervised SVM algorithm and its application in an EEG-based brain computer interface speller system Pattern Recognition Letters 29 9 2008 1285 1294
58. M. Lindenbaum, S. Markovitch, and D. Rusakov Selective sampling for nearest neighbor classifiers Machine Learning 54 2 2004 125 152
59. Luo, Z.-P., Zhang, X.-M., 2008. A semi-supervised learning based relevance feedback algorithm in content-based image retrieval. In: Chinese Conference on Pattern Recognition (CCPR '08), pp. 1-4.
60. J. Malkin, A. Subramanya, and J. Bilmes On the semi-supervised learning of multi-layered perceptrons INTERSPEECH 2009 ISCA 660 663
61. McCallum, A., Nigam, K., 1998. Employing em and pool-based active learning for text classification. In: Proc. of the 15th International Conference on, Machine Learning (ICML'98), pp. 350-358.
62. S. Meudt, and F. Schwenker On instance selection in audio based emotion recognition N. Mana, F. Schwenker, E. Trentin, Artificial Neural Networks in Pattern Recognition (ANNPR'12) LNAI vol. 7477 2012 Springer 186 192
63. D.J. Miller, and H.S. Uyar A mixture of experts classifier with learning based on both labelled and unlabelled data Advances in Neural Information Processing Systems 9 1997 571 577
64. G. Nagy, and G. Shelton Self-corrective character recognition system IEEE Transactions on Information Theory 12 2 1966 215 222
65. Nigam, K., 2001. Using unlabeled data to improve text classification. Ph.D. Thesis, School of Computer Science, Carnegie Mellon University, Pittsburgh, USA.
66. Nigam, K., Ghani, R., 2000. Analyzing the effectiveness and applicability of co-training. In: Proc. of the Ninth International Conference on Information and, Knowledge Management, pp. 86-93.
67. K. Nigam, A.K. McCallum, S. Thrun, and T. Mitchell Text classification from labeled and unlabeled documents using EM Machine Learning 39 2-3 2000 103 134 (Pubitemid 30594822)
68. S. Patra, and L. Bruzzone A cluster-assumption based batch mode active learning technique Pattern Recognition Letters 33 9 2012 1042 1048
69. S. Patra, S. Ghosh, and A. Ghosh Semi-supervised learning with multilayer perceptron for detecting changes of remote sensing images A. Ghosh, R. De, S. Pal, Pattern Recognition and Machine Intelligence vol. 4815 2007 Springer Berlin, Heidelberg 161 168
70. B. Peng, G. Qian, and Y. Ma Recognizing body poses using multilinear analysis and semi-supervised learning Pattern Recognition Letters 30 14 2009 1289 1294
71. J.C. Platt Probabilistic outputs for support vector machines and comparisons to regularized likelihood methods Advances in Large Margin Classifiers 1999 MIT Press 61 74
72. Poggio, T., Girosi, F., 1989. A theory of networks for approximation and learning. Laboratory, Massachusetts Institute of Technology, Technical Report 1140.
73. Y. Qi, O. Tastan, J.G. Carbonell, J. Klein-Seetharaman, and J. Weston Semi-supervised multi-task learning for predicting interactions between HIV-1 and human proteins Bioinformatics 26 18 2010 i645 i652
74. M.A. Ranzato, and M. Szummer Semi-supervised learning of compact document representations with deep networks Proc. of the 25th International conference on Machine Learning (ICML'08) 2008 ACM New York, NY, USA 792 799
75. A. Raychaudhuri, and J. Dutta Image binarization using multi-layer perceptron: a semi-supervised approach International Journal of Engineering Innovations and Research 1 2 2012 134 139
76. T. Scheffer, C. Decomain, and S. Wrobel Active hidden Markov models for information extraction F. Hoffmann, D.J. Hand, N.M. Adams, D.H. Fisher, G. Guimarães, Advances in Intelligent Data Analysis (IDA'01) LNCS vol. 2189 2001 Springer 309 318 (Pubitemid 33348510)
77. S. Scherer, M. Glodek, G. Layher, M. Schels, M. Schmidt, T. Brosch, S. Tschechne, F. Schwenker, H. Neumann, and G. Palm A generic framework for the inference of user states in human computer interaction: how patterns of low level communicational cues support complex affective states Journal on Multimodal User Interfaces 6 3 2012 117 141
78. S. Scherer, J. Kane, C. Gobl, and F. Schwenker Investigating fuzzy-input fuzzy-output support vector machines for robust voice quality classification Computer Speech & Language 27 1 2013 263 287
79. F. Schwenker, H.A. Kestler, and G. Palm Three learning phases for radial-basis-function networks Neural Networks 14 4-5 2001 439 458 (Pubitemid 32475857)
80. Seeger, M., 2002. Learning with labeled and unlabeled data. Technical Report, University of Edinburgh, Institute for Adaptive and Neural Computation.
81. S. Seo, M. Bode, and K. Obermayer Soft nearest prototype classification IEEE Transactions on Neural Networks 14 2 2003 390 398
82. Settles, B., 2009. Active learning literature survey. Tech. rep., Department of Computer Sciences, University of Wisconsin-Madison, Madison, WI.
83. B. Settles, and M. Craven An analysis of active learning strategies for sequence labeling tasks Proceedings of the Conference on Empirical Methods in Natural Language Processing (EMNLP'08) 2008 Association for Computational Linguistics 1070 1079
84. H.S. Seung, M. Opper, and H. Sompolinsky Query by committee Proc. of the Fifth Annual Workshop on Computational Learning Theory (COLT '92) 1992 ACM 287 294 (Pubitemid 23615454)
85. B. Shahshahani, and D. Landgrebe The effect of unlabeled samples in reducing the small sample size problem and mitigating the Hughes phenomenon IEEE Transactions on Geoscience and Remote Sensing 32 5 1994 1087 1095
86. F. Shen, H. Yu, K. Sakurai, and O. Hasegawa An incremental online semi-supervised active learning algorithm based on self-organizing incremental neural network Neural Computing and Applications 20 7 2011 1061 1074
87. Sindhwani, V., Keerthi, S., Chapelle, O., 2006. Deterministic annealing for semi-supervised kernel machines. In: International Conference on, Machine Learning (ICML'06), pp. 841-848.
88. A. Singh, R. Nowak, and X. Zhu Unlabeled data: now it helps, now it doesn't Advances in Neural Information Processing Systems (NIPS'08) 21 2008 1513 1520
89. M. Soleymani Baghshah, and S. Bagheri Shouraki Kernel-based metric learning for semi-supervised clustering Neurocomputing 73 7 2010 1352 1361
90. Y. Song, F. Nie, and C. Zhang Semi-supervised sub-manifold discriminant analysis Pattern Recognition Letters 29 13 2008 1806 1813
91. C. Thiel, F. Giacco, F. Schwenker, and G. Palm Comparison of neural classification algorithms applied to land cover mapping B. Apolloni, S. Bassis, M. Marinaro, New Directions in Neural Networks (WIRN 2008) Frontiers in Artificial Intelligence and Applications vol. 193 2009 IOS Press 254 263
92. C. Thiel, S. Scherer, and F. Schwenker Fuzzy-input fuzzy-output one-against-all support vector machines B. Apolloni, R.J. Howlett, L.C. Jain, Knowledge-Based Intelligent Information and Engineering Systems (KES'07) LNCS vol. 4694 2007 Springer 156 165
93. C. Thiel, B. Sonntag, and F. Schwenker Experiments with supervised fuzzy LVQ L. Prevost, S. Marinai, F. Schwenker, Artificial Neural Networks in Pattern Recognition (ANNPR'08) LNAI vol. 5064 2008 Springer 125 132
94. S. Tong, and E. Chang Support vector machine active learning for image retrieval Proc.of the 9th ACM International Conference on Multimedia (MULTIMEDIA'01) 2001 ACM 107 118 (Pubitemid 32962460)
95. S. Tong, and D. Koller Support vector machine active learning with applications to text classification Journal Machine Learning Research 2 2002 45 66
96. E. Trentin Simple and effective connectionist nonparametric estimation of probability density functions Proc. of the IAPR Workshop on Artificial Neural Networks in Pattern Recognition (ANNPR'06) 2006 Springer 1 10 (Pubitemid 44502867)
97. E. Trentin, L. Lusnig, and F. Cavalli Comparison of combined probabilistic connectionist models in a forensic application Proc. of the First IAPR Workshop on Partially Supervised Learning (PSL'11) 2011 Springer 128 137
98. D. Tuia, M. Volpi, L. Copa, M. Kanevski, and J. Munoz-Mari A survey of active learning algorithms for supervised remote sensing image classification IEEE Journal of Selected Topics in Signal Processing 5 3 2011 606 617
99. V. Vapnik The Nature of Statistical Learning Theory 1995 Springer-Verlag
100. A. Verikas, A. Gelzinis, and K. Malmqvist Using unlabelled data to train a multilayer perceptron Neural Processing Letters 14 3 2001 179 201 (Pubitemid 33110850)
101. Villmann, T., Schleif, F.-M., Hammer, B., 2005. Fuzzy labeled soft nearest neighbor classification with relevance learning. In: Proc.of the Fourth International Conference on Machine Learning and Applications (ICMLA'05), pp. 11-15.
102. K.L. Wagstaff Constrained clustering Encyclopedia of Machine Learning 2010 Springer 220 221
103. Wagstaff, K., Cardie, C., Schroedl, S., 2001. Constrained k-means clustering with background knowledge. In: Proc. of the 18th International Conference on, Machine Learning (ICML'01), pp. 577-584.
104. J. Weston, F. Ratle, and R. Collobert Deep learning via semi-supervised embedding Proc. of the 25th international Conference on Machine Learning (ICML'08) 2008 ACM New York, NY, USA 1168 1175
105. T. Young, and A. Farjo On decision directed estimation and stochastic approximation IEEE Transactions on Information Theory 18 5 1972 671 673
106. Z. Yu, L. Su, L. Li, Q. Zhao, C. Mao, and J. Guo Question classification based on co-training style semi-supervised learning Pattern Recognition Letters 31 13 2010 1975 1980
107. L. Zhang Contextual and active learning-based affect-sensing from virtual drama improvisation ACM Transaction on Speech Language Processing 9 4 2013 8:1 8:25
108. Q. Zhang, and S. Sun Multiple-view multiple-learner active learning Pattern Recognition 43 9 2010 3113 3119
109. Z.-H. Zhou Ensemble Methods: Foundations and Algorithms 2012 Chapman Hall/CRC
110. Y. Zhou, and S. Goldman Democratic co-learning Proc. of the 16th IEEE International Conference on Tools with Artificial Intelligence (ICTAI'04) 2004 IEEE Computer Society 594 602 (Pubitemid 40455067)
111. Z.-H. Zhou, and M. Li Tri-training: exploiting unlabeled data using three classifiers IEEE Transactions on Knowledge and Data Engineering 17 11 2005 1529 1541 (Pubitemid 41704840)
112. Zhou, Z.-H., Li, M., 2005. Semi-supervised regression with co-training. In: Proc. of the 19th International Joint Conference on Artificial Intelligence (IJCAI'05), pp. 908-913.
113. Z.-H. Zhou, and M. Li Semi-supervised learning by disagreement Knowledge and Information Systems 24 3 2010 415 439
114. D. Zhou, O. Bousquet, T.N. Lal, J. Weston, and B. Schölkopf Learning with local and global consistency Advances in Neural Information Processing Systems 16 2004 753 760
115. Zhou, Z.-H., Zhang, D., Chen, S., 2007. Semi-supervised dimensionality reduction. In: Proc. of the Seventh SIAM International Conference on Data Mining (SDM'07), pp. 629-634.
116. S. Zhou, Q. Chen, and X. Wang Active deep networks for semi-supervised sentiment classification Proc.of the 23rd International Conference on Computational Linguistics (COLING '10) 2010 Association for Computational Linguistics Stroudsburg, PA, USA 1515 1523
117. Zhu, X., 2008. Semi-supervised learning literature survey. Technical Report 1530.
118. Zhu, X., Ghahramani, Z., Lafferty, J., 2003. Semi-supervised learning using gaussian fields and harmonic functions. In: Proc. of the 20th International Conference on, Machine Learning (ICML'03), pp. 912-919.