A Survey on semi-supervised feature selection methods

Pattern Recognition

Volumn 64, Issue , 2017, Pages 141-158

  Sheikhpour, Razieh ^a   Sarram, Mehdi Agha ^a   Gharaghani, Sajjad ^b   Chahooki, Mohammad Ali Zare ^a  

^a YAZD UNIVERSITY   (Iran)

^b UNIVERSITY OF TEHRAN   (Iran)

Author keywords

Feature selection; Semi supervised learning; Survey

Indexed keywords

DATA MINING; LEARNING ALGORITHMS; LEARNING SYSTEMS; SUPERVISED LEARNING; SURVEYING; SURVEYS; TAXONOMIES;

FEATURE SELECTION METHODS; HIERARCHICAL STRUCTURES; LABELED AND UNLABELED DATA; LEARNING PERFORMANCE; MACHINE LEARNING APPLICATIONS; REDUNDANT FEATURES; SEMI- SUPERVISED LEARNING; SEMI-SUPERVISED LEARNING METHODS;

FEATURE EXTRACTION;

EID: 85007404250     PISSN: 00313203     EISSN: None     Source Type: Journal    
DOI: 10.1016/j.patcog.2016.11.003     Document Type: Article

References (96)

1. [1] Kalakech, M., Biela, P., Macaire, L., Hamad, D., Constraint scores for semi-supervised feature selection: a comparative study. Pattern Recognit. Lett. 32 (2011), 656–665, 10.1016/j.patrec.2010.12.014.
2. [2] Zhao, M., Zhang, Z., Chow, T.W.S., Trace ratio criterion based generalized discriminative learning for semi-supervised dimensionality reduction. Pattern Recognit. 45 (2012), 1482–1499, 10.1016/j.patcog.2011.10.008.
3. [3] M.Hindawi, K.Allab, K.Benabdeslem, Constraint selection-based semi-supervised feature selection, in: Proceedings of the ICDM, IEEE, 2011, pp. 1080–1085.
4. [4] Shen, K.-Q., Ong, C.-J., Li, X.-P., Wilder-Smith, E.P.V., Feature selection via sensitivity analysis of SVM probabilistic outputs. Mach. Learn. 70 (2008), 1–20, 10.1007/s10994-007-5025-7.
5. [5] Benabdeslem, K., Hindawi, M., Efficient semi-supervised feature selection: constraint, relevance, and redundancy. IEEE Trans. Knowl. Data Eng. 26 (2014), 1131–1143, 10.1109/TKDE.2013.86.
6. [6] Zhang, D., Chen, S., Zhou, Z.-H., Constraint score: a new filter method for feature selection with pairwise constraints. Pattern Recognit. 41 (2008), 1440–1451, 10.1016/j.patcog.2007.10.009.
7. [7] Reif, M., Shafait, F., Efficient feature size reduction via predictive forward selection. Pattern Recognit. 47 (2014), 1664–1673, 10.1016/j.patcog.2013.10.009.
8. [8] Xue, B., Zhang, M., Member, S., Browne, W.N., Particle swarm optimization for feature selection in classification: a multi-objective approach. IEEE Trans. Cybern. 43 (2013), 1656–1671.
9. [9] Zhang, X., Wu, G., Dong, Z., Crawford, C., Embedded feature-selection support vector machine for driving pattern recognition. J. Frankl. Inst. 352 (2015), 669–685, 10.1016/j.jfranklin.2014.04.021.
10. [10] Yang, J.D., Xu, H., Jia, P.F., Effective search for genetic-based machine learning systems via estimation of distribution algorithms and embedded feature reduction techniques. Neurocomputing 113 (2013), 105–121, 10.1016/J.Neucom.2013.01.014.
11. [11] H.Cheng, W.Deng, C.Fu, Y.Wang, Z.Qin, Graph-based semi-supervised feature selection with application to automatic spam image identification, in: Proceedings of the Computer Science for Environmental Engineering and EcoInformatics, Springer, 2011, pp. 259–264.
12. [12] Chen, X., Fang, T., Huo, H., Li, D., Semisupervised feature selection for unbalanced sample sets of VHR images. IEEE Geosci. Remote Sens. Lett. 7 (2010), 781–785, 10.1109/LGRS.2010.2048197.
13. [13] Sun, Y., Wen, G., Emotion recognition using semi-supervised feature selection with speaker normalization. Int. J. Speech Technol., 2015, 1–15, 10.1007/s10772-015-9272-x.
14. [14] Chen, C.-H., A semi-supervised feature selection method using a non-parametric technique with pairwise instance constraints. J. Inf. Sci. 39 (2013), 359–371, 10.1177/0165551512456502.
15. [15] L. Yang, L. Wang, Simultaneous feature selection and classification via semi-supervised models, in: Proceedings of the Third International Conference on Natural Computation, ICNC 2007, 2007, pp. 646–650. 〈 http://dx.doi.org/10.1109/ICNC.2007.666〉.
16. [16] Mitra, P., Murthy, C.A., Pal, S.K., Unsupervised feature selection using feature similarity. IEEE Trans. Pattern Anal. Mach. Intell. 24 (2002), 301–312, 10.1109/34.990133.
17. [17] Maldonado, S., Weber, R., Basak, J., Simultaneous feature selection and classification using kernel-penalized support vector machines. Inf. Sci. 181 (2011), 115–128, 10.1016/j.ins.2010.08.047.
18. [18] Uǧuz, H., A two-stage feature selection method for text categorization by using information gain, principal component analysis and genetic algorithm. Knowl. Based Syst. 24 (2011), 1024–1032, 10.1016/j.knosys.2011.04.014.
19. [19] Hall, M., Holmes, G., Benchmarking attribute selection techniques for discrete class data mining. IEEE Trans. Knowl. Data Eng. 15 (2003), 1437–1447.
20. [20] Unler, A., Murat, A., Chinnam, R.B., Mr2PSO: a maximum relevance minimum redundancy feature selection method based on swarm intelligence for support vector machine classification. Inf. Sci. 181 (2011), 4625–4641, 10.1016/j.ins.2010.05.037.
21. [21] Chen, C.-H., A hybrid intelligent model of analyzing clinical breast cancer data using clustering techniques with feature selection. Appl. Soft Comput. 20 (2014), 4–14.
22. [22] Pohjalainen, J., Räsänen, O., Kadioglu, S., Feature selection methods and their combinations in high-dimensional classification of speaker likability, intelligibility and personality traits. Comput. Speech Lang. 29 (2015), 145–171.
23. [23] Zhao, Z., Fu, G., Liu, S., Elokely, K.M., Doerksen, R.J., Chen, Y., et al. Drug activity prediction using multiple-instance learning via joint instance and feature selection. BMC Bioinform., 14(Suppl 1), 2013, S16, 10.1186/1471-2105-14-S14-S16.
24. [24] Xue, B., Zhang, M., Browne, W.N., Particle swarm optimisation for feature selection in classification: novel initialisation and updating mechanisms. Appl. Soft Comput. 18 (2014), 261–276, 10.1016/j.asoc.2013.09.018.
25. [25] Peng, Y., Wu, Z., Jiang, J., A novel feature selection approach for biomedical data classification. J. Biomed. Inform. 43 (2010), 15–23, 10.1016/j.jbi.2009.07.008.
26. [26] Nowotny, T., Berna, A.Z., Binions, R., Trowell, S., Optimal feature selection for classifying a large set of chemicals using metal oxide sensors. Sens. Actuators B Chem. 187 (2013), 471–480, 10.1016/j.snb.2013.01.088.
27. [27] Unler, A., Murat, A., A discrete particle swarm optimization method for feature selection in binary classification problems. Eur. J. Oper. Res. 206 (2010), 528–539, 10.1016/j.ejor.2010.02.032.
28. [28] Rashedi, E., Nezamabadi-Pour, H., Saryazdi, S., A simultaneous feature adaptation and feature selection method for content-based image retrieval systems. Knowl. Based Syst. 39 (2013), 85–94, 10.1016/j.knosys.2012.10.011.
29. [29] Chen, H.-L., Yang, B., Liu, J., Liu, D.-Y., A support vector machine classifier with rough set-based feature selection for breast cancer diagnosis. Expert Syst. Appl. 38 (2011), 9014–9022, 10.1016/j.eswa.2011.01.120.
30. [30] Kersten, J., Simultaneous feature selection and Gaussian mixture model estimation for supervised classification problems. Pattern Recognit. 47 (2014), 2582–2595.
31. [31] Peralta, B., Soto, A., Embedded local feature selection within mixture of experts. Inf. Sci. 269 (2014), 176–187, 10.1016/j.ins.2014.01.008.
32. [32] Wang, S., Li, D., Song, X., Wei, Y., Li, H., A feature selection method based on improved fisher's discriminant ratio for text sentiment classification. Expert Syst. Appl. 38 (2011), 8696–8702, 10.1016/j.eswa.2011.01.077.
33. [33] Akay, M.F., Support vector machines combined with feature selection for breast cancer diagnosis. Expert Syst. Appl. 36 (2009), 3240–3247, 10.1016/j.eswa.2008.01.009.
34. [34] Bamakan, S.M.H., Gholami, P., A novel feature selection method based on an integrated data envelopment analysis and entropy model. Procedia Comput. Sci. 31 (2014), 632–638, 10.1016/j.procs.2014.05.310.
35. [35] Nakariyakul, S., Suboptimal branch and bound algorithms for feature subset selection: a comparative study. Pattern Recognit. Lett. 45 (2014), 62–70, 10.1016/j.ins.2014.03.072.
36. [36] Yang, J., Liu, Y., Liu, Z., Zhu, X., Zhang, X., A new feature selection algorithm based on binomial hypothesis testing for spam filtering. Knowl. Based Syst. 24 (2011), 904–914, 10.1016/j.knosys.2011.04.006.
37. [37] Li, G.-Z., Meng, H.-H., Lu, W.-C., Yang, J.Y., Yang, M., Asymmetric bagging and feature selection for activities prediction of drug molecules. BMC Bioinform., 9, 2008, 10.1186/1471-2105-9-S6-S7.
38. [38] Shi, P., Ray, S., Zhu, Q., Kon, M.A., Top scoring pairs for feature selection in machine learning and applications to cancer outcome prediction. BMC Bioinform., 12, 2011, 375, 10.1186/1471-2105-12-375.
39. [39] Zhou, W., Dickerson, J.A., A novel class dependent feature selection method for cancer biomarker discovery. Comput. Biol. Med. 47 (2014), 66–75, 10.1016/j.compbiomed.2014.01.014.
40. [40] K.Benabdeslem, M.Hindawi, Constrained laplacian score for semi-supervised feature selection, in: Proceedings of the Machine Learning and Knowledge Discovery in Databases, Springer, 2011, pp. 204–218.
41. [41] Sheikhpour, R., Sarram, M.A., Sheikhpour, R., Particle swarm optimization for bandwidth determination and feature selection of kernel density estimation based classifiers in diagnosis of breast cancer. Appl. Soft Comput. 40 (2016), 113–131, 10.1016/j.asoc.2015.10.005.
42. [42] Chin, A., Mirzal, A., Haron, H., Hamed, H., Supervised, unsupervised and semi-supervised feature selection: a review on gene selection. IEEE/ACM Trans. Comput. Biol. Bioinform., 2015, 10.1109/TCBB.2015.2478454.
43. [43] Bolón-Canedo, V., Sánchez-Maroño, N., Alonso-Betanzos, A., Benítez, J.M., Herrera, F., A review of microarray datasets and applied feature selection methods. Inf. Sci. 282 (2014), 111–135, 10.1016/j.ins.2014.05.042.
44. [44] Chandrashekar, G., Sahin, F., A survey on feature selection methods. Comput. Electr. Eng. 40 (2014), 16–28, 10.1016/j.compeleceng.2013.11.024.
45. [45] Saeys, Y., Inza, I., Larrañaga, P., A review of feature selection techniques in bioinformatics. Bioinformatics 23 (2007), 2507–2517, 10.1093/bioinformatics/btm344.
46. [46] Guyon, I., Elisseeff, a., An introduction to variable and feature selection. J. Mach. Learn. Res. 3 (2003), 1157–1182, 10.1162/153244303322753616.
47. [47] G.-Z.Yang, X.-P.Hu, Feature Selection, U.S. Pat. Appl. 12/064,993, 2006.
48. [48] Song, X., Zhang, J., Han, Y., Jiang, J., Semi-supervised feature selection via hierarchical regression for web image classification. Multimed. Syst., 2014, 10.1007/s00530-014-0390-0.
49. [49] Han, Y., Yang, Y., Yan, Y., Ma, Z., Sebe, N., Member, S., Semisupervised feature selection via spline regression for video semantic recognition. IEEE Trans. Neural Netw. Learn. Syst. 26 (2015), 252–264.
50. [50] X. Chang, Y. Yang, Semi-supervised Feature Analysis by Mining Correlations among Multipe Tasks, 2014, 11. 〈 http://arxiv.org/abs/1411.6232〉.
51. [51] Z.Zhao, H.Liu, Semi-supervised feature selection via spectral snalysis, in: Proceedings of the 7th SIAM International Conference Data Mining, SIAM, 2007, pp. 641–646.
52. [52] Bellal, F., Elghazel, H., Aussem, A., A semi-supervised feature ranking method with ensemble learning. Pattern Recognit. Lett. 33 (2012), 1426–1433, 10.1016/j.patrec.2012.03.001.
53. [53] J.Ren, Z.Qiu, W.Fan, H.Cheng, P.S.Yu, S.Y.Philip, Forward semi-supervised feature selection, in: Proceedings of the Advances in Knowledge Discovery and Data Mining, Springer, 2008, pp. 970–976.
54. [54] Y. Han, K. Park, Y.K. Lee, Confident wrapper-type semi-supervised feature selection using an ensemble classifier, in: Proceedings of the 2011 2nd Artificial Intelligence, Management Science and Electronic Commerce, AIMSEC 2011, pp. 4581–4586. 〈 http://dx.doi.org/10.1109/AIMSEC.2011.6010202〉.
55. [55] H. Barkia, H. Elghazel, A. Aussem, Semi-supervised feature importance evaluation with ensemble learning, in: Proceedings of the International Conference on Data Mining, ICDM 2011, 2011, pp. 31–40. 〈 http://dx.doi.org/10.1109/icdm.2011.129〉.
56. [56] Zuo, L., Li, L., Chen, C., The graph based semi-supervised algorithm with ℓ1-regularizer. Neurocomputing 149 (2015), 966–974, 10.1016/j.neucom.2014.07.037.
57. [57] Zhang, K., Lan, L., Kwok, J.T., Vucetic, S., Parvin, B., Scaling up graph-based semisupervised learning via prototype vector machines. IEEE Trans. Neural Netw. Learn. Syst. 26 (2015), 444–457, 10.1109/TNNLS.2014.2315526.
58. [58] N.N.Pise, P.Kulkarni, A survey of semi-supervised learning methods, in: Proceedings of the International Conference Computational Intelligence and Security, CIS’08, IEEE, 2008, pp. 30–34. 〈 http://dx.doi.org/10.1109/CIS.2008.204〉.
59. [59] Chapelle, O., Schölkopf, B., Zien, A., Semi-Supervised Learning. 2006, MIT Press, Cambridge.
60. [60] Chahooki, M.A.Z., Charkari, N.M., Unsupervised manifold learning based on multiple feature spaces. Mach. Vis. Appl. 25 (2014), 1053–1065, 10.1007/s00138-014-0604-7.
61. [61] M.A.Z.Chahooki, N.M.Charkari, Improvement of supervised shape retrieval by learning the manifold space, in: Proceedings of the Iranian Conference on Machine Vision and Image Processing, MVIP 2011, IEEE, 2011, pp. 1–4.
62. [62] Halder, A., Ghosh, S., Ghosh, A., Aggregation pheromone metaphor for semi-supervised classification. Pattern Recognit. 46 (2013), 2239–2248, 10.1016/j.patcog.2013.01.002.
63. [63] X. Zhu, A.B. Goldberg, Introduction to Semi-Supervised Learning, 2009. 〈 http://dx.doi.org/10.2200/S00196ED1V01Y200906AIM006〉.
64. [64] X. Zhu, Semi-Supervised Learning Literature Survey, 2008. 〈 http://dx.doi.org/10.1.1.146.2352〉.
65. [65] Prakash, V.J., Nithya, L.M., A survey On semi-supervised learning techniques. Int. J. Comput. Trends Technol. 8 (2014), 25–29.
66. [66] Zhao, J., Lu, K., He, X., Locality sensitive semi-supervised feature selection. Neurocomputing 71 (2008), 1842–1849, 10.1016/j.neucom.2007.06.014.
67. [67] G. Doquire, M. Verleysen, Graph laplacian for semi-supervised feature selection in regression problems, Lect. Notes Comput. Sci. (Including Subser. Lect. Notes Artif. Intell. Lect. Notes Bioinformatics), 2011, pp. 248–255. 〈 http://dx.doi.org/10.1007/978-3-642-21501-8_31〉.
68. [68] Doquire, G., Verleysen, M., A graph laplacian based approach to semi-supervised feature selection for regression problems. Neurocomputing 121 (2013), 5–13, 10.1016/j.neucom.2012.10.028.
69. [69] L.C.L. Chen, R.H.R. Huang, W.H.W. Huang, Graph-based semi-supervised weighted band selection for classification of hyperspectral data, in: Proceedings of the International Conference on Audio, Language and Image Processing, ICALIP 2010, pp. 1123–1126. 〈 http://dx.doi.org/10.1109/ICALIP.2010.5685086〉.
70. [70] M.Yang, Y.Chen, G.Ji, Semi_fisher score : a semi-supervised method for feature selection, in: Proceedings of the International Conference on Machine Learning and Cybernetics, 2010, pp. 527–532.
71. [71] S.Lv, H.Jiang, L.Zhao, D.Wang, M.Fan, Manifold based fisher method for semi-supervised feature selection, in: Proceedings of the 10th International Conference on Fuzzy Systems and Knowledge Discovery, 2013, pp. 664–668.
72. [72] W. Yang, C. Hou, Y. Wu, A semi-supervised method for feature selection, in: Proceedings of the International Conference on Computer and Information Science and Technology, 2011, pp. 329–332. 〈 http://dx.doi.org/10.1109/ICCIS.2011.54〉.
73. [73] Liu, Y., Nie, F., Wu, J., Chen, L., Efficient semi-supervised feature selection with noise insensitive trace ratio criterion. Neurocomputing 105 (2013), 12–18, 10.1016/j.neucom.2012.05.031.
74. [74] Y.Liu, F.Nie, J.Wu, L.Chen, Semi-supervised feature selection based on label propagation and subset selection, in: Proceedings of the International Conference on Computer and Information Application, IEEE, 2010, pp. 293–296.
75. [75] J.Li, Semi-supervised feature selection under logistic I-RELIEF framework, in: Proceedings of the 19th International Conference Pattern Recognition, 2008, pp. 1–4. 〈 http://dx.doi.org/10.1109/ICPR.2008.4761687〉.
76. [76] Ma, Z., Nie, F., Yang, Y., Uijlings, J.R.R., Sebe, N., Member, S., et al. Discriminating joint feature analysis for multimedia data understanding. IEEE Trans. Multimed. 14 (2012), 1662–1672.
77. [77] Shi, C., Ruan, Q., An, G., Sparse feature selection based on graph Laplacian for web image annotation. Image Vis. Comput. 32 (2014), 189–201, 10.1016/j.imavis.2013.12.013.
78. [78] Z. Ma, Y. Yang, F. Nie, J. Uijlings, N. Sebe, Exploiting the entire feature space with sparsity for automatic image annotation, in: Proceedings of the 19th ACM Multimedia Conference, MM’11, 2011, p. 283. 〈 http://dx.doi.org/10.1145/2072298.2072336〉.
79. [79] Xu, Z., King, I., Lyu, M.R.T., Jin, R., Discriminative semi-supervised feature selection via manifold regularization. IEEE Trans. Neural Netw. 21 (2010), 1033–1047, 10.1109/TNN.2010.2047114.
80. [80] Ang, J.C., B, H.H., Nuzly, H., Hamed, A., Haron, H., Hamed, H.N.A., Semi-supervised SVM-based feature felection for cancer classification using microarray gene expression data. Curr. Approaches Appl. Artif. Intell., 2015, 468–477, 10.1007/978-3-319-19066-2.
81. [81] Dai, K., Yu, H.-Y., Li, Q., A semisupervised feature selection with support vector machine. J. Appl. Math., 2013, 2013.
82. [82] Bishop, C.M., Neural Networks for Pattern Recognition. 1995, Clarendon Press, Oxford.
83. [83] X. He, D. Cai, P. Niyogi, Laplacian Score for Feature Selection, in: Proceedings of the Advances in Neural Information Processing Systems, 2005, pp. 507–514. 〈 http://books.nips.cc/papers/files/nips18/NIPS2005_0149.pdf〉.
84. [84] Q. Gu, Z. Li, J. Han, Generalized Fisher Score for Feature Selection, CoRR, abs/1202.3, 2012.
85. [85] Zeng, Z., Wang, X., Zhang, J., Wu, Q., Semi-supervised feature selection based on local discriminative information. Neurocomputing, 2015, 10.1016/j.neucom.2015.05.119.
86. [86] X. Chang, F. Nie, Y. Yang, H. Huang, A convex formulation for semi-supervised multi-label feature selection, in: Proceedings of the Twenty-Eighth AAAI Conference on Artificial Intelligence, 2014, pp. 1171–1177.
87. [87] Foucart, S., Lai, M.-J., Sparsest solutions of underdetermined linear systems via ℓq-minimization for 0
88. [88] D.Krishnan, R.Fergus, Fast image deconvolution using hyper-Laplacian priors, in: Proceedings of the Advances in Neural Information Processing Systems, 2009, pp. 1033–1041.
89. [89] Chartrand, R., Exact reconstruction of sparse signals via nonconvex minimization. IEEE Signal Process. Lett. 14 (2007), 707–710.
90. [90] R.Chartrand, Fast algorithms for nonconvex compressive sensing: MRI reconstruction from very few data, in: Proceedings of the International Symposium on Biomedical Imaging From Nano to Macro, ISBI’09, 2009, pp. 262–265.
91. [91] Zongben, X., Xiangyu, C., Fengmin, X., Hai, Z., l1/2 regularization: a thresholding representation theory and a fast solver. IEEE Trans. Neural Netw. Learn. Syst. 23 (2012), 1013–1027.
92. [92] F.Nie, H.Huang, X.Cai, C.H.Ding, Efficient and robust feature selection via joint ℓ2, 1-norms minimization, in: Proceedings of the Advances in Neural Information Processing Systems, 2010, pp. 1813–1821.
93. [93] Z.Zhao, L.Wang, H.Liu, Efficient spectral feature selection with minimum redundancy, in: Proceedings of the AAAI Conference on Artificial Intelligence, Citeseer, 2010.
94. [94] X.Zhu, Z.Ghahramani, J.Lafferty, Semi-supervised learning using gaussian fields and harmonic functions, in: Proceedings of the International Conference on Machine Learning, ICML, 2003, pp. 912–919.
95. [95] Nie, F., Xu, D., Tsang, I.W.-H., Zhang, C., Flexible manifold embedding: a framework for semi-supervised and unsupervised dimension reduction. Image Process. IEEE Trans. 19 (2010), 1921–1932.
96. [96] Y.Ren, G.Zhang, G.Yu, Random subspace based semi-supervised feature selection, in: Proceedings of the 2011 International Conference on Machine Learning and Cybernetics, 2011, pp. 113–118.