High-dimensional feature selection via feature grouping: A Variable Neighborhood Search approach

Information Sciences

Volumn 326, Issue , 2016, Pages 102-118

  García Torres, Miguel ^a   Gómez Vela, Francisco ^a   Melián Batista, Belén ^b   Moreno Vega, J Marcos ^b  

^a UNIVERSIDAD PABLO DE OLAVIDE   (Spain)

^b UNIVERSIDAD DE LA LAGUNA   (Spain)

Author keywords

Feature grouping; Feature selection; High dimensionality; Metaheuristic

Indexed keywords

CLUSTERING ALGORITHMS; DATA MINING; OPTIMIZATION;

APPROXIMATE ALGORITHMS; FEATURE GROUPING; HIGH DIMENSIONAL DATA; HIGH DIMENSIONAL DATASETS; HIGH DIMENSIONAL FEATURE; HIGH DIMENSIONALITY; METAHEURISTIC; VARIABLE NEIGHBORHOOD SEARCH;

FEATURE EXTRACTION;

EID: 84943748390     PISSN: 00200255     EISSN: None     Source Type: Journal    
DOI: 10.1016/j.ins.2015.07.041     Document Type: Article

References (79)

1. U. Alon, N. Barkai, D.A. Notterman, K. Gishdagger, S. Ybarradagger, D. Mackdagger, and A.J. Levine Broad patterns of gene expression revealed by clustering analysis of tumor and normal colon tissues probed by oligonucleotide arrays Proc. Natl. Acad. Sci. USA 96 12 1999 6745 6750
2. R. Bekkerman, R. El-Yaniv, N. Tishby, and Y. Winter Distributional word clusters vs. words for text categorization J. Mach. Learn Res. 3 2003 1183 1208
3. D. Bell, and H. Wang A formalism for relevance and its application in feature subset selection Mach Learn 41 2 2000 175 195
4. A. Ben-Dor, R. Shamir, and Z. Yakhini Clustering gene expression patterns J. Comput Biol 6 1999 281 297
5. A.L. Blum, and P. Langley Selection of relevant features and examples in machine learning Artif. Intell. 97 1-2 1997 245 271
6. A.L. Blum, and R.L. Rivest Training a 3-node neural networks is NP-complete Neural Netw. 5 1992 117 127
7. M.E. Burczynski, R.L. Peterson, N.C. Twine, K.A. Zuberek, B.J. Brodeur, L. Casciotti, V. Maganti, P.S. Reddy, A. Strahs, F. Immermann, W. Spinelli, U. Schwertschlag, A.M. Slager, M.M. Cotreau, and A.J. Dorner Molecular classification of crohn's disease and ulcerative colitis patients using transcriptional profiles in peripheral blood mononuclear cells J. Mol. Diagn. 8 1 2006 51 61
8. R. Caruana, and D. Freitag How useful is relevance? Proceedings of Working Notes of the AAAI Fall Symposium on Relevance 1994 25 29
9. G. Chandrashekar, and F. Sahin A survey on feature selection methods Comput. Electr. Eng. 40 1 2014 16 28
10. K. Chin, S. DeVries, J. Fridlyand, P.T. Spellman, R. Roydasgupta, W.-L. Kuo, A. Lapuk, R.M. Neve, Z. Qian, T. Ryder, F. Chen, H. Feiler, T. Tokuyasu, C. Kingsley, S. Dairkee, Z. Meng, K. Chew, D. Pinkel, A. Jain, B.M. Ljung, L. Esserman, D.G. Albertson, F.M. Waldman, and J.W. Gray Genomic and transcriptional aberrations linked to breast cancer pathophysiologies Cancer Cell 10 6 2006 529 541
11. D. Chowdary, J. Lathrop, J. Skelton, K. Curtin, T. Briggs, Y. Zhang, J. Yu, Y. Wang, and A. Mazumder Prognostic gene expression signatures can be measured in tissues collected in RNAlater preservative J. Mol. Diagn. 8 1 2006 31 39
12. A. Dasgupta, P. Drineas, B. Harb, V. Josifovski, and M.W. Mahoney Feature selection methods for text classification Proceedings of the 13th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, KDD 2007 230 239
13. M. Dash, and H. Liu Feature selection for classification Intell. Data Anal. 1 1997 131 156
14. J. Demšar Statistical comparisons of classifiers over multiple data sets J. Mach. Learn. Res. 7 2006 1 30
15. M. Dettling, and P. Buhlmann Supervised clustering of genes Genome Biol. 3 12 2002 0069.1 0069.15
16. M. Dettling, and P. Buhlmann Finding predictive gene groups from microarray data J. Multivar. Anal. 90 1 2004 106 131
17. I.S. Dhillon, S. Mallela, and R. Kumar A divisive information theoretic feature clustering algorithm for text classification J. Mach. Learn. Res. 3 2003 1265 1287
18. C.H.Q. Ding, and H. Peng Minimum redundancy feature selection from microarray gene expression data J. Bioinform. Comput. Biol. 3 2 2005 185 206
19. J. Fan, and J. Lv A selective overview of variable selection in high dimensional feature space Stat. Sin. 20 2010 101 148
20. M. García-Torres, R. Armañanzas, C. Bielza, and P. Larrañaga Comparison of metaheuristic strategies for peakbin selection in proteomic mass spectrometry data Inf. Sci. 222 2013 229 246
21. G.J. Gordon, R.V. Jensen, L.L. Hsiao, S.R. Gullans, J.E. Blumenstock, S. Ramaswamy, W.G. Richards, D.J. Sugarbaker, and R. Bueno Translation of microarray data into clinically relevant cancer diagnostic tests using gene expression ratios in lung cancer and mesothelioma Cancer Res. 62 17 2002 4963 4967
22. Gravier, Eleonore, G. Pierron, A. Vincent-Salomon, N. gruel, V. Raynal, A. Savignoni, Y. De Rycke, J.-Y. Pierga, C. Lucchesi, F. Reyal, A. Fourquet, S. Roman-Roman, F. Radvanyi, X. Sastre-Garau, B. Asselain, and O. Delattre A prognostic DNA signature for T1T2 node-negative breast cancer patients. Genes Chromosomes Cancer 49 12 2010 1125
23. I. Guyon, and A. Elisseeff An introduction to variable and feature selection J. Mach. Learn. Res. 3 2003 1157 1182
24. M.A. Hall Correlation-based Feature Subset Selection for Machine Learning 1999 University of Waikato Hamilton, New Zealand (Ph.D. thesis)
25. M.A. Hall, E. Frank, G. Holmes, B. Pfahringer, P. Reutemann, and I.H. Witten The WEKA data mining software: an update SIGKDD Explor. 11 1 2009 10 18
26. X. Han, X. Chang, L. Quan, X. Xiong, J. Li, Z. Zhang, and Y. Liu Feature subset selection by gravitational search algorithm optimization Inf. Sci. 281 2014 128 146
27. P. Hansen, and N. Mladenović Variable neighborhood search Comput. Oper. Res. 24 1997 1097 1100
28. H.H. Hsu, C.W. Hsieh, and M.D. Lu Hybrid feature selection by combining filters and wrappers Expert Syst. Appl. 38 7 2011 8144 8150
29. J. Huang, J.L. Horowitz, and S. Ma Asymptotic properties of bridge estimators in sparse high-dimensional regression models Ann. Stat. 36 2 2008 587 613
30. T. Jirapech-Umpai, and S. Aitken Feature selection and classification for microarray data analysis: evolutionary methods for identifying predictive genes BMC Bioinform. 6 148 2005 1 11
31. G.H. John, R. Kohavi, and K. Pfleger Irrelevant feature and the subset selection problem Proceedings of the Eleventh International Conference on Machine Learning 1994 121 129
32. R. Jörnsten, and B. Yu Simultaneous gene clustering and subset selection for sample classification via MDL Bioinformatics 19 9 2003 1100 1109
33. W. Junyun Study and analyze on feature selection in text categorization for engineering domain Adv. Mater. Res. 487 2012 383 386
34. A. Kalousis, J. Prados, and M. Hilario Stability of feature selection algorithms: a study on high-dimensional spaces Knowl. Inf. Syst. 12 1 2007 95 116
35. R. Kohavi, and G.H. John Wrappers for feature subset selection Artif. Intell. 97 1-2 1997 273 324
36. D. Koller, and M. Sahami Toward optimal feature selection Proceedings of the Thirteenth International Conference on Machine Learning 1996 284 292
37. C. Krier, D. Francois, F. Rossi, and M. Verleysen Feature clustering and mutual information for the selection of variables in spectral data Proceedings of the European Symposium on Artificial Neural Networks 2007 157 162
38. P. Krízek, J. Kittler, and V. Hlavác Improving stability of feature selection methods Computer Analysis of Images and Patterns Lecture Notes in Computer Science 4673 2007 Springer 929 936
39. L.I. Kuncheva A stability index for feature selection Proceedings of the 25th IASTED International Multi-Conference 2007 390 395
40. N. Lavrac Selected techniques for data mining in medicine Artif. Intell. Med. 16 1999 3 23
41. J. Lee, and D. Kim Memetic feature selection algorithm for multi-label classification Inf. Sci. 293 2015 80 96
42. P.M. Lewis The characteristic selection problem in recognition systems. IRE Trans. Inf. Theory 8 2 1962 171 178
43. J. Li, and H. Zha Simultaneous classification and feature clustering using discriminant vector quantization with applications to microarray data analysis 1st IEEE Computer Society Bioinformatics Conference, CSB, 14-16 August 2002 2002 IEEE Computer Society 246 255 10.1109/CSB.2002.1039347
44. H. Liu, and H. Motada Feature Selection for Knowledge Discovery and Data Mining 1998 Kluwer Academic Publishers
45. H. Liu, and H. Motoda On issues of instance selection Data Min. Knowl. Discov. 6 2 2002 115 130
46. H.C. Liu, P.C. Peng, T.C. Hsieh, T.C. Yeh, C.J. Lin, C.Y. Chen, J.Y. Hou, L.Y. Shih, and D.C. Liang Comparison of feature selection methods for cross-laboratory microarray analysis IEEE/ACM Trans. Comput. Biol. Bioinform. 10 3 2013 593 604
47. S. Loscalzo, L. Yu, and C. Ding Consensus group stable feature selection Proceedings of the 15th ACM SIGKDD International Conference on Knowledge Discovery and Data mining 2009 International Conference on Knowledge Discovery and Data Mining 567 576
48. R. Ltd., I. Carnegie Group, Reuters-21578, 1995.
49. S. Ma, and J. Huang Penalized feature selection and classification in bioinformatics. Brief. Bioinform. 9 5 2008 392 403
50. A. McCallum, and K. Nigam A comparison of event models for naive bayes text classification Provceedings of AAA Workshop on Learning for Text Categorization 1998 AAAI Press 41 48
51. A. Mitchell, A. Divoli, J. Kim, M. Hilario, I. Selimas, and T.K. Attwood METIS: multiple extraction techniques for informative sentences. Bioinformatics 21 22 2005 4196 4197
52. C.H. Ooi, M. Chetty, and S.W. Teng Relevance, redundancy and differential priorization in feature selection for multiclass gene expression data Proceedings of the 6th International Symposium on Biological and Medical Data Analysis Lecture Notes in Computer Science/Lecture Notes in Bioinformatics 3745 2005 Springer 367 378
53. H.L. Peng, and C.F. Ding Feature selection based on mutual information: criteria of max-dependency, max-relevance, and min-redundancy IEEE Trans. Pattern Anal. Mach. Intell. 27 8 2005 1226 1238
54. F. Pereira, N. Tishby, and L. Lee Distributional clustering of English words Proceedings of Association for Computational Linguistics, ACL 1993 183 190
55. S.L. Pomeroy, P. Tamayo, M. Gaasenbeek, L.M. Sturla, M.A.M.E. McLaughlin, J.Y.H. Kim, L.C. Goumnerova, P.M. Black, C. Lau, J.C. Allen, D. Zagzag, J.M. Olson, T. Curran, C. Wetmore, J.A. Biegel, T. Poggio, S. Mukherjee, R. Rifkin, A. Califano, G. Stolovitzky, D.N. Louis, J.P. Mesirov, E.S. Lander, and T.R. Golub Prediction of central nervous system embryonal tumour outcome based on gene expression Nature 415 6870 2002 436 442
56. Y. Saeys, T. Abeel, and Y. van de Peer Robust feature selection using ensemble feature selection techniques Proceedings of the European Conference on Machine Learning and Knowledge Discovery in Databases Lecture Notes In Artificial Intelligence 5212 2008 313 325
57. Y. Saeys, I. Inza, and P. Larrañaga A review of feature selection techniques in bioinformatics. Bioinformatics 23 19 2007 2507 2517
58. M. Sebban, and R. Nock A hybrid filter/wrapper approach of feature selection using information theory Pattern Recogn. 35 2002 835 846
59. G.S. Sebestyen Decision-Making Processes in Pattern Recognition ACM monograph series 1962 Macmillan
60. X. Shen, and H.-C. Huang Grouping pursuit through a regularization solution surface J. Am. Stat. Assoc. 105 490 2010 727 739
61. M.A. Shipp, K.N. Ross, P. Tamayo, A.P. Weng, J.L. Kutok, R.C. Aguiar, M. Gaasenbeek, M. Angelo, M. Reich, G.S. Pinkus, T.S. Ray, M.A. Koval, K.W. Last, A. Norton, T.A. Lister, J. Mesirov, D.S. Neuberg, E.S. Lander, J.C. Aster, and T.R. Golub Diffuse large B-cell lymphoma outcome prediction by gene-expression profiling and supervised machine learning. Nat. Med. 8 1 2002 68 74
62. B. Silva, and N. Marques Feature clustering with self-organizing maps and an application to financial time-series for portfolio selection Proceeding of the Sixth International Conference on Neural Computation 2010 301 309
63. D. Singh, P.G. Febbo, K. Ross, D.G. Jackson, J. Manola, C. Ladd, P. Tamayo, A.A. Renshaw, A.V. D'Amico, and J.P. Richie Gene expression correlates of clinical prostate cancer behavior Cancer Cell 1 2 2002 203 209
64. N. Slonim, and N. Tishby The power of word clusters for text classification Proceedings of the23rd European Colloquium on Information Retrieval Research 2001
65. Q. Song, J. Ni, and G. Wang A fast clustering-based feature subset selection algorithm for high-dimensional data IEEE Trans. Knowl. Data Eng. 25 1 2013 1 14
66. J. Tang, S. Alelyani, H. Liu, Data Classification: Algorithms and Applications, Data Mining and Knowledge Discovery Series, CRC Press, pp. 37-64.
67. R. Tibshirani Regression shrinkage and selection via the lasso J. R. Stat. Soc. Ser. B 58 1994 267 288
68. R. Tibshirani, M. Saunders, S. Rosset, J. Zhu, and K. Knight Sparsity and smoothness via the fused lasso J. R. Stat. Soc. Ser. B 67 1 2005 91 108
69. A. Unler, A. Murat, and R.B. Chinnam mr2pso: a maximum relevance minimum redundancy feature selection method based on swarm intelligence for support vector machine classification Inf. Sci. 181 20 2011 4625 4641
70. H. U uz A two-stage feature selection method for text categorization by using information gain, principal component analysis and genetic algorithm Knowl.-Based Syst. 24 7 2011 1024 1032
71. J.R. Vergara, and P.A. Estvez A review of feature selection methods based on mutual information. Neural Comput. Appl. 24 1 2013 175 186
72. C.M.M. Wahid, A.B.M.S. Ali, and K.S. Tickle A novel hybrid approach of feature selection through feature clustering using microarray gene expression data Proceedings of Hybrid Intelligent Systems, HIS 2011 IEEE 121 126
73. H. Wang, D. Bell, and F. Murtagh Axiomatic approach to feature subset selection based on relevance IEEE Trans. Pattern Anal. Mach. Intell. 21 3 1999 271 277
74. L. Wang, F. Chu, and W. Xie Accurate cancer classification using expressions of very few genes IEEE/ACM Trans. Comput. Biol. Bioinform. 4 1 2007 40 53
75. C.H. Yang, L.Y. Chuang, and C.H. Yang Ig-ga: a hybrid filter/wrapper method for feature selection of microarray data J. Med. Biol. Eng. 30 1 2010 23 28
76. L. Yu, C. Ding, and S. Loscalzo Stable feature selection via dense feature groups Proceedings of the 14th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining 2008 803 811
77. L. Yu, and H. Liu Efficient feature selection via analysis of relevance and redundancy J. Mach. Learn. Res. 5 2004 1205 1224
78. H. Zou The Adaptive Lasso and Its Oracle Properties J. Am. Stat. Assoc. 101 476 2006 1418 1429
79. H. Zou, and T. Hastie Regularization and variable selection via the elastic net J. R. Stat. Soc. Ser. B 67 2005 301 320