A fast clustering-based feature subset selection algorithm for high-dimensional data

IEEE Transactions on Knowledge and Data Engineering

Volumn 25, Issue 1, 2013, Pages 1-14

  Song, Qinbao ^a   Ni, Jingjie ^a   Wang, Guangtao ^a  

^a XI'AN JIAOTONG UNIVERSITY   (China)

Author keywords

feature clustering; Feature subset selection; filter method; graph based clustering

Indexed keywords

CLUSTERING METHODS; EMPIRICAL STUDIES; FAST ALGORITHMS; FEATURE CLUSTERING; FEATURE SELECTION ALGORITHM; FEATURE SUBSET SELECTION; FILTER METHOD; GRAPH-BASED CLUSTERING; GRAPH-THEORETIC CLUSTERING; HIGH DIMENSIONAL DATA; HIGH PROBABILITY; HIGH-DIMENSIONAL IMAGES; NAIVE BAYES; RELIEFF; RULE BASED; TARGET CLASS; TEXT DATA; TREE-BASED;

EFFICIENCY; FEATURE EXTRACTION; GRAPH THEORY; SET THEORY;

CLUSTERING ALGORITHMS;

EID: 84870441851     PISSN: 10414347     EISSN: None     Source Type: Journal    
DOI: 10.1109/TKDE.2011.181     Document Type: Article

References (76)

1. H. Almuallim and T.G. Dietterich, "Algorithms for Identifying Relevant Features," Proc. Ninth Canadian Conf. Artificial Intelligence, pp. 38-45, 1992.
2. H. Almuallim and T.G. Dietterich, "Learning Boolean Concepts in the Presence of Many Irrelevant Features," Artificial Intelligence, vol. 69, nos. 1/2, pp. 279-305, 1994.
3. A. Arauzo-Azofra, J.M. Benitez, and J.L. Castro, "A Feature Set Measure Based on Relief," Proc. Fifth Int'l Conf. Recent Advances in Soft Computing, pp. 104-109, 2004.
4. L.D. Baker and A.K. McCallum, "Distributional Clustering of Words for Text Classification," Proc. 21st Ann. Int'l ACM SIGIR Conf. Research and Development in information Retrieval, pp. 96-103, 1998.
5. R. Battiti, "Using Mutual Information for Selecting Features in Supervised Neural Net Learning," IEEE Trans. Neural Networks, vol. 5, no. 4, pp. 537-550, July 1994.
6. D.A. Bell and H. Wang, "A Formalism for Relevance and Its Application in Feature Subset Selection," Machine Learning, vol. 41, no. 2, pp. 175-195, 2000.
7. J. Biesiada and W. Duch, "Features Election for High-Dimensional data a Pearson Redundancy Based Filter," Advances in Soft Computing, vol. 45, pp. 242-249, 2008.
8. R. Butterworth, G. Piatetsky-Shapiro, and D.A. Simovici, "On Feature Selection through Clustering," Proc. IEEE Fifth Int'l Conf. Data Mining, pp. 581-584, 2005. (Pubitemid 47385745)
9. C. Cardie, "Using Decision Trees to Improve Case-Based Learning," Proc. 10th Int'l Conf. Machine Learning, pp. 25-32, 1993.
10. P. Chanda, Y. Cho, A. Zhang, and M. Ramanathan, "Mining of Attribute Interactions Using Information Theoretic Metrics," Proc. IEEE Int'l Conf. Data Mining Workshops, pp. 350-355, 2009.
11. S. Chikhi and S. Benhammada, "ReliefMSS: A Variation on a Feature Ranking Relieff Algorithm," Int'l J. Business Intelligence and Data Mining, vol. 4, nos. 3/4, pp. 375-390, 2009.
12. W. Cohen, "Fast Effective Rule Induction," Proc. 12th Int'l Conf. Machine Learning (ICML '95), pp. 115-123, 1995.
13. M. Dash and H. Liu, "Feature Selection for Classification," Intelligent Data Analysis, vol. 1, no. 3, pp. 131-156, 1997.
14. M. Dash, H. Liu, and H. Motoda, "Consistency Based Feature Selection," Proc. Fourth Pacific Asia Conf. Knowledge Discovery and Data Mining, pp. 98-109, 2000.
15. S. Das, "Filters, Wrappers and a Boosting-Based Hybrid for Feature Selection," Proc. 18th Int'l Conf. Machine Learning, pp. 74- 81, 2001.
16. M. Dash and H. Liu, "Consistency-Based Search in Feature Selection," Artificial Intelligence, vol. 151, nos. 1/2, pp. 155-176, 2003.
17. J. Demsar, "Statistical Comparison of Classifiers over Multiple Data Sets," J. Machine Learning Res., vol. 7, pp. 1-30, 2006. (Pubitemid 43022939)
18. I.S. Dhillon, S. Mallela, and R. Kumar, "A Divisive Information Theoretic Feature Clustering Algorithm for Text Classification," J. Machine Learning Research, vol. 3, pp. 1265-1287, 2003.
19. E.R. Dougherty, "Small Sample Issues for Microarray-Based Classification," Comparative and Functional Genomics, vol. 2, no. 1, pp. 28-34, 2001. (Pubitemid 32236919)
20. U. Fayyad and K. Irani, "Multi-Interval Discretization of Continuous- Valued Attributes for Classification Learning," Proc. 13th Int'l Joint Conf. Artificial Intelligence, pp. 1022-1027, 1993.
21. D.H. Fisher, L. Xu, and N. Zard, "Ordering Effects in Clustering," Proc. Ninth Int'l Workshop Machine Learning, pp. 162-168, 1992.
22. F. Fleuret, "Fast Binary Feature Selection with Conditional Mutual Information," J. Machine Learning Research, vol. 5, pp. 1531-1555, 2004.
23. G. Forman, "An Extensive Empirical Study of Feature Selection Metrics for Text Classification," J. Machine Learning Research, vol. 3, pp. 1289-1305, 2003.
24. M. Friedman, "A Comparison of Alternative Tests of Significance for the Problem of m Ranking," Annals of Math. Statistics, vol. 11, pp. 86-92, 1940.
25. S. Garcia and F. Herrera, "An Extension on Statistical Comparisons of Classifiers over Multiple Data Sets for All Pairwise Comparisons," J. Machine Learning Res., vol. 9, pp. 2677-2694, 2008.
26. M.R. Garey and D.S. Johnson, Computers and Intractability: A Guide to the Theory of NP-Completeness. W.H. Freeman & Co, 1979.
27. T.R. Golub, D.K. Slonim, P. Tamayo, C. Huard, M. Gaasenbeek, J.P. Mesirov, H. Coller, M.L. Loh, J.R. Downing, and M.A. Caligiuri, "Molecular Classification of Cancer: Class Discovery and Class Prediction by Gene Expression Monitoring," Science, vol. 286, no. 5439, pp. 531-537, 1999.
28. I. Guyon and A. Elisseeff, "An Introduction to Variable and Feature Selection," J. Machine Learning Research, vol 3, pp. 1157- 1182, 2003.
29. M.A. Hall, "Correlation-Based Feature Subset Selection for Machine Learning," PhD dissertation, Univ. of Waikato, 1999.
30. M.A. Hall and L.A. Smith, "Feature Selection for Machine Learning: Comparing a Correlation-Based Filter Approach to the Wrapper," Proc. 12th Int'l Florida Artificial Intelligence Research Soc. Conf., pp. 235-239, 1999.
31. M.A. Hall, "Correlation-Based Feature Selection for Discrete and Numeric Class Machine Learning," Proc. 17th Int'l Conf. Machine Learning, pp. 359-366, 2000.
32. J.W. Jaromczyk and G.T. Toussaint, "Relative Neighborhood Graphs and Their Relatives," Proc. IEEE, vol. 80, no. 9, pp. 1502- 1517, Sept. 1992.
33. G.H. John, R. Kohavi, and K. Pfleger, "Irrelevant Features and the Subset Selection Problem," Proc. 11th Int'l Conf. Machine Learning, pp. 121-129, 1994.
34. K. Kira and L.A. Rendell, "The Feature Selection Problem: Traditional Methods and a New Algorithm," Proc. 10th Nat'l Conf. Artificial Intelligence, pp. 129-134, 1992. (Pubitemid 23633599)
35. R. Kohavi and G.H. John, "Wrappers for Feature Subset Selection," Artificial Intelligence, vol. 97, nos. 1/2, pp. 273-324, 1997. (Pubitemid 127401107)
36. D. Koller and M. Sahami, "Toward Optimal Feature Selection," Proc. Int'l Conf. Machine Learning, pp. 284-292, 1996.
37. I. Kononenko, "Estimating Attributes: Analysis and Extensions of RELIEF," Proc. European Conf. Machine Learning, pp. 171-182, 1994.
38. C. Krier, D. Francois, F. Rossi, and M. Verleysen, "Feature Clustering and Mutual Information for the Selection of Variables in Spectral Data," Proc. European Symp. Artificial Neural Networks Advances in Computational Intelligence and Learning, pp. 157-162, 2007.
39. P. Langley, "Selection of Relevant Features in Machine Learning," Proc. AAAI Fall Symp. Relevance, pp. 1-5, 1994.
40. P. Langley and S. Sage, "Oblivious Decision Trees and Abstract Cases," Proc. AAAI-94 Case-Based Reasoning Workshop, pp. 113-117, 1994.
41. M. Last, A. Kandel, and O. Maimon, "Information-Theoretic Algorithm for Feature Selection," Pattern Recognition Letters, vol. 22, nos. 6/7, pp. 799-811, 2001. (Pubitemid 32411936)
42. H. Liu and R. Setiono, "A Probabilistic Approach to Feature Selection: A Filter Solution," Proc. 13th Int'l Conf. Machine Learning, pp. 319-327, 1996.
43. H. Liu, H. Motoda, and L. Yu, "Selective Sampling Approach to Active Feature Selection," Artificial Intelligence, vol. 159, nos. 1/2, pp. 49-74, 2004.
44. T.M. Mitchell, "Generalization as Search," Artificial Intelligence, vol. 18, no. 2, pp. 203-226, 1982.
45. M. Modrzejewski, "Feature Selection Using Rough Sets Theory," Proc. European Conf. Machine Learning, pp. 213-226, 1993.
46. L.C. Molina, L. Belanche, and A. Nebot, "Feature Selection Algorithms: A Survey and Experimental Evaluation," Proc. IEEE Int'l Conf. Data Mining, pp. 306-313, 2002.
47. B. Nemenyi, "Distribution-Free Multiple Comparison," PhD thesis, Princeton Univ., 1963.
48. D. Newman, "The Distribution of Range in Samples from a Normal Population, Expressed in Terms of an Independent Estimate of Standard Deviation," Biometrika, vol. 31, pp. 20-30, 1939.
49. A.Y. Ng, "On Feature Selection: Learning with Exponentially Many Irrelevant Features as Training Examples," Proc. 15th Int'l Conf. Machine Learning, pp. 404-412, 1998.
50. A.L. Oliveira and A.S. Vincentelli, "Constructive Induction Using a Non-Greedy Strategy for Feature Selection," Proc. Ninth Int'l Conf. Machine Learning, pp. 355-360, 1992.
51. H. Park and H. Kwon, "Extended Relief Algorithms in Instance- Based Feature Filtering," Proc. Sixth Int'l Conf. Advanced Language Processing and Web Information Technology (ALPIT '07), pp. 123-128, 2007.
52. F. Pereira, N. Tishby, and L. Lee, "Distributional Clustering of English Words," Proc. 31st Ann. Meeting on Assoc. for Computational Linguistics, pp. 183-190, 1993.
53. W.H. Press, B.P. Flannery, S.A. Teukolsky, and W.T. Vetterling, Numerical Recipes in C. Cambridge Univ. Press 1988.
54. R.C. Prim, "Shortest Connection Networks and Some Generalizations," Bell System Technical J., vol. 36, pp. 1389-1401, 1957.
55. J.R. Quinlan, C4.5: Programs for Machine Learning. Morgan Kaufman, 1993.
56. B. Raman and T.R. Ioerger, "Instance-Based Filter for Feature Selection," J. Machine Learning Research, vol. 1, pp. 1-23, 2002.
57. M. Robnik-Sikonja and I. Kononenko, "Theoretical and Empirical Analysis of Relief and ReliefF," Machine Learning, vol. 53, pp. 23- 69, 2003.
58. P. Scanlon, G. Potamianos, V. Libal, and S.M. Chu, "Mutual Information Based Visual Feature Selection for Lipreading," Proc. Int'l Conf. Spoken Language Processing, 2004.
59. M. Scherf and W. Brauer, "Feature Selection by Means of a Feature Weighting Approach," Technical Report FKI-221-97, Institut fur Informatik, Technische Universitat Munchen, 1997.
60. J.C. Schlimmer, "Efficiently Inducing Determinations: A Complete and Systematic Search Algorithm that Uses Optimal Pruning," Proc. 10th Int'l Conf. Machine Learning, pp. 284-290, 1993.
61. C. Sha, X. Qiu, and A. Zhou, "Feature Selection Based on a New Dependency Measure," Proc. Fifth Int'l Conf. Fuzzy Systems and Knowledge Discovery, vol. 1, pp. 266-270, 2008.
62. J. Sheinvald, B. Dom, and W. Niblack, "A Modelling Approach to Feature Selection," Proc. 10th Int'l Conf. Pattern Recognition, vol. 1, pp. 535-539, 1990.
63. J. Souza, "Feature Selection with a General Hybrid Algorithm," PhD dissertation, Univ. of Ottawa, 2004.
64. G. Van Dijck and M.M. Van Hulle, "Speeding Up the Wrapper Feature Subset Selection in Regression by Mutual Information Relevance and Redundancy Analysis," Proc. Int'l Conf. Artificial Neural Networks, 2006.
65. G.I. Webb, "Multiboosting: A Technique for Combining Boosting and Wagging," Machine Learning, vol. 40, no. 2, pp. 159-196, 2000.
66. E. Xing, M. Jordan, and R. Karp, "Feature Selection for High- Dimensional Genomic Microarray Data," Proc. 18th Int'l Conf. Machine Learning, pp. 601-608, 2001.
67. J. Yu, S.S.R. Abidi, and P.H. Artes, "A Hybrid Feature Selection Strategy for Image Defining Features: Towards Interpretation of Optic Nerve Images," Proc. Int'l Conf. Machine Learning and Cybernetics, vol. 8, pp. 5127-5132, 2005. (Pubitemid 41734822)
68. L. Yu and H. Liu, "Feature Selection for High-Dimensional Data: A Fast Correlation-Based Filter Solution," Proc. 20th Int'l Conf. Machine Leaning, vol. 20, no. 2, pp. 856-863, 2003.
69. L. Yu and H. Liu, "Efficiently Handling Feature Redundancy in High-Dimensional Data," Proc. Ninth ACM SIGKDD Int'l Conf. Knowledge Discovery and Data Mining (KDD '03), pp. 685-690, 2003.
70. L. Yu and H. Liu, "Redundancy Based Feature Selection for Microarray Data," Proc. 10th ACM SIGKDD Int'l Conf. Knowledge Discovery and Data Mining, pp. 737-742, 2004. (Pubitemid 40115000)
71. L. Yu and H. Liu, "Efficient Feature Selection via Analysis of Relevance and Redundancy," J. Machine Learning Research, vol. 10, no. 5, pp. 1205-1224, 2004.
72. Z. Zhao and H. Liu, "Searching for Interacting Features," Proc. 20th Int'l Joint Conf. Artificial Intelligence, 2007.
73. Z. Zhao and H. Liu, "Searching for Interacting Features in Subset Selection," J. Intelligent Data Analysis, vol. 13, no. 2, pp. 207-228, 2009.
74. M.F. Usama and B. Keki, "Irani: Multi-Interval Discretization of Continuousvalued Attributes for Classification Learning," Proc. 13th Int'l Joint Conf. Artificial Intelligence, pp. 1022-1027, 1993.
75. F. Wilcoxon, "Individual Comparison by Ranking Methods," Biometrics, vol. 1, pp. 80-83, 1945.
76. J. Demsar, "Statistical Comparison of Classifiers over Multiple Data Sets," J. Machine Learning Res., vol. 7, pp. 1-30, 2006. (Pubitemid 43022939)