Divergence-based feature selection for separate classes

Neurocomputing

Volumn 101, Issue , 2013, Pages 32-42

  Zhang, Yishi ^a   Li, Shujuan ^a   Wang, Teng ^a   Zhang, Zigang ^a  

^a HUAZHONG UNIVERSITY OF SCIENCE AND TECHNOLOGY   (China)

Author keywords

Feature selection; Kullback Leibler divergence; Redundancy; Relevance; Separate classes

Indexed keywords

CLASS LABELS; EMPIRICAL STUDIES; FEATURE SELECTION ALGORITHM; FEATURE SELECTION METHODS; KULLBACK LEIBLER DIVERGENCE; RELEVANCE; RELEVANCE AND REDUNDANCIES; SELECTION FRAMEWORK; SEPARATE CLASSES;

FEATURE EXTRACTION; PATTERN RECOGNITION SYSTEMS; REDUNDANCY; SEPARATION;

ALGORITHMS;

ACCURACY; ARTICLE; CALCULATION; EMPIRICISM; FEATURE SELECTION; INTERMETHOD COMPARISON; KULLBACK LEIBLER DIVERGENCE; LEARNING ALGORITHM; MATHEMATICAL COMPUTING; MATHEMATICAL PARAMETERS; PRIORITY JOURNAL; PROBABILITY; RELIABILITY; STATISTICAL PARAMETERS;

EID: 84868628924     PISSN: 09252312     EISSN: 18728286     Source Type: Journal    
DOI: 10.1016/j.neucom.2012.06.036     Document Type: Article

References (38)

1. Yu L., Liu H. Efficient feature selection via analysis of relevance and redundancy. J. Mach. Learn. Res. 2004, 5:1205-1224.
2. Guyon I., Elisseeff A. An introduction to variable and features election. J. Mach. Learn. Res. 2003, 3:1157-1182.
3. Liu H., Yu L. Toward integrating features election algorithms for classification and clustering. IEEE Trans. Knowl. Data Eng. 2005, 17(4):491-502.
4. Quinlan R. C4.5. Programs for Machine Learning 1993, Morgan Kaufmann Publishers, San Mateo, CA, USA.
5. Furey T.S., Cristianini N., Duffy N., Bednarski D.W., Schummer M., Haussler D. Support vector machine classification and validation of cancer tissue samples using microarray expression data. Bioinformatics 2000, 16(10):906-914.
6. Qu G., Hariri S., Yousif M. A new dependency and correlation analysis for features. IEEE Trans. Knowl. Data Eng. 2005, 17(9):1199-1207.
7. Peng H., Long F., Ding C. Feature selection based on mutual information. criteria of max-dependency, max-relevance, and min-redundancy. IEEE Trans. Pattern Anal. Mach. Intell. 2005, 27(8):1226-1238.
8. Huang D., Chow T.W.S. Effective feature selection scheme using mutual information. Neurocomputing 2005, 63:325-343.
9. Huang J.J., Cai Y.Z., Xu X.M. A parameterless feature ranking algorithm based on mi. Neurocomputing 2008, 71:1656-1668.
10. K. Kira, L. Rendell, Apractical approach to feature selection, in: Proceedings of the 9th International Workshop on Machine Learning (ML'92), Morgan Kaufmann, San Francisco, CA, USA, 1992, pp. 249-256.
11. I. Kononenko, Estimating attributes: analysis and extensions of relief, in: Proceedings of European Conference on Machine Learning (ECML'94), Springer-Verlag New York, Inc., Secaucus, NJ, USA, 1994, pp. 171-182.
12. Battiti R. Using mutual information for selecting features in supervised neural net learning. IEEE Trans. Neural Networks 1994, 5(4):537-550.
13. M.A. Hall, Correlation-based feature selection for discrete and numeric class machine learning, in: Proceedings of the 7th International Conference on Machine Learning (ICML'00), Morgan Kaufmann, Los Altos, CA, USA, 2000, pp. 359-366.
14. Flueret F. Fast binary feature selection with conditional mutual information. J. Mach. Learn. Res. 2004, 5:1531-1555.
15. Tsamardinos I., Aliferis C., Statnikov A. Local causal and Markov blanket induction for causal discovery and feature selection for classification Part I. algorithms and empirical evaluation. J. Mach. Learn. Res. 2010, 11:171-234.
16. X. Wu, K. Yu, H. Wang, W. Ding, Online streaming feature selection, in: Proceedings of the 27th International Conference on Machine Learning (ICML'10), Omnipress, Madison, WI, USA, 2010, pp. 1159-1166.
17. Albrecht A.A. Stochastic local search for the feature set problem, with applications to microarray data. Appl. Math. Comput. 2006, 183(2):1148-1164.
18. Kwak N., Choi C.H. Input feature selection for classification problems. IEEE Trans. Neural Networks 2002, 13(1):143-159.
19. Zhang Y., Zhang Z. Feature subset selection with cumulate conditional mutual information minimization. Expert Syst. with Appl. 2012, 39:6078-6088.
20. Pudil P., Ferri F.J., Novovicova J., Kittler J. Floating search methods for feature selection with nonmonotonic criterion functions. Pattern Recognition 1994, 2:279-283.
21. Russell S.J., Norvig P. Artificial Intelligence. A Modern Approach 2003, Prentice Hall, Upper Saddle River, NJ, USA. 2nd edition.
22. C. Ding, H. Peng, Minimum redundancy feature selection from microarray gene expression data, in: Proceedings of the IEEE Computer Society Conference on Bioinformatics (CSB'03), IEEE Computer Society, Washington, DC, USA, 2003, pp. 523-528.
23. G. Wang, F.H. Lochovsky, Q. Yang, Feature selection with conditional mutual information maximin in text categorization, in: Proceedings of the 19th ACM International Conference on Information and Knowledge Management (CIKM'04), ACM Press, New York, NY, USA, 2004, pp. 342-349.
24. Sotoca J.M., Pla F. Supervised feature selection by clustering using conditional mutual information-based distances. Pattern Recognition 2010, 43(6):2068-2081.
25. Pearl J. Probabilistic Reasoning in Intelligent Systems. Networks of Plausible Inference 1988, Morgan Kaufmann, San Francisco, CA, USA.
26. D. Koller, M. Sahami, Toward optimal feature selection, in: Proceedings of International Conference on Machine Learning (ICML'96), Morgan Kaufmann, Los Altos, CA, USA, 1996, pp. 284-292.
27. I. Tsamardinos, C.F. Aliferis, A. Statnikov, Algorithms for large scale Markov blanket discovery, in: Proceedings of the 16th International Florida Artificial Intelligence Research Society Conference (FLAIRS'03), AAAI Press, Menlo Park, CA, USA, 2003, pp. 376-381.
28. S. Yaramakala, D. Margaritis, Speculative Markov blanket discovery for optimal feature selection, in: Proceedings of the 5th IEEE International Conference on Data Mining (ICDM'05), IEEE Computer Society Press, Washington, DC, USA, 2005, pp. 809-812.
29. Tsamardinos I., Brown L.E., Aliferis C.F. The max-min hillclimbing Bayesian network structure learning algorithm. Mach. Learn. 2006, 65(1):31-78.
30. Peòa J.M., Nilsson R., Björkegren J., Tegnér J. Towards scalable and data efficient learning of Markov boundaries. Int. J. Approx. Reason. 2007, 45(2):211-232.
31. Wu J., Xiong H., Chen J. Cog. local decomposition for rare class analysis. Data Min. Knowl. Discovery 2010, 20(2):191-220.
32. G.H. John, R. Kohavi, K. Pfleger, Irrelevant features and the subset selection problem, in: Proceedings of the 11th International Conference on Machine Learning (ICML'94), Morgan Kaufmann Publishers, San Francisco, CA, 1994, pp. 121-129.
33. Robnik-Sikonja M., Kononenko I. Theoretical and empirical analysis of ReliefF and RReliefF. Mach. Learn. 2003, 53:23-69.
34. Witten H.I., Frank E. Data Mining. Practical Machine Learning Tools and Techniques with Java Implementations 2000, Morgan Kaufmann, San Francisco, CA, USA.
35. U.M. Fayyad, K.B. Irani, Multi-interval discretization of continuous valued attributes for classification learning, in: Proceedings of the 13th International Joint Conference on Artificial Intelligence (IJCAI'93), 1993, pp. 1022-1027.
36. Aha D., Kibler D. Instance-based learning algorithms. Mach. Learn. 1991, 6:37-66.
37. Cristianini N., Shawe-Taylor J. An Introduction to Support Vector Machines and other Kernel-based Learning Methods 2000, Cambridge University Press, Cambridge, UK.
38. Hsu C.W., Lin C.J. A comparison of methods for multi-class support vector machines. IEEE Transactions on Neural Networks 2002, 13:415-425.