Feature subset selection with cumulate conditional mutual information minimization

Expert Systems with Applications

Volumn 39, Issue 5, 2012, Pages 6078-6088

  Zhang, Yishi ^a   Zhang, Zigang ^a  

^a HUAZHONG UNIVERSITY OF SCIENCE AND TECHNOLOGY   (China)

Author keywords

Classification; Conditional mutual information; Feature selection; Redundancy; Relevance

Indexed keywords

CONDITIONAL MUTUAL INFORMATION; DATA SETS; EMPIRICAL RESULTS; FAST IMPLEMENTATION; FEATURE SELECTION ALGORITHM; FEATURE SUBSET SELECTION; MINIMAL INFORMATION; MUTUAL INFORMATIONS; REDUNDANT FEATURES; RELEVANCE;

ALGORITHMS; CLASSIFICATION (OF INFORMATION); LEARNING SYSTEMS; PATTERN RECOGNITION SYSTEMS; REDUNDANCY;

FEATURE EXTRACTION;

EID: 84855872857     PISSN: 09574174     EISSN: None     Source Type: Journal    
DOI: 10.1016/j.eswa.2011.12.003     Document Type: Article

References (46)

1. D. Aha, and D. Kibler Instance-based learning algorithms Machine Learning 6 1991 37 66
2. E. Amaldi, and V. Kann On the approximation of minimizing non zero variables or unsatisfied relations in linear systems Theoretical Computer Science 209 1-2 1998 237 260
3. R. Battiti Using mutual information for selecting features in supervised neural net learning IEEE Transactions on Neural Networks 5 4 1994 537 550
4. R. Bellman Adaptive control processes: A guided tour 1961 Princeton University Press Princeton, NJ, USA
5. A. Blum, and P. Langley Selection of relevant features and examples in machine learning Artificial Intelligence 97 1997 245 271
6. T.M. Cover The best two independent measurements are not the two best IEEE Transactions on Systems, Man, and Cybernetics 4 1974 116 117
7. T.M. Cover, and J.A. Thomas Elements of information theory 1991 Wiley NewYork, NY, USA
8. M. Dash, and H. Liu Feature selection for classification Intelligent Data Analysis 1 1997 131 156
9. Ding, C.; & Peng, H. (2003). Minimum redundancy feature selection from microarray gene expression data. In: Proceedings of the IEEE computer society conference on bioinformatics. CSB'03. IEEE computer society (pp. 523-528). Washington, DC, USA.
10. Fayyad, U.M.; & Irani, K.B. (1993). Multi-interval discretization of continuous valued attributes for classification learning. In: Proceedings of the thirteenth international joint conference on artificial intelligence. IJCAI'93 p. 1022C1027.
11. F. Flueret Fast binary feature selection with conditional mutual information Journal of Machine Learning Research 5 2004 1531 1555
12. T.S. Furey, N. Cristianini, N. Duffy, D.W. Bednarski, M. Schummer, and D. Haussler Support vector machine classification and validation of cancer tissue samples using microarray expression data Bioinformatics 16 10 2000 906 914
13. I. Guyon, and A. Elisseeff An introduction to variable and features election Journal of Machine Learning Research 3 2003 1157 1182
14. Hall, M.A. (2000). Correlation-based feature selection for discrete and numeric class machine learning. In: Proceedings of the seventh international conference on machine learning. ICML'00 (pp. 359-366). Morgan Kaufmann, Los Altos, CA, USA.
15. D. Huang, and T.W.S. Chow Effective feature selection scheme using mutual information Neurocomputing 63 2005 325 343
16. J.J. Huang, Y.Z. Cai, and X.M. Xu A parameterless feature ranking algorithm based on mi Neurocomputing 71 2008 1656 1668
17. A.K. Jain, R.P.W. Duin, and J. Mao Statistical pattern recognition: A review IEEE Transactions on Pattern Analysis and Machine Intelligence 22 1 2000 4 37
18. Kira, K.; & Rendell, L. (1992). Apractical approach to feature selection. In: Proceedingsof the ninth international workshop on machine learning. ML'92 (pp. 249-256). Morgan Kaufmann, San Francisco, CA, USA.
19. Koller, D.; & Sahami, M. (1996). Toward optimal feature selection. In: Proceedings of International Conference on Machine Learning. ICML'96 (pp. 284-292). Morgan Kaufmann, Los Altos, CA, USA.
20. Kononenko, I. (1994). Estimating attributes: analysis and extensions of relief. In: Proceedings of European Conference on Machine Learning. ECML'94 (pp. 171-182). Secaucus, NJ, USA: Springer-Verlag New York, Inc.
21. N. Kwak, and C.H. Choi Input feature selection for classification problems IEEE Transactions on Neural Networks 13 1 2002 143 159
22. H. Liu, and L. Yu Toward integrating features election algorithms for classification and clustering IEEE Transactions on Knowledge and Data Engineering 17 4 2005 491 502
23. Margaritis, D.; & Thrun, S. (1999). Bayesian network induction via local neighborhoods. Technical report, CMU-CS-99-134, Carnegie Mellon University.
24. Miyahara, K.; & Pazzani, M.J. (2000). Collaborative filtering with the simple bayesian classifier. In: Proceedings of pacific rim international conference on artificial intelligence. PRICAI'00. (pp. 679-689). Berlin, Heidelberg: Springer-Verlag.
25. J. Pearl Probabilistic reasoning in intelligent systems: Networks of plausible inference 1988 Morgan Kaufmann San Francisco, CA, USA
26. H. Peng, F. Long, and C. Ding Feature selection based on mutual information: criteria of max-dependency, max-relevance, and min-redundancy IEEE Transactions on Pattern Analysis and Machine Intelligence 27 8 2005 1226 1238
27. J.M. Pea, R. Nilsson, J. Björkegren, and J. Tegnér Towards scalable and data efficient learning of markov boundaries International Journal of Approximate Reasoning 45 2 2007 211 232
28. P. Pudil, F.J. Ferri, J. Novovicova, and J. Kittler Floating search methods for feature selection with nonmonotonic criterion functions Pattern Recognition 2 1994 279 283
29. G. Qu, S. Hariri, and M. Yousif A new dependency and correlation analysis for features IEEE Transactions on Knowledge and Data Engineering 17 9 2005 1199 1207
30. R. Quinlan C4.5: Programs for machine learning 1993 Morgan Kaufmann Publishers San Mateo, CA, USA
31. M. Robnik-Sikonja, and I. Kononenko Theoretical and empirical analysis of relief and relieff Machine Learning 53 2003 23 69
32. S.J. Russell, and P. Norvig Artificial intelligence: A modern approach 2nd ed. 2003 Prentice Hall Upper Saddle River, NJ, USA
33. Song, L.; Smola, A.J.; Borgwardt, K.M.; & Bedo, J. (2007). Supervised feature selection via dependence estimation. In: Proceedings of the twentyfourth international conference on machine learning. ICML'07 (pp. 823-830). New York, NY, USA: ACM Press.
34. J.M. Sotoca, and F. Pla Supervised feature selection by clustering using conditional mutual information-based distances Pattern Recognition 43 6 2010 2068 2081
35. K. Torkkola Feature extraction by non-parametric mutual information maximization Journal of Machine Learning Research 3 2003 1415 1438
36. I. Tsamardinos, C. Aliferis, and A. Statnikov Local causal and markov blanket induction for causal discovery and feature selection for classification part i: Algorithms and empirical evaluation Journal of Machine Learning Research 11 2010 171 234
37. Tsamardinos, I.; Aliferis, C.F.; & Statnikov, A. (2003). Algorithms for large scale markov blanket discovery. In: Proceedings of the sixteenth international florida artificial intelligence research society conference. FLAIRS'03 (pp. 376-381). Menlo Park, CA, USA: AAAI Press.
38. I. Tsamardinos, L.E. Brown, and C.F. Aliferis The max-min hillclimbing bayesian network structure learning algorithm Machine Learning 65 1 2006 31 78
39. Wang, G.; Lochovsky, F.H.; & Yang, Q.; 2004. Feature selection with conditional mutual information maximin in text categorization. In: Proceedings of the nineteenth ACM international conference on information and knowledge management. CIKM'04 (pp. 342-349). New York, NY, USA: ACM Press.
40. A.R. Webb Statistical pattern recognition 2nd ed. 2002 John Wiley & Sons Ltd Chichester, West Sussex, England
41. H.I. Witten, and E. Frank Data mining: practical machine learning tools and techniques with Java implementations 2000 Morgan Kaufmann San Francisco, CA, USA
42. Wu, X.; Yu, K.; Wang, H.; & Ding, W. (2010). Online streaming feature selection. In: Proceedings of the seventyseventh international conference on machine learning. ICML'10 (pp. 1159-1166). Madison, WI, USA: Omnipress.
43. Yang, H.H.; & Moody, J. (1999). Feature selection based on joint mutual information. In: Proceedings of international ICSC symposium on advances in intelligent data analysis (pp. 22-25).
44. Yaramakala, S.; & Margaritis, D. (2005). Speculative markov blanket discovery for optimal feature selection. In: Proceedings of the 5th IEEE International Conference on Data Mining. ICDM'05 (pp. 809-812). Washington, DC, USA: IEEE Computer Society Press.
45. L. Yu, and H. Liu Efficient feature selection via analysis of relevance and redundancy Journal of Machine Learning Research 5 2004 1205 1224
46. Zhao, Z.; & Liu, H. (2007). Spectral feature selection for supervised and unsupervised learning. In: Proceedings of the twentyfourth international conference on machine learning. ICML'07 (pp. 1151-1157). New York, NY, USA: ACM Press.