Measuring relevance between discrete and continuous features based on neighborhood mutual information

Expert Systems with Applications

Volumn 38, Issue 9, 2011, Pages 10737-10750

  Hu, Qinghua ^a,b   Zhang, Lei ^b   Zhang, David ^b   Pan, Wei ^b   An, Shuang ^a   Pedrycz, Witold ^c  

^a HARBIN INSTITUTE OF TECHNOLOGY   (China)

^b HONG KONG POLYTECHNIC UNIVERSITY   (Hong Kong)

^c UNIVERSITY OF ALBERTA   (Canada)

Author keywords

Continuous feature; Feature selection; Neighborhood entropy; Neighborhood mutual information; Relevance

Indexed keywords

CONTINUOUS FEATURE; FEATURE SELECTION; NEIGHBORHOOD ENTROPY; NEIGHBORHOOD MUTUAL INFORMATION; RELEVANCE;

ALGORITHMS; DECISION TREES; ENTROPY; PROBABILITY DENSITY FUNCTION; REGRESSION ANALYSIS;

FEATURE EXTRACTION;

EID: 79955613590     PISSN: 09574174     EISSN: None     Source Type: Journal    
DOI: 10.1016/j.eswa.2011.01.023     Document Type: Article

References (42)

1. R. Battiti Using mutual information for Selecting features in supervised neural net learning IEEE Transactions on Neural Networks 5 1994 537 550
2. D. Bell, and H. Wang A formalism for relevance and its application in feature subset selection Machine Learning 41 2 2000 175 195
3. Blake, C. L.; & Merz, C. J. (1998). UCI repository of machine learning databases. .
4. L. Breiman Classification and regression trees 1993 Chapman and Hall Boca Raton
5. M. Dash, and H. Liu Consistency-based search in feature selection Artificial Intelligence 151 1-2 2003 155 176
6. I. Düntsch, and G. Gediga Statistical evaluation of rough set dependency analysis International Journal of Human Computer Studies 46 5 1997 589 604 (Pubitemid 127402946)
7. Fayyad, U.; & Irani, K. (1993). Multi-interval discretization of continuous-valued attributes for classification learning. In Proceedings of 13th international joint conference on artificial intelligence. San Mateo, CA: Morgan Kaufmann (pp. 1022-1027).
8. U.M. Fayyad, and K.B. Irani On the handling of continuous-valued attributes in decision tree generation Machine Learning 8 1992 87 102
9. F. Fleuret Fast binary feature selection with conditional mutual information Journal of Machine Learning Research 5 2004 1531 1555
10. I. Guyon, and A. Elisseeff An introduction to variable and feature selection Journal of Machine Learning Research 3 2003 1157 1182
11. Hall, M. A. (1999). Correlation-based feature subset selection for machine learning, Ph. D. dissertation, Univ. Waikato, Waikato, New Zealand.
12. Hall, M. A. (2000). Correlation-based feature selection for discrete and numeric class machine learning. In Proceedings 17th international conference on machine learning (pp. 359-366).
13. J.J. Huang, Y.Z. Cai, and X.M. Xu A parameterless feature ranking algorithm based on MI Neurocomputing 71 1-2 2008 1656 1668
14. X.H. Hu, and N. Cercone Learning in relational databases: A rough set approach Computational Intelligence 12 2 1995 323 338
15. Q.H. Hu, Z.X. Xie, and D.R. Yu Hybrid attribute reduction based on a novel fuzzy-rough model and information granulation Pattern Recognition 40 12 2007 3509 3521
16. Q.H. Hu, D.R. Yu, J.F. Liu, and C. Wu Neighborhood rough set based heterogeneous feature subset selection Information Sciences 178 18 2008 3577 3594
17. Q.H. Hu, D.R. Yu, and Z.X. Xie Information-preserving hybrid data reduction based on fuzzy-rough techniques Pattern Recognition Letters 27 5 2006 414 423
18. Q.H. Hu, D.R. Yu, Z.X. Xie, and X.D. Li EROS: Ensemble rough subspaces Pattern Recognition 40 12 2007 3728 3739
19. Q.H. Hu, D.R. Yu, Z.X. Xie, and J.F. Liu Fuzzy probabilistic approximation spaces and their information measures IEEE Transactions on Fuzzy Systems 14 2 2006 191 201
20. A. Kalousis, J. Prados, and M. Hilario Stability of feature selection algorithms: A study on high-dimensional spaces Knowledge and Information Systems 12 1 2007 95 116
21. N. Kwak, and C.-H. Choi Input feature selection for classification problems IEEE Transactions on Neural Networks 13 1 2002 143 159 (Pubitemid 34236844)
22. Nojun Kwak, and Chong-Ho Choi Input feature selection by mutual information based on Parzen window IEEE Transactions on Pattern Analysis and Machine Intelligence 24 12 2002 1667 1671
23. H. Liu, F. Hussain, and M. Dash Discretization: An enabling technique Data Mining and Knowledge Discovery 6 2002 393 423 (Pubitemid 37113864)
24. X.X. Liu, A. Krishnan, and A. Mondry An entropy-based gene selection method for cancer classification using microarray data BMC Bioinformatics 6 2005 76 10.1186/1471-2105-6-76
25. H. Liu, and R. Setiono Feature selection via discretization of numeric attributes IEEE Transactions on Knowledge and Data Engineering 9 4 1997 642 645 (Pubitemid 127776408)
26. H. Liu, and L. Yu Toward integrating feature selection algorithms for classification and clustering IEEE Transactions on Knowledge and Data Engineering 17 4 2005 491 502 (Pubitemid 40495592)
27. Z. Pawlak Rough sets: Theoretical aspects of reasoning about data 1991 Kluwer Academic Publishers Dordrecht
28. Z. Pawlak, and C. Rauszer Dependency of attributes in information systems Bulletin of the Polish Academy of Sciences Mathematics 33 1985 551 559
29. H.C. Peng, F.H. Long, and Chris Ding Feature selection based on mutual information: Criteria of max-dependency, max-relevance, and min-redundancy IEEE Transactions on Pattern Analysis and Machine Intelligence 27 8 2005 1226 1238 (Pubitemid 41245053)
30. Y.H. Qian, J.Y. Liang, and C.Y. Dang Knowledge structure, knowledge granulation and knowledge distance in a knowledge base International Journal of Approximate Reasoning 50 2009 174 188
31. G. Qu, S. Hariri, and M. Yousif A new dependency and correlation analysis for features IEEE Transactions on Knowledge and Data Engineering 17 9 2005 1199 1207
32. J.R. Quinlan Induction of decision trees 1 1 1986 81 106
33. J.R. Quinlan C4. 5: Programming for machine learning 1993 Morgan Kauffmann
34. S.J. Raudys, and A.K. Jain Small sample size effects in statistical pattern recognition: recommendations for practitioners IEEE Transactions on Pattern Analysis and Machine Intelligence 13 3 1991 252 264
35. C.E. Shannon A mathematical theory of communication The Bell System Technical Journal 27 1948 379 423 623-656
36. M.R. Sikonja, and I. Kononenko Theoretical and empirical analysis of ReliefF and RReliefF Machine Learning 53 2003 23 69
37. H. Wang Nearest neighbors by neighborhood counting IEEE Transactions on Pattern Analysis and Machine Intelligence 28 6 2006 942 953
38. H. Wang, D. Bell, and F. Murtagh Axiomatic approach to feature subset selection based on relevance IEEE Transactions on Pattern Analysis and Machine Intelligence 21 3 1999 271 277
39. D. Wettschereck, D.W. Aha, and T. Mohri review and comparative evaluation of feature weighting methods for lazy learning algorithms Artificial Intelligence Review 11 1-5 1997 273 314 (Pubitemid 127508242)
40. Y.Y. Yao, and Y. Zhao Attribute reduction in decision-theoretic rough set models Information Sciences 178 17 2008 3356 33731
41. L. Yu, and H. Liu Efficient feature selection via analysis of relevance and redundancy Journal of Machine Learning Research 5 Oct 2004 1205 1224
42. D.R. Yu, Q.H. Hu, and C.X. Wu Uncertainty measures for fuzzy relations and their applications Apply Soft Computing 7 2007 1135 1143