A parameterless feature ranking algorithm based on MI

Neurocomputing

Volumn 71, Issue 7-9, 2008, Pages 1656-1668

  Huang, Jin Jie ^a,b   Cai, Yun Ze ^b   Xu, Xiao Ming ^b,c  

^a HARBIN UNIVERSITY OF SCIENCE AND TECHNOLOGY   (China)

^b SHANGHAI JIAO TONG UNIVERSITY   (China)

^c UNIVERSITY OF SHANGHAI FOR SCIENCE AND TECHNOLOGY   (China)

Author keywords

Feature ranking; Feature selection; Machine learning; Mutual information; Pattern classification

Indexed keywords

ALGORITHMS; ESTIMATION; LEARNING SYSTEMS;

FEATURE RANKING; GREEDY FEATURE SELECTION ALGORITHM; MUTUAL INFORMATION;

FEATURE EXTRACTION;

BIOLOGICAL MARKER;

ANALYTICAL ERROR; ARTICLE; BAYES THEOREM; COMPUTER PROGRAM; ENTROPY; INFORMATION PROCESSING; KERNEL METHOD; MACHINE LEARNING; MATHEMATICAL ANALYSIS; MATHEMATICAL COMPUTING; PATTERN RECOGNITION; PRIORITY JOURNAL; PROSTATE CANCER; QUALITY CONTROL; SUPPORT VECTOR MACHINE;

EID: 40649115462     PISSN: 09252312     EISSN: None     Source Type: Journal    
DOI: 10.1016/j.neucom.2007.04.012     Document Type: Article

References (50)

1. Amaldi E., and Kann V. On the approximation of minimizing non zero variables or unsatisfied relations in linear systems. Theoret. Comput. Sci. 209 (1998) 237-260
2. Battiti R. Using MI for selecting features in supervised neural net learning. IEEE Trans. Neural Networks 5 4 (1994) 537-550
3. Bhanu B., and Lin Y. Genetic algorithm based feature selection for target detection in SAR images. Image Vis. Comput. 21 7 (2003) 591-608
4. Bhatt R.B., and Gopal M. On fuzzy-rough sets approach to feature selection. Pattern Recogn. Lett. 26 7 (2005) 965-975
5. B.V. Bonnlander, A.S. Weigend, Selecting input variables using MI and nonparametric density estimation, in: Proceedings of International Symposium on Artificial Neural Networks, Taiwan, 1994, pp. 42-50.
6. Castellano G., and Fanelli A.M. Variable selection using neural-network models. Neurocomputing 31 3 (2000) 1-13
7. Chakraborty D., and Pal N.R. A neuro-fuzzy scheme for simultaneous feature selection and fuzzy rule-based classification. IEEE Trans. Neural Networks 15 1 (2004) 110-123
8. Cover T.M., and Thomas J.A. Elements of Information Theory (1991), Wiley, New York
9. Dash M., and Liu H. Feature selection for classification. Intell. Data Anal. 1 (1997) 131-156
10. Dash M., and Liu H. Consistency-based search in feature selection. Artif.Intell. 151 1-2 (2003) 155-176
11. Fano R.M. Transmission of Information: A Statistical Theory of Communications (1961), Wiley, New York
12. M.E. Farmer, S. Bapna, A.K. Jain, Large scale feature selection using modified random mutation hill climbing, in: Proceedings of the 17th International Conference on Pattern Recognition, ICPR 2004, vol. 2, Cambridge, UK, 2004, pp. 287-290.
13. Febbo S., et al. Gene expression correlates of clinical prostate cancer behavior. Cancer Cell March (2002). 〈http://www.broad.mit.edu/mpr/publications/projects/Prostate_Cancer/Prostate_TN_final0701_allmeanScale.res〉
14. Francesco M. Feature selection based on a modified fuzzy C-means algorithm with supervision. Inform. Sci. 151 (2003) 201-226
15. Fraser A.M., and Swinney H.L. Independent coordinates for strange attractors from MI. Phys. Rev. A 33 2 (1986) 1134-1140
16. Hellman M.E., and Raviv J. Probability of error, equivocation and the chernoff bound. IEEE Transactions on Information Theory 16 4 (1970) 368-372
17. Guyon I., and Elisseeff A. An introduction to variable and feature selection. J. Mach. Learn. Res. 3 (2003) 1157-1182
18. Hsing T., Liu L.-Y., Brun M., and Dougherty E.R. The coefficient of intrinsic dependence (feature selection using el CID). Pattern Recogn. 38 5 (2005) 623-636
19. Huang D., and Chow T.W.S. Effective feature selection scheme using MI. Neurocomputing 63 (2005) 325-343
20. Jain A., and Zongker D. Feature selection: evaluation, application, and small sample performance. IEEE Trans. Pattern Anal. Mach. Intell. 19 2 (1997) 153-158
21. Kapur J.N. Measures of Information and their Applications (1994), Wiley, New Delhi, India
22. Kohavi R., and John G.H. Wrappers for feature subset selection. Artif. Intell. 97 (1997) 273-324
23. D. Koller, M. Sahami, Toward optimal feature selection, in: Proceedings of International Conference on Machine Learning, Bari, Italy, 1996, pp. 284-292.
24. Kwak N., and Choi C.H. Input feature selection by MI based on Parzen window. IEEE Trans. Pattern Anal. Mach. Intell. 24 12 (2002) 1667-1671
25. Kwak N., and Choi C.H. Input feature selection for classification problems. IEEE Trans. Neural Networks 13 1 (2002) 143-159
26. Last M., and Maimon O. A compact and accurate model for classification. IEEE Trans. Knowl. Data Eng. 16 2 (2004) 203-215
27. Lazzerini B., and Marcelloni F. Feature selection based on similarity. Electron. Lett. 38 3 (2002) 121-122
28. Li R.P., Mukaidono M., and Turksen I.B. A fuzzy neural network for pattern classification and feature selection. Fuzzy Sets Systems 130 1 (2002) 101-108
29. Liu H., and Motoda H. Feature Selection for Knowledge Discovery and Data Mining (1998), Kluwer Academic Publishers, Boston
30. Malhi A., and Gao R.X. PCA-based feature selection scheme for machine defect classification. IEEE Trans. Instrum. Meas. 53 6 (2004) 1517-1525
31. Merz C.J., and Murphy P.M. UCI Repository of Machine Learning Databases (1996), University of California, Department of Information and Computer Science, Irvine, CA. 〈http://www.ics.uci.edu/~mlearn/MLRepository.html〉
32. Narendra P., and Fukunaga K. A branch and bound algorithm for feature subset selection. IEEE Trans. Comput. 26 9 (1977) 917-922
33. Oh I.-S., Lee J.-S., and Moon B.-R. Hybrid genetic algorithms for feature selection. IEEE Trans. Pattern Anal. Mach. Intell. 26 11 (2004) 1424-1437
34. Pudil P., Novovicova J., and Kittler J. Floating search methods in feature selection. Pattern Recogn. Lett. 15 11 (1994) 1119-1125
35. A. Renyi, On measures of entropy and information, in: Proceedings of the Fourth Berkeley Symposium on Mathematical Statistics and Probability, University of California Press, 1961, pp. 547-561.
36. Sant'anna A.P., and Taneja I.J. Trigonometric entropies, Jensen difference divergences and error bounds. Inform. Sci. 35 (1985) 145-156
37. J.C. Schlimmer, Efficiently inducing determinations: a complete and systematic search algorithm that uses optimal pruning, in: Proceedings of Tenth International Conference on Machine Learning, Morgan Kaufmann, Amherst, MA, 1993, pp. 284-290.
38. Shannon C.E. A mathematical theory of communication. The Bell System Tech. J. 27 (1948) 379-423 623-656
39. Somol P., Pudil P., Novoviova J., and Paclik P. Adaptive floating search methods in feature selection. Pattern Recogn. Lett. 20 11-13 (1999) 1157-1163
40. Somol P., Pudil P., and Kittler J. Fast branch and bound algorithms for optimal feature selection. IEEE Trans. Pattern Anal. Mach. Intell. 26 7 (2004) 900-912
41. Swiniarski R.W., and Andrzej S. Rough set methods in feature selection and recognition. Pattern Recogn. Lett. 24 6 (2003) 833-849
42. Torkkola K. Feature extraction by non-parametric mi maximization. J. Mach. Learn. Res. 3 (2003) 1415-1438
43. Vapnik V. Statistical Learning Theory (1998), Wiley, New York
44. Verikas A., and Bacauskiene M. Feature selection with neural networks. Pattern Recogn. Lett. 23 11 (2002) 1323-1335
45. Wang X., and Paliwal K.K. Feature extraction and dimensionality reduction algorithms and their applications in vowel recognition. Pattern Recogn. 36 10 (2003) 2429-2439
46. Wang H., Bell D., and Murtagh F. Axiomatic approach to feature subset selection based on relevance. IEEE Trans. Pattern Anal. Mach. Intell. 21 3 (1999) 271-277
47. Yang S., and Gu J. Feature selection based on MI and redundancy-synergy coefficient. J. Zhejiang Univ. Sci. 5 11 (2004) 1382-1391
48. Yang J., and Honavar V. Feature subset selection using a genetic algorithm. IEEE Intell. Systems 13 (1998) 44-49
49. H. Yang, J. Moody, Data visualization and feature selection: New algorithms for nongaussian data, in: Advances in Neural Information Processing Systems vol. 12, MIT Press, 2000, pp. 687-693.
50. H. Ye, H. Liu, A SOM-based method for feature selection, in: Proceedings of the 9th International Conference on Neural Information Processing vol. 3, 2002, pp. 1295-1299.