Feature selection for classificatory analysis based on information-theoretic criteria

Zidonghua Xuebao/Acta Automatica Sinica

Volumn 34, Issue 3, 2008, Pages 383-392

  Huang, Jin Jie ^a   Lu, Ning ^a   Li, Shuang Quan ^a   Cai, Yun Ze ^b  

^a HARBIN UNIVERSITY OF SCIENCE AND TECHNOLOGY   (China)

^b SHANGHAI JIAO TONG UNIVERSITY   (China)

Author keywords

Data mining; Feature selection; Information theoretic measures; Pattern classification

Indexed keywords

DATA MINING; INFORMATION THEORY; NEURAL NETWORKS; PRINCIPAL COMPONENT ANALYSIS;

DATA SETS; FEATURE SELECTION; INFORMATION THEORETIC MEASURES; MUTUAL INFORMATION; PATTERN CLASSIFICATION;

FEATURE EXTRACTION;

EID: 42449117307     PISSN: 02544156     EISSN: None     Source Type: Journal    
DOI: 10.3724/SP.J.1004.2008.00383     Document Type: Article

References (40)

1. Guyon I, Elisseeff A. An introduction to variable and feature selection. The Journal of Machine Learning Research, 2003, 3(7-8): 1157-1182
2. Dash M, Liu H. Feature selection for classification. Intelligent Data Analysis, 1997, 1(3): 131-156
3. Ahmad A, Dey L. A feature selection technique for classificatory analysis. Pattern Recognition Letters, 2005, 26(1): 43-56
4. Liu H, Motoda H. Feature Selection for Knowledge Discovery and Data Mining. Boston, USA: Kluwer Academic Publishers, 1998
5. Malhi A, Gao R X. PCA-based feature selection scheme for machine defect classification. IEEE Transactions on Instrumentation and Measurement, 2004, 53(6): 1517-1525
6. Salcedo-Sanz S, Camps-Valls G, Perez-Cruz F, Sepulveda-Sanchis J, Bousono-Calzon C. Enhancing genetic feature selection through restricted search and Walsh analysis. IEEE Transactions on Systems, Man, and Cybernetics-Part C: Applications and Reviews, 2004, 34(4): 398-406
7. Verikas A, Bacauskiene M. Feature selection with neural networks. Pattern Recognition Letters, 2002, 23(11): 1323-1335
8. Shima K, Todoriki M, Suzuki A. SVM-based feature selection of latent semantic features. Pattern Recognition Letters, 2004, 25(9): 1051-1057
9. Jensen R, Shen Q. Semantics-preserving dimensionality reduction: rough and fuzzy-rough-based approaches. IEEE Transactions on Knowledge and Data Engineering, 2004, 16(12): 1457-1471
10. Swiniarski R W, Skowron A. Rough set methods in feature selection and recognition. Pattern Recognition Letters, 2003, 24(6): 833-849
11. Chakraborty D, Pal N R. A neuro-fuzzy scheme for simultaneous feature selection and fuzzy rule-based classification. IEEE Transactions on Neural Networks, 2004, 15(1): 110-123
12. Marcelloni F. Feature selection based on a modified fuzzy C-means algorithm with supervision. Information Sciences, 2003, 151(5): 201-226
13. Ye H L, Liu H C. A SOM-based method for feature selection. In: Proceedings of the 9th International Conference on Neural Information Processing. Singapore, Singapore: IEEE, 2002. 1295-1299
14. Farmer M E, Bapna S, Jain A K. Large scale feature selection using modified random mutation hill climbing. In: Proceedings of the 17th International Conference on Pattern Recognition. Cambridge, UK: IEEE, 2004. 287-290
15. Somol P, Pudil P, Kittler J. Fast branch and bound algorithms for optimal feature selection. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2004, 26(7): 900-912
16. Bhanu B, Lin Y. Genetic algorithm based feature selection for target detection in SAR images. Image and Vision Computing, 2003, 21(7): 591-608
17. Oh I S, Lee J S, Moon B R. Hybrid genetic algorithms for feature selection. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2004, 26(11): 1424-1437
18. Kohavi R, John G H. Wrappers for feature subset selection. Artificial Intelligence, 1997, 97(1-2): 273-324
19. Wang X C, Paliwal K K. Feature extraction and dimensionality reduction algorithms and their applications in vowel recognition. Pattern Recognition, 2003, 36(10): 2429-2439
20. Lazzerini B, Marcelloni F. Feature selection based on similarity. Electronics Letters, 2002, 38(3): 121-122
21. Dash M, Liu H. Consistency-based search in feature selection. Artificial Intelligence, 2003, 151(1): 155-176
22. Shannon C E, Weaver W. The Mathematical Theory of Communication. Urbana, Israel: University of Illinois Press, 1949
23. Cover T M, Thomas J A. Elements of Information Theory. New York, USA: Wiley-Interscience Press, 1991
24. Last M, Maimon O. A compact and accurate model for classification. IEEE Transactions on Knowledge and Data Engineering, 2004, 16(2): 203-215
25. Koller D, Sahami M. Toward optimal feature selection. In: Proceedings of International Conference on Machine Learning. Bari, Italy: Morgan Kaufmann, 1996. 284-292
26. Battiti R. Using mutual information for selecting features in supervised neural net learning. IEEE Transactions on Neural Networks, 1994, 5(4): 537-550
27. Kwak N, Choi C H. Input feature selection for classification problems. IEEE Transactions on Neural Networks, 2002, 13(1): 143-159
28. Grail-Maes E, Beauseroy P. Mutual information-based feature extraction on the time-frequency plane. IEEE Transactions on Signal Processing, 2002, 50(4): 779-790
29. Quinlan J R. Improved use of continuous attributes in C4.5. Journal of Artificial Intelligence Research, 1996, 96(4): 77-90
30. Fano R M. Transmission of Information: A Statistical Theory of Communications. New York, USA: Wiley, 1961
31. Fraser A M, Swinney H L. Independent coordinates for strange attractors from mutual information. Physical Review A, 1986, 33(2): 1134-1140
32. Hellman M E, Raviv J. Probability of error, equivocation, and the Chernoff bound. IEEE Transactions on Information Theory, 1970, 16(4): 368-372
33. Feder M, Merhav N. Relations between entropy and error probability. IEEE Transactions on Information Theory, 1994, 40(1): 259-266
34. Han T S, Verdu S. Generalizing the Fano inequality. IEEE Transactions on Information Theory, 1994, 40(4): 1247-1251
35. Erdogmus D, Principe J. Lower and upper bounds for mis-classification probability based on Renyi's information. The Journal of VLSI Signal Processing, 2004, 37(2-3): 305-317
36. Silverman B W. Density Estimation for Statistics and Data Analysis. London, UK: Chapman and Hall, 1993
37. Biau G, Devroye L. Density estimation by the penalized combinatorial method. Journal of Multivariate Analysis, 2005, 94(1): 196-208
38. Browne M. A geometric approach to non-parametric density estimation. Pattern Recognition, 2007, 40(1): 134-140
39. Merz C J, Murphy P M. UCI repository of machine learning databases [online]. available: http://www.ics.uci.edu/{mlearn/MLRepository.html, October 10, 2007
40. Vapnik V N. Statistical Learning Theory. New York: Wiley-Interscience, 1998