Supervised feature selection by clustering using conditional mutual information-based distances

Pattern Recognition

Volumn 43, Issue 6, 2010, Pages 2068-2081

  Martínez Sotoca, José ^a   Pla, Filiberto ^a  

^a UNIVERSITAT JAUME I   (Spain)

Author keywords

Clustering; Conditional mutual information; Supervised feature selection

Indexed keywords

CLASS LABELS; CLASSIFICATION ACCURACY; CONDITIONAL MUTUAL INFORMATION; DISCRETE DATA; FEATURE SELECTION; HIER-ARCHICAL CLUSTERING; INFORMATION MEASURES; INFORMATION THEORETIC MEASURE; STATE-OF-ART METHODS;

CLUSTERING ALGORITHMS; INFORMATION THEORY; INFORMATION USE;

FEATURE EXTRACTION;

EID: 76749129275     PISSN: 00313203     EISSN: None     Source Type: Journal    
DOI: 10.1016/j.patcog.2009.12.013     Document Type: Article

References (30)

1. L.D. Baker, A. McCallum, Distributional clustering of words for text classification, in: 21th Annual International ACM SIGIR, ACM, August 1998, pp. 96-103.
2. Battiti R. Using mutual information for selection features in supervised neural net learning. IEEE Trans. Neural Networks 5 4 (1994) 537-550
3. Blum A.L., and Langley P. Selection of relevant features and examples in machine learning. Artif. Intell. 97 1-2 (1997) 245-271
4. Breiman L., Friedman J., Stone C.J., and Olshen R.A. Classification and Regression Trees (1984), CRC Press, Boca Raton
5. Cover T., and Hart P. Nearest neighbor pattern classification. IEEE Trans. Inf. Theory IT-13 1 (1967) 21-27
6. Cover T.M., and Thomas J.A. Elements of Information Theory (1991), Wiley, New York
7. Cover T.M. The best two independent measurements are not the two best. IEEE Trans. Syst. Man Cybern. 4 (1974) 116-117
8. Cristianini N., and Shawe-Taylor J. An Introduction to Support Vector Machines and Other Kernel-Based Learning Methods (2000), Cambridge University Press, Cambridge, UK
9. Dhillong I., Mallela S., and Kumar R. A divisive information-theoretic feature clustering algorithm for text classification. J. Mach. Learn. Res. 3 (2003) 1265-1287
10. Friedman M. The use of ranks to avoid the assumption of normality implicit in the analysis of variance. J. Am. Stat. Assoc. 32 200 (1937) 675-701
11. I. Guyon, A. Elisseeff, An introduction to variable and feature selection, Special issue on variable and feature selection, J. Mach. Learn. Res. 3 (2003) 1157-1182.
12. I. Guyon, S. Gunn, M. Nikravesh, L. Zadeh (Eds.), Feature extraction, foundations and applications, in: Series Studies in Fuzziness and Soft Computing, Physica-Verlag, Springer, 2006.
13. Jain A.K., Duin R.P.W., and Mao J. Statistical pattern recognition: a review. IEEE Trans. Pattern Anal. Mach. Intell. 22 1 (2000) 4-37
14. John G.H., Kohavi R., and Pfleger K. Irrelevant features and the subset selection problem. 11th International Conference on Machine Learning (1994), Morgan Kaufmann, Los Altos, CA 121-129
15. Kohavi R., and John G.H. Wrapper for feature subset selection. Artif. Intell. 97 1-2 (1997) 273-324
16. Kudo M., and Sklansky J. Comparison of algorithms that select features for pattern classifiers. Pattern Recognition 33 (2000) 25-41
17. Kwak N., and Choi C.-H. Input feature selection by mutual information based on parzen window. IEEE Trans. Pattern Anal. Mach. Intell. 24 12 (2002) 1667-1671
18. Kwak N., and Choi C.-H. Input feature selection for classification problems. IEEE Trans. Neural Networks 13 1 (2002) 143-159
19. Hsu C.W., and Lin C.J. A comparison of methods for multi-class support vector machines. IEEE Trans. Neural Networks 13 (2002) 415-425. http://www.csie.ntu.edu.tw/~cjlin/libsvm [Online]. Available: 〈www.csie.ntu.edu.tw/∼cjlin/libsvm〉
20. Li J. Divergence measures based on the Shannon entropy. IEEE Trans. Inf. Theory 37 1 (1991) 145-151
21. Dash M., and Liu H. Feature selection for classification. Intelligent Data Anal. 1 (1997) 131-156
22. P.M. Murphy, UCI Repository of Machine Learning 〈http://archive.ics.uci.edu/ml/〉, Department of Information and Computer Science, University of California, Irvine, CA, 1995.
23. Peng H., Long F., and Ding C. Feature selection based on mutual information: criteria of max-dependency, max-relevance, and min-redundance. IEEE Trans. Pattern Anal. Mach. Intell. 27 8 (2005) 1226-1238
24. F.C. Pereira, N. Tishby, L. Lee, Distributional clustering of English words, in: 30th Annual Meeting of the Association for Computational Linguistics, Columbus, Ohio, 1993, pp. 183-190.
25. Pudil P., Ferri F.J., Novovicova J., and Kittler J. Floating search methods for feature selection with nonmonotonic criterion functions. Pattern Recognition 2 (1994) 279-283
26. Quinlan J.R. Improved use of continuous attributes in C4.5. J. Artif. Intell. Res. 4 (1996) 77-90
27. N. Slonim, N. Tishby, Agglomerative information bottleneck, in: Proceedings of Neural Information Processing Systems (NIPS99), 1999, pp. 617-623.
28. Tishby N., Pereira F., and Bialek W. The information bottleneck method. In: Hajek B., and Sreenivas R.S. (Eds). 37th Annual Allerton Conference Urbana (1999), University of Illinois
29. Ward J.H. Hierarchical grouping to optimize an objective function. Am. Stat. Assoc. 58 301 (1963) 236-244
30. R.W. Yeung, A First Course in Information Theory, Information Technology: Transmision, Processing, and Storage (series), Springer Science + Business Media, LLC, 2002.