Feature selection with missing data using mutual information estimators

Neurocomputing

Volumn 90, Issue , 2012, Pages 3-11

  Doquire, Gauthier ^a   Verleysen, Michel ^a  

^a UNIVERSITÉ CATHOLIQUE DE LOUVAIN   (Belgium)

Author keywords

Feature selection; Missing data; Mutual information

Indexed keywords

IMPUTATION ALGORITHM; MISSING DATA; MUTUAL INFORMATIONS; NEAREST NEIGHBORS; PREDICTION MODEL; PREDICTION PROBLEM; RANDOM DATA; REAL-WORLD DATASETS; REAL-WORLD PROBLEM;

FEATURE EXTRACTION; MATHEMATICAL MODELS; PATTERN RECOGNITION;

ESTIMATION;

ALGORITHM; ARTICLE; DATA BASE; DATA PROCESSING; FEATURE SELECTION; INFORMATION SYSTEM; MACHINE LEARNING; PREDICTION; PRIORITY JOURNAL; PROBABILITY; STATISTICAL ANALYSIS;

EID: 84860237900     PISSN: 09252312     EISSN: 18728286     Source Type: Journal    
DOI: 10.1016/j.neucom.2012.02.031     Document Type: Article

References (46)

1. Schafer J.L., Graham J.W. Missing data: our view of the state of the art. Psycholog. Methods 2002, 7(2):147-177.
2. Little R.J.A., Rubin D.B. Statistical Analysis with Missing Data 2002, Wiley-Interscience. 2nd ed.
3. King G., Honaker J., Joseph A., Scheve K. Analyzing incomplete political science data: an alternative algorithm for multiple imputation. Am. Polit. Sci. Rev. 2000, 95:49-69.
4. Rubin D.B. Inference and missing data. Biometrika 1976, 63:581-592. 10.1093/biomet/63.3.581.
5. Acock A.C. Working with missing values. J. Marriage Family 2005, 67:1012-1028.
6. Guyon I., Elisseeff A. An introduction to variable and feature selection. J. Mach. Learn. Res. 2003, 3:1157-1182.
7. Verleysen M. Learning high-dimensional data. Limit. Future Trends Neural Comput. 2003, 186:141-162.
8. Saeys Y., Inza I.n., Larrañaga P. A review of feature selection techniques in bioinformatics. Bioinformatics 2007, 23:2507-2517.
9. Y. Yang, J.O. Pedersen, A comparative study on feature selection in text categorization, in: Proceedings of the Fourteenth International Conference on Machine Learning, ICML '97, Morgan Kaufmann Publishers Inc., San Francisco, CA, USA, 1997, pp. 412-420.
10. Rossi F., Lendasse A., Francois D., Wertz V., Verleysen M. Mutual information for the selection of relevant variables in spectrometric nonlinear modelling. Chemomet. Intell. Lab. Syst. 2006, 80:215-226.
11. Kraskov A., Stögbauer H., Grassberger P. Estimating mutual information. Phys. Rev. E 2004, 69(6):066138.
12. G. Doquire, M. Verleysen, Mutual information for feature selection with missing data, in: Proceedings of the European Symposium on Artificial Neural Networks (ESANN 2011), 2011.
13. J.R. Quinlan, Unknown attribute values in induction, in: Proceedings of the Sixth International Machine Learning Workshop, Morgan Kaufmann, 1989, pp. 164-168.
14. Troyanskaya O., Cantor M., Sherlock G., Brown P., Hastie T., Tibshirani R., Botstein D., Altman R.B. Missing value estimation methods for DNA microarrays. Bioinformatics (Oxford, England) 2001, 17(6):520-525.
15. Junninen H., Niska H., Tuppurainen K., Ruuskanen J., Kolehmainen M. Methods for imputation of missing values in air quality data sets. Atmos. Environ. 2004, 38(18):2895-2907.
16. J.V. Hulse, T.M. Khoshgoftaar, Incomplete-case nearest neighbor imputation in software measurement data, in: Proceedings of the IEEE International Conference on Information Reuse and Integration, IRI 2007, 2007, pp. 630-637.
17. T. Schneider, Analysis of incomplete climate data: estimation of mean values and covariance matrices and imputation of missing values, J. Clim. 14 (5) (2001) 853-871.
18. Rubin D.B. Multiple Imputation for Nonresponse in Surveys 1987, John Wiley & Sons, New York.
19. D. Williams, X. Liao, Y. Xue, L. Carin, Incomplete-data classification using logistic regression, in: In ICML, ACM Press, 2005, pp. 972-979.
20. K. Wagstaff, Clustering with missing values: no imputation required, in: Proceedings of the Meeting of the International Federation of Classification Societies, 2004, pp. 649-658.
21. Kohavi R., John G.H. Wrappers for feature subset selection. Artif. Intell. 1997, 97:273-324.
22. M. Hall, Correlation-based Feature Selection for Machine Learning, Ph.D. Thesis, University of Waikato, 1999.
23. Peng H., Long F., Ding C. Feature selection based on mutual information criteria of max-dependency, max-relevance, and min-redundancy. IEEE Trans. Pattern Anal. Mach. Intell. 2005, 27(8):1226-1238.
24. Steuer R., Kurths J., Daub C.O., Weise J., Selbig J. The mutual information: detecting and evaluating dependencies between variables. Bioinformatics 2002, 18(Suppl. 2):S231-S240.
25. Brown G. A new perspective for information theoretic feature selection. J. Mach. Learn. Res.: Proc. Track 2009, 5:49-56.
26. Zou H., Hastie T. Regularization and variable selection via the elastic net. J. R. Stat. Soc. B 2005, 67:301-320.
27. Yuan M., Lin Y. Model selection and estimation in regression with grouped variables. J. R Stat. Soc., Ser. B 2006, 68:49-67.
28. M. Zaffalon, M. Hutter, Robust feature selection by mutual information distributions, in: Proceedings of the 18th International Conference on Uncertainty in Artificial Intelligence (UAI-2002), Morgan Kaufmann, 2002, pp. 577-584.
29. P. Meesad, K. Hengpraprohm, Combination of knn-based feature selection and knn-based missing-value imputation of microarray data, in: Proceedings of the 2008 Third International Conference on Innovative Computing Information and Control, IEEE Computer Society, Washington, DC, USA, 2008, pp. 341-344.
30. Shannon C.E. A mathematical theory of communication. Bell Syst. Tech. J. 1948, 27:379-423.
31. Battiti R. Using mutual information for selecting features in supervised neural net learning. IEEE Trans. Neural Networks 1994, 5:537-550. 10.1109/72.298224.
32. Fleuret F. Fast binary feature selection with conditional mutual information. J. Mach. Learn. Res. 2004, 5:1531-1555.
33. Kullback S., Leibler R.A. On information and sufficiency. Ann. Math. Stat. 1951, 22(1):79-86.
34. Parzen E. On estimation of a probability density function and mode. Ann. Math. Stat. 1962, 33(3):1065-1076.
35. Rossi F., François D., Wertz V., Verleysen M. Fast selection of spectral variables with b-spline compression. Chemomet. Intell. Lab. Syst. 2007, 86(2):208-218.
36. Kozachenko L.F., Leonenko N. Sample estimate of the entropy of a random vector. Prob. Inf. Transm. 1987, 23:95-101.
37. Francois D., Rossi F., Wertz V., Verleysen M. Resampling methods for parameter-free and robust feature selection with mutual information. Neurocomputing 2007, 70(7-9):1276-1288.
38. Hathaway R.J., Bezdek J.C. Fuzzy c-means clustering of incomplete data. IEEE Trans. Syst. Man Cybern. B: Cybern. 2001, 31(5):735-744.
39. R.J. Almeida, U. Kaymak, J.M.C. Sousa, A new approach to dealing with missing values in data-driven fuzzy modeling, in: Proceedings of the IEEE International Conference on Fuzzy Systems (FUZZ), 2010, pp. 1-7.
40. A. Sorjamaa, A. Lendasse, E. Séverin, Combination of SOMs for fast missing value imputation, in: Proceedings of MASHS, Models and Learnings in Human and social Sciences, 2010.
41. Guyon I., Gunn S., Nikravesh M., Zadeh L.A. Feature Extraction: Foundations and Applications 2006, Springer-Verlag, Inc., New York.
42. Friedman J.H. Multivariate adaptive regression splines. Ann. Stat. 1991, 19(1):1-67.
43. Rossi F., Delannay N., Conan-Guez B., Verleysen M. Representation of functional data in neural networks. Neurocomputing 2005, 64:183-210. (Trends in Neurocomputing: 12th European Symposium on Artificial Neural Networks 2004).
44. Pelckmans K., De Brabanter J., Suykens J.A.K., De Moor B. Handling missing values in support vector machine classifiers. Neural Networks 2005, 18(5-6):684-692.
45. Benoudjit N., Verleysen M. On the kernel widths in radial-basis function networks. Neural Process. Lett. 2003, 18(2):139-154. http://dx.doi.org/10.1023/A:1026289910256.
46. Gomez-Verdejo V., Verleysen M., Fleury J. Information-theoretic feature selection for functional data classification. Neurocomputing 2009, 72(16-18):3580-3589.