Resampling methods for parameter-free and robust feature selection with mutual information

Neurocomputing

Volumn 70, Issue 7-9, 2007, Pages 1276-1288

  Francois D ^a   Rossi, F ^b   Wertz, V ^a   Verleysen, M ^a  

^a UNIVERSITÉ CATHOLIQUE DE LOUVAIN   (Belgium)

^b INRIA   (France)

Author keywords

Feature selection; Mutual information; Permutation test

Indexed keywords

FEATURE EXTRACTION; PARAMETER ESTIMATION; PARAMETER EXTRACTION; SAMPLING;

MUTUAL INFORMATION; PERMUTATION TEST; RESAMPLING METHOD;

NEURAL NETWORKS;

ACCURACY; ARTICLE; AUTOMATION; BOOTSTRAPPING; CONFIDENCE INTERVAL; CRITERION VARIABLE; INFORMATION PROCESSING; NONLINEAR SYSTEM; NONPARAMETRIC TEST; PREDICTION; PRIORITY JOURNAL; PROBABILITY; PROCESS OPTIMIZATION; RELIABILITY; SENSITIVITY ANALYSIS; STATISTICAL SIGNIFICANCE; VALIDATION PROCESS;

EID: 33847674996     PISSN: 09252312     EISSN: None     Source Type: Journal    
DOI: 10.1016/j.neucom.2006.11.019     Document Type: Article

References (35)

1. Andreas Hahn P.T., Rahnenfhrer J., and Lengauer T. Confirmation of human protein interaction data by human expression data. BMC Bioinformatics 6 (2005) 112
2. Battiti R. Using the mutual information for selecting features in supervised neural net learning. IEEE Trans. Neural Networks 5 (1994) 537-550
3. Bellmann R. Adaptive Control Processes: A Guided Tour (1961), Princeton University Press, Princeton, NJ
4. Benoudjit N., and Verleysen M. On the kernel widths in radial-basis function networks. Neural Process. Lett. 18 (2003) 139-154
5. Bonnlander B.V., and Weigend A.S. Selecting input variables using mutual information and nonparametric density estimation. Proceedings of the 1994 International Symposium on Artificial Neural Networks (ISANN 94). Tainan, Taiwan (1994) 42-50
6. Conrad C., Erfle H., Warnat P., Daigle N., Lorch T., Ellenberg J., Pepperkok R., and Eils R. Automatic identification of subcellular phenotypes on human cell arrays. Genome Res. 14 6 (2004) 1130-1136
7. Craddock R., Taylor R., Broderick G., Whistler T., Klimas N., and Unger E. Exploration of statistical dependence between illness parameters using the entropy correlation coefficient. Pharmacogenomics 7 3 (2006) 421-428
8. Diks C., and Manzan S. Tests for serial independence and linearity based on correlation integrals. Studies Nonlinear Dyn. Econometrics 6 2 (2002) Article 2
9. Dijck G.V., and Hulle M.M.V. Speeding up the wrapper feature subset selection in regression by mutual information relevance and redundancy analysis. ICANN 2006: International Conference in Aritificial Neural Networks. Athens, Greece (September 2006)
10. Fleuret F. Fast binary feature selection with conditional mutual information. J. Mach. Learn. Res. 5 (2004) 1531-1555
11. François D., Wertz V., and Verleysen M. The permutation test for feature selection by mutual information. ESANN 2006, European Symposium on Artificial Neural Networks (2006), Bruges, Belgium 239-244
12. Frank E., and Witten I.H. Using a permutation test for attribute selection in decision trees. Proceedings 15th International Conference on Machine Learning. Morgan Kaufmann, San Francisco, CA (1998) 152-160
13. Friedman J. Multivariate adaptive regression splines (with discussion). Ann. Stat. 9 1 (1991) 1-141
14. Fung G.M., and Mangasarian O. A feature selection newton method for support vector machine classification. Comput. Optim. Appl. 28 2 (2004) 185-202
15. Good P. Permutation Tests (1994), Springer, New York
16. Guyon I., and Elisseeff A. An introduction to variable and feature selection. J. Mach. Learn. Res. 3 (2003) 1157-1182
17. Guyon I., Gunn S., Nikravesh M., and Zadeh L.A. Feature Extraction: Foundations and Applications (Studies in Fuzziness and Soft Computing) (2006), Springer, Berlin
18. Hammer B., and Villmann T. Generalized relevance learning vector quantization. Neural Networks 15 10 (2002) 1059-1068
19. Hild K., Erdogmus D., and Principe J. Blind source separation using renyiś mutual information. IEEE Signal Process. Lett. 8 6 (2001) 174-176
20. Hoffman N., Schiffer C., and Swanstrom R. Covariation of amino acid positions in HIV-1 protease. Virology 314 2 (2003) 536-548
21. Hummel J., Keshvari N., Weckwerth W., and Selbig J. Species-specific analysis of protein sequence motifs using mutual information. BMC Bioinformatics 6 (2005) 164
22. Kraskov A., Stögbauer H., and Grassberger P. Estimating mutual information. Phys. Rev. E 69 (2004) 066138
23. Kwak N., and Choi C.-H. Input feature selection by mutual information based on parzen window. IEEE Trans. Pattern Anal. Mach. Intell. 24 12 (2002) 1667-1671
24. Kwak N., and Choi C.-H. Input feature selection for classification problems. IEEE Trans. Neural Networks 13 1 (2002) 143-159
25. Nicolaou N., and Nasuto S.J. Mutual information for eeg analysis. Proceedings of Fourth IEEE EMBSS UKRI Postgraduate Conference on Biomedical Engineering and Medical Physics (PGBIOMED'05). Reading, UK, 18-20 (July, 2005) 23-24
26. Opdyke J. Fast permutation tests that maximize power under conventional Monte Carlo sampling for pairwise and multiple comparisons. J. Mod. Appl. Stat. Methods 2 1 (2003) 27-49
27. Purushothaman G., and Bradley D.C. Neural population code for fine perceptual decisions in area MT. Nat. Neurosci. 8 1 (2005) 99-106
28. P. Radivojac, Z. Obradovic, K. Dunker, S. Vucetic, Feature selection filters based on the permutation test, in: J.-F. Boulicaut, F. Esposito, F. Giannotti, D. Pedreschi (Eds.), Machine Learning: ECML 2004, 15th European Conference on Machine Learning, Pisa, Italy, September 20-24, 2004, Lecture Notes in Computer Science, vol. 3201, Springer, Berlin, 2004, pp. 334-346.
29. Rossi F., Delannay N., Conan-Guez B., and Verleysen M. Representation of functional data in neural networks. Neurocomputing 64 (2005) 183-210
30. Rossi F., Lendasse A., François D., Wertz V., and Verleysen M. Mutual information for the selection of relevant variables in spectrometric nonlinear modelling. Chemometrics Intelligent Lab. Syst. 2 (2005) 215-226
31. Rossi F., Lendasse A., François D., Wertz V., and Verleysen M. Mutual information for the selection of relevant variables in spectrometric nonlinear modelling. Chemometrics Intelligent Lab. Syst. 2 80 (2006) 215-226
32. Scott D. Multivariable Density Estimation: Theory, Practice, and Visualization (1992), Wiley, New York
33. Stefansson A., Koncar N., and Jones A.J. A note on the gamma test. Neural Comput. Appl. 5 3 (1997) 131-133
34. Verikas A., and Bacauskiene M. Feature selection with neural networks. Pattern Recognition Lett. 23 13 (September 2002) 1323-1335
35. Zhou X., Wang X., Dougherty E., and Suh D.R.E. Gene clustering based on clusterwide mutual information. J. Comput. Biol. 11 1 (2004) 147-161