Ranking a random feature for variable and feature selection

Journal of Machine Learning Research

Volumn 3, Issue , 2003, Pages 1399-1414

  Stoppiglia, Hervé ^a   Dreyfus, Gérard ^b   Dubois, Rémi ^b   Oussar, Yacine ^b  

^a Informatique Caisse des Dépôts et Consignations ^*  (France)

^b PSL RESEARCH UNIVERSITY   (France)

Author keywords

Classification; Feature selection; Gram Schmidt orthogonalization; Information filtering; Kernel; Leave one out; Model selection; Neural networks; Statistical tests; Variable selection

Indexed keywords

GRAM-SCHMIDT ORTHOGONALIZATIONS; KERNEL; LEAVE-ONE-OUT; MODEL SELECTION; VARIABLE SELECTION;

ARTIFICIAL INTELLIGENCE; CLASSIFICATION (OF INFORMATION); FEATURE EXTRACTION; INFORMATION FILTERING; NEURAL NETWORKS; SOFTWARE ENGINEERING;

STATISTICAL TESTS;

EID: 2942701493     PISSN: 15324435     EISSN: 15337928     Source Type: Journal    
DOI: None     Document Type: Article

References (29)

1. A. Bjoerck. Solving linear least squares problems by gram-schmidt orthogonalization. Nordisk Tidshrift for Informationsbehadlung, 7:1-21, 1967.
2. N. Bodor, M. J. Huang, and A. Harget. Neural network studies. 3. prediction of partition coefficients. J. Mol. Struct. (Theochem.), 309:259-266, 1994.
3. C. Breneman, K. Bennett, M. Embrechts, S. Cramer, M. Song, and J. Bi. Descriptor generation, selection and model building in quantitative structure-property analysis. In J. Crawse, editor, Experimental Design for Combinatorial and High Throughput Materials Development. Wiley (to be publised), 2002.
4. S. Chen, S.A. Billings, and W. Luo. Orthogonal least squares methods and their application to non-linear system identification. International Journal of Control, 50:1873-1896, 1989.
5. S. Chen, F. Cowan, and P. Grant. Orthogonal least squares learning algorithm for radial basis function networks. IEEE Transactions on Neural Networks, 2:302-309, 1991.
6. J. de Lagarde. Initiation à l'analyse des données. Dunod, Paris, 1983.
7. A. Duprat, T. Huynh, and G. Dreyfus. Towards a principled methodology for neural network design and performance evaluation in qsar; application to the prediction of logp. J. Chem. Inf. Comp. Sci., 38:586-594, 1998.
8. G. C. Goodwin and R.L. Payne. Dynamic system identification: Experiment design and data analysis. Mathematics in Science and Engineering, Academic Press, 136, 1977.
9. I. Guyon, J. Weston, S. Barnhill, and V. Vapnik. Gene selection for cancer classification using support vector machines. Machine Learning, 46:389-422, 2002.
10. C. Hansch and A. Leo. Exploring qsar, fundamentals and applications in chemistry and biology. American Chemical Society, 1995.
11. H. Jing and E. Tzoukermann. Information retrieval based on context distance and morphology. In Proceedings of the 22nd Annual International Conference on Research and Development in Information Retrieval (SIGIR '99), pages 90-96, 1999.
12. I. J. Leontaritis and S. A. Billings. Model selection and validation methods for non-linear systems. International Journal of Control, 45:311-341, 1987.
13. G. Monari. Sélection de modèles non linéaires par leave-one-out. Etude théorique et application au procédé de soudage par points. PhD thesis, Université Pierre et Marie Curie, 1999.
14. G. Monari and G. Dreyfus. Withdrawing an example from the training set: an analytic estimation of its effect on a nonlinear parameterised model. Neurocomputing, 35:195-201, 2000.
15. G. Monari and G. Dreyfus. Local overfitting control via leverages. Neural Computation, 14(6): 1481-1506, 2002.
16. A. Mood, F. A. Graybill, and D. C. Boes. Introduction to the Theory of Statistics. MacGraw-Hill International, 1974.
17. R. H. Myers. Classical and Modern Regression with Applications. Duxbury Press, 1990.
18. L. Oukhellou, P. Aknin, H. Stoppiglia, and G. Dreyfus. A new decision criterion for feature selection: Application to the classification of non destructive testing signatures. In European SIgnal Processing COnference (EUSIPCO'98), 1998.
19. Y. Oussar. Réseaux d'ondelettes et réseaux de neurones pour la modélisation statique et dynamique de processus. PhD thesis, Université Pierre et Marie Curie, 1998.
20. Y. Oussar and G. Dreyfus. Initialization by selection for wavelet network training. Neurocomputing, 34:131-143, 2000.
21. R. Reed. Pruning algorithms - a survey. IEEE Transactions on Neural Networks, 4:740-747, 1993.
22. T. Soederstroem. On model structure testing in system identification. International Journal of Control, 26:1-18, 1977.
23. H. Stoppiglia. Méthodes Statistiques de Sélection deModèles Neuronaux; Applications Financières et Bancaires. PhD thesis, Université Pierre et Marie Curie, Paris, 1997.
24. M. Stricker. Réseaux de neurones pour le traitement automatique du langage : conception et réalisation de filtres d'informations. PhD thesis, Université Pierre et Marie Curie, Paris, 2000.
25. M. Stricker, F. Vichot, G. Dreyfus, and F. Wolinski. Two-step feature selection and neural network classification for the trec-8 routing. In Proceedings of the Eighth Text Retrieval Conference, 1999.
26. D. Urbani, P. Roussel-Ragot, L. Personnaz, and G. Dreyfus. The selection of neural models of nonlinear dynamical systems by statistical tests. In J. Vlontzos, J.Hwang, and E. Wilson, editors, Neural Networks for Signal Processing IV, pages 229-237, 1993.
27. P. Vincent and Y. Bengio. Kernel matching pursuit. Machine Learning, 48:165-187, 2001.
28. J. Weston, S. Mukherjee, O. Chapelle, M. Pontil, T. Poggio, and V. Vapnik. Feature selection for svms. In Neural Information Processing Systems 14, 2001.
29. F. Wolinski, F. Vichot, and M. Stricker. Using learning-based filters to detect rule-based filtering obsolescence. In Proceedings RIAO'2000, 2000.