A penalized criterion for variable selection in classification

Journal of Multivariate Analysis

Volumn 98, Issue 4, 2007, Pages 695-705

  Mary Huard, Tristan ^a   Robin, Stéphane ^a   Daudin, Jean Jacques ^a  

^a AGROPARISTECH   (France)

Author keywords

Oracle inequality; Penalized criterion; Statistical learning; Variable selection

Indexed keywords

EID: 33846902129     PISSN: 0047259X     EISSN: 10957243     Source Type: Journal    
DOI: 10.1016/j.jmva.2006.06.003     Document Type: Article

References (34)

1. P.L. Bartlett, S. Boucheron, G. Lugosi, Model selection and error estimation, Technical Report 508, Department of Economics and Business, Universitat Pompeu Fabra, October 2000.
2. Birgé L., and Massart P. Gaussian model selection. J. European Math. Soc. 3 (2001) 203-268
3. p criterion for Gaussian model selection, Technical Report, Publication No. 647, Universités de Paris 6 & Paris 7, 2001.
4. L. Birgé, Y. Rozenholc, How many bins should be put in a regular histogram, Technical Report, Publication No. 721, Université Paris 6 & Paris 7, 2002.
5. G. Blanchard, O. Bousquet, P. Massart, Statistical performance of support vector machines, submitted for publication.
6. Boser B., Guyon I., and Vapnik V. A training algorithm for optimal margin classifier. Fifth Annual Workshop on Computational Learning Theory (1992), ACM, New York 144-152
7. Brown M., Grundy W., Lin D., Cristianini N., Sugnet C., Furey T., Jr M., and Haussler D. Knowledge-based analysis of microarray gene expression data by using support vector machines. Proc. Nat. Acad. Sci. U.S.A. 97 (2000) 262-267
8. Castellan G. Modified akaike's criterion for histogram density estimation (1999), Prépublication d'Orsay
9. Costanza C.M., and Afifi A.A. Comparison of stopping rules in forward stepwise discriminant analysis. J. Amer. Statist. Assoc. 74 (1979) 777-785
10. DeRisi J., Penland L., Brown P.O., Bittner M.L., Meltzer P.S., Ray M., Chen Y., Su Y.A., and Trent J.M. Use of a cdna microarray to analyse gene expression patterns in human cancer. Nat. Genet. 14 (1996) 457
11. Devroye L., Gyorfi L., and Lugosi G. A Probabilistic Theory of Pattern Recognition (1996), Springer, Berlin
12. Fisher R.A. The use of multiple measurements in taxinomic problems. Ann. Eugenics 7 (1936) 179-188
13. Fukumizu K., Bach F.R., and Jordan M.I. Dimensionality reduction for supervised learning with reproducing kernel hilbert spaces. J. Machine Learning Res. 5 (2004) 73-99
14. S. Gey, E. Lebarbier, A cart based algorithm for detection of multiple change-points in the mean for large samples, Technical Report, Publication Université Paris-VI 10, 2002.
15. Golub T.R., Slonim D.K., Tamayo P., Huard C., Gaasenbeek M., Mesirov J.P., Coller H., Loh M.L., Downing J.R., Caligiuri M.A., Bloomfield C.D., and Lander E.S. Class prediction and discovery using gene expression data. Science 286 (1999) 531-537
16. Guyon I., and Elisseeff A. An introduction to variable and feature selection. J. Machine Learning Res. 3 (2003) 1157-1182
17. Guyon I., Weston J., Barnhill S., and Vapnik V. Gene selection for cancer classification using support vector machines. Machine Learning 46 (2002) 389-422
18. Hand D.J. Discrimination and Classification, Wiley Series in Probability and Mathematical Statistics (1981), Wiley, Chichester
19. Kohavi R., and John G. Wrappers for feature subset selection. Artificial Intelligence 97 (1997) 273-324
20. Krishnapuram B., Carin L., and Hartemink A. Gene expression analysis: joint feature selection and classifier design, Kernel Methods in Computational Biology (2004), MIT Press, Cambridge, MA
21. Krishnapuram B., Hartemink A.J., Carin L., and Figueiredo M.A.T. A bayesian approach to joint feature selection and classifier design. IEEE Trans. Pattern Anal. Machine Intell. 26 (2004) 1105-1111
22. E. Lebarbier, Quelques approaches pour la détection de ruptures à horizon fini, Ph.D. Thesis, Université Paris XI, 2002.
23. Lugosi G., and Zeger K. Concept learning using complexity regularization. IEEE Trans. Inform. Theory 42 1 (1995) 48-54
24. Massart P. Some applications of concentration inequalities to statistics. Ann. Fac. Sci. Toulouse IX (2000) 245-303
25. McHenry C.E. Computation of a best subset in multivariate analysis 27 3 (1978) 291-296
26. Rakotomamonjy A. Variable selection using SVM-based criteria. J. Machine Learning Res. 3 (2003) 1357-1370
27. Rao C.R. Linear Statistical Inference and its Applications (1965), Wiley, New York
28. Rao C.R. Inference on discriminant function coefficients, Essays in Probability and Statistics (1970), The University of North Carolina Press, Chapell Hill
29. Schölkopf B., and Smola A.J. Learning with Kernels, Support Vector Machines, Regularization, Optimization and Beyond (2002), MIT Press, Cambridge, MA
30. Seber G.A.F. Multivariate Observations (1984), Wiley, New York
31. Vapnik V. Statistical Learning Theory (1998), Wiley, New York
32. Weston J., Mukherjee S., Chapelle O., Pontil M., Poggio T., and Vapnik V. Feature selection for SVMs, Advances in Neural Information Processing Systems (2000), MIT Press, Cambridge, MA
33. Xiong M., Li W., Zhao J., Jin L., and Boerwinkle E. Feature (gene) selection in gene expression-based tumor classification. Mol. Genet. Metab. 73 3 (2001) 239-247
34. Zhu J., and Hastie T. Classification of gene microarrays by penalyzed logistic regression. Biostatistics 3 (2003) 427-443