Variable selection using random forests

Pattern Recognition Letters

Volumn 31, Issue 14, 2010, Pages 2225-2236

  Genuer, Robin ^a   Poggi, Jean Michel ^a,b   Tuleau Malot, Christine ^c  

^a UNIVERSITÉ PARIS SUD   (France)

^b UNIVERSITÉ PARIS DESCARTES   (France)

^c UNIVERSITÉ DE NICE SOPHIA ANTIPOLIS   (France)

Author keywords

Classification; High dimensional data; Random forests; Regression; Variable importance; Variable selection

Indexed keywords

CLASSIFICATION; HIGH DIMENSIONAL DATA; RANDOM FORESTS; REGRESSION; VARIABLE IMPORTANCE; VARIABLE SELECTION;

CLASSIFICATION (OF INFORMATION); CLUSTERING ALGORITHMS; MATHEMATICAL MODELS; REGRESSION ANALYSIS;

DECISION TREES;

EID: 77957922514     PISSN: 01678655     EISSN: None     Source Type: Journal    
DOI: 10.1016/j.patrec.2010.03.014     Document Type: Article

References (38)

1. A.A. Alizadeh Distinct types of diffuse large b-cell lymphoma identified by gene expression profiling Nature 403 2000 503 511
2. U. Alon, N. Barkai, D.A. Notterman, K. Gish, S. Ybarra, D. Mack, and A.J. Levine Broad patterns of gene expression revealed by clustering analysis of tumor and normal colon tissues probed by oligonucleotide arrays Proc. Natl. Acad. Sci. USA 96 12 1999 6745 6750
3. K.J. Archer, and R.V. Kimes Empirical characterization of random forest variable importance measures Comput. Statist. Data Anal. 52 2008 2249 2260
4. A. Ben Ishak, and B. Ghattas Sélection de variables pour la classification binaire en grande dimension: comparaisons et application aux données de biopuces J. de la SFdS 149 3 2008 43 66
5. G. Biau, L. Devroye, and G. Lugosi Consistency of random forests and other averaging classifiers J. Machine Learn. Res. 9 2008 2039 2057
6. L. Breiman Bagging predictors Machine Learn. 26 2 1996 123 140
7. L. Breiman Random Forests Machine Learn. 45 2001 5 32
8. Breiman, L., 2004. Consistency for a simple model of random forests. Technical Report 670, Berkeley.
9. Breiman, L., Cutler, A., 2005. Random forests, Berkeley. Available from: http://www.stat.berkeley.edu/users/breiman/RandomForests/.
10. L. Breiman, J.H. Friedman, R.A. Olshen, and C.J. Stone Classification And Regression Trees 1984 Chapman & Hall New York
11. P. Bühlmann, and B. Yu Analyzing bagging The Ann. Statist. 30 4 2002 927 961
12. N. Cheze, J.M. Poggi, and B. Portier Partial and recombined estimators for nonlinear additive models Statist. Inference Stoch. Process. 6 2 2003 155 197
13. A. Cutler, and G. Zhao Pert - perfect random tree ensembles Comput. Sci. Statist. 33 2001 490 497
14. R. Díaz-Uriarte, and S. Alvarez de Andrés Gene selection and classification of microarray data using random forest BMC Bioinf. 7 2006 3
15. T. Dietterich An experimental comparison of three methods for constructing ensembles of decision trees: Bagging, boosting and randomization Machine Lear. 40 1999 139 157
16. T. Dietterich Ensemble methods in machine learning Lecture Notes Comput. Sci. 1857 2000 1 15
17. B. Efron, T. Hastie, I. Johnstone, and R. Tibshirani Least angle regression Ann. Statist. 32 2 2004 407 499
18. J. Fan, and R. Li Variable selection via nonconcave penalized likelihood and its oracle properties J. Amer. Statist. Assoc. 96 2001 1348 1359
19. J. Fan, and J. Lv Sure independence screening for ultra-high dimensional feature space J. Roy. Statist. Soc. Ser. B. 70 2008 849 911
20. Genuer, R., Poggi, J.-M., Tuleau, C., 2008. Random Forests: Some methodological insights. Research report INRIA Saclay, RR-6729. Available from: http://hal.inria.fr/inria-00340725/fr/.
21. Genuer, R., Michel, V., Eger, E., Thirion, B., 2010. Random forests based feature selection for decoding fMRI data. In: Proceedings Compstat 2010, August 22-27, Paris, France. Paper 267, pp. 1-8.
22. T.R. Golub, D.K. Slonim, P. Tamayo, C. Huard, M. Gaasenbeek, J.P. Mesirov, H. Coller, M.L. Loh, J.R. Downing, M.A. Caligiuri, C.D. Bloomfield, and E.S. Lander Molecular classification of cancer: Class discovery and class prediction by gene expression monitoring Science 286 1999 531 537
23. U. Grömping Estimators of relative importance in linear regression based on variance decomposition The Am. Statistician 61 2007 139 147
24. U. Grömping Variable importance assessment in regression: Linear regression versus random forest The Am. Statistician 63 4 2009 308 319
25. I. Guyon, and A. Elisseff An introduction to variable and feature selection J. Machine Learn. Res. 3 2003 1157 1182
26. I. Guyon, J. Weston, S. Barnhill, and V.N. Vapnik Gene selection for cancer classification using support vector machines Machine Learn. 46 1-3 2002 389 422
27. T. Hastie, R. Tibshirani, and J. Friedman The Elements of Statistical Learning 2001 Springer Berlin, Heidelberg, New York
28. Jolliffe, I.T., 2002. Principal Component Analysis. Springer series in statistics.
29. R. Kohavi, and G.H. John Wrappers for feature subset selection Artif. Intell. 97 1-2 1997 273 324
30. A. Liaw, and M. Wiener Classification and regression by random forest R News 2 3 2002 18 22
31. M.Y. Park, and T. Hastie An L1 regularization-path algorithm for generalized linear models J. Roy. Statist. Soc. Ser. B. 69 2007 659 677
32. J.M. Poggi, and C. Tuleau Classification supervisée en grande dimension Application l'agrément de conduite automobile. Revue de Statistique Appliquée. LIV 4 2006 39 58
33. A. Rakotomamonjy Variable selection using SVM-based criteria J. Machine Learn. Res. 3 2003 1357 1370
34. D. Singh, P.G. Febbo, K. Ross, D.G. Jackson, J. Manola, C. Ladd, P. Tamayo, A.A. Renshaw, A.V. D'Amico, J.P. Richie, E.S. Lander, M. Loda, P.W. Kantoff, T.R. Golub, and W.R. Sellers Gene expression correlates of clinical prostate cancer behavior Cancer Cell 1 2002 203 209
35. I.M. Sobol' Sensitivity analysis for nonlinear mathematical models Math. Model. Comput. Exp. 1 4 1993 407 414
36. C. Strobl, A.-L. Boulesteix, A. Zeileis, and T. Hothorn Bias in random forest variable importance measures: Illustrations, sources and a solution BMC Bioinf. 8 2007 25
37. C. Strobl, A.-L. Boulesteix, T. Kneib, T. Augustin, and A. Zeileis Conditional variable importance for random forests BMC Bioinf. 9 2008 307
38. J. Weston, A. Elisseff, B. Schoelkopf, and M. Tipping Use of the zero norm with linear models and Kernel methods J. Machine Learn. Res. 3 2003 1439 1461