Analysis of a random forests model

Journal of Machine Learning Research

Volumn 13, Issue , 2012, Pages 1063-1095

  Biau, Gérard ^a  

^a SORBONNE UNIVERSITÉ   (France)

Author keywords

Consistency; Dimension reduction; Random forests; Randomization; Rate of convergence; Sparsity

Indexed keywords

CONSISTENCY; DIMENSION REDUCTION; RANDOM FORESTS; RANDOMIZATION; RATE OF CONVERGENCE; SPARSITY;

ALGORITHMS; APPROXIMATION THEORY; FORESTRY;

DECISION TREES;

ALGORITHMS; DATA PROCESSING; DIMENSIONS; PROCEDURES; RANDOM PROCESSES;

EID: 84860701629     PISSN: 15324435     EISSN: 15337928     Source Type: Journal    
DOI: None     Document Type: Review

References (38)

1. D. Amaratunga, J. Cabrera, and Y.S. Lee. Enriched random forests. Bioinformatics, 24:2010-2014, 2008.
2. Y. Amit. 2D Object Detection and Recognition: Models, Algorithms, and Networks. The MIT Press, Cambridge, 2002.
3. Y. Amit and D. Geman. Shape quantization and recognition with randomized trees. Neural Computation, 9:1545-1588, 1997. (Pubitemid 127462804)
4. G. Biau and L. Devroye. On the layered nearest neighbour estimate, the bagged nearest neighbour estimate and the random forest method in regression and classification. Journal of Multivariate Analysis, 101:2499-2518, 2010.
5. G. Biau, L. Devroye, and G. Lugosi. Consistency of random forests and other averaging classifiers. Journal of Machine Learning Research, 9:2015-2033, 2008.
6. G. Biau, F. Cérou, and A. Guyader. On the rate of convergence of the bagged nearest neighbor estimate. Journal of Machine Learning Research, 11:687-712, 2010.
7. P. Bickel, Y. Ritov, and A. Tsybakov. Simultaneous analysis of Lasso and Dantzig selector. The Annals of Statistics, 37:1705-1732, 2009.
8. G. Blanchard. Different paradigms for choosing sequential reweighting algorithms. Neural Computation, 16:811-836, 2004. (Pubitemid 38318133)
9. L. Breiman. Bagging predictors. Machine Learning, 24:123-140, 1996. (Pubitemid 126724382)
10. L. Breiman. Some Infinity Theory for Predictor Ensembles. Technical Report 577, UC Berkeley, 2000. URL http://www.stat.berkeley.edu/breiman.
11. L. Breiman. Random forests. Machine Learning, 45:5-32, 2001. (Pubitemid 32933532)
12. L. Breiman. Consistency For a Simple Model of Random Forests. Technical Report 670, UC Berkeley, 2004. URL http://www.stat.berkeley.edu/breiman.
13. L. Breiman, J.H. Friedman, R.A. Olshen, and C.J. Stone. Classification and Regression Trees. Chapman & Hall, New York, 1984.
14. A.M. Bruckstein, D.L. Donoho, and M. Elad. From sparse solutions of systems of equations to sparse modeling of signals and images. SIAM Review, 51:34-81, 2009.
15. P. Bühlmann and B. Yu. Analyzing bagging. The Annals of Statistics, 30:927-961, 2002. (Pubitemid 37095335)
16. A. Buja and W. Stuetzle. Observations on bagging. Statistica Sinica, 16:323-352, 2006.
17. F. Bunea, A. Tsybakov, and M. Wegkamp. Sparsity oracle inequalities for the Lasso. Electronic Journal of Statistics, 1:169-194, 2007.
18. E.J. Candès and T. Tao. The Dantzig selector: Statistical estimation when p is much larger than n. The Annals of Statistics, 35:2313-2351, 2005.
19. L. Devroye, L. Györfi, and G. Lugosi. A Probabilistic Theory of Pattern Recognition. Springer- Verlag, New York, 1996.
20. R. Diaz-Uriarte and S.A. de Andrés. Gene selection and classification of microarray data using random forest. BMC Bioinformatics, 7:1471-2105, 2006.
21. T.G. Dietterich. An experimental comparison of three methods for constructing ensembles of decision trees: Bagging, boosting, and randomization. Machine Learning, 40:139-157, 2000.
22. Y. Freund and R. Shapire. Experiments with a new boosting algorithm. In L. Saitta, editor, Machine Learning: Proceedings of the 13th International Conference, pages 148-156, San Francisco, 1996. Morgan Kaufmann.
23. J.H. Friedman. Multivariate adaptive regression splines. The Annals of Statistics, 19:1-67, 1991.
24. R. Genuer, J.-M. Poggi, and C. Tuleau. Random Forests: Some Methodological Insights. arXiv:0811.3619, 2008.
25. R. Genuer, J.-M. Poggi, and C. Tuleau-Malot. Variable selection using random forests. Pattern Recognition Letters, 31:2225-2236, 2010.
26. L. Gyrfi, M. Kohler, A. Krzyžak, and H. Walk. A Distribution-Free Theory of Nonparametric Regression. Springer-Verlag, New York, 2002.
27. T.K. Ho. The random subspace method for constructing decision forests. IEEE Transactions on Pattern Analysis and Machine Intelligence, 20:832-844, 1998. (Pubitemid 128741345)
28. I.A. Ibragimov and R.Z. Khasminskii. On nonparametric estimation of regression. Doklady Akademii Nauk SSSR, 252:780-784, 1980.
29. I.A. Ibragimov and R.Z. Khasminskii. Statistical Estimation: Asymptotic Theory. Springer-Verlag, New York, 1981.
30. I.A. Ibragimov and R.Z. Khasminskii. On the bounds for quality of nonparametric regression function estimation. Theory of Probability and its Applications, 27:81-94, 1982.
31. A.N. Kolmogorov and V.M. Tihomirov. e-entropy and e-capacity of sets in functional spaces. American Mathematical Society Translations, 17:277-364, 1961.
32. Y. Lin and Y. Jeon. Random forests and adaptive nearest neighbors. Journal of the American Statistical Association, 101:578-590, 2006. (Pubitemid 43972300)
33. N. Meinshausen. Quantile regression forests. Journal of Machine Learning Research, 7:983-999, 2006. (Pubitemid 43902307)
34. W. Rudin. Real and Complex Analysis, 3rd Edition. McGraw-Hill, New York, 1987.
35. J. Shawe-Taylor and N. Cristianini. Kernel Methods for Pattern Analysis. Cambridge University Press, Cambridge, 2004.
36. V. Svetnik, A. Liaw, C. Tong, J. Culberson, R. Sheridan, and B. Feuston. Random forest: A classification and regression tool for compound classification and QSAR modeling. Journal of Chemical Information and Computer Sciences, 43:1947-1958, 2003.
37. R. Tibshirani. Regression shrinkage and selection via the Lasso. Journal of the Royal Statistical Society, Series B, 58:267-288, 1996.
38. L.J. van't Veer, H. Dai, M.J. van de Vijver, Y.D. He, A.A.M. Hart, M. Mao, H.L. Peterse, K. van der Kooy, M.J. Marton, A.T. Witteveen, G.J. Schreiber, R.M. Kerkhoven, C. Roberts, P.S. Linsley, R. Bernards, and S.H. Friend. Gene expression profiling predicts clinical outcome of breast cancer. Nature, 415:530-536, 2002. (Pubitemid 34130608)