Consistency of random forests

Annals of Statistics

Volumn 43, Issue 4, 2015, Pages 1716-1741

  Scornet, Erwan ^a   Biau, Gerard ^a   Vert, Jean Philippe ^b,c,d  

^a SORBONNE UNIVERSITÉ   (France)

^b PSL RESEARCH UNIVERSITY   (France)

^c INSTITUT CURIE   (France)

^d INSERM   (France)

Author keywords

Additive model; Consistency; Dimension reduction; Random forests; Randomization; Sparsity

Indexed keywords

EID: 84933565370     PISSN: 00905364     EISSN: 21688966     Source Type: Journal    
DOI: 10.1214/15-AOS1321     Document Type: Article

References (41)

1. AMARATUNGA, D., CABRERA, J. and LEE, Y.-S. (2008). Enriched random forests. Bioinformatics 24 2010-2014.
2. BAI, Z.-D., DEVROYE, L., HWANG, H.-K. and TSAI, T.-H. (2005). Maxima in hypercubes. Random Structures Algorithms 27 290-309. MR2162600
3. BARNDORFF-NIELSEN, O. and SOBEL, M. (1966). On the distribution of the number of admissible points in a vector random sample. Teor. Verojatnost. i Primenen. 11 283-305. MR0207003
4. BIAU, G. (2012). Analysis of a random forests model. J. Mach. Learn. Res. 13 1063-1095. MR2930634
5. BIAU, G. and DEVROYE, L. (2010). On the layered nearest neighbour estimate, the bagged nearest neighbour estimate and the random forest method in regression and classification. J. Multivariate Anal. 101 2499-2518. MR2719877
6. BIAU, G., DEVROYE, L. and LUGOSI, G. (2008). Consistency of random forests and other averaging classifiers. J. Mach. Learn. Res. 9 2015-2033. MR2447310
7. BOUCHERON, S., LUGOSI, G. and MASSART, P. (2013). Concentration Inequalities: A Nonasymptotic Theory of Independence. Oxford Univ. Press, Oxford. MR3185193
8. BREIMAN, L. (1996). Bagging predictors. Mach. Learn. 24 123-140.
9. BREIMAN, L. (2001). Random forests. Mach. Learn. 45 5-32.
10. BREIMAN, L. (2004). Consistency for a simple model of random forests. Technical Report 670, Univ. California, Berkeley, CA.
11. BREIMAN, L., FRIEDMAN, J. H., OLSHEN, R. A. and STONE, C. J. (1984). Classification and Regression Trees. Wadsworth Advanced Books and Software, Belmont, CA. MR0726392
12. BÜHLMANN, P. and YU, B. (2002). Analyzing bagging. Ann. Statist. 30 927-961. MR1926165
13. CLÉMENÇON, S., DEPECKER, M. and VAYATIS, N. (2013). Ranking forests. J. Mach. Learn. Res. 14 39-73. MR3033325
14. CUTLER, D. R., EDWARDS, T. C. JR, BEARD, K. H., CUTLER, A., HESS, K. T., GIBSON, J. and LAWLER, J. J. (2007). Random forests for classification in ecology. Ecology 88 2783-2792.
15. DENIL, M., MATHESON, D. and FREITAS, N. D. (2013). Consistency of online random forests. In Proceedings of the ICML Conference. Available at arXiv:1302.4853.
16. DEVROYE, L., GYÖRFI, L. and LUGOSI, G. (1996). A Probabilistic Theory of Pattern Recognition. Applications of Mathematics (New York) 31. Springer, New York. MR1383093
17. DÍAZ-URIARTE, R. and ALVAREZ DE ANDRÉS, S. (2006). Gene selection and classification of microarray data using random forest. BMC Bioinformatics 7 1-13.
18. EFRON, B. (1982). The Jackknife, the Bootstrap and Other Resampling Plans. CBMS-NSF Regional Conference Series in Applied Mathematics 38. SIAM, Philadelphia. MR0659849
19. GENUER, R. (2012). Variance reduction in purely random forests. J. Nonparametr. Stat. 24 543-562. MR2968888
20. GEURTS, P., ERNST, D. and WEHENKEL, L. (2006). Extremely randomized trees. Mach. Learn. 63 3-42.
21. GYÖRFI, L., KOHLER, M., KRZYŻAK, A. and WALK, H. (2002). A Distribution-Free Theory of Nonparametric Regression. Springer, New York. MR1920390
22. HASTIE, T. and TIBSHIRANI, R. (1986). Generalized additive models. Statist. Sci. 1 297-318. MR0858512
23. HASTIE, T., TIBSHIRANI, R. and FRIEDMAN, J. (2009). The Elements of Statistical Learning: Data Mining, Inference, and Prediction, 2nd ed. Springer, New York. MR2722294
24. ISHWARAN, H. and KOGALUR, U. B. (2010). Consistency of random survival forests. Statist. Probab. Lett. 80 1056-1064. MR2651045
25. ISHWARAN, H., KOGALUR, U. B., BLACKSTONE, E. H. and LAUER, M. S. (2008). Random survival forests. Ann. Appl. Stat. 2 841-860. MR2516796
26. KLEINER, A., TALWALKAR, A., SARKAR, P. and JORDAN, M. I. (2014). A scalable bootstrap for massive data. J. R. Stat. Soc. Ser. B. Stat. Methodol. 76 795-816. MR3248677
27. LIN, Y. and JEON, Y. (2006). Random forests and adaptive nearest neighbors. J. Amer. Statist. Assoc. 101 578-590. MR2256176
28. MEINSHAUSEN, N. (2006). Quantile regression forests. J. Mach. Learn. Res. 7 983-999. MR2274394
29. MENTCH, L. and HOOKER, G. (2014). Ensemble trees and clts: Statistical inference for supervised learning. Available at arXiv:1404.6473.
30. NOBEL, A. (1996). Histogram regression estimation using data-dependent partitions. Ann. Statist. 24 1084-1105. MR1401839
31. POLITIS, D. N., ROMANO, J. P. and WOLF, M. (1999). Subsampling. Springer, New York. MR1707286
32. PRASAD, A. M., IVERSON, L. R. and LIAW, A. (2006). Newer classification and regression tree techniques: Bagging and random forests for ecological prediction. Ecosystems 9 181-199.
33. SCORNET, E. (2014). On the asymptotics of random forests. Available at arXiv:1409.2090.
34. SCORNET, E., BIAU, G. and VERT, J. (2015). Supplement to "Consistency of random forests." DOI:10.1214/15-AOS1321SUPP.
35. SHOTTON, J., SHARP, T., KIPMAN, A., FITZGIBBON, A., FINOCCHIO, M., BLAKE, A., COOK, M. and MOORE, R. (2013). Real-time human pose recognition in parts from single depth images. Comm. ACM 56 116-124.
36. STONE, C. J. (1977). Consistent nonparametric regression. Ann. Statist. 5 595-645. MR0443204
37. STONE, C. J. (1985). Additive regression and other nonparametric models. Ann. Statist. 13 689-705. MR0790566
38. SVETNIK, V., LIAW, A., TONG, C., CULBERSON, J. C., SHERIDAN, R. P. and FEUSTON, B. P. (2003). Random forest: A classification and regression tool for compound classification and QSAR modeling. J. Chem. Inf. Comput. Sci. 43 1947-1958.
39. WAGER, S. (2014). Asymptotic theory for random forests. Available at arXiv:1405.0352.
40. WAGER, S., HASTIE, T. and EFRON, B. (2014). Confidence intervals for random forests: The jackknife and the infinitesimal jackknife. J. Mach. Learn. Res. 15 1625-1651. MR3225243
41. ZHU, R., ZENG, D. and KOSOROK, M. R. (2012). Reinforcement learning trees. Technical report, Univ. North Carolina.