Confidence intervals for random forests: The jackknife and the infinitesimal jackknife

Journal of Machine Learning Research

Volumn 15, Issue , 2014, Pages 1625-1651

  Wager, Stefan ^a   Hastie, Trevor ^a   Efron, Bradley ^a  

^a STANFORD UNIVERSITY   (United States)

Author keywords

Bagging; Jackknife methods; Monte Carlo noise; Variance estimation

Indexed keywords

DECISION TREES; MONTE CARLO METHODS;

BAGGING; BOOTSTRAP REPLICATES; CONFIDENCE INTERVAL; JACK-KNIFE METHOD; SAMPLING DISTRIBUTION; SIMULATION STUDIES; VARIANCE ESTIMATE; VARIANCE ESTIMATION;

STATISTICAL METHODS;

EID: 84902788102     PISSN: 15324435     EISSN: 15337928     Source Type: Journal    
DOI: None     Document Type: Article

References (28)

1. Kevin Bache and Moshe Lichman. UCI machine learning repository, 2013. URL http://archive.ics.uci.edu/ml.
2. Gérard Biau. Analysis of a random forests model. The Journal of Machine Learning Research, 13(4):1063-1095, 2012.
3. Gérard Biau, Luc Devroye, and Gábor Lugosi. Consistency of random forests and other averaging classifiers. The Journal of Machine Learning Research, 9:2015-2033, 2008.
4. Leo Breiman. Bagging predictors. Machine Learning, 24(2):123-140, 1996. (Pubitemid 126724382)
5. Leo Breiman. Random forests. Machine Learning, 45(1):5-32, 2001. (Pubitemid 32933532)
6. Peter Bühlmann and Bin Yu. Analyzing bagging. The Annals of Statistics, 30(4):927-961, 2002. (Pubitemid 37095335)
7. Andreas Buja and Werner Stuetzle. Observations on bagging. Statistica Sinica, 16(2):323, 2006.
8. Song Xi Chen and Peter Hall. Effects of bagging and bias correction on estimators defined by estimating equations. Statistica Sinica, 13(1):97-110, 2003.
9. Thomas G Dietterich. An experimental comparison of three methods for constructing ensembles of decision trees: Bagging, boosting, and randomization. Machine Learning, 40 (2):139-157, 2000.
10. Jiangtao Duan. Bootstrap-Based Variance Estimators for a Bagging Predictor. PhD thesis, North Carolina State University, 2011.
11. Bradley Efron. Jackknife-after-bootstrap standard errors and influence functions. Journal of the Royal Statistical Society. Series B (Methodological), pages 83-127, 1992.
12. Bradley Efron. Estimation and accuracy after model selection. Journal of the American Statistical Association, (just-accepted), 2013.
13. Bradley Efron and David Feldman. Compliance as an explanatory variable in clinical trials. Journal of the American Statistical Association, 86(413):9-17, 1991.
14. Bradley Efron and Charles Stein. The jackknife estimate of variance. The Annals of Statistics, pages 586-596, 1981.
15. Jerome H Friedman. Stochastic gradient boosting. Computational Statistics & Data Analysis, 38(4):367-378, 2002. (Pubitemid 34197167)
16. Jerome H Friedman and Peter Hall. On bagging and nonlinear estimation. Journal of Statistical Planning and Inference, 137(3):669-683, 2007. (Pubitemid 44660748)
17. Pierre Geurts, Damien Ernst, and Louis Wehenkel. Extremely randomized trees. Machine Learning, 63(1):3-42, 2006.
18. Trevor Hastie, Robert Tibshirani, and Jerome Friedman. The Elements of Statistical Learning. New York: Springer, 2009.
19. Louis A Jaeckel. The Infinitesimal Jackknife. 1972.
20. Andy Liaw and Matthew Wiener. Classification and regression by random Forest. R News, 2(3):18-22, 2002. URL http://CRAN.R-project.org/doc/Rnews/.
21. Yi Lin and Yongho Jeon. Random forests and adaptive nearest neighbors. Journal of the American Statistical Association, 101(474):578-590, 2006. (Pubitemid 43972300)
22. Colin L Mallows. Some comments on Cp. Technometrics, 15(4):661-675, 1973.
23. Nicolai Meinshausen. Quantile regression forests. The Journal of Machine Learning Research, 7:983-999, 2006. (Pubitemid 43902307)
24. Joseph Sexton and Petter Laake. Standard errors for bagged and random forest estimators. Computational Statistics & Data Analysis, 53(3):801-811, 2009.
25. Marina Skurichina and Robert PW Duin. Bagging for linear classifiers. Pattern Recognition, 31(7):909-930, 1998. (Pubitemid 128428102)
26. Thomas A Stamey, John N Kabalin, John E McNeal, Iain M Johnstone, Fuad Freiha, EA Redwine, and N Yang. Prostate specific antigen in the diagnosis and treatment of adenocarcinoma of the prostate. II. radical prostatectomy treated patients. The Journal of Urology, 141(5):1076-1083, 1989. (Pubitemid 19124201)
27. Carolin Strobl, Anne-Laure Boulesteix, Achim Zeileis, and Torsten Hothorn. Bias in random forest variable importance measures: Illustrations, sources and a solution. BMC Bioinformatics, 8(1):25, 2007.
28. William N Venables and Brian D Ripley. Modern Applied Statistics with S. Springer, New York, fourth edition, 2002. URL http://www.stats.ox.ac.uk/pub/ MASS4. ISBN 0-387-95457-0.