On the layered nearest neighbour estimate, the bagged nearest neighbour estimate and the random forest method in regression and classification

Journal of Multivariate Analysis

Volumn 101, Issue 10, 2010, Pages 2499-2518

  Biau, Gérard ^a,b   Devroye, Luc ^c  

^a SORBONNE UNIVERSITÉ   (France)

^b ECOLE NORMALE SUPÉRIEURE   (France)

^c MCGILL UNIVERSITY   (Canada)

Author keywords

Bagging; Layered nearest neighbours; One nearest neighbour estimate; Random forests; Regression estimation

Indexed keywords

EID: 77956747417     PISSN: 0047259X     EISSN: 10957243     Source Type: Journal    
DOI: 10.1016/j.jmva.2010.06.019     Document Type: Article

References (24)

1. Amit Y., Geman D. Shape quantization and recognition with randomized trees. Neural Computation 1997, 9:1545-1588.
2. Bai Z.-D., Chao C.-C., Hwang H.-K., Liang W.-Q. On the variance of the number of maxima in random vectors and its applications. The Annals of Applied Probability 1998, 8:886-895.
3. Bai Z.-D., Devroye L., Hwang H.-K., Tsai T.-S. Maxima in hypercubes. Random Structures and Algorithms 2005, 27:290-309.
4. Barndorff-Nielsen O., Sobel M. On the distribution of admissible points in a vector random sample. Theory of Probability and its Applications 1966, 11:249-269.
5. Biau G., Devroye L., Lugosi G. Consistency of random forests and other averaging classifiers. Journal of Machine Learning Research 2008, 9:2015-2033.
6. Breiman L. Bagging predictors. Machine Learning 1996, 24:123-140.
7. L. Breiman, Some infinite theory for predictor ensembles, Technical Report 577, Statistics Department, UC Berkeley, 2000. http://www.stat.berkeley.edu/~breiman.
8. Breiman L. Random forests. Machine Learning 2001, 45:5-32.
9. L. Breiman, Consistency for a simple model of random forests, Technical Report 670, Statistics Department, UC Berkeley, 2004. http://www.stat.berkeley.edu/~breiman.
10. Cover T.M., Hart P.E. Nearest neighbour pattern classification. IEEE Transactions on Information Theory 1967, 13:21-27.
11. Cutler A., Zhao G. Pert-Perfect random tree ensembles. Computing Science and Statistics 2001, 33:490-497.
12. Devroye L. The uniform convergence of nearest neighbour regression function estimators and their application in optimization. IEEE Transactions on Information Theory 1978, 24:142-151.
13. Devroye L., Györfi L., Lugosi G. A Probabilistic Theory of Pattern Recognition 1996, Springer-Verlag, New York.
14. Dietterich T.G. An experimental comparison of three methods for constructing ensembles of decision trees: Bagging, boosting, and randomization. Machine Learning 2000, 40:139-157.
15. Doeblin W. Exposé de la théorie des chaǐnes simples constantes de Markov à un nombre fini d'états. Revue Mathématique de l'Union Interbalkanique 1937, 2:77-105.
16. E. Fix, J. Hodges, Discriminatory analysis. Nonparametric discrimination: Consistency properties, Technical Report 4, Project Number 21-49-004, USAF School of Aviation Medicine, Randolph Field, Texas, 1951.
17. Györfi L., Kohler M., Krzyzak A., Walk H. A Distribution-Free Theory of Nonparametric Regression 2002, Springer-Verlag, New York.
18. Lin Y., Jeon Y. Random forests and adaptive nearest neighbors. Journal of the American Statistical Association 2006, 101:578-590.
19. Marcinkiewicz J., Zygmund A. Sur les fonctions indépendantes. Fundamenta Mathematicae 1937, 29:60-90.
20. Petrov V.V. Sums of Independent Random Variables 1975, Springer-Verlag, Berlin.
21. Rachev S.T., Rüschendorf L. Mass Transportation Problems, Volume I: Theory 1998, Springer, New York.
22. Steele B.M. Exact bootstrap k-nearest neighbor learners. Machine Learning 2009, 74:235-255.
23. Stone C.J. Consistent nonparametric regression. The Annals of Statistics 1977, 5:595-645.
24. Wheeden R.L., Zygmund A. Measure and Integral. An Introduction to Real Analysis 1977, Marcel Dekker, New York.