On the rate of convergence of the bagged nearest neighbor estimate

Journal of Machine Learning Research

Volumn 11, Issue , 2010, Pages 687-712

  Biau, Gérard ^a   Cérou, Frédéric ^b   Guyader, Arnaud ^c  

^a SORBONNE UNIVERSITÉ   (France)

^b INRIA   (France)

^c UNIVERSITÉ RENNES 2   (France)

Author keywords

Bagging; Nearest neighbor estimate; Rates of convergence; Resampling

Indexed keywords

DATA SETS; NEAREST NEIGHBORS; OPTIMAL RATE OF CONVERGENCE; RATE OF CONVERGENCE; RATES OF CONVERGENCE; REGRESSION ESTIMATES; RESAMPLING; STATISTICAL ANALYSIS;

APPROXIMATION THEORY;

EID: 77949521444     PISSN: 15324435     EISSN: 15337928     Source Type: Journal    
DOI: None     Document Type: Article

References (28)

1. G. Biau and L. Devroye. On the Layered Nearest Neighbour Estimate, the Bagged Nearest Neighbour Estimate and the Random Forest Method in Regression and Classification. Technical Report, Université Pierre et Marie Curie, Paris, 2008. URL http://www.lsta.upmc.fr/BIAU/bd4.pdf.
2. L. Breiman. Random forests. Machine Learning, 45:5-32, 2001.
3. L. Breiman. Bagging predictors. Machine Learning, 24:123-140, 1996.
4. P. Bühlmann and B. Yu. Analyzing bagging. The Annals of Statistics, 30:927-961, 2002.
5. A. Buja and W. Stuetzle. Observations on bagging. Statistica Sinica, 16:323-352, 2006.
6. T.M. Cover. Estimation by the nearest neighbor rule. IEEE Transactions on Information Theory, 14:50-55, 1968a.
7. T.M. Cover. Rates of convergence for nearest neighbor procedures. In Proceedings of the Hawaii International Conference on Systems Sciences, pages 413-415, Honolulu, 1968b.
8. T.M. Cover and P.E. Hart. Nearest neighbor pattern classification. IEEE Transactions on Information Theory, 13:21-27, 1967.
9. L. Devroye. On the almost everywhere convergence of nonparametric regression function estimates. The Annals of Statistics, 9:1310-1319, 1981.
10. L. Devroye, L. Györfi, and G. Lugosi. A Probabilistic Theory of Pattern Recognition. Springer-Verlag, New York, 1996.
11. T.G. Dietterich. Ensemble methods in machine learning. In J. Kittler and F. Roli, editors, First International Workshop on Multiple Classifier Systems, Lecture Notes in Computer Science, pages 1-15, New York, 2000. Springer-Verlag.
12. E. Fix and J.L. Hodges. Discriminatory Analysis. Nonparametric Discrimination: Consistency Properties. Technical Report 4, Project Number 21-49-1004, USAF School of Aviation Medicine, Randolph Field, Texas, 1951.
13. E. Fix and J.L. Hodges. Discriminatory analysis: Small sample performance. Technical Report 11, Project Number 21-49-1004, USAF School of Aviation Medicine, Randolph Field, Texas, 1952.
14. J.H. Friedman and P. Hall. On bagging and nonlinear estimation. Journal of Statistical Planning and Inference, 137:669-683, 2000.
15. I.S. Gradshteyn and I.M. Ryzhik. Table of Integrals, Series, and Products. Academic Press, New York, 2007.
16. L. Györfi. On the rate of convergence of nearest neighbor rules. IEEE Transactions on Information Theory, 29:509-512, 1978.
17. L. Györfi, M. Kohler, A. Krzyżak, and H. Walk. A Distribution-Free Theory of Nonparametric Regression. Springer-Verlag, New York, 2002.
18. P. Hall and R.J. Samworth. Properties of bagged nearest neighbour classifiers. Journal of the Royal Statistical Society B, 67:363-379, 2005.
19. I.A. Ibragimov and R.Z. Khasminskii. On nonparametric estimation of regression. Doklady Akademii Nauk SSSR, 252:780-784, 1980.
20. I.A. Ibragimov and R.Z. Khasminskii. Statistical Estimation: Asymptotic Theory. Springer-Verlag, New York, 1981.
21. I.A. Ibragimov and R.Z. Khasminskii. On the bounds for quality of nonparametric regression function estimation. Theory of Probability and its Applications, 27:81-94, 1982.
22. A.N. Kolmogorov and V.M. Tihomirov. e-entropy and e-capacity of sets in functional spaces. American Mathematical Society Translations, 17:277-364, 1961.
23. S.R. Kulkarni and S.E. Posner. Rates of convergence of nearest neighbor estimation under arbitrary sampling. IEEE Transactions on Information Theory, 41:1028-1039, 1995.
24. Y. Lin and Y. Jeon. Random forests and adaptive nearest neighbours. Journal of the American Statistical Association, 101:578-590, 2006.
25. D. Psaltis, R.R. Snapp, and S.S. Venkatesh. On the finite sample performance of the nearest neighbor classifier. IEEE Transactions on Information Theory, 40:820-837, 1994.
26. B.M. Steele. Exact bootstrap k-nearest neighbor learners. Machine Learning, 74:235-255, 2009.
27. C.J. Stone. Consistent nonparametric regression. The Annals of Statistics, 5:595-645, 1977.
28. S.S. Venkatesh, R.R. Snapp, and D. Psaltis. Bellman strikes again! The growth rate of sample complexity with dimension for the nearest neighbor classifier. In J. Kittler and F. Roli, editors, Proceedings of the Fifth Annual Workshop on Computational Learning Theory, pages 93-102, Pittsburgh, 1992.