Optimal weighted nearest neighbour classifiers

Annals of Statistics

Volumn 40, Issue 5, 2012, Pages 2733-2763

  Samworth, Richard J ^a  

^a UNIVERSITY OF CAMBRIDGE   (United Kingdom)

Author keywords

Bagging; Classification; Nearest neighbours; Weighted nearest neighbour classifiers

Indexed keywords

EID: 84873393113     PISSN: 00905364     EISSN: None     Source Type: Journal    
DOI: 10.1214/12-AOS1049     Document Type: Article

References (42)

1. ATHITSOS, V. and SCLAROFF, S. (2005). Boosting nearest neighbour classifiers for multiclass recognition. In Proc. IEEE Computer Society Conference on Computer Vision and Pattern Recognition 45-55. IEEE Computer Society, Washington, DC
2. AUDIBERT, J.-Y. and TSYBAKOV, A. B. (2007). Fast learning rates for plug-in classifiers. Ann. Statist. 35 608-633. MR2336861
3. BAILEY, T. and JAIN, A. (1978). A note on distance-weighted k-nearest neighbour rules. Transactions on Systems, Man, and Cybernetics 8 311-313.
4. BIAU, G., CÉROU, F. and GUYADER, A. (2010). On the rate of convergence of the bagged nearest neighbour estimate. J. Mach. Learn. Res. 11 687-712. MR2600626
5. BIAU, G. and DEVROYE, L. (2010). On the layered nearest neighbour estimate, the bagged nearest neighbour estimate and the random forest method in regression and classification. J. Multivariate Anal. 101 2499-2518. MR2719877
6. BLANCHARD, G., BOUSQUET, O. and MASSART, P. (2008). Statistical performance of support vector machines. Ann. Statist. 36 489-531. MR2396805
7. BOUCHERON, S., BOUSQUET, O. and LUGOSI, G. (2005). Theory of classification: A survey of some recent advances. ESAIM Probab. Stat. 9 323-375. MR2182250
8. BREIMAN, L. (1996). Heuristics of instability and stabilization in model selection. Ann. Statist. 24 2350-2383. MR1425957
9. BREIMAN, L. (1999). Using adaptive bagging to debias regressions. Technical report, Dept. Statistics, Univ. California, Berkeley.
10. BREIMAN, L. (2001). Random forests. Mach. Learn. 45 5-32.
11. CHANDA, K. C. and RUYMGAART, F. H. (1989). Asymptotic estimate of probability of misclassification for discriminant rules based on density estimates. Statist. Probab. Lett. 8 81-88. MR1006427
12. CORTES, C. and VAPNIK, V. (1995). Support-vector networks. Mach. Learn. 20 273-297.
13. COVER, T. M. and HART, P. E. (1967). Nearest neighbour pattern classification. IEEE Trans. Inform. Theory 13 21-27.
14. DEVROYE, L., GYÖRFI, L. and LUGOSI, G. (1996). A Probabilistic Theory of Pattern Recognition. Applications of Mathematics (New York) 31. Springer, New York. MR1383093
15. FIX, E. and HODGES, J. L. (1951). Discriminatory analysis - Nonparametric discrimination: Consistency properties. Technical Report 4, Project no. 21-29-004, USAF School of Aviation Medicine, Randolph Field, Texas.
16. FIX, E. and HODGES, J. L. (1989). Discriminatory analysis - Nonparametric discrimination: Consistency properties. Internat. Statist. Rev. 57 238-247.
17. FRANK, A. and ASUNCION, A. (2010). UCI machine learning repository. School of Information and Computer Sciences, Univ. California, Irvine, CA. Available at http://archive.ics.uci.edu/ml.
18. GORDON, A. D. (1999). Classification, 2nd ed. Chapman & Hall, London. MR0637465
19. GRAY, A. (2004). Tubes, 2nd ed. Progress in Mathematics 221. Birkhäuser, Basel. MR2024928
20. GUILLEMIN, V. and POLLACK, A. (1974). Differential Topology. Prentice-Hall, Englewood Cliffs, NJ. MR0348781
21. HALL, P. and KANG, K.-H. (2005). Bandwidth choice for nonparametric classification. Ann. Statist. 33 284-306. MR2157804
22. HALL, P., PARK, B. U. and SAMWORTH, R. J. (2008). Choice of neighbour order in nearestneighbour classification. Ann. Statist. 36 2135-2152. MR2458182
23. HALL, P. and SAMWORTH, R. J. (2005). Properties of bagged nearest neighbour classifiers. J. R. Stat. Soc. Ser. B Stat. Methodol. 67 363-379. MR2155343
24. HAND, D. J. (1981). Discrimination and Classification. Wiley, Chichester. MR0634676
25. IBRAGIMOV, I. A. and KHASMINSKIǏ, R. Z. (1980). Nonparametric regression estimation. Dokl. Akad. Nauk SSSR 252 780-784. MR0580829
26. IBRAGIMOV, I. A. and KHASMINSKIǏ, R. Z. (1981). Statistical Estimation. Applications of Mathematics 16. Springer, New York. MR0620321
27. IBRAGIMOV, I. A. and KHASMINSKIǏ, R. Z. (1982). On the bounds for quality of nonparametric regression function estimation. Theory Probab. Appl. 27 81-94.
28. LEPSKIǏ, O. V. (1991). Asymptotically minimax adaptive estimation. I. Upper bounds. Optimally adaptive estimates. Theory Probab. Appl. 36 682-697.
29. MAMMEN, E. and TSYBAKOV, A. B. (1999). Smooth discrimination analysis. Ann. Statist. 27 1808- 1829. MR1765618
30. MARTÍNEZ-MUÑOZ, G. and SUÁREZ, A. (2010). Out-of-bag estimation of the optimal sample size in bagging. Pattern Recognition 43 143-152.
31. MOORE, J. D. (1992). Book review: Tubes. Bull. Amer. Math. Soc. (N.S.) 27 311-313. MR1567997
32. POLONIK, W. (1995). Measuring mass concentrations and estimating density contour clusters - An excess mass approach. Ann. Statist. 23 855-881. MR1345204
33. RAUDYS, Š. and YOUNG, D. M. (2004). Results in statistical discriminant analysis: A review of the former Soviet Union literature. J. Multivariate Anal. 89 1-35. MR2041207
34. RIGOLLET, P. and VERT, R. (2009). Optimal rates for plug-in estimators of density level sets. Bernoulli 15 1154-1178. MR2597587
35. ROYALL, R. (1966). A class of nonparametric estimators of a smooth regression function. Ph.D. thesis, Stanford Univ., Stanford, CA
36. SAMWORTH, R. J. (2012). Supplement to "Optimal weighted nearest neighbour classifiers." DOI:10.1214/12-AOS1049SUPP
37. SAMWORTH, R. J. and WAND, M. P. (2010). Asymptotics and optimal bandwidth selection for highest density region estimation. Ann. Statist. 38 1767-1792. MR2662359
38. SHORACK, G. R. and WELLNER, J. A. (1986). Empirical Processes with Applications to Statistics. Wiley, New York. MR0838963
39. STEELE, B. M. (2009). Exact bootstrap k-nearest neighbour learners. Mach. Learn. 74 235-255.
40. STEINWART, I. and CHRISTMANN, A. (2008). Support Vector Machines. Springer, New York. MR2450103
41. STONE, C. J. (1977). Consistent nonparametric regression. Ann. Statist. 5 595-645. MR0443204
42. TSYBAKOV, A. B. (2004). Optimal aggregation of classifiers in statistical learning. Ann. Statist. 32 135-166. MR2051002