Bandwidth choice for nonparametric classification

Annals of Statistics

Volumn 33, Issue 1, 2005, Pages 284-306

  Hall, Peter ^a   Kang, Kee Hoon ^b  

^a AUSTRALIAN NATIONAL UNIVERSITY   (Australia)

^b Hankuk University of Foreign Studies   (South Korea)

Author keywords

Bayes risk; Bootstrap; Classification error; Cross validation; Discrimination; Error rate; Kernel methods; Nonparametric density estimation

Indexed keywords

EID: 18444375293     PISSN: 00905364     EISSN: None     Source Type: Journal    
DOI: 10.1214/009053604000000959     Document Type: Article

References (44)

1. ANCUKIEWICZ, M. (1998). An unsupervised and nonparametric classification procedure based on mixtures with known weights. J. Classification 15 129-141.
2. BAEK, S. and SUNG, K. M. (2000). Fast K -nearest-neighbour search algorithm for nonparametric classification. Electronics Letters 36 1821-1822.
3. BREIMAN, L. (1998). Arcing classifiers (with discussion). Ann. Statist. 26 801-849.
4. BREIMAN, L. (2001). Random forests. Machine Learning 45 5-32.
5. CHANDA, K. C. and RUYMGAART, F. H. (1989). Asymptotic estimate of probability of misclassification for discriminant rules based on density estimates. Statist. Probab. Lett. 8 81-88.
6. COVER, T. M. (1968). Rates of convergence for nearest neighbor procedures. In Proc. Hawaii International Conference on System Sciences (B. K. Kinariwala and F. F. Kuo, eds.) 413-415. Univ. Hawaii Press, Honolulu.
7. DEVROYE, L. (1982). Any discrimination rule can have an arbitrarily bad probability of error for finite sample size. IEEE Trans. Pattern Anal. Machine Intelligence 4 154-157.
8. DEVROYE, L., GYÖRFI, L. and LUGOSI, G. (1996). A Probabilistic Theory of Pattern Recognition. Springer, New York.
9. DUDOIT, S., FRIDLYAND, J. and SPEED, T. P. (2002). Comparison of discrimination methods for the classification of tumors using gene expression data. J. Amer. Statist. Assoc. 97 77-87.
10. EFRON, B. (1983). Estimating the error rate of a prediction rule: Improvement on cross-validation. J. Amer. Statist. Assoc. 78 316-331.
11. EFRON, B. and TIBSHIRANI, R. (1997). Improvements on cross-validation: The .632+ bootstrap method. J. Amer. Statist. Assoc. 92 548-560.
12. EHRENFEUCHT, A., HAUSSLER, D., KEARNS, M. and VALIANT, L. (1989). A general lower bound on the number of examples needed for learning. Inform. and Comput. 82 247-261.
13. Also published in Proc. 1988 Workshop on Computational Learning Theory (D. Haussler and L. Pitt, eds.) 139-154. Morgan Kaufmann, San Mateo, CA.
14. FIX, E. and HODGES, J. (1951). Discriminatory analysis. Nonparametric discrimination: Consistency properties. Technical Report No. 4, Project No. 21-49-004, USAF School of Aviation Medicine, Randolph Field, TX.
15. FRIEDMAN, J. H. (1997). On bias, variance, 0/1-loss, and the curse-of-dimensionality. Data Min. Knowl. Discov. 1 55-77.
16. FRIEDMAN, J. H., HASTIE, T. and TIBSHIRANI, R. (2000). Additive logistic regression: A statistical view of boosting (with discussion). Ann. Statist. 28 337-407.
17. FUKUNAGA, K. and FLICK, T. E. (1984). Classification error for a very large number of classes. IEEE Trans. Pattern Anal. Machine Intelligence 6 779-788.
18. FUKUNAGA, K. and HUMMELS, D. M. (1987). Bias of nearest neighbor estimates. IEEE Trans. Pattern Anal. Machine Intelligence 9 103-112.
19. GYÖRFI, L., KOHLER, M., KRZYŻAK, A. and WALK, H. (2002). A Distribution-Free Theory of Nonparametric Regression. Springer, New York.
20. HALL, P. (1983). Large sample optimality of least squares cross-validation in density estimation. Ann. Statist. 11 1156-1174.
21. HALL, P. and KANG, K.-H. (2002). Effect of bandwidth choice on Bayes risk in nonparametric classification. Available at http://stats.hufs.ac.kr/~khkang.
22. HALL, P. and SCHUCANY, W. R. (1989). A local cross-validation algorithm. Statist. Probab. Lett. 8 109-117.
23. HÄRDLE, W. and KELLY, G. (1987). Nonparametric kernel regression estimation - optimal choice of bandwidth. Statistics 18 21-35.
24. 1 error of a multivariate kernel density estimator. J. Multivariate Anal. 42 245-266.
25. JIANG, W. X. (2002). On weak base hypotheses and their implications for boosting regression and classification Ann. Statist. 30 51-73.
26. KHARIN, YU. S. (1983). Analysis and optimization of Rosenblatt-Parzen classifier with the aid of asymptotic expansions. Automat. Remote Control 44 72-80.
27. KHARIN, YU. S. and DUCINSKAS, K. (1979). The asymptotic expansion of the risk for classifiers using maximum likelihood estimates. Statist. Problemy Upravleniya - Trudy Sem. Protsessy Optimal. Upravleniya V Sektsiya 38 77-93. (In Russian.)
28. KIM, H. and LOH, W.-Y. (2001). Classification trees with unbiased multiway splits. J. Amer. Statist. Assoc. 96 589-604.
29. KRZYŻAK, A. (1991). On exponential bounds on the Bayes risk of the nonparametric classification rules. In Nonparametric Functional Estimation and Related Topics (G. Roussas, ed.) 347-360. Kluwer, Dordrecht.
30. LAPKO, A. V. (1993). Nonparametric Classification Methods and Their Application. VO Nauka, Novosibirsk. (In Russian.)
31. LIN, C.-T. (2001). Nonparametric classification on two univariate distributions. Comm. Statist. Theory Methods 30 319-330.
32. LUGOSI, G. and NOBEL, A. (1996). Consistency of data-driven histogram methods for density estimation and classification. Ann. Statist. 24 687-706.
33. LUGOSI, G. and PAWLAK, M. (1994). On the posterior-probability estimate of the error rate of nonparametric classification rules. IEEE Trans. Inform. Theory 40 475-481.
34. MAMMEN, E. and TSYBAKOV, A. B. (1999). Smooth discrimination analysis. Ann. Statist. 27 1808-1829.
35. MARRON, J. S. (1983). Optimal rates on convergence to Bayes risk in nonparametric discrimination. Ann. Statist. 11 1142-1155.
36. MIELNICZUK, J., SARDA, P. and VIEU, P. (1989). Local data-driven bandwidth choice for density estimation. J. Statist. Plann. Inference 23 53-69.
37. PAWLAK, M. (1993). Kernel classification rules from missing data. IEEE Trans. Inform. Theory 39 979-988.
38. PSALTIS, D., SNAPP, R. R. and VENKATESH, S. S. (1994). On the finite sample performance of the nearest neighbor classifier. IEEE Trans. Inform. Theory 40 820-837.
39. SCHAPIRE, R. E., FREUND, Y., BARTLETT, P. and LEE, W. S. (1998). Boosting the margin: A new explanation for the effectiveness of voting methods. Ann. Statist. 26 1651-1686.
40. STEELE, B. M. and PATTERSON, D. A. (2000). Ideal bootstrap estimation of expected prediction error for k-nearest neighbor classifiers: Applications for classification and error assessment. Statist. Comput. 10 349-355.
41. STOLLER, D. S. (1954). Univariate two-population distribution-free discrimination. J. Amer. Statist. Assoc. 49 770-777.
42. STONE, C. J. (1984). An asymptotically optimal window selection rule for kernel density estimates. Ann. Statist. 12 1285-1297.
43. YANG, Y. H. (1999a). Minimax nonparametric classification. I. Rates of convergence. IEEE Trans. Inform. Theory 45 2271-2284.
44. YANG, Y. H. (1999b). Minimax nonparametric classification. II. Model selection for adaptation. IEEE Trans. Inform. Theory 45 2285-2292.