Bayes Error Estimation Using Parzen and k-NN Procedures

IEEE Transactions on Pattern Analysis and Machine Intelligence

Volumn PAMI-9, Issue 5, 1987, Pages 634-643

  Fukunaga, Keinosuke ^a   Hummels, Donald M ^a  

^a PURDUE UNIVERSITY   (United States)

Author keywords

Bayes error estimation; finite sample; k NN; nearest; neighbor; nonparametric error estimation; Parzen

Indexed keywords

EID: 0000079087     PISSN: 01628828     EISSN: None     Source Type: Journal    
DOI: 10.1109/TPAMI.1987.4767958     Document Type: Article

References (28)

1. E. Fix and J. L. Hodges,“Discriminatory analysis, nonparametric discrimination, consistency properties,” School of Aviation Medicine, Randolph Field, TX, Report No. 4, Project 21-49–004, 1951.
2. ——, “Nonparametric discrimination small sample performance,” School of Aviation Medicine, Randolph Field, TX, Report No. 11, Project 21-49-004, 1952.
3. T. M. Cover and P. E. Hart, “Nearest neighbor pattern classification,” IEEE Trans. Inform. Theory, vol. IT-13, pp. 21–27, Jan. 1967.
4. 14] M. Rosenblatt, “Remarks on some nonparametric estimates of a density function,” Ann. Math. Stat., vol. 27, pp. 832–837, 1956.
5. E. Parzen, “On the estimation of a probability density function and the mode,” Ann. Math. Stat., vol. 33, pp. 1065–1076, 1962.
6. T. Cacoullos, “Estimation of a mulitvariate density,” Ann. Inst. Stat. Math., vol. 18, pp. 178–189, 1966.
7. K. Fukunaga and D. M. Hummels, “Bias of nearest neighbor error estimates,” IEEE Trans. Pattern Anal. Mach. Intell., vol. PAMI-9, pp. 103–112, Jan. 1986.
8. L. Devroye and L. Györfi, Nonparametric Density Estimation: The L, View. New York: Wiley, 1985.
9. D. O. Loftsgaarden and C. P. Quesenberry, “A nonparametric estimate of a multivariate density function,” Ann. Math. Stat., vol. 36, pp. 1049–1051, 1965.
10. D. S. Moore and J. W. Yackel, “Consistency properties of nearest neighbor density estimates,” Ann. Stat., vol. 5, pp. 143–154, 1977.
11. L. Devroye and T. J. Wagner, “The strong uniform consistency of nearest neighbor density estimates,” Ann. Stat., vol. 5, pp. 536–540, 1977.
12. L. Devroye, “On the almost everywhere convergence of nonparametric metric regression function estimates,” Ann. Stat., vol. 9, pp. 1310–1319, 1319, 1981.
13. C. J. Stone, “Consistent nonparametric regression,” Ann. Stat., vol. 5, pp. 595–645, 1977.
14. L. Devroye, “Necessary and sufficient conditions for the pointwise convergence of nearest neighbor regression function estimates,” Zeitschrift schrift Wahrscheinlichkeitstheorie verwandte Gebiete, vol. 61, pp. 467-481, 1982.
15. “On the inequality of Cover and Hart in nearest neighbor discrimination,” IEEE Trans. Pattern Anal. Mach. Intell., vol. PAMI-3, 3, pp. 75–78, Jan. 1981.
16. K. Fukunaga, Introduction to Statistical Pattern Recognition. New York: Academic, 1972.
17. K. Fukunaga and L. D. Hostetler, “Optimization of k-nearest neighbor density estimates,” IEEE Trans. Inform. Theory, vol. IT-19, pp. 320–326, 1973.
18. R. P. W. Duin, “On the choice of smoothing parameters for Parzen estimators of probability density functions,” IEEE Trans. Comput., vol. C-25, pp. 1175–1179, 1976.
19. J. D. F. Habbema, J. Hermans, and K. Vandenbroek, “A stepwise discriminant analysis program using density estimation,” in COMPSTAT STAT 1974, G. Bruckmann, Ed. Wien, Austria: Physica Verlag, 1974, pp. 101–110.
20. J. VanNess, “On the dominance of non-parametric Bayes rule discriminant algorithms in high dimensions,” Pattern Recognition, vol. 12, pp. 355–368, 1980.
21. Y. S. Chow, S. German, and L. D. Wu, “Consistent cross-validated density estimation,” Ann. Stat., vol. 11, pp. 25–38, 1983.
22. P. Hall, “Large sample optimality of least squares cross-validation in density estimation,” Ann. Stat., vol. 11, pp. 1156–1174, 1983.
23. T. J. Wagner, “Nonparametric estimates of probability densities,” IEEE Trans. Inform. Theory, vol. IT-21, pp. 438–440, 1975.
24. B. W. Silverman, “Choosing the window width when estimating a density,” Biometrika, vol. 65, pp. 1–11, 1978.
25. D. J. Hand, Kernel Discriminant Analysis. Chichester, UK: Research Studies Press, 1982.
26. K. Fukunaga and T. E. Flick, “Classification error for a very large number of classes,” IEEE Trans. Pattern Anal. Mach. Intell., vol. PAMI-6, pp. 779–788, 1984.
27. B. Chandrasekaran and A. K. Jain, “On balancing decision functions,” J. Cybernet. Inform. Sci., vol. 2, pp. 12–15, 1979.
28. K. Fukunaga and J. M. Matlock, “A nonparametric two-dimensional display for classification,” IEEE Trans. Pattern Anal. Mach. Intell., vol. PAMI-4, pp. 427–436, July 1982.