Minimax nonparametric classification - Part I: Rates of convergence

IEEE Transactions on Information Theory

Volumn 45, Issue 7, 1999, Pages 2271-2284

  Yang, Yuhong ^a  

^a Department of Statistics Center for Statistics and Applications in Forensic Evidence Iowa State University 2438 Osborn Dr Ames   (United States)

Author keywords

[No Author keywords available]

Indexed keywords

APPROXIMATION THEORY; CONVERGENCE OF NUMERICAL METHODS; ERROR ANALYSIS; NEURAL NETWORKS; PROBABILITY;

CONDITIONAL PROBABILITY ESTIMATION; METRIC ENTROPY; MINIMAX NONPARAMETRIC CLASSIFICATION;

INFORMATION THEORY;

EID: 0033321586     PISSN: 00189448     EISSN: None     Source Type: Journal    
DOI: 10.1109/18.796368     Document Type: Article

References (42)

1. A. R. Barron, "Are Bayes rules consistent in information?" in Open Problems in Communication and Computation, T. M. Cover and B. Gopinath Eds. New York: Springer-Verlag, 1987, pp. 85-91.
2. _, "Complexity regularization," in Nonparametric Functional Estimation and Related Topics, G. Roussas, Ed. Dordrecht, The Netherlands: Kluwer, 1991, pp. 561-576.
3. _, "Neural net approximation," in Proc. Yale Workshop Adaptive Learning Syst., K. Narendra, Ed. New Haven, CT: Yale Univ. Press, May 1992.
4. _, "Universal approximation bounds for superpositions of a sigmoidal function," IEEE Trans. Inform. Theory, vol. 39, pp. 930-945, 1993.
5. _, "Approximation and estimation bounds for artificial neural networks," Machine Learning, vol. 14, pp. 115-133, 1994.
6. A. R. Barron and T. M. Cover, "Minimum complexity density estimation," IEEE Trans. Inform. Theory, vol. 37, pp. 1034-1054, 1991.
7. L. Birgé, "Approximation dans les espaces metriques et theorie de l'estimation," Z. Wahrscheinlichkeitstheor. Verw. Geb., vol. 65, pp. 181-237, 1983.
8. _, "On estimating a density using Hellinger distance and some other strange facts," Probab. Theory Related Fields, vol. 71, pp. 271-291, 1986.
9. p," Matem. Sbornik, N.S., vol. 2, pp. 295-317, 1967.
10. _, "Quantitative analysis in Sobolev imbedding theorems and application to spectral theory," in 10th Math. School Kiev, 1974, pp. 5-189;
11. English transl.: Amer. Math. Soc. Transl., vol. 114, 1980, pp. 1-132.
12. J. Bretagnolle and C. Huber, "Estimation des densites: Risque minimax," Z. Wahrscheinlichkeitstheor. Verw. Geb., vol. 47, pp. 119-137, 1979.
13. B. Carl, "Entropy numbers of embedding maps between Besov spaces with an application to eigenvalue problems," Proc. Roy. Soc. Edinburgh, vol. 90A, pp. 63-70 1981.
14. T. M. Cover and J. A. Thomas, Elements of Information Theory. New York: Wiley, 1991.
15. R. A. DeVore and G. G. Lorentz, Constructive Approximation. New York: Springer-Verlag, 1993.
16. 1 error of partitioning estimates of a regression functions," in Proc. 4th Pannonian Symp. Mathematical Statistics, F. Konecny, J. Mogyoródi, and W. Wertz, Eds. Budapest, Hungary: Akadémiai Kiadó, 1983, pp. 67-76.
17. L. Devroye, L. Györfi, and G. Lugosi, A Probabilistic Theory of Pattern Recognition. New York: Springer-Verlag, 1996.
18. L. Gorden and R. Olshen, "Asymptotically efficient solutions to the classification problem," Ann. Statist., vol. 6, pp. 515-533, 1978.
19. W. Greblicki, "Asymptotic efficiency of classifying procedures using the Hermite series estimate of multivariate probability densities," IEEE Trans. Inform. Theory, vol. IT-27, pp. 364-366, 1981.
20. n-NN regression estimates and classification rules," IEEE Trans. Inform. Theory, vol. IT-27, pp. 362-364, 1981.
21. C. Huber, "Lower bounds for function estimation," in Research Papers in Probability and Statistics: Festschrift for Lucien Le Cam, D. Pollard, E. Torgersen, and G. Yang, Eds. New York: Springer-Verlag, 1996, pp. 245-258.
22. I. A. Ibragimov and R. Z. Hasminskii, "On the estimation of an infinite-dimensional parameter in Gaussian white noise," Sov. Math.-Dokl., vol. 18, pp. 1307-1309, 1977.
23. G. Kerkyacharian and D. Picard, "Density estimation in Besov spaces," Statist. Probab. Lett., vol. 13, pp. 15-24, 1992.
24. A. N. Kolmogorov and V. M. Tihomirov, "ε-entropy and ε-capacity of sets in function spaces," Usp. Mat. Nauk, vol. 14, pp. 3-86, 1959;
25. English transl.: Amer. Math. Soc. Transl., vol. 17, pp. 277-364, 1961.
26. A. Krzyzak, "The rate of convergence of kernel regression and classification rules," IEEE Trans. Inform. Theory, vol. IT-32, pp. 668-679, 1986.
27. G. G. Lorentz, "Metric entropy and approximation," Bull. Amer. Math. Soc., vol. 72, pp. 903-937, 1966.
28. G. G. Lorentz, M. V. Golitschek, and Y. Makovoz, Constructive Approximation: Advanced Problems. New York: Springer-Verlag, 1996.
29. G. Lugosi and A. Nobel, "Consistency of data-driven histogram methods for density estimation and classification," Ann. Statist., vol. 24, pp. 687-706, 1996.
30. Y. Makovoz, "Random approximants and neural networks," J. Approx. Theory, vol. 85, pp. 98-109, 1996.
31. J. S. Marron, "Optimal rates of convergence to Bayes risk in nonparametric discrimination," Ann. Statist., vol. 11, pp. 1142-1155, 1983.
32. B. S. Mitjagin, "The approximation dimension and bases in nuclear spaces," Usp. Mat. Nauk, vol. 16, pp. 63-132, 1961.
33. 2 metric," Dokl. Akad. Nauk SSSR, vol. 131, pp. 30-33, 1960.
34. English transl.: in Sov. Math.-Dolk., vol. 1, pp. 192-195, 1960.
35. C. J. Stone, "Consistent nonparametric regression," Ann. Statist., vol. 8, pp. 1348-1360, 1977.
36. A. F. Timan, Theory of Approximation of Functions of a Real Variable. New York: MacMillan, 1963.
37. H. Triebel, "Interpolation properties of ε-entropy and diameters. Geometric characteristics of imbedding for function spaces of Sobolev-Besov type," Mat. Sbornik, vol. 98, pp. 27-41, 1975;
38. English Transl.: in Math. USSR Sb., vol. 27, pp. 23-37, 1977.
39. _, Theory of Function Spaces II. Basel and Boston: Birkhauser, 1992.
40. Y. Yang and A. R. Barron, "Information-theoretic determination of minimax rates of convergence," Ann. Statistics, 1999, accepted for publication.
41. Y. Yang, "Minimax nonparametric classification - Part II: Model selection for adaptation," this issue, pp. 0000-0000.
42. Y. G. Yatracos, "Rates of convergence of minimum distance estimators and Kolmogorov's entropy," Ann. Statist., vol. 13, 768-774, 1985.