An asymptotic property of model selection criteria

IEEE Transactions on Information Theory

Volumn 44, Issue 1, 1998, Pages 95-116

  Yang, Yuhong ^a   Barron, Andrew R ^a  

^a Department of Statistics Center for Statistics and Applications in Forensic Evidence Iowa State University 2438 Osborn Dr Ames   (United States)

Author keywords

[No Author keywords available]

Indexed keywords

COMPUTATIONAL COMPLEXITY; CONVERGENCE OF NUMERICAL METHODS; MATHEMATICAL MODELS; NEURAL NETWORKS; PARAMETER ESTIMATION;

MODEL SELECTION CRITERIA;

PROBABILITY DENSITY FUNCTION;

EID: 0031672252     PISSN: 00189448     EISSN: None     Source Type: Journal    
DOI: 10.1109/18.650993     Document Type: Article

References (48)

1. H. Akaike, "Information theory and an extension of the maximum likelihood principle," in Proc. 2nd Int. Symp. on Information Theory, B. N. Petrov and F. Csaki, Eds. Budapest, Hungary: Akademia Kiado, 1973, pp. 267-281
2. A.R. Barron, "Logistically smooth density estimation," Ph.D. dissertation, Department of Electrical Engineering, Stanford University, Stanford, CA, 1985
3. A.R. Barron and C.-H Sheu, "Approximation of density function by sequences of exponential families," A. Statist., vol. 19, pp. 1347-1369, 1991
4. A.R. Barron and T. M. Cover, "Minimum complexity density estimation," IEEE Trans. Inform. Theory, vol. 37, pp. 1034-1054, 1991.
5. A.R. Barron, "Universal approximation bounds for superpositions of a sigmoidal function," IEEE Trans. Inform. Theory, vol. 39, pp. 930-945, 1993.
6. -, "Approximation and estimation bounds for artificial neura networks," Mach. Learn., vol. 14, pp. 115-133, 1994.
7. A.R. Barron, Y. Yang, and B. Yu, "Asymptotically optimal function estimation by minimum complexity criteria," in Proc. 1994 Int. Symp. on Information Theory (Trondheim, Norway, 1994), p. 38.
8. A.R. Barron, L. Birgé, and P. Massart, "Risk bounds for model selectio via penalization," Prob. Theory Related Fields, 1996, to be published.
9. J.M. Bernardo, "Reference prior distributions for Bayesian inference," J. Roy. Statist. Soc., vol. 41, ser. B, pp. 113-147, 1979
10. L. Birgé, "Approximation dans les espaces métriques et théorie de l'estimation," Z. Wahrscheinlichkeitstheor. Verw. Geb., vol. 65, pp. 181-237, 1983
11. L. Birgé and P. Massart, "Rates of convergence for minimum contrast estimators," Prob. Theory Related Fields, vol. 97, pp. 113-150, 1993
12. -, "Minimum contrast estimators on sieves: Exponential bounds and rates of convergence," Tech. Rep., Université Paris-Sud, 1996
13. -, "From model selection to adaptive estimation," in Research Papers in Probability and Statistics: Festschrift for Lucien Le Cam, D. Pollard and G. Yang, Eds. New York: Springer, 1996
14. N.N. Cencov, Statistical Decision Rules and Optimal Inference. Providence, RI: Amer. Math. Soc. Transi., 1982, vol. 53
15. H. Chernoff, "A measure of asymptotic efficiency for tests of a hypothesis based on the sum of observations," Ann. Math. Statist., vol. 23, pp. 493-507, 1952
16. C.K. Chui, An Introduction to Wavelets. New York: Academic, 1991
17. B. Clarke and A. R. Barron, "Information-theoretic asymptotics of Bayes methods," IEEE Trans. Inform. Theory, vol. 36, pp. 453-4171, 1990
18. __, "Jeffrey's prior is asymptotically least favorable under entropy risk," J. Statist. Planning Infer., vol. 41, pp. 37-410, 1994
19. C. de Boor and G. J. Fix, "Spline approximation by quasiinterpolents," J. Approx. Theory, vol. 8, pp. 19-45, 1973
20. C. de Boor, A Practical Guide to Splines. New York: Springer-Verlag, 1978
21. D.L. Donoho, "Unconditional bases are optimal bases for data compression and for statistical estimation," Appl. and Comput. Harmonic Anal, vol. 1, pp. 100-115, 1993.
22. D.L. Donoho, I. M. Johnstone, G. Kerkyacharian, and D. Picard "Density estimation by wavelet thresholding," Ann. Statist., vol. 24, pp 508-539, 1996.
23. D.L. Donoho and I. M. Johnstone, "Minimax estimation via wavele shrinkage," 1994, unpublished manuscript.
24. S. Yu. Efroimovich, "Nonparametric estimation of a density of unknow smoothness," Theory Probab. Appl., vol. 30, pp. 557-568, 1985.
25. D. Haughton, "Size of the error in the choice of a model to fit dat from an exponential family," Sankhys: Indian J. Statist. Ser. A, vol. 51 pp. 45-58, 1989
26. L.M. Le Cam, "Convergence of estimates under dimensionality restrictions," Ann. Statist., vol. 1, pp. 38-53, 1973.
27. W.S. Lee, P. L. Bartlett, and R. C. Williamson, "Efficient agnosti leaning of neural networks with bounded fan-in," IEEE Trans. Inform Theory, to be published.
28. K.C. Li, "Asymptotic optimality for Cp, CL, cross-validation an generalized cross-validation: Discrete index set," Ann. Statist., vol. 15 pp. 958-975, 1987.
29. D.S. Modha and E. Masry, "Rates of convergence in density estimatio using neural networks," 1994, manuscript
30. A. Nemirovskii, "Nonparametric estimation of smooth regression functions," J. Comput. Syst. Sci., vol. 23, pp. 1-11, 1986
31. D. Pollard, Convergence of Stochastic Processes. New York: SpringerVerlag, 1984.
32. B.T. Polyak and A. B. Tsybakov, "Asymptotic optimality of the Cp test for the orthogonal series estimation of regression," Theory Probab Appl., vol. 35, pp. 293-306, 1990.
33. S. Portnoy, "Asymptotic behavior of likelihood methods for exponentia families when the number of parameters tend to infinity," Ann. Statist. vol. 16, pp. 356-366, 1988.
34. J. Rissanen, "Universal coding, information, prediction, and estimation, IEEE Trans. Inform. Theory, vol. IT-30, pp. 629-636, 1984.
35. __, "Fisher information and stochastic complexity," IEEE Trans Inform. Theory, vol. 40, pp. 40-47, 1996.
36. R. Shibata, "An optimal selection of regression variables," Biometrika vol. 68, pp. 45-54, 1981.
37. G. Schwartz, "Estimating the dimension of a model," Ann. Statist., vol 6, pp. 461-464, 1978.
38. T.P. Speed and B. Yu, "Model selection and prediction: Norma regression," Ann. Inst. Stat. Math., vol. 45, pp. 35-54, 1993.
39. C.J. Stone, "The dimensionality reduction principle for generalize additive models," A. Statist., vol. 14, no. 2, pp. 590-606, 1986.
40. -, " Large-sample inference for log-spline models," Ann. Statist. vol. 18, pp. 717-741, 1990.
41. __, "The use of polynomial splines and their tensor products i multivariate function estimation," Ann. Statist., vol. 22, pp. 118-184 1994.
42. M. Talagrand, "Sharper bounds for Gaussian and empirical processes, Ann. Probab., vol. 22, pp. 28-76, 1994
43. S. Van de Geer, "Hellinger consistency of certain nonparametric maximum likelihood estimates," A. Statist., vol. 21, pp. 14-414, 1993.
44. C.S. Wallace and D. M. Boulton, "An invariant Bayes method for poin estimation," Classification Soc. Bull., vol. 3, pp. 11-34, 1975.
45. C.S. Wallace and P. R. Freeman, "Estimation and inference by compac coding," J. Roy. Statist. Soc. B, vol. 49, pp. 240-265, 1987.
46. W.H. Wong and X. Shen, "Probability inequalities for likelihoo ratios and convergence rates of sieve MLEs," Ann. Statist., vol. 23 pp. 339-362, 1995.
47. Y. Yang, "Complexity-based model selection," prospectus submitted t Department of Statistics, Yale University, New Haven, CT, 1993.
48. Y. Yang and A. R. Barron, "Information-theoretic determination o minimax rates of convergence," submitted lo Ann. Statist., 1996.