From ε-entropy to KL-entropy: Analysis of minimum information complexity density estimation

Annals of Statistics

Volumn 34, Issue 5, 2006, Pages 2180-2210

  Zhang, Tong ^a  

^a YAHOO RESEARCH   (United States)

Author keywords

Bayesian posterior distribution; Density estimation; Minimum description length

Indexed keywords

EID: 33847361463     PISSN: 00905364     EISSN: None     Source Type: Journal    
DOI: 10.1214/009053606000000704     Document Type: Article

References (18)

1. BARRON, A. and COVER, T. (1991). Minimum complexity density estimation. IEEE Trans. Inform. Theory 37 1034-1054. MR1111806
2. BARRON, A., SCHERVISH, M. J. and WASSERMAN, L. (1999). The consistency of posterior distributions in nonparametric problems. Ann. Statist. 27 536-561. MR1714718
3. CATONI, O. (2004). A PAC-Bayesian approach to adaptive classification. Available at www.proba.jussieu.fr/users/catoni/homepage/classif. pdf.
4. GHOSAL, S., GHOSH, J. K. and VAN DER VAART, A. W. (2000). Convergence rates of posterior distributions. Ann. Statist. 28 500-531. MR1790007
5. LE CAM, L. (1973). Convergence of estimates under dimensionality restrictions. Ann. Statist. 1 38-53. MR0334381
6. LE CAM, L. (1986). Asymptotic Methods in Statistical Decision Theory. Springer, New York. MR0856411
7. LI, J. (1999). Estimation of mixture models. Ph.D. dissertation, Dept. Statistics, Yale Univ.
8. MEIR, R. and ZHANG, T. (2003). Generalization error bounds for Bayesian mixture algorithms. J. Mach. Learn. Res. 4 839-860.
9. RÉNYI, A. (1961). On measures of entropy and information. Proc. Fourth Berkeley Symp. Math. Statist. Probab. 1547-561. Univ. California Press, Berkeley. MR0132570
10. RISSANEN, J. (1989). Stochastic Complexity in Statistical Inquiry. World Scientific, Singapore. MR1082556
11. SEEGER, M. (2002). PAC-Bayesian generalization error bounds for Gaussian process classification. J. Mach. Learn. Res. 3 233-269. MR1971338
12. SHEN, X. and WASSERMAN, L. (2001). Rates of convergence of posterior distributions. Ann. Statist. 29 687-714. MR1865337
13. VAN DE GEER, S. (2000). Empirical Processes in M-Estimation. Cambridge Univ. Press.
14. VAN DER VAART, A. W. and WELLNER, J. A. (1996). Weak Convergence and Empirical Processes. With Applications to Statistics. Springer, New York. MR1385671
15. WALKER, S. and HJORT, N. (2001). On Bayesian consistency. J. R. Stat. Soc. Ser. B Stat. Methodol. 63 811-821. MR1872068
16. YANG, Y. and BARRON, A. (1999). Information-theoretic determination of minimax rates of convergence. Ann. Statist. 27 1564-1599. MR1742500
17. ZHANG, T. (1999). Theoretical analysis of a class of randomized regularization methods. In Proc. Twelfth Annual Conference on Computational Learning Theory 156-163. ACM Press, New York. MR1811611
18. ZHANG, T. (2004). Learning bounds for a generalized family of Bayesian posterior distributions. In Advances in Neural Information Processing Systems 16 (S. Thrun, L. K. Saul and B. Schölkopf, eds.) 1149-1156. MIT Press, Cambridge, MA.