A Bernstein-Von mises theorem for discrete probability distributions

Electronic Journal of Statistics

Volumn 3, Issue , 2009, Pages 114-148

  Boucheron, S ^a   Gassiat, E ^b  

^a UNIVERSITÉ PARIS DESCARTES   (France)

^b UNIVERSITÉ PARIS SACLAY   (France)

Author keywords

Bernstein VonMises theorem; Concentration inequalities; Discrete models; Entropy estimation; Non parametric bayesian statistics

Indexed keywords

EID: 82655164891     PISSN: 19357524     EISSN: None     Source Type: Journal    
DOI: 10.1214/08-EJS262     Document Type: Article

References (35)

1. A. Antos and I. Kontoyiannis. Convergence properties of functional estimates for discrete distributions. Random Struct. & Algorithms, 19(3-4):163-193, 2001.
2. S. Boucheron, A. Garivier, and E. Gassiat. Coding over infinite alphabets. IEEE Trans. Inform. Theory, 55:to appear, 2009.
3. B. Clarke and A. Barron. Information-theoretic asymptotics of bayes methods. IEEE Trans. Inform. Theory, 36:453-471, 1990.
4. B. Clarke and A. Barron. Jeffrey’s prior is asymptotically least favorable under entropy risk. J. Stat. Planning and Inference, 41:37-60, 1994.
5. T. Cover and J. Thomas. Elements of information theory. John Wiley & sons, 1991.
6. I. Csiszár and J. Körner. Information Theory: Coding Theorems for Discrete Memoryless Channels. Academic Press, 1981.
7. J. Doob. Application of the theory of martingales. In Le Calcul des Probabilités et ses Applications., Colloques Internationaux du Centre National de la Recherche Scientifique, no. 13, pages 23-27. Centre National de la Recherche Scientifique, Paris, 1949.
8. D. Dubhashi and D. Ranjan. Balls and bins: A study in negative dependence. Random Struct. & Algorithms, 13(2):99-124, 1998.
9. R. M. Dudley. Real analysis and probability, volume 74 of Cambridge Studies in Advanced Mathematics. Cambridge University Press, Cambridge, 2002.
10. J. Fan. Local linear regression smoothers and their minimax efficiency. Annals of Statistics, 21:196-216, 1993.
11. J. Fan and Y. K. Truong. Nonparametric regression with errors in variables. Annals of Statistics, 21(4):1900-1925, 1993.
12. J. Fan, C. Zhang, and J. Zhang. Generalized likelihood ratio statistics and wilks phenomenon. Annals of Statistics, 29(1):153-193, 2001.
13. D. A. Freedman. On the asymptotic behavior of Bayes’ estimates in the discrete case. Ann. Math. Statist., 34:1386-1403, 1963.
14. D. A. Freedman. On the asymptotic behavior of Bayes estimates in the discrete case. II. Ann. Math. Statist., 36:454-456, 1965.
15. R. G. Gallager. Information theory and reliable communication. John Wiley & sons, 1968.
16. S. Ghosal. Asymptotic normality of posterior distributions for exponential families when the number of parameters tends to infinity. J. Multivariate Anal., 74(1):49-68, 2000.
17. S. Ghosal and A. van der Vaart. Convergence rates of posterior distributions for non-i.i.d. observations. Annals of Statistics, 35(1):192-223, 2007a.
18. S. Ghosal and A. van der Vaart. Posterior convergence rates of Dirichlet mixtures at smooth densities. Annals of Statistics, 35(2):697-723, 2007b.
19. S. Ghosal and A. W. van der Vaart. Entropies and rates of convergence for maximum likelihood and Bayes estimation for mixtures of normal densities. Annals of Statistics, 29(5):1233-1263, 2001.
20. S. Ghosal, J. Ghosh, and A. van der Vaart. Convergence rates of posterior distributions. Annals of Statistics, 28(2):500-531, 2000.
21. L. Gyorfi, I. Pali, and E. van der Meulen. On universal noiseless source coding for infinite source alphabets. Eur. Trans. Telecommun. & Relat. Technol., 4 (2):125-132, 1993.
22. J. C. Kieffer. A unified approach to weak universal source coding. IEEE Trans. Inform. Theory, 24(6):674-682, 1978.
23. L. Le Cam and G. Yang. Asymptotics in Statistics: Some Basic Concepts. Springer, 2000.
24. A. Lo. A large sample study of the Bayesian bootstrap. Ann. Statist., 15(1): 360-375, 1987.
25. A. Lo. A Bayesian bootstrap for a finite population. Ann. Statist., 16(4):1684-1695, 1988.
26. P. Massart. Ecole d’Eté de Probabilité de Saint-Flour XXXIII, chapter Concentration inequalities and model selection. LNM. Springer-Verlag, 2003.
27. L. Paninski. Estimating entropy on m bins given fewer than m samples. IEEE Trans. Inform. Theory, 50(9):2200-2203, 2004.
28. S. Portnoy. Asymptotic behavior of likelihood methods for exponential families when the number of parameters tends to infinity. Annals of Statistics, 16: 356-366, 1988.
29. D. Rubin. The Bayesian bootstrap. Annals of Statistics, 9(1):130, 1981.
30. L. Schwartz. On Bayes procedures. Z. Wahrscheinlichkeitstheorie und Verw. Gebiete, 4:10-26, 1965.
31. A. Tsybakov. Introduction à l’estimation non-paramétrique, volume 41 of Mathématiques & Applications. Springer-Verlag, Berlin, 2004.
32. A. van der Vaart. The statistical work of Lucien Le Cam. Annals of Statistics, 30(3):631-682, 2002.
33. A. van der Vaart. Asymptotic statistics. Cambridge University Press, 1998.
34. A. van der Vaart and J. Wellner. Weak convergence and empirical processes. Springer Series in Statistics. Springer-Verlag, New York, 1996.
35. C.-S. Weng. On a second-order asymptotic property of the Bayesian bootstrap mean. Ann. Statist., 17(2):705-710, 1989.