Entropies and rates of convergence for maximum likelihood and Bayes estimation for mixtures of normal densities

Annals of Statistics

Volumn 29, Issue 5, 2001, Pages 1233-1263

  Ghosal, Subhashis ^a   Van Der Vaart, Aad W ^b  

^a UNIVERSITY OF MINNESOTA   (United States)

^b VU UNIVERSITY AMSTERDAM   (Netherlands)

Author keywords

Bracketing; Dirichlet mixture; Entropy; Maximum likelihood; Mixture of normals; Posterior distribution; Rate of convergence; Sieve

Indexed keywords

EID: 0035470893     PISSN: 00905364     EISSN: None     Source Type: Journal    
DOI: 10.1214/aos/1013203452     Document Type: Article

References (34)

1. BANFIELD, J. and RAFTERY, A. (1993). Model based Gaussian and non-Gaussian clustering. Biometrics 49 803-821.
2. BARKON, A., SCHERVISH, M. and WASSERMAN, L. (1999). The consistency of posterior distributions in nonparametric problems. Ann. Statist. 27 536-561.
3. BIRGÉ, L. and MASSART, P. (1998). Minimum contract estimators on sieves: exponential bounds and rates of convergence. Bernoulli 4 329-375.
4. ESCOBAR, M. and WEST, M. (1995). Bayesian density estimation and inference using mixtures. J. Amer. Statist. Assoc. 90 577-588.
5. FERGUSON, T. S. (1967). Mathematical Statistics: A Decision Theoretic Approach. Academic Press, New York.
6. FERGUSON, T. S. (1983). Bayesian density estimation by mixtures of Normal distributions. In Recent Advances in Statistics (M. Rizvi, J. Rustagi and D. Siegmund, eds.) 287-302. Academic Press, New York.
7. GEMAN, S. and HWANG, C. (1982). Nonparametric maximum likelihood estimation by the method of sieves. Ann. Statist. 10 401-414.
8. GENOVESE, C. and WASSERMAN, L. (2000). Rates of convergence for the Gaussian mixture sieve. Ann. Statist. 28 1105-1127.
9. GHOSAL, S., GHOSH, J. K. and RAMAMOORTHI, R. V. (1999a). Posterior consistency of Dirichlet mixtures in density estimation. Ann. Statist. 27 143-158.
10. GHOSAL, S., GHOSH, J. K. and RAMAMOORTHI, R. V. (1999b). Consistency issues in Bayesian Non-parametrics. In Asymptotics, Nonparametrics and Time Series: A Tribute to Madan Lal Puri (S. Ghosh, ed.) 639-668. Dekker, New York.
11. GHOSAL, S., GHOSH, J. K. and VAN DER VAART, A. W. (2000). Convergence rates of posterior distributions. Ann. Statist. 28 500-531.
12. GRENANDER, U. (1981). Abstract Inference. Wiley, New York.
13. IBRAGIMOV, I. A. (2001). Estimation of analytic functions. In State of the Art in Probability and Statistics. Festschrift for W. R. van Zwet. IMS, Hayward, CA.
14. IBRAGIMOV, I. A. and KHASMINSKII, R. Z. (1982). An estimate of the density of a distribution belonging to a class of entire functions. Theory Probab. Appl. 27 514-524 (in Russian).
15. KOLMOGOROV, A. N. and TIHOMIROV. V. M. (1961). ε-entropy and ε-capacity of sets in function spaces. Amer. Math. Soc. Transl. Ser. 2 17 277-364. [Translated from Russian (1959) Uspekhi Mat. Nauk 14 3-86.]
16. KOLMOGOROV, A. N. and TIHOMIROV. V. M. (1961). ε-entropy and ε-capacity of sets in function spaces. Amer. Math. Soc. Transl. Ser. 2 17 277-364. [Translated from Russian (1959) Uspekhi Mat. Nauk 14 3-86.]
17. LINDSAY, B. (1995). Mixture Models: Theory, Geometry and Applications. IMS, Hayward, CA.
18. LO, A. Y. (1984). On a class of Bayesian nonparametric estimates I: Density estimates. Ann. Statist. 12 351-357.
19. MCLACHLAN, G. and BASFORD, K. (1988). Mixture Models. Inference and Applications to Clustering. Dekker, New York.
20. PRIEBE, C. E. (1994). Adaptive mixtures. J. Amer. Statist. Assoc. 89 796-806.
21. ROBERT, C. (1996). Mixtures of distributions: inference and estimation. In Markov Chain Monte Carlo in Practice (W. Gilks, S. Richardson and D. Spiegelhalter, eds.) 441-464. Chapman and Hall, London.
22. ROEDER, K. (1992). Semiparametric estimation of normal mixture densities. Ann. Statist. 20 929-943.
23. ROEDER, K. and WASSERMAN, L. (1997). Practical Bayesian density estimation using mixtures of normals. J. Amer. Statist. Assoc. 92 894-902.
24. RUDIN, W. (1987). Real and Complex Analysis, 3rd ed. McGraw-Hill, New York.
25. SCHWARTZ, L. (1965). On Bayes procedures. Z. Wahrsch. Verw. Gebiete 4 10-26.
26. SHEN, X. and WASSERMAN, L. (2001). Rates of convergence of posterior distributions. Ann. Statist. 29 687-714.
27. SHEN, X. and WONG, W. H. (1994). Convergence rate of sieve estimates. Ann. Statist. 22 580-615.
28. VAN DE GEER, S. (1993). Hellinger consistency of certain nonparametric maximum likelihood estimators. Ann. Statist. 21 14-44.
29. VAN DE GEER, S. (1996). Rates of convergence for the maximum likelihood estimator in mixture models. J. Nonparametr. Statist. 6 293-310.
30. VAN DER VAART, A. W. and WELLNER, J. A. (1996). Weak Convergence and empirical Processes. Springer, New York.
31. WASSERMAN, L. (1998). Asymptotic properties of nonparametric Bayesian procedures. Practical Nonparametric and Semiparametric Bayesian Statistics. Lecture Notes in Statist. 133 293-304. Springer, New York.
32. WEST, M. (1992). Modeling with mixtures. In Bayesian Statistics 4 (J. M. Bernardo, J. O. Berger, A. P. Dawid and A. F. M. Smith, eds.) 503-524. Oxford Univ. Press.
33. WEST, M., MULLER, P. and ESCOBAR, M. D. (1994). Hierarchical priors and mixture models, with applications in regression and density estimation. In Aspects of Uncertainty: A Tribute to D. V. Lindley (P. R. Freeman and A. F. M. Smith, eds.) 363-386. Wiley, New York.
34. WONG, W. H. and SHEN, X. (1995). Probability inequalities for likelihood ratios and convergence rates of sieve MLEs. Ann. Statist. 23 339-362.