An empirical comparison of EM, SEM and MCMC performance for problematic Gaussian mixture likelihoods

Statistics and Computing

Volumn 14, Issue 4, 2004, Pages 323-332

  Dias, José G ^a   Wedel, Michel ^b  

^a Department of Quantitative Methods   (Portugal)

^b UNIVERSITY OF MICHIGAN   (United States)

Author keywords

conjugate prior; EM algorithm; Gaussian mixture models; hierarchical prior; label switching; loss functions; MCMC; SEM algorithm

Indexed keywords

EID: 4444302656     PISSN: 09603174     EISSN: None     Source Type: Journal    
DOI: 10.1023/B:STCO.0000039481.32211.5a     Document Type: Article

References (29)

1. Celeux G. 1998. Bayesian inference for mixture: The label switching problem. In: Payne R. and Green P. (Eds.), COMPSTAT 98. Physica-Verlag, Heidelberg, pp. 227-232.
2. Celeux G. and Diebolt J. 1985. The SEM algorithm: A probabilistic teacher algorithm derived from the EM algorithm for the mixture problem. Computational Statistics Quarterly 2: 73-82.
3. Celeux G., Chauveau D., and Diebolt J. 1996. Stochastic versions of the EM algorithm: An experimental study in the mixture case. Journal of Statistical Computation and Simulations 55: 287-314.
4. Celeux G., Hurn M., and Robert C.P. 2000. Computational and inferential difficulties with mixture posterior distributions. Journal of the American Statistical Association 95: 957-970.
5. Cowles M.K. and Carlin B.P. 1996. Markov chain Monte Carlo convergence diagnostics: A comparative review. Journal of the American Statistical Association 91: 883-904.
6. Dempster A.P., Laird N.M., and Rubin D.B. 1977. Maximum likelihood from incomplete data via the EM algorithm (with discussion). Journal of the Royal Statistical Society B 39: 1-38.
7. Diebolt J. and Ip E.H.S. 1996. Stochastic EM: Method and application. In: Gilks W.R., Richardson S.T., and Spiegelhalter D.J. (Eds.), Markov Chain Monte Carlo in Practice. Chapman & Hall, London, pp. 259-273.
8. Diebolt J. and Robert C.P. 1994. Estimation of finite mixture distributions through Bayesian sampling. Journal of the Royal Statistical Society B 56(2): 363-375.
9. Escobar M.D. and West M. 1995. Bayesian density estimation and inference using mixtures. Journal of the American Statistical Association 90: 577-588.
10. Fletcher R. 1980. Practical Methods of Optimization. Vol. 1 Unconstrained Optimization. John Wiley & Sons, Chichester.
11. Frigessi A. and Rue H. 1997. Bayesian image classification with Baddeley's delta loss. Journal of Computational and Graphical Statistics 6(1): 55-73.
12. Gelfand A.E. and Smith A.F.M. 1990. Sampling based approaches to calculating marginal densities. Journal of the American Statistical Association 85: 398-409.
13. Gelman A., Carlin J.B., Stern H.S., and Rubin D.B. 1995. Bayesian Data Analysis. Chapman & Hall, Boca Raton.
14. Hurn M., Justel A., and Robert C.P. 2003. Estimating mixtures of regressions. Journal of Computational and Graphical Statistics 12(1): 55-79.
15. McLachlan G.J. and Krishnan T. 1997. The EM Algorithm and Extensions. John Wiley & Sons, New York.
16. McLachlan G.J. and Peel D. 2000. Finite Mixture Models. John Wiley & Sons, New York.
17. Richardson S. and Green P.J. 1997. On Bayesian analysis of mixtures with an unknown number of components (with discussion). Journal of the Royal Statistical Society B 59(4): 731-792.
18. Robert C.P. 1996. Mixtures of distributions: Inference and estimation. In: Gilks W.R., Richardson S., and Spiegelhalter D.J. (Eds.), Markov Chain Monte Carlo in Practice. Chapman & Hall, London, pp. 441-464.
19. Roeder K. and Wasserman L. 1997. Practical Bayesian density estimation using mixtures of normals. Journal of the American Statistical Association, 92: 894-902.
20. Rue H. 1995. New loss functions in Bayesian imaging. Journal of the American Statistical Association 90(431): 900-908.
21. Sahu S.K. and Roberts G.O. 1999. On convergence of the EM algorithm and the Gibbs sampler. Statistics and Computing 9: 55-64.
22. Stephens M. 1997a. Discussion on 'On Bayesian analysis of mixtures with an unknown number of components (with discussion)'. Journal of Royal Statistical Society B 59(4): 768-769.
23. Stephens M. 1997b. Bayesian methods for mixtures of normal distributions, DPhil Thesis Oxford. University of Oxford.
24. Stephens M. 2000. Dealing with label switching in mixture models, Journal of the Royal Statistical Society B 62(4): 795-809.
25. Tanner M.A. and Wong W.H. 1987. The calculation of posterior distributions by data augmentation. Journal of the American Statistical Association 82: 528-540.
26. UNDP. 2000. Human Development Report 2000. Human Rights and Human Development United Nations Development Programme, Oxford University Press, New York.
27. Vlassis N. and Likas A. 2002. A greedy EM algorithm for Gaussian mixture learning. Neural Processing Letters 15: 77-87.
28. Wedel M. and DeSarbo WS. 1995. A mixture likelihood approach for generalized linear models. Journal of Classification 12: 21-55.
29. Wu C.F.J. 1983. On the convergence properties of the EM algorithm. Annals of Statistics 11: 95-103.