Learning a multivariate Gaussian mixture model with the reversible jump MCMC algorithm

Statistics and Computing

Volumn 14, Issue 4, 2004, Pages 343-355

  Zhang, Zhihua ^a   Chan, Kap Luk ^b   Wu, Yiming ^b   Chen, Chibiao ^b  

^a HONG KONG UNIVERSITY OF SCIENCE AND TECHNOLOGY   (Hong Kong)

^b NANYANG TECHNOLOGICAL UNIVERSITY   (Singapore)

Author keywords

Bayesian inference; Gaussian mixture model; Markov chain Monte Carlo; model selection; reversible jump methodology; split and combine moves

Indexed keywords

EID: 4444233747     PISSN: 09603174     EISSN: None     Source Type: Journal    
DOI: 10.1023/B:STCO.0000039484.36470.41     Document Type: Article

References (30)

1. Andrieu C., de Freitas N., and Doucet A. 2001. Robust full Bayesian learning for radial basis networks. Neural Computation 13: 2359-2407.
2. Andrieu C. and Doucet A. 1999. Joint Bayesian model selection and estimation of noisy sinusoids via reversible jump MCMC. IEEE Transactions Signal Processing 47: 2667-2676.
3. Attial H. 2000. A Variational Bayesian framework for graphical models. Advances in Neural Information Processing Systems 12: 21-30.
4. Banfield J.D. and Raftery A.E. 1993. Model-based Gaussian and non-Gaussian clustering. Biometrics 49: 803-821.
5. Bensmail H. and Celeux G. 1996. Regularized Gaussian discriminant analysis through eigenvalue decomposition. Journal of the American Statistical Association 91(436): 1743-1748.
6. Bensmail H., Celeux G., Raftery A.E., and Robert C.P. 1997. Inference in model-based cluster analysis. Statistics and Computing 7: 1-10.
7. Cappé O., Robert C.P., and Rydén T. 2003. Reversible jump, birth-and-death and more general continuous time Markov Chain Monte Carlo samplers. Journal of the Royal Statistical Society, B 65: 679-700.
8. Celeux G., Hurn M., and Robert C.P. 2000. Computational and inferential difficulties with mixture posterior distributions. Journal of the American Statistical Association 95(451): 957-970.
9. Dempster A.P., Laird N.M., and Rubin D.B. 1977. Maximum likelihood from incomplete data via the EM algorithm. Journal of the Royal Statistical Society Series B 39(1): 1-38.
10. Diebolt J. and Robert C.P., 1994, Estimation of finite mixture distribution through Bayesian sampling. Journal of the Royal Statistical Society Series B 56(2): 363-375.
11. Gilks W.R., Richardson S., and Spiegelhalter D.J. 1996. Markov Chain Monte Carlo in practice. Chapman and Hall, London.
12. Golub G.H. and VanLoan, C.F. 1996. Matrix Computations, 3rd edition. The Johns Hopkins University Press, Baltimore.
13. Green P.J. 1994. Discussion on representations of knowledge in complex systems (by U. Grenander). Journal of the Royal Statistial Society Series B 56: 589-590.
14. Green P.J. 1995. Reversible jump Markov Chain Monte Carlo computation and Bayesian model determination. Biometrika 82: 711-732.
15. Grenander U. and Miller M.I. 1994. Representations of knowledge in complex systems (with discussion). Journal of the Royal Statistical Society Series B 56: 549-603.
16. Hastie T. and Tibshirani R. 1996. Discriminant analysis by Gaussian mixtures. Journal of the Royal Statistical Society Series B 58: 155-176.
17. Holmes C.C. and Mallick B.K. 1998. Bayesian radial basis functions of variable dimension. Neural Computation 10: 1217-1233.
18. Larocque J.R. and Reilly J.P. 2002. Reversible jump MCMC for joint detection and estimation of sources in colored noise. IEEE Transactions on Signal Processing 50(2): 231-240.
19. McLachlan G. and Krishnan T. 1997. The EM Algorithm and Extensions. John Wiley and Sons, New York.
20. McLachlan G. and Peel D. 2000. Finite Mixture Models. John Wiley and Sons, New York.
21. Redner R.A. and Walker H.F. 1984. Mixture densities, maximum likelihood and the EM algorithm. SIAM Review 26(2): 195-239.
22. Richardson S. and Green P.J. 1997. On Bayesian analysis of mixtures with an unknown number of components (with discussion). Journal of the Royal Statistical Society Series B 59: 731-792.
23. Richardson S. and Green P.J. 1998. Corrigendum: On Bayesian analysis of mixtures with an unknown number of components (with discussion). Journal of the Royal Statistical Society Series B 60: 661.
24. Robert C.P. 1996. Mixtures of distributions: Inference and estimation. In Gilks W.R., Richardson S., and Spiegelhalter D.J. (eds.), Markov Chain Monte Carlo in Practice. Chapman and Hall, London. Chapt. 24, pp. 441-464.
25. Robert C.P., Rydén T., and Titterington D.M. 2000. Bayesian inference in hidden Markov models through the reversible jump Markov Chain Monte Carlo method. Journal of the Royal Statistical Society Series B 62: 57-75.
26. Sato M. 2001. On-line model selection based on the variational Bayes. Neural Computation 13(7): 1649-1681.
27. Stephens M. 2000a. Bayesian analysis of mixtures with an unknown number of components - An llternative to reversible jump methods. Annals of Statistics 28: 40-74.
28. Stephens M. 2000b. Dealing with label switching in mixture models. Journal of the Royal Statistical Society Series B 62: 795-809.
29. Titterington D.M., Smith A.F.M., and Makov U.E. 1985. Statistical Analysis of Finite Mixture Distributions, Wiley Series in Probability and Mathematical Statistics. Wiley, New York.
30. Zhang Z., Chen C., Sun J., and Chan K.L. 2003. EM algorithms for Gaussian mixtures with split-and-merge operation. Pattern Recognition 36: 1973-1983.