Convergence properties of a general algorithm for calculating variational Bayesian estimates for a normal mixture model

Bayesian Analysis

Volumn 1, Issue 3, 2006, Pages 625-650

  Wang, Bo ^a   Titteringtony, D M ^b  

^a NEWCASTLE UNIVERSITY   (United Kingdom)

^b UNIVERSITY OF GLASGOW   (United Kingdom)

Author keywords

Laplace approximation; Local convergence; Mixture model; Variational bayes

Indexed keywords

EID: 33745841556     PISSN: 19360975     EISSN: 19316690     Source Type: Journal    
DOI: 10.1214/06-BA121     Document Type: Article

References (24)

1. Attias, H. (1999). "Inferring parameters and structure of latent variable models by variational Bayes." In Prade, H. and Laskey, K. (eds.), Proc. 15th Conference on Uncertainty in Artificial Intelligence, 21-30. Stockholm, Sweden: Morgan Kaufmann Publishers. 625, 629
2. Attias, H. (2000). "A variational Bayesian framework for graphical models." In Solla, S., Leen, T., and Muller, K.-R. (eds.), Advances in Neural Information Processing Systems 12, 209-215. Cambridge, MA: MIT Press. 625, 629
3. Beal, M. J. (2003). "Variational Algorithms for Approximate Bayesian Inference." Ph.D. thesis, University College London. 625
4. Bhatia, R. (1997). Matrix Analysis. New York: Springer-Verlag. 636, 637
5. Celeux, G., Hurn, M., and Robert, C. P. (2000). "Computational and inferential dif- ficulties with mixture posterior distributions." Journal of the American Statistical Association, 95(957-979). 625
6. Corduneanu, A. and Bishop, C. M. (2001). "Variational Bayesian model selection for mixture distributions." In Richardson, T. and Jaakkola, T. (eds.), Proceedings Eighth International Conference on Artificial Intelligence and Statistics, 27-34. Morgan Kauf- mann. 625, 629
7. Duda, R. O. and Hart, P. E. (1973). Pattern Classification and Scene Analysis. New York: John Wiley. 632
8. Evans, M. and Swartz, T. (2000). Approximating Integrals via Monte Carlo and Deter- ministic Methods. New York: Oxford University Press. 641, 644
9. Ghahramani, Z. and Beal, M. J. (2000). "Variational inference for Bayesian mixtures of factor analysers." In Solla, S., Leen, T., and Muller, K.-R. (eds.), Advances in Neural Information Processing Systems 12, 449-455. Cambridge, MA: MIT Press. 625
10. Ghahramani, Z. (2001). "Propagation algorithms for variational Bayesian learning." In Leen, T., Di- etterich, T., and Tresp, V. (eds.), Advances in Neural Information Processing Systems 13, 507-513. Cambridge, MA: MIT Press. 625
11. Hall, P., Humphreys, K., and Titterington, D. M. (2002). "On the adequacy of varia- tional lower bound functions for likelihood-based inference in Markovian models with missing values." Journal of the Royal Statistical Society Series B, 64: 549-564. 626
12. Humphreys, K. and Titterington, D. M. (2000). "Approximate Bayesian inference for simple mixtures." In Bethlehem, J. G. and van der Heijden, P. G. M. (eds.), COMP- STAT2000, 331-336. Heidelberg: Physica-Verlag. 625, 629
13. Jordan, M. I. (2004). "Graphical Models." Statistical Science, 19(1): 140-155. 625
14. MacKay, D. J. C. (1997). "Ensemble learning for hidden Markov models." Technical report, Cavendish Laboratory, University of Cambridge. 625
15. McLachlan, G. J. and Peel, D. (2000). Finite Mixture Models. New York: Wiley. 627
16. Penny, W. D. and Roberts, S. J. (2000). "Variational Bayes for 1-dimensional mixture models." Technical Report PARG-2000-01, Oxford University. 625, 629
17. Peters, B. C. and Walker, H. F. (1978). "An iterative procedure for obtaining maximum- likelihood estimates of the parameters for a mixture of normal distributions." SIAM J. Appl. Math., 35: 362-378. 627, 632, 635, 641, 647
18. Redner, R. A. and Walker, H. F. (1984). "Mixture densities, Maximum likelihood and the EM Algorithm." SIAM Review, 26: 195-239. 633
19. Salakhutdinov, R. and Roweis, S. (2003). "Adaptive overrelaxed bound optimization methods." In Fawcett, T. and Mishra, N. (eds.), Proceedings of the Twentieth Inter- national Conference on Machine Learning, 664-671. AAAI Press. 627
20. Titterington, D. M. (2004). "Bayesian methods for neural networks and related models." Statistical Science, 19(1): 128-139. 625
21. Ueda, N. and Ghahramani, Z. (2003). "Bayesian model search for mixture models based on optimizing variational bounds." Neural Networks, 15: 1223-1241. 625, 629
22. Wang, B. and Titterington, D. M. (2003). "Local convergence of variational Bayes estimators for mixing coefficients." Technical Report 03-4, University of Glasgow. Http://www.stats.gla.ac.uk/Research/TechRep2003/03-4.pdf. 627
23. Wang, B. (2004). "Lack of consistency of mean field and variational Bayes approximations for state space models." Neural Processing Letters, 20: 151-170. 626, 635
24. Wang, B. (2005). "Inadequacy of interval estimates corresponding to variational Bayesian approximations." In Cowell, R. G. and Ghahramani, Z. (eds.), Proceedings of the Tenth International Workshop on Artificial Intelligence and Statistics, Jan 6-8, 2005, Savannah Hotel, Barbados, 373-380. Society for Artificial Intelligence and Statistics. (Available electronically at http://www.gatsby.ucl.ac.uk/aistats/). 635