Variational approximations in Bayesian model selection for finite mixture distributions

Computational Statistics and Data Analysis

Volumn 51, Issue 11, 2007, Pages 5352-5367

  McGrory, C A ^a   Titterington, D M ^a  

^a UNIVERSITY OF GLASGOW   (United Kingdom)

Author keywords

Bayesian analysis; Deviance information criterion (DIC); Mixtures; Variational approximations

Indexed keywords

BAYESIAN NETWORKS; COMPUTATIONAL COMPLEXITY; COMPUTER SIMULATION; DATA REDUCTION; LEARNING SYSTEMS; MATHEMATICAL MODELS; MIXTURES; NEURAL NETWORKS;

BAYESIAN ANALYSIS; DEVIANCE INFORMATION CRITERION (DIC); VARIATIONAL APPROXIMATIONS;

APPROXIMATION THEORY;

EID: 34247869715     PISSN: 01679473     EISSN: None     Source Type: Journal    
DOI: 10.1016/j.csda.2006.07.020     Document Type: Article

References (35)

1. Andrieu C., de Freitas N., Doucet A., and Jordan M.I. An introduction to MCMC for machine learning. Mach. Learning 50 (2003) 5-43
2. Attias, H., 1999. Inferring parameters and structure of latent variable models by variational Bayes. In: Proceedings of Conference on Uncertainty in Artificial Intelligence.
3. Berg A., Meyer R., and Yu J. Deviance information criterion for comparing stochastic volatility models. J. Bus. Econ. Statist. 22 (2004) 107-119
4. Bishop, C.M., Winn, J., 2003. Structural variational distributions in VIBES, in: Bishop, C.M., Frey, B. (Eds.), Proceedings of Artificial Intelligence, Florida.
5. Celeux, G., Forbes, F., Robert, C., Titterington, D.M., 2006. Deviance information criteria for missing data models, Bayesian Anal., 1, to appear.
6. Corduneanu A., and Bishop C.M. Variational Bayesian model selection for mixture distributions. In: Jaakkola T., and Richardson T. (Eds). Artificial Intelligence and Statistics (2001), Morgan Kaufmann, Los Altos, CA 27-34
7. Dempster, A.P., 1974. The direct use of likelihood for significance testing, in: Proceedings of Conference on Foundational Questions in Statistical Inference, University of Aarhus, pp. 335-352.
8. Doucet, A., de Freitas, N., Gordon, N.J. (Eds.), 2001. Sequential Monte Carlo Methods in Practice, Springer, Berlin.
9. Gelfand A.E., and Smith A.F.M. Sampling-based approaches to calculating marginal densities. J. Amer. Statist. Assoc. 85 (1990) 398-409
10. Geyer C. Practical Markov chain Monte Carlo (with discussion). Statist. Sci. 7 (1992) 473-483
11. Ghahramani Z., and Beal M. Propagation algorithms for variational learning. In: Touretzky D.S., Mozer M.C., and Hasselmo M.E. (Eds). Advances in Neural Information Processing vol. 13 (2001), MIT Press, Cambridge, MA
12. Gilks W.R., Best N.G., and Tan K.K.C. Adaptive rejection Metropolis sampling within Gibbs sampling. J. Appl. Statist. 44 (1995) 455-472
13. Gilks, W.R., Richardson, S., Spiegelhalter, D.J. (Eds.), 1996. Markov Chain Monte Carlo in Practice, Chapman & Hall, London.
14. Green P.J. Reversible jump Markov chain Monte Carlo computation and Bayesian model determination. Biometrika 82 (1995) 711-732
15. Green P.J., and Richardson S. Hidden Markov models and disease mapping. J. Amer. Statist. Assoc. 97 (2002) 1055-1070
16. Han C., and Carlin B.P. Markov chain Monte Carlo methods for computing Bayes factors: a comparative review. J. Amer. Statist. Assoc. 96 (2001) 1122-1132
17. Jordan M.I. Graphical models. Statist. Sci. 19 (2004) 140-155
18. Jordan M.I., Ghahramani Z., Jaakkola T.S., and Saul L.K. An introduction to variational methods for graphical models. Mach. Learning 37 (1999) 183-233
19. McGrory, C.A., Titterington, D.M., 2006. Bayesian analysis of hidden Markov models using variational approximations. Submitted for publication.
20. Neal, R.M., 1996. Bayesian Learning for Neural Networks, Lecture Notes in Statistics, vol. 118, Springer, New York.
21. Postman M., Huchra J.P., and Geller M.J. Probes of large-scale structure in the Corona Borealis region. Astronomical J. 92 (1986) 1238-1247
22. Richardson S., and Green P.J. On Bayesian analysis of mixtures with an unknown number of components (with discussion). J. Roy. Statist. Soc. Ser. B 59 (1997) 731-792
23. Robert C.P., Rydén T., and Titterington D.M. Bayesian inference in hidden Markov models through the reversible jump Markov chain Monte Carlo method. J. Roy. Statist. Soc. Ser. B 62 (2000) 57-75
24. Roeder K. Density estimation with confidence sets exemplified by superclusters and voids in the galaxies. J. Amer. Statist. Assoc. 85 (1990) 617-624
25. Schwarz G. Estimating the dimension of a model. Ann. Statist. 6 (1978) 461-464
26. Spiegelhalter D.J., Best N.G., Carlin B.P., and Van der Linde A. Bayesian measures of model complexity and fit (with discussion). J. Roy. Statist. Soc. Ser. B 64 (2002) 583-639
27. Stephens M. Bayesian analysis of mixture models with an unknown number of components-an alternative to reversible jump methods. Ann. Statist. 28 (2000) 40-74
28. Tierney L. Markov chains for exploring posterior distributions. Ann. Statist. 22 (1994) 1701-1728
29. Titterington D.M. Bayesian methods for neural networks and related models. Statist. Sci. 19 (2004) 128-139
30. Titterington D.M., Smith A.F.M., and Makov U.E. Statistical Analysis of Finite Mixture Distributions (1985), Wiley, Chichester
31. Ueda N., and Ghahramani Z. Bayesian model search for mixture models based on optimizing variational bounds. Neural Networks 15 (2002) 1223-1241
32. Wang, B., Titterington, D.M., 2006. Convergence properties of a general algorithm for calculating variational Bayesian estimates for a normal mixture model. Bayesian Anal., 1, 625-650.
33. Waterhouse S., MacKay D., and Robinson T. Bayesian methods for mixtures of experts. In: Touretzky D.S., et al. (Ed). Advances in Neural Information Processing Systems vol. 8 (1996), MIT Press, Cambridge, MA
34. Winn J., and Bishop C.M. Variational message passing. J. Machine Learning Res. 6 (2005) 661-694
35. Zhu L., and Carlin B.P. Comparing hierarchical models for spatio-temporally misaligned data using the deviance information criterion. Statist. Med. 19 (2000) 2265-2278