Stochastic complexities of general mixture models in variational Bayesian learning

Neural Networks

Volumn 20, Issue 2, 2007, Pages 210-219

  Watanabe, Kazuho ^a   Watanabe, Sumio ^a  

^a Tokyo Institute of Technology ^*  (Japan)

Author keywords

Bayesian learning; Exponential family; Free energy; Kullback information; Mixture model; Non regular model; Stochastic complexity; Variational Bayes

Indexed keywords

ASYMPTOTIC STABILITY; COMPUTATIONAL COMPLEXITY; MATHEMATICAL MODELS; PARAMETER ESTIMATION; STATISTICAL METHODS; STOCHASTIC CONTROL SYSTEMS;

BAYESIAN LEARNING; EXPONENTIAL FAMILY DISTRIBUTIONS; KULLBACK INFORMATION; MIXTURE MODELS; NON REGULAR MODELS; STOCHASTIC COMPLEXITY; VARIATIONAL BAYES;

LEARNING SYSTEMS;

ARTICLE; ARTIFICIAL NEURAL NETWORK; BAYESIAN LEARNING; MONTE CARLO METHOD; PATTERN RECOGNITION; PRIORITY JOURNAL; PROBABILITY; STATISTICAL MODEL;

ANIMALS; ARTIFICIAL INTELLIGENCE; BAYES THEOREM; HUMANS; LEARNING; NEURAL NETWORKS (COMPUTER); STOCHASTIC PROCESSES;

EID: 33847097822     PISSN: 08936080     EISSN: None     Source Type: Journal    
DOI: 10.1016/j.neunet.2006.05.030     Document Type: Article

References (22)

1. Akaho S., and Kappen H.J. Nonmonotonic generalization bias of Gaussian mixture models. Neural Computation 12 4 (2000) 1411-1427
2. Akaike H. Likelihood and Bayes procedure. In: Bernald J.M. (Ed). Bayesian statistics (1980), University Press, Valencia, Spain 143-166
3. Alzer H. On some inequalities for the Gamma and Psi functions. Mathematics of Computation 66 217 (1997) 373-389
4. Attias, H. (1999). Inferring parameters and structure of latent variable models by variational Bayes. In Proceedings of uncertainty in artificial intelligence (pp. 21-30)
5. Beal, M. J. (2003). Variational algorithms for approximate Bayesian inference. Ph.D. thesis. University College London
6. Brown, L. D. (1986). Fundamentals of statistical exponential families. IMS Lecture Notes-Monograph Series: Vol. 9
7. Ghahramani Z., and Beal M.J. Graphical models and variational methods. In: Saad D., and Opper M. (Eds). Advanced mean field methods-Theory and practice (2000), MIT Press
8. Hartigan, J. A. (1985). A failure of likelihood asymptotics for normal mixtures. In Proceedings of the Berkeley conference in honor of J. Neyman and J. Kiefer (pp. 807-810)
9. Liu X., Pasarica C., and Shao Y. Testing homogeneity in gamma mixture models. Scandinavian Journal of Statistics 30 1 (2003) 227-239
10. Mackay D.J. Bayesian interpolation. Neural Computation 4 2 (1992) 415-447
11. McLachlan G., and Peel D. Finite mixture models (2000), Wiley, New York
12. Rissanen J. Stochastic complexity and modeling. Annals of Statistics 14 3 (1986) 1080-1100
13. Sato M. Online model selection based on the variational Bayes. Neural Computation 13 7 (2001) 1649-1681
14. Schwarz G. Estimating the dimension of a model. Annals of Statistics 6 2 (1978) 461-464
15. Ueda N., and Ghahramani Z. Bayesian model search for mixture models based on optimizing variational bounds. International Journal of Neural Networks 15 (2002) 1223-1241
16. Watanabe, K., & Watanabe, S. (2005). Stochastic complexity for mixture of exponential families in variational Bayes. In Proceedings of the 16th international conference on algorithmic learning theory (pp. 107-121)
17. Watanabe K., and Watanabe S. Stochastic complexities of Gaussian mixtures in variational Bayesian approximation. Journal of Machine Learning Research 7 4 (2006) 625-644
18. Watanabe K., and Watanabe S. Variational Bayesian stochastic complexity of mixture models. Advances in Neural Information Processing Systems 18 (2006) 1465-1472
19. Watanabe S. Algebraic analysis for non-identifiable learning machines. Neural Computation 13 4 (2001) 899-933
20. Yamazaki, K., Nagata, K., & Watanabe, S. (2005). A new method of model selection based on learning coefficient. In Proceedings of international symposium on nonlinear theory and its applications (pp. 389-392)
21. Yamazaki K., and Watanabe S. Singularities in mixture models and upper bounds of stochastic complexity. International Journal of Neural Networks 16 (2003) 1023-1038
22. Yamazaki, K., & Watanabe, S. (2003b). Stochastic complexity of Bayesian networks. In Proceedings of uncertainty in artificial intelligence (pp. 592-599)