Stochastic complexities of Gaussian mixtures in variational Bayesian approximation

Journal of Machine Learning Research

Volumn 7, Issue , 2006, Pages 625-644

  Watanabe, Kazuho ^a   Watanabe, Sumio ^a  

^a TOKYO INSTITUTE OF TECHNOLOGY   (Japan)

Author keywords

Gaussian mixture model; Stochastic complexity; Variational Bayesian learning

Indexed keywords

APPROXIMATION THEORY; COMPUTATION THEORY; COMPUTATIONAL COMPLEXITY; DATA STRUCTURES; MATHEMATICAL MODELS; PROBLEM SOLVING; STOCHASTIC CONTROL SYSTEMS;

GAUSSIAN MIXTURE MODELS; STOCHASTIC COMPLEXITY; VARIATIONAL BAYESIAN LEARNING; VARIATIONAL FREE ENERGY;

LEARNING SYSTEMS;

EID: 33646348720     PISSN: 15337928     EISSN: 15337928     Source Type: Journal    
DOI: None     Document Type: Article

References (17)

1. H. Akaike, Likelihood and bayes procedure. In Bayesian Statistics, pages 143-166, Valencia, Spain, 1980, (Bernald J.M. eds.), University Press.
2. H. Alzer, On some inequalities for the Gamma and Psi functions. Mathematics of computation, 66 (217):373-389, 1997.
3. H. Attias. Inferring parameters and structure of latent variable models by variational bayes. In Proceedings of Uncertainty in Artificial Intelligence(UAI'99), 1999.
4. M. J. Beal. Variational Algorithms for approximate Bayesian inference. PhD thesis, University College London, 2003.
5. D. Dacunha-Castelle and E. Gassiat. Testing in locally conic models, and application to mixture models. Probability and Statistics, 1:285-317, 1997.
6. Z. Ghahramani and M, J, Beal. Graphical models and variational methods. Advanced Mean Field Methods - Theory and Practice, eds. D. Saad and M. Opper; MIT Press, 2000.
7. J. A. Hartigan, A failure of likelihood asymptotics for normal mixtures. In Proceedings of the Berkeley Conference in Honor of J.Neyman and J.Kiefer, pages 807-810, 1985.
8. E. Levin, N, Tishby, and S. A. Solla. A statistical approaches to learning and generalization in layered neural networks. Proc. of IEEE, 78(10): 1568-1674, 1990.
9. D. J. Mackay. Bayesian interpolation. Neural Computation, 4(2):415-447, 1992.
10. J. Rissanen. Stochastic complexity and modeling. Annals of Statistics, 14(3): 1080-1100, 1986.
11. M. Sato. Online model selection based on the variational bayes. Neural Computation, 13(7): 1649-1681, 2001.
12. G. Schwarz. Estimating the dimension of a model. Annals of Statistics, 6(2):461-464, 1978.
13. K. Watanabe and S. Watanabe. Lower bounds of stochastic complexities in variational bayes learning of gaussian mixture models. In Proceedings of IEEE conference on Cybernetics and Intelligent Systems (CIS04), pages 99-104, Singapore, 2004.
14. S. Watanabe. Algebraic analysis for non-identifiable learning machines. Neural Computation, 13 (4):899-933, 2001.
15. S. Watanabe, K. Yamazaki, and M. Aoyagi. Kullback information of normal mixture is not an analytic function. Technical Report of IEICE (in Japanese), NC2004-50:41-46, 2004.
16. K. Yamazaki and S. Watanabe. Singularities in mixture models and upper bounds of stochastic complexity. International Journal of Neural Networks, 16:1023-1038, 2003a.
17. K. Yamazaki and S. Watanabe. Stochastic complexity of bayesian networks. In Proceedings of Uncertainty in Artificial Intelligence (UAI'03), 2003b.