Bayesian model search for mixture models based on optimizing variational bounds

Neural Networks

Volumn 15, Issue 10, 2002, Pages 1223-1241

  Ueda, Naonori ^a   Ghahramani, Zoubin ^b  

^a NTT Communication Science Laboratories   (Japan)

^b UNIVERSITY COLLEGE LONDON   (United Kingdom)

Author keywords

Bayesian model search; EM algorithm; Mixture models; Mixture of experts models; Variational Bayes

Indexed keywords

ALGORITHMS; OPTIMIZATION; VARIATIONAL TECHNIQUES;

VARIATIONAL BOUNDS;

NEURAL NETWORKS;

ALGORITHM; ANALYTIC METHOD; ARTICLE; BAYES THEOREM; CALCULATION; MODEL; PRIORITY JOURNAL; PROBABILITY;

ALGORITHMS; BAYES THEOREM; NEURAL NETWORKS (COMPUTER);

EID: 0036887504     PISSN: 08936080     EISSN: None     Source Type: Journal    
DOI: 10.1016/S0893-6080(02)00040-0     Document Type: Article

References (25)

1. Akaike H. A new look at the statistical model identification. IEEE Transactions on AC. 19-6:1974;716-723.
2. Attias H. (1999). Learning parameters and structure of latent variable models by variational Bayes. Proceedings of Uncertainty in Artificial Intelligence (UAI).
3. Bernardo J.M., Smith A.F.M. Bayesian theory. 1994;Wiley, New York.
4. Bishop C.M., Bayesian P.C.A. Advances in neural information processing systems. MIT Press. NIPS11:1999;382-388.
5. Dempster A.P., Laird N.M., Rubin D.B. Maximum likelihood from incomplete data via the EM algorithm. Journal of the Royal Statistical Society, Series B. 39:1977;1-38.
6. Gamerman D. Markov chain Monte Carlo. 1997;Chapman & Hall, London.
7. Ghahramani Z., Beal M.J. Variational inference for Bayesian mixture of factor analyzers. Advances in Neural Information Processing Systems. NIPS12:2000;449-455. MIT Press.
8. Ghahramani Z., Beal M.J. Propagation algorithm for variational Bayesian learning. Advances in Neural Information Processing Systems. NIPS13:2001;507-513. MIT Press.
9. Jaakkola, T. (1997). Variational methods for inference and learning in graphical models. PhD thesis, MIT Press.
10. Jebara T., Pentland A. Maximum conditional likelihood via bound maximization and the CEM algorithm. Advances in Neural Information Processing Systems. NIPS11:2000;494-500.
11. Jebara T., Pentland A. On reversing Jensen's inequality. Advances in Neural Information Processing Systems. NIPS13:2002;. in press.
12. MacKay D.J.C. Bayesian interpolation. Neural Computation. 4:1992;405-447.
13. MacKay D.J.C. A practical Bayesian framework for backpropagation networks. Neural Computation. 4:1992;448-472.
14. MacKay D.J.C. Bayesian non-linear modeling for the prediction competition. ASHRAE Transactions. 100:1994;1053-1062.
15. Neal R.M. Lecture Notes in Statistics. Lecture Notes in Statistics. Vol. 118:1996;Springer, Berlin.
16. Rasmussen, C. E., Neal, R. M., Hinton, G. E., Camp, D. van, Revow, M., Ghahramani, Z., Kustra, R., & Tibshirani, R. (1996). The DELVE Manual. http://www.cs.utoronto.ca/~delve/.
17. Seber G.A.F., Wild C.J. Nonlinear regression. 1989;Wiley, New York.
18. Stone M. Cross-validatory choice and assessment of statistical predictions. Journal of the Royal Statistical Society, Series B. 39:1974;44-47.
19. Takeuchi K. On the selection of statistical models by AIC. Journal of the Society of Instrument and Control Engineering. 22:(5):1983;445-453. in Japanese.
20. Tipping M.E. The relevance vector machine. Advances in Neural Information Processing Systems. NIPS12:2000;652-658.
21. Ueda N. Variational Bayesian learning for optimal model search. Journal of Japanese Society for Artificial Intelligence. 16:(2):2000;SP-F. in Japanese.
22. Ueda N., Nakano R., Ghahramani Z., Hinton G.E. SMEM algorithm for mixture models. Advances in Neural Information Processing Systems. NIPS11:1999;599-605.
23. Ueda N., Nakano R., Ghahramani Z., Hinton G.E. SMEM algorithm for mixture models. Neural Computation. 12:(9):2000;2109-2128.
24. Waterhouse S.R., MacKay D., Robinson A.J. Bayesian methods for mixture of experts. Advances in Neural Information Processing Systems. NIPS8:1995;351-357.
25. Xu L., Jordan M.I., Hinton G.E. An alternative model for mixtures of experts. Advances in Neural Information Processing Systems. NIPS7:1994;633-640.