Variational em algorithms for non-Gaussian latent variable models

Advances in Neural Information Processing Systems

Volumn , Issue , 2005, Pages 1059-1066

  Palmer, J A ^a   Wipf, D P ^a   Kreutz Delgado, K ^a   Rao, B D ^a  

^a UNIVERSITY OF CALIFORNIA   (United States)

Author keywords

[No Author keywords available]

Indexed keywords

BAYES METHOD; BOUNDING METHOD; ENSEMBLE LEARNING; LATENT VARIABLE; LATENT VARIABLE MODELS; NON-GAUSSIAN; VARIATIONAL EM;

EQUIVALENCE CLASSES; GAUSSIAN NOISE (ELECTRONIC);

ALGORITHMS;

EID: 84864068448     PISSN: 10495258     EISSN: None     Source Type: Conference Proceeding    
DOI: None     Document Type: Conference Paper

References (32)

1. D. F. Andrews and C. L. Mallows. Scale mixtures of normal distributions. J. Roy. Statist. Soc. Ser. B, 36:99-102, 1974.
2. H. Attias. Independent factor analysis. Neural Computation, 11:803-851, 1999.
3. H. Attias. A variational Bayesian framework for graphical models. In Advances in Neural Information Processing Systems 12. MIT Press, 2000.
4. M. J. Beal and Z. Ghahrarmani. The variational Bayesian EM algorithm for incomplete data: with application to scoring graphical model structures. In Bayesian Statistics 7, pages 453-464. University of Oxford Press, 2002.
5. A. Benveniste, M. Métivier, and P. Priouret. Adaptive algorithms and stochastic approximations. Springer-Verlag, 1990.
6. C. M. Bishop and M. E. Tipping. Variational relevance vector machines. In C. Boutilier and M. Goldszmidt, editors, Proceedings of the 16th Conference on Uncertainty in Artificial Intelligence, pages 46-53. Morgan Kaufmann, 2000.
7. S. Bochner. Harmonic analysis and the theory of probability. University of California Press, Berkeley and Los Angeles, 1960.
8. S. Boyd and L. Vandenberghe. Convex Optimization. Cambridge University Press, 2004.
9. A. P. Dempster, N. M. Laird, and D. B. Rubin. Maximum likelihood from incomplete data via the EM algorithm. Journal of the Royal Statistical Society, Series B, 39:1-38, 1977.
10. A. P. Dempster, N. M. Laird, and D. B. Rubin. Iteratively reweighted least squares for linear regression when errors are Normal/Independent distributed. In P. R. Krishnaiah, editor, Multivariate Analysis V, pages 35-57. North Holland Publishing Company, 1980.
11. M. Figueiredo. Adaptive sparseness using Jeffreys prior. In T. G. Dietterich, S. Becker, and Z. Ghahramani, editors, Advances in Neural Information Processing Systems 14, Cambridge, MA, 2002. MIT Press.
12. D. Geman and G. Reynolds. Constrained restoration and the recovery of discontinuities. IEEE Trans. Pattern Analysis and Machine Intelligence, 14(3):367-383, 1992.
13. Z. Ghahramani and M. J. Beal. Variational inference for Bayesian mixtures of factor analysers. In Advances in Neural Information Processing Systems 12. MIT Press, 2000.
14. M. Girolami. A variational method for learning sparse and overcomplete representations. Neural Computation, 13:2517-2532, 2001.
15. A. Honkela and H. Valpola. Unsupervised variational Bayesian learning of nonlinear models. In Advances in Neural Information Processing Systems 17. MIT Press, 2005.
16. T. S. Jaakkola. Variational Methods for Inference and Estimation in Graphical Models. PhD thesis, Massachusetts Institute of Technology, 1997.
17. T. S. Jaakkola and M. I. Jordan. A variational approach to Bayesian logistic regression models and their extensions. In Proceedings of the 1997 Conference on Artificial Intelligence and Statistics, 1997.
18. M. I. Jordan, Z. Ghahramani, T. S. Jaakkola, and L. K. Saul. An introduction to variational methods for graphical models. In M. I. Jordan, editor, Learning in Graphical Models. Kluwer Academic Publishers, 1998.
19. J. Keilson and F. W. Steutel. Mixtures of distributions, moment inequalities, and measures of exponentiality and Normality. The Annals of Probability, 2:112-130, 1974.
20. H. Lappalainen. Ensemble learning for independent component analysis. In Proceedings of the First International Workshop on Independent Component Analysis, 1999.
21. D. J. C. MacKay. Bayesian interpolation. Neural Computation, 4(3):415-447, 1992.
22. D. J. C. MacKay. Ensemble learning and evidence maximization. Unpublished manuscript, 1995.
23. D. J. C. Mackay. Comparison of approximate methods for handling hyperparameters. Neural Computation, 11(5):1035-1068, 1999.
24. R. M. Neal and G. E. Hinton. A view of the EM algorithm that justifies incremental, sparse, and other variants. In M. I. Jordan, editor, Learning in Graphical Models, pages 355-368. Kluwer, 1998.
25. B. D. Rao, K. Engan, S. F. Cotter, J. Palmer, and K. Kreutz-Delgado. Subset selection in noise based on diversity measure minimization. IEEE Trans. Signal Processing, 51(3), 2003.
26. B. D. Rao and I. F. Gorodnitsky. Sparse signal reconstruction from limited data using FOCUSS: a re-weighted minimum norm algorithm. IEEE Trans. Signal Processing, 45:600-616, 1997.
27. R. T. Rockafellar. Convex Analysis. Princeton, 1970.
28. Sam Roweis and Zoubin Ghahramani. A unifying review of linear gaussian models. Neural Computation, 11(5):305-345, 1999.
29. L. K. Saul, T. S. Jaakkola, and M. I. Jordan. Mean field theory for sigmoid belief networks. Journal of Artificial Intelligence Research, 4:61-76, 1996.
30. M. E. Tipping. Sparse Bayesian learning and the Relevance Vector Machine. Journal of Machine Learning Research, 1:211-244, 2001.
31. D. V. Widder. The Laplace Transform. Princeton University Press, 1946.
32. D. Wipf, J. Palmer, and B. Rao. Perspectives on sparse bayesian learning. In S. Thrun, L. Saul, and B. Schölkopf, editors, Advances in Neural Information Processing Systems 16, Cambridge, MA, 2003. MIT Press.