Linear response methods for accurate covariance estimates from mean field variational bayes

Advances in Neural Information Processing Systems

Volumn 2015-January, Issue , 2015, Pages 1441-1449

  Giordano, Ryan ^a   Broderick, Tamara ^b   Jordan, Michael ^a  

^a UNIVERSITY OF CALIFORNIA   (United States)

^b MASSACHUSETTS INSTITUTE OF TECHNOLOGY   (United States)

Author keywords

[No Author keywords available]

Indexed keywords

INFORMATION SCIENCE;

APPROXIMATION METHODS; CONJUGATE MODELS; EXPONENTIAL FAMILY; LARGE SCALE DATA SETS; LINEAR RESPONSE METHOD; STATISTICAL PHYSICS; UNCERTAINTY ESTIMATES; VARIATIONAL BAYES;

UNCERTAINTY ANALYSIS;

EID: 84965143892     PISSN: 10495258     EISSN: None     Source Type: Conference Proceeding    
DOI: None     Document Type: Conference Paper

References (24)

1. D. M. Blei, A. Y. Ng, and M. I. Jordan. Latent Dirichlet allocation. Journal of Machine Learning Research, 3:993-1022, 2003.
2. D. M. Blei and M. I. Jordan. Variational inference for Dirichlet process mixtures. Bayesian Analysis, 1(1):121-143, 2006.
3. M. D. Hoffman, D. M. Blei, C. Wang, and J. Paisley. Stochastic variational inference. Journal of Machine Learning Research, 14(1):1303-1347, 2013.
4. D. J. C. MacKay. Information Theory, Inference, and Learning Algorithms. Cambridge University Press, 2003. Chapter 33.
5. C. M. Bishop. Pattern Recognition and Machine Learning. Springer, New York, 2006. Chapter 10.
6. R. E. Turner and M. Sahani. Two problems with variational expectation maximisation for time-series models. In D. Barber, A. T. Cemgil, and S. Chiappa, editors, Bayesian Time Series Models. 2011.
7. B. Wang and M. Titterington. Inadequacy of interval estimates corresponding to variational Bayesian approximations. In Workshop on Artificial Intelligence and Statistics, pages 373-380, 2004.
8. H. Rue, S. Martino, and N. Chopin. Approximate Bayesian inference for latent Gaussian models by using integrated nested Laplace approximations. Journal of the Royal Statistical Society: Series B (statistical methodology), 71(2):319-392, 2009.
9. G. Parisi. Statistical Field Theory, volume 4. Addison-Wesley New York, 1988.
10. M. Opper and O. Winther. Variational linear response. In Advances in Neural Information Processing Systems, 2003.
11. M. Opper and D. Saad. Advanced mean field methods: Theory and practice. MIT press, 2001.
12. T. Tanaka. Information geometry of mean-field approximation. Neural Computation, 12(8):1951-1968, 2000.
13. H. J. Kappen and F. B. Rodriguez. Efficient learning in Boltzmann machines using linear response theory. Neural Computation, 10(5):1137-1156, 1998.
14. M. Welling and Y. W. Teh. Linear response algorithms for approximate inference in graphical models. Neural Computation, 16(1):197-221, 2004.
15. P. A. d. F. R. Højen-Sørensen, O. Winther, and L. K. Hansen. Mean-field approaches to independent component analysis. Neural Computation, 14(4):889-918, 2002.
16. T. Tanaka. Mean-field theory of Boltzmann machine learning. Physical Review E, 58(2):2302, 1998.
17. M. J. Wainwright and M. I. Jordan. Graphical models, exponential families, and variational inference. Foundations and Trends® in Machine Learning, 1(1-2):1-305, 2008.
18. J. D. Hadfield. MCMC methods for multi-response generalized linear mixed models: The MCMCglmm R package. Journal of Statistical Software, 33(2):1-22, 2010.
19. M. Lubin and I. Dunning. Computing in operations research using Julia. INFORMS Journal on Computing, 27(2):238-248, 2015.
20. D. Bates and D. Eddelbuettel. Fast and elegant numerical linear algebra using the RcppEigen package. Journal of Statistical Software, 52(5):1-24, 2013.
21. Y. LeCun, L. Bottou, Y. Bengio, and P. Haffner. Gradient-based learning applied to document recognition. Proceedings of the IEEE, 86(11):2278-2324, 1998.
22. M. Plummer, N. Best, K. Cowles, and K. Vines. CODA: Convergence diagnosis and output analysis for MCMC. R News, 6(1):7-11, 2006.
23. X. L. Meng and D. B. Rubin. Using EM to obtain asymptotic variance-covariance matrices: The SEM algorithm. Journal of the American Statistical Association, 86(416):899-909, 1991.
24. A. Wächter and L. T. Biegler. On the implementation of an interior-point filter line-search algorithm for large-scale nonlinear programming. Mathematical Programming, 106(1):25-57, 2006.