Stein variational gradient descent: A general purpose Bayesian inference algorithm

Advances in Neural Information Processing Systems

Volumn , Issue , 2016, Pages 2378-2386

  Liu, Qiang ^a   Wang, Dilin ^a  

^a Dartmouth Department of Computer Science_Dartmouth College   (United States)

Author keywords

[No Author keywords available]

Indexed keywords

BAYESIAN NETWORKS; ITERATIVE METHODS; OPTIMIZATION;

BAYESIAN INFERENCE; EMPIRICAL STUDIES; FUNCTIONAL GRADIENT DESCENT; GRADIENT DESCENT; KL-DIVERGENCE; REAL-WORLD; STATE-OF-THE-ART METHODS; VARIATIONAL INFERENCE;

INFERENCE ENGINES;

EID: 85018878907     PISSN: 10495258     EISSN: None     Source Type: Conference Proceeding    
DOI: None     Document Type: Conference Paper

References (34)

1. M. D. Hoffman, D. M. Blei, C. Wang, and J. Paisley. Stochastic variational inference. JMLR, 2013.
2. M. Welling and Y. W. Teh. Bayesian learning via stochastic gradient Langevin dynamics. In ICML, 2011.
3. D. Maclaurin and R. P. Adams. Firefly Monte Carlo: Exact MCMC with subsets of data. In UAI, 2014.
4. R. Ranganath, S. Gerrish, and D. M. Blei. Black box variational inference. In AISTATS, 2014.
5. S. Gershman, M. Hoffman, and D. Blei. Nonparametric variational inference. In ICML, 2012.
6. A. Kucukelbir, R. Ranganath, A. Gelman, and D. Blei. Automatic variational inference in STAN. In NIPS, 2015.
7. B. Dai, N. He, H. Dai, and L. Song. Provable Bayesian inference via particle mirror descent. In AISTATS, 2016.
8. Q. Liu, J. D. Lee, and M. I. Jordan. A kernelized Stein discrepancy for goodness-of-fit tests and model evaluation. arXiv preprint arXiv:1602.03253, 2016.
9. C. J. Oates, M. Girolami, and N. Chopin. Control functionals for Monte Carlo integration. Journal of the Royal Statistical Society, Series B, 2017.
10. K. Chwialkowski, H. Strathmann, and A. Gretton. A kernel test of goodness-of-fit. arXiv preprint arXiv:1602.02964, 2016.
11. J. Gorham and L. Mackey. Measuring sample quality with Stein's method. In NIPS, pages 226-234, 2015.
12. C. Villani. Optimal transport: old and new, Volume 338. Springer Science & Business Media, 2008.
13. D. J. Rezende and S. Mohamed. Variational inference with normalizing flows. In ICML, 2015.
14. Y. Marzouk, T. Moselhy, M. Parno, and A. Spantini. An introduction to sampling via measure transport. arXiv preprint arXiv:1602.05023, 2016.
15. P. Del Moral. Mean field simulation for Monte Carlo integration. CRC Press, 2013.
16. M. Kac. Probability and related topics in physical sciences, Volume 1. American Mathematical Soc., 1959.
17. A. Rahimi and B. Recht. Random features for large-scale kernel machines. In NIPS, pages 1177-1184, 2007.
18. D. Tran, R. Ranganath, and D. M. Blei. Variational Gaussian process. In ICLR, 2016.
19. M. Titsias and M. Lázaro-Gredilla. Doubly stochastic variational Bayes for non-conjugate inference. In ICML, pages 1971-1979, 2014.
20. E. Challis and D. Barber. Affine independent variational inference. In NIPS, 2012.
21. S. Han, X. Liao, D. B. Dunson, and L. Carin. Variational Gaussian copula inference. In AISTATS, 2016.
22. D. Tran, D. M. Blei, and E. M. Airoldi. Copula variational inference. In NIPS, 2015.
23. C. M. B. N. Lawrence and T. J. M. I. Jordan. Approximating posterior distributions in belief networks using mixtures. In NIPS, 1998.
24. T. S. Jaakkola and M. I. Jordon. Improving the mean field approximation via the use of mixture distributions. In Learning in graphical models, pages 163-173. MIT Press, 1999.
25. N. D. Lawrence. Variational inference in probabilistic models. PhD thesis, University of Cambridge, 2001.
26. T. D. Kulkarni, A. Saeedi, and S. Gershman. Variational particle approximations. arXiv preprint arXiv:1402.5715, 2014.
27. C. Robert and G. Casella. Monte Carlo statistical methods. Springer Science & Business Media, 2013.
28. A. Smith, A. Doucet, N. de Freitas, and N. Gordon. Sequential Monte Carlo methods in practice. Springer Science & Business Media, 2013.
29. M. D. Hoffman and A. Gelman. The No-U-Turn sampler: Adaptively setting path lengths in Hamiltonian Monte Carlo. The Journal of Machine Learning Research, 15(1):1593-1623, 2014.
30. J. M. Hernández-Lobato and R. P. Adams. Probabilistic backpropagation for scalable learning of Bayesian neural networks. In ICML, 2015.
31. C. Stein, P. Diaconis, S. Holmes, G. Reinert, et al Use of exchangeable pairs in the analysis of simulations. In Stein's Method, pages 1-25. Institute of Mathematical Statistics, 2004.
32. Y. Li, J. M. Hernández-Lobato, and R. E. Turner. Stochastic expectation propagation. In NIPS, 2015.
33. Y. Li and R. E. Turner. Variational inference with Renyi divergence. arXiv preprint arXiv:1602.02311, 2016.
34. Y. Gal and Z. Ghahramani. Dropout as a Bayesian approximation: Representing model uncertainty in deep learning. arXiv preprint arXiv:1506.02142, 2015.