Learning to draw samples with amortized stein variational gradient descent

Uncertainty in Artificial Intelligence - Proceedings of the 33rd Conference, UAI 2017

Volumn , Issue , 2017, Pages

  Feng, Yihao ^a   Wang, Dilin ^a   Liu, Qiang ^a  

^a Dartmouth College   (United States)

Author keywords

[No Author keywords available]

Indexed keywords

ARTIFICIAL INTELLIGENCE; BAYESIAN NETWORKS; INFERENCE ENGINES; NEURAL NETWORKS; PROBABILITY DISTRIBUTIONS; STOCHASTIC SYSTEMS;

BAYESIAN INFERENCE; GRADIENT DESCENT; GRADIENT DIRECTION; LATENT SPACE STRUCTURE; NEURAL NETWORK PARAMETERS; PROBABILISTIC INFERENCE; STOCHASTIC NEURAL NETWORK; UNNORMALIZED DENSITY;

ITERATIVE METHODS;

EID: 85031125843     PISSN: None     EISSN: None     Source Type: Conference Proceeding    
DOI: None     Document Type: Conference Paper

References (32)

1. Agakov, Felix V and Barber, David. An auxiliary variational method. In International Conference on Neural Information Processing, pp. 561-566. Springer, 2004.
2. Andrieu, Christophe and Thoms, Johannes. A tutorial on adaptive mcmc. Statistics and Computing, 18(4): 343-373, 2008.
3. Balan, Anoop Korattikara, Rathod, Vivek, Murphy, Kevin P, and Welling, Max. Bayesian dark knowledge. In Advances in Neural Information Processing Systems, pp. 3438-3446, 2015.
4. Briol, François-Xavier, Oates, Chris, Girolami, Mark, Osborne, Michael A, Sejdinovic, Dino, et al. Probabilistic integration: A role for statisticians in numerical analysis? arXiv preprint http://arxiv.org/abs/1512.00933, 2015.
5. Burda, Yuri, Grosse, Roger, and Salakhutdinov, Ruslan. Importance weighted autoencoders. arXiv preprint arXiv: 1509.00519, 2015.
6. Chwialkowski, Kacper, Strathmann, Heiko, and Gretton, Arthur. A kernel test of goodness of fit. In Proceedings of the International Conference on Machine Learning (ICML), 2016.
7. Delyon, Bernard and Portier, François. Integral approximation by kernel smoothing. arXiv preprint arXiv: 1409.0733, 2014.
8. Gershman, Samuel J and Goodman, Noah D. Amortized inference in probabilistic reasoning. In Proceedings of the 36th Annual Conference of the Cognitive Science Society, 2014.
9. Goodfellow, Ian, Pouget-Abadie, Jean, Mirza, Mehdi, Xu, Bing, Warde-Farley, David, Ozair, Sherjil, Courville, Aaron, and Bengio, Yoshua. Generative adversarial nets. In Advances in Neural Information Processing Systems, pp. 2672-2680, 2014.
10. Henmi, Masayuki, Yoshida, Ryo, and Eguchi, Shinto. Importance sampling via the estimated sampler. Biometrika, 94(4): 985-991, 2007.
11. Hoffman, Matthew D, Blei, David M, Wang, Chong, and Paisley, John. Stochastic variational inference. JMLR, 2013.
12. Kingma, Diederik P and Welling, Max. Auto-encoding variational Bayes. In Proceedings of the International Conference on Learning Representations (ICLR), 2013.
13. Kingma, Diederik P, Salimans, Tim, andWelling, Max. Improving variational inference with inverse autoregressive flow. arXiv preprint arXiv: 1606.04934, 2016.
14. Li, Yingzhen, Turner, Richard E, and Liu, Qiang. Approximate inference with amortised mcmc. arXiv preprint arXiv: 1702.08343, 2017.
15. Liu, Qiang and Lee, Jason D. Black-box importance sampling. https://arxiv.org/abs/1610.05247, 2016.
16. Liu, Qiang and Wang, Dilin. Stein variational gradient descent: A general purpose Bayesian inference algorithm. arXiv preprint arXiv: 1608.04471, 2016.
17. Liu, Qiang, Lee, Jason D, and Jordan, Michael I. A kernelized Stein discrepancy for goodness-of-fit tests. In Proceedings of the International Conference on Machine Learning (ICML), 2016.
18. Maclaurin, Dougal, Duvenaud, David, and Adams, Ryan P. Early stopping is nonparametric variational inference. arXiv preprint arXiv: 1504.01344, 2015.
19. Mandt, Stephan, Hoffman, Matthew D, and Blei, David M. A variational analysis of stochastic gradient algorithms. arXiv preprint arXiv: 1602.02666, 2016.
20. Mescheder, Lars, Nowozin, Sebastian, and Geiger, Andreas. Adversarial variational bayes: Unifying variational autoencoders and generative adversarial networks. arXiv preprint arXiv: 1701.04722, 2017.
21. Mohamed, Shakir and Lakshminarayanan, Balaji. Learning in implicit generative models. arXiv preprint arXiv: 1610.03483, 2016.
22. Oates, Chris J, Girolami, Mark, and Chopin, Nicolas. Control functionals for Monte Carlo integration. Journal of the Royal Statistical Society, Series B, 2017.
23. O'Hagan, Anthony. Monte Carlo is fundamentally unsound. Journal of the Royal Statistical Society. Series D (The Statistician), 36(2/3): 247-249, 1987.
24. O'Hagan, Anthony. Bayes-hermite quadrature. Journal of statistical planning and inference, 29(3): 245-260, 1991.
25. Paige, Brooks andWood, Frank. Inference networks for sequential monte carlo in graphical models. arXiv preprint arXiv: 1602.06701, 2016.
26. Ranganath, R., Altosaar, J., Tran, D., and Blei, D.M. Operator variational inference. 2016.
27. Rezende, Danilo Jimenez and Mohamed, Shakir. Variational inference with normalizing flows. In Proceedings of the International Conference on Machine Learning (ICML), 2015.
28. Roberts, Gareth O and Rosenthal, Jeffrey S. Examples of adaptive mcmc. Journal of Computational and Graphical Statistics, 18(2): 349-367, 2009.
29. Salimans, Tim et al. Markov chain monte carlo and variational inference: Bridging the gap. In International Conference on Machine Learning, 2015.
30. Snelson, Edward and Ghahramani, Zoubin. Compact approximations to Bayesian predictive distributions. In Proceedings of the 22nd international conference on Machine learning, pp. 840-847. ACM, 2005.
31. Welling, Max and Teh, Yee W. Bayesian learning via stochastic gradient Langevin dynamics. In Proceedings of the International Conference on Machine Learning (ICML), 2011.
32. Wu, Yuhuai, Burda, Yuri, Salakhutdinov, Ruslan, and Grosse, Roger. On the quantitative analysis of decoder-based generative models. arXiv preprint arXiv: 1611.04273, 2016.