Variational inference with normalizing flows

32nd International Conference on Machine Learning, ICML 2015

Volumn 2, Issue , 2015, Pages 1530-1538

  Rezende, Danilo Jimenez ^a   Mohamed, Shakir ^a  

^a DEEPMIND   (United Kingdom)

Author keywords

[No Author keywords available]

Indexed keywords

ARTIFICIAL INTELLIGENCE;

CORE PROBLEMS; INITIAL DENSITY; MEAN FIELD; NEW APPROACHES; POSTERIOR DISTRIBUTIONS; VARIATIONAL APPROACHES; VARIATIONAL INFERENCE; VARIATIONAL METHODS;

LEARNING SYSTEMS;

EID: 84969776493     PISSN: None     EISSN: None     Source Type: Conference Proceeding    
DOI: None     Document Type: Conference Paper

References (37)

1. Ahn, S., Korattikara, A., and Welling, M. Bayesian posterior sampling via stochastic gradient Fisher scoring. In ICML, 2012.
2. Baird, L., Smalenberger, D., and Ingkiriwang, S. One-step neural network inversion with PDF learning and emulation. In IJCNN, volume 2, pp. 966-971. IEEE, 2005.
3. Bishop, C. M. Pattern recognition and machine learning. springer New York, 2006.
4. Challis, E. and Barber, D. Affine independent variational inference. In NIPS, 2012.
5. Dayan, P. Helmholtz machines and wake-sleep learning. Handbook of Brain Theory and Neural Network. MIT Press, Cambridge, MA, 44(0), 2000.
6. Dinh, L., Krueger, D., and Bengio, Y. NICE: Non-linear independent components estimation. arXiv preprint arXiv: 1410.8516, 2014.
7. Duchi, J., Hazan, E., and Singer, Y. Adaptive subgradient methods for online learning and stochastic optimization. Journal of Machine Learning Research, 12:2121-2159, 2010.
8. Genz, A. Methods for generating random orthogonal matrices. Monte Carlo and Quasi-Monte Carlo Methods, 1998.
9. Gershman, S., Hoffman, M., and Blei, D. Nonparametric variational inference. In ICML, 2012.
10. Gershman, S. J. and Goodman, N. D. Amortized inference in probabilistic reasoning. In Annual Conference of the Cognitive Science Society, 2014.
11. Goodfellow, I. J., Warde-Farley, D., Mirza, M., Courville, A., and Bengio, Y. Maxout networks. ICML, 2013.
12. Gregor, K., Mnih, A., and Wierstra, D. Deep autoregressive networks. In ICML, 2014.
13. Hoffman, M. D., Blei, D. M, Wang, C., and Paisley, J. Stochastic variational inference. The Journal of Machine Learning Research, 14(1):1303-1347, 2013.
14. Jaakkola, T. S. and Jordan, M. I. Improving the mean field approximation via the use of mixture distributions. In Learning in graphical models, pp. 163-173. 1998.
15. Jordan, M. I., Ghahramani, Z., Jaakkola, T. S., and Saul, L. K. An introduction to variational methods for graphical models. Machine learning, 37(2): 183-233, 1999.
16. Kingma, D. P. and Welling, M. Auto-encoding variational Bayes. In ICLR, 2014.
17. Kingma, D. P., Mohamed, S., Rezende, D. J., and Welling, M. Semi-supervised learning with deep generative models. In NIPS, pp. 3581-3589, 2014.
18. Krizhevsky, A. and Hinton, G. Convolutional deep belief networks on CIFAR-10. Unpublished manuscript, 2010.
19. LeCun, Y. and Cortes, C. The MNIST database of handwritten digits, 1998.
20. Mnih, A. and Gregor, K. Neural variational inference and learning in belief networks. In ICML, 2014.
21. Neal, R. M. MCMC using hamiltonian dynamics. Handbook of Markov Chain Monte Carlo, 2011.
22. Papaspiliopoulos, O., Roberts, G. O., and Sköld, M. Non-centered parameterisations for hierarchical models and data augmentation. In Bayesian Statistics 7, 2003.
23. Ranganath, R., Gerrish, S., and Blei, D. M. Black box variational inference. In AISTATS, 2013.
24. Rezende, D. J., Mohamed, S., and Wierstra, D. Stochastic backpropagation and approximate inference in deep generative models. In ICML, 2014.
25. Rippel, O. and Adams, R. P. High-dimensional probability estimation with deep density models. arXiv preprint arXiv: 1302.5125, 2013.
26. Salimans, T., Kingma, D. P., and Welling, M. Markov chain Monte Carlo and variational inference: Bridging the gap. In ICML, 2015.
27. Stuhlmüller, A., Taylor, J., and Goodman, N. Learning stochastic inverses. In MPS, pp. 3048-3056, 2013.
28. Suykens, J. A. K., Verrelst, H., and Vandewalle, J. Online learning Fokker-Planck machine. Neural processing letters, 7(2):81-89, 1998.
29. Tabak, E. G. and Turner, C. V. A family of nonparametric density estimation algorithms. Communications on Pure and Applied Mathematics, 66(2): 145-164, 2013.
30. Tabak, E. G and Vanden-Eijnden, E. Density estimation by dual ascent of the log-likelihood. Communications in Mathematical Sciences, 8(1):217-233, 2010.
31. Titsias, M. and Lazaro-Gredilla, M. Doubly stochastic variational Bayes for non-conjugate inference. In ICML, 2014.
32. Turner, R. E. and Sahani, M. Two problems with variational expectation maximisation for time-series models. In Barber, D., Cemgil, T., and Chiappa, S. (eds.), Bayesian Time series models, chapter 5, pp. 109-130. Cambridge University Press, 2011.
33. Uria, B., Murray, I., and Larochelle, H. A deep and tractable density estimator. In ICML, 2014.
34. Wang, B. and Titterington, D. M. Convergence and asymptotic normality of variational Bayesian approximations for exponential family models with missing values. In UAI, 2004.
35. Welling, M. and Teh, Y. W. Bayesian learning via stochastic gradient Langevin dynamics. In ICML, 2011.
36. Williams, Ronald J. Simple statistical gradient-following algorithms for connectionist reinforcement learning. Machine learning, 8(3-4):229-256, 1992.
37. Wingate, D. and Weber, T. Automated variational inference in probabilistic programming. In NIPS Workshop on Probabilistic Programming, 2013.