Improved variational inference with inverse autoregressive flow

Advances in Neural Information Processing Systems

Volumn , Issue , 2016, Pages 4743-4751

  Kingma, Diederik P ^d   Salimans, Tim ^d   Jozefowicz, Rafal ^d   Chen, Xi ^d   Sutskever, Ilya ^d   Welling, Max ^a,b,c  

^a UNIVERSITY OF AMSTERDAM   (Netherlands)

^b UNIVERSITY OF CALIFORNIA   (United States)

^c CANADIAN INSTITUTE FOR ADVANCED RESEARCH   (Canada)

^d NONE

Author keywords

[No Author keywords available]

Indexed keywords

AUTO-REGRESSIVE; AUTOREGRESSIVE NEURAL NETWORKS; HIGH-DIMENSIONAL; LATENT VARIABLE; LOG LIKELIHOOD; NATURAL IMAGES; NEURAL AUTOREGRESSIVE MODEL; VARIATIONAL INFERENCE;

EID: 85018920337     PISSN: 10495258     EISSN: None     Source Type: Conference Proceeding    
DOI: None     Document Type: Conference Paper

References (30)

1. Blei, D. M., and Jordan., M. I., and Paisley, J. W. (2012). Variational Bayesian inference with Stochastic Search. In Proceedings of the 29th International Conference on Machine Learning (ICML-12), pages 1367-1374.
2. Bowman, S. R., Vilnis, L., Vinyals, O., Dai, A. M., Jozefowicz, R., and Bengio, S. (2015). Generating sentences from a continuous space. arXiv preprint arXiv:1511.06349.
3. Burda, Y., Grosse, R., and Salakhutdinov, R. (2015). Importance weighted autoencoders. arXiv preprint arXiv:1509.00519.
4. Clevert, D.-A., Unterthiner, T., and Hochreiter, S. (2015). Fast and accurate deep network learning by Exponential Linear Units (ELUs). arXiv preprint arXiv:1511.07289.
5. Deco, G. and Brauer, W. (1995). Higher order statistical decorrelation without information loss. Advances in Neural Information Processing Systems, pages 247-254.
6. Dinh, L., Krueger, D., and Bengio, Y. (2014). Nice: non-linear independent components estimation. arXiv preprint arXiv:1410.8516.
7. Dinh, L., Sohl-Dickstein, J., and Bengio, S. (2016). Density estimation using Real NVP. arXiv preprint arXiv:1605.08803.
8. Germain, M., Gregor, K., Murray, I., and Larochelle, H. (2015). Made: Masked autoencoder for distribution estimation. arXiv preprint arXiv:1502.03509.
9. Gregor, K., Besse, F., Rezende, D. J., Danihelka, I., and Wierstra, D. (2016). Towards conceptual compression. arXiv preprint arXiv:1604.08772.
10. Gregor, K., Mnih, A., and Wierstra, D. (2013). Deep AutoRegressive Networks. arXiv preprint arXiv:1310.8499.
11. He, K., Zhang, X., Ren, S., and Sun, J. (2015). Deep residual learning for image recognition. arXiv preprint arXiv:1512.03385.
12. He, K., Zhang, X., Ren, S., and Sun, J. (2016). Identity mappings in deep residual networks. arXiv preprint arXiv:1603.05027.
13. Hoffman, M. D., and Blei., D. M., Wang, C., and Paisley, J. (2013). Stochastic variational inference. The Journal of Machine Learning Research, 14(1):1303-1347.
14. Jozefowicz, R., Zaremba, W., and Sutskever, I. (2015). An empirical exploration of recurrent network architectures. In Proceedings of the 32nd International Conference on Machine Learning (ICML-15), pages 2342-2350.
15. Kaae Sønderby, C., Raiko, T., Maaløe, L., Kaae Sønderby, S., and Winther, O. (2016). How to train deep variational autoencoders and probabilistic ladder networks. arXiv preprint arXiv:1602.02282.
16. Kingma, D. P. and Welling, M. (2013). Auto-encoding variational Bayes. Proceedings of the 2nd International Conference on Learning Representations.
17. Ranganath, R., Tran, D., and Blei, D. M. (2015). Hierarchical variational models. arXiv preprint arXiv:1511.02386.
18. Rezende, D. and Mohamed, S. (2015). Variational inference with normalizing flows. In Proceedings of The 32nd International Conference on Machine Learning, pages 1530-1538.
19. Rezende, D. J., Mohamed, S., and Wierstra, D. (2014). Stochastic backpropagation and approximate inference in deep generative models. In Proceedings of the 31st International Conference on Machine Learning (ICML-14), pages 1278-1286.
20. Salimans, T. (2016). A structured variational auto-encoder for learning deep hierarchies of sparse features. arXiv preprint arXiv:1602.08734.
21. Salimans, T. and Kingma, D. P. (2016). Weight normalization: A simple reparameterization to accelerate training of deep neural networks. arXiv preprint arXiv:1602.07868.
22. Salimans, T., and Kingma., D. P., and Welling, M. (2014). Markov chain Monte Carlo and variational inference: Bridging the gap. arXiv preprint arXiv:1410.6460.
23. Sohl-Dickstein, J., Weiss, E. A., Maheswaranathan, N., and Ganguli, S. (2015). Deep unsupervised learning using nonequilibrium thermodynamics. arXiv preprint arXiv:1503.03585.
24. Tran, D., Ranganath, R., and Blei, D. M. (2015). Variational gaussian process. arXiv preprint arXiv:1511.06499.
25. van den Oord, A., Dieleman, S., Zen, H., Simonyan, K., Vinyals, O., Graves, A., Kalchbrenner, N., Senior, A., and Kavukcuoglu, K. (2016a). Wavenet: A generative model for raw audio. arXiv preprint arXiv:1609.03499.
26. van den Oord, A., Kalchbrenner, N., and Kavukcuoglu, K. (2016b). Pixel recurrent neural networks. arXiv preprint arXiv:1601.06759.
27. van den Oord, A., Kalchbrenner, N., Vinyals, O., Espeholt, L., Graves, A., and Kavukcuoglu, K. (2016c). Conditional image generation with pixelcnn decoders. arXiv preprint arXiv:1606.05328.
28. van den Oord, A. and Schrauwen, B. (2014). Factoring variations in natural images with deep gaussian mixture models. In Advances in Neural Information Processing Systems, pages 3518-3526.
29. Zagoruyko, S. and Komodakis, N. (2016). Wide residual networks. arXiv preprint arXiv:1605.07146.
30. Zeiler, M. D., Krishnan, D., Taylor, G. W., and Fergus, R. (2010). Deconvolutional networks. In Computer Vision and Pattern Recognition (CVPR), 2010 IEEE Conference on, pages 2528-2535. IEEE.