Iterative refinement of the approximate posterior for directed belief networks

Advances in Neural Information Processing Systems

Volumn , Issue , 2016, Pages 4698-4706

  Hjelm, R Devon ^a   Cho, Kyunghyun ^b   Chung, Junyoung ^c   Salakhutdinov, Russ ^d   Calhoun, Vince ^a   Jojic, Nebojsa ^e  

^a UNIVERSITY OF NEW MEXICO   (United States)

^b NEW YORK UNIVERSITY   (United States)

^c UNIVERSITÉ DE MONTRÉAL   (Canada)

^d CARNEGIE MELLON UNIVERSITY   (United States)

^e MICROSOFT RESEARCH   (United States)

Author keywords

[No Author keywords available]

Indexed keywords

GENERATIVE MODEL; GRADIENT ESTIMATES; GRAPHICAL MODEL; ITERATIVE REFINEMENT; MONTE CARLO ESTIMATES; SAMPLE SIZES; STATE-OF-THE-ART METHODS; VARIATIONAL METHODS;

ITERATIVE METHODS;

EID: 85018918401     PISSN: 10495258     EISSN: None     Source Type: Conference Proceeding    
DOI: None     Document Type: Conference Paper

References (26)

1. Jørg Bornschein and Yoshua Bengio. Reweighted wake-sleep. arXiv preprint arXiv:1406.2751, 2014.
2. Yuri Burda, Roger Grosse, and Ruslan Salakhutdinov. Importance weighted autoencoders. arXiv preprint arXiv:1509.00519, 2015.
3. Peter Dayan, Geoffrey E Hinton, Radford M Neal, and Richard S Zemel. The helmholtz machine. Neural computation, 7(5):889-904, 1995.
4. Arthur P Dempster, Nan M Laird, and Donald B Rubin. Maximum likelihood from incomplete data via the em algorithm. Journal of the royal statistical society. Series B (methodological), 1977.
5. Arnaud Doucet, Nando De Freitas, and Neil Gordon. An introduction to sequential monte carlo methods. In Sequential Monte Carlo methods in practice, pages 3-14. Springer, 2001.
6. Neil J Gordon, David J Salmond, and Adrian FM Smith. Novel approach to nonlinear/non-gaussian bayesian state estimation. In Radar and Signal Processing, IEE Proceedings F, Volume 140, pages 107-113. IET, 1993.
7. Karol Gregor, Ivo Danihelka, Andriy Mnih, Charles Blundell, and Daan Wierstra. Deep autoregressive networks. arXiv preprint arXiv:1310.8499, 2013.
8. Shixiang Gu, Zoubin Ghahramani, and Richard E Turner. Neural adaptive sequential monte carlo. In Advances in Neural Information Processing Systems, pages 2611-2619, 2015.
9. Shixiang Gu, Sergey Levine, Ilya Sutskever, and Andriy Mnih. Muprop: Unbiased backpropagation for stochastic neural networks. arXiv preprint arXiv:1511.05176, 2015.
10. Geoffrey Hinton. Neural networks for machine learning. Coursera, video lectures, 2012.
11. Geoffrey E Hinton, Simon Osindero, and Yee-Whye Teh. A fast learning algorithm for deep belief nets. Neural computation, 18(7):1527-1554, 2006.
12. Diederik Kingma and Max Welling. Auto-encoding variational bayes. arXiv preprint arXiv:1312.6114, 2013.
13. Andriy Mnih and Karol Gregor. Neural variational inference and learning in belief networks. In Proceedings of the 31st International Conference on Machine Learning (ICML-14), pages 1791-1799, 2014.
14. Andriy Mnih and Danilo J Rezende. Variational inference for monte carlo objectives. arXiv preprint arXiv:1602.06725, 2016.
15. Radford M Neal. Connectionist learning of belief networks. Artificial intelligence, 56(1), 1992.
16. Radford M Neal and Geoffrey E Hinton. A view of the em algorithm that justifies incremental, sparse, and other variants. In Learning in graphical models, pages 355-368. Springer, 1998.
17. Man-Suk Oh and James O Berger. Adaptive importance sampling in monte carlo integration. Journal of Statistical Computation and Simulation, 41(3-4):143-168, 1992.
18. Brooks Paige and Frank Wood. Inference networks for sequential monte carlo in graphical models. arXiv preprint arXiv:1602.06701, 2016.
19. Tapani Raiko, Mathias Berglund, Guillaume Alain, and Laurent Dinh. Techniques for learning binary stochastic feedforward neural networks. arXiv preprint arXiv:1406.2989, 2014.
20. Danilo J Rezende, Shakir Mohamed, and Daan Wierstra. Stochastic backpropagation and approximate inference in deep generative models. In Proceedings of the 31st International Conference on Machine Learning (ICML-14), pages 1278-1286, 2014.
21. Danilo Jimenez Rezende and Shakir Mohamed. Variational inference with normalizing flows. arXiv preprint arXiv:1505.05770, 2015.
22. Ruslan Salakhutdinov and Hugo Larochelle. Efficient learning of deep boltzmann machines. In International Conference on Artificial Intelligence and Statistics, pages 693-700, 2010.
23. Ruslan Salakhutdinov and Iain Murray. On the quantitative analysis of deep belief networks. In Proceedings of the 25th international conference on Machine learning, pages 872-879. ACM, 2008.
24. Tim Salimans, Diederik Kingma, and Max Welling. Markov chain monte carlo and variational inference: Bridging the gap. In David Blei and Francis Bach, editors, Proceedings of the 32nd International Conference on Machine Learning (ICML-15), pages 1218-1226. JMLR Workshop and Conference Proceedings, 2015. URL http://jmlr.org/proceedings/papers/v37/salimans15.pdf.
25. Lawrence K Saul, Tommi Jaakkola, and Michael I Jordan. Mean field theory for sigmoid belief networks. Journal of artificial intelligence research, 4(1):61-76, 1996.
26. Yichuan Tang and Ruslan R Salakhutdinov. Learning stochastic feedforward neural networks. In Advances in Neural Information Processing Systems, pages 530-538, 2013.