Neural variational inference and learning in belief networks

31st International Conference on Machine Learning, ICML 2014

Volumn 5, Issue , 2014, Pages 3800-3809

  Mnih, Andriy ^a   Gregor, Karol ^a  

^a DEEPMIND   (United Kingdom)

Author keywords

[No Author keywords available]

Indexed keywords

ARTIFICIAL INTELLIGENCE; LEARNING SYSTEMS;

APPROXIMATE INFERENCE; EXACT INFERENCE; EXACT SAMPLINGS; FEED-FORWARD NETWORK; INFERENCE NETWORK; LATENT VARIABLE MODELS; VARIANCE REDUCTION TECHNIQUES; VARIATIONAL INFERENCE;

ITERATIVE METHODS;

EID: 84919786239     PISSN: None     EISSN: None     Source Type: Conference Proceeding    
DOI: None     Document Type: Conference Paper

References (26)

1. Bradley, David M and Bagnell, J Andrew. Differential sparse coding. In Advances in Neural Information Processing Systems, volume 20, 2008.
2. Dayan, Peter and Hinton, Geoffrey E. Varieties of helmholtz machine. Neural Networks, 9(8): 1385-1403, 1996.
3. Dayan, Peter, Hinton, Geoffrey E, Neal, Radford M, and Zemel, Richard S. The helmholtz machine. Neural computation, 7(5):889-904, 1995.
4. Greensmith, Evan, Bartlett, Peter L., and Baxter, Jonathan. Variance reduction techniques for gradient estimates in reinforcement learning. Journal of Machine Learning Research, 5:1471-1530, 2004.
5. Gregor, Karol and LeCun, Yann. Learning fast approximations of sparse coding. In Proc. International Conference on Machine learning (ICML'10), 2010.
6. Gregor, Karol, Mnih, Andriy, and Wierstra, Daan. Deep autoregressive networks. arXiv preprint arXiv: 1310.8499, 2013.
7. Hinton, Geoffrey E and Zemel, Richard S. Autoencoders, minimum description length, and Helmholtz free energy. In Advances in Neural Information Processing Systems, 1994.
8. Hinton, Geoffrey E, Dayan, Peter, Frey, Brendan J, and Neal, Radford M. The "wake-sleep" algorithm for unsupervised neural networks. Science, 268(5214): 1158-1161, 1995.
9. Hinton, Geoffrey E., Osindero, Simon, and Teh, Yee Whye. A fast learning algorithm for deep belief nets. Neural Computation, 18(7): 1527-1554, 2006.
10. Jordan, Michael I., Ghahramani, Zoubin, Jaakkola, Tommi S., and Saul, Lawrence K. An introduction to variational methods for graphical models. Machine Learning, 37(2): 183-233, 1999.
11. Kavukcuoglu, Koray, Ranzato, Marc'Aurelio, and LeCun, Yann. Fast inference in sparse coding algorithms with applications to object recognition. Technical report, Courant Institute, NYU, 2008.
12. Kingma, Diederik P and Welling, Max. Auto-encoding variational bayes. arXiv preprint arXiv.1312.6114, 2013.
13. Larochelle, Hugo and Lauly, Stanislas. A neural autoregressive topic model. In Advances in Neural Information Processing Systems, pp. 2717-2725, 2012.
14. Larochelle, Hugo and Murray, Iain. The neural autoregressive distribution estimator. JMLR: W&CP, 15:29-37, 2011.
15. Neal, Radford M. Connectionist learning of belief networks. Artificial intelligence, 56(1):71-113, 1992.
16. Paisley, John William, Blei, David M., and Jordan, Michael I. Variational bayesian inference with stochastic search. In ICML, 2012.
17. Ranganath, Rajesh, Gerrish, Sean, and Blei, David M. Black box variational inference. arXiv preprint arXiv:1401.0118, 2013.
18. Rezende, Danilo Jimenez, Mohamed, Shakir, and Wierstra, Daan. Stochastic back-propagation and variational inference in deep latent gaussian models. arXiv preprint arXiv. 1401.4082, 2014.
19. Salakhutdinov, Ruslan and Hinton, Geoffrey E. Deep boltz-mann machines. In International Conference on Artificial Intelligence and Statistics, pp. 448-455, 2009a.
20. Salakhutdinov, Ruslan and Hinton, Geoffrey E. Replicated softmax: an undirected topic model. In Advances in neural information processing systems, 2009b.
21. Salakhutdinov, Ruslan and Larochelle, Hugo. Efficient learning of deep boltzmann machines. In International Conference on Artificial Intelligence and Statistics, pp. 693-700, 2010.
22. Salakhutdinov, Ruslan and Murray, Iain. On the quantitative analysis of Deep Belief Networks. In Proceedings of the 25th Annual International Conference on Machine Learning (ICML 2008), 2008.
23. Saul, Lawrence K., Jaakkola, Tommi, and Jordan, Michael I. Mean field theory for sigmoid belief networks. Journal of Artificial Intelligence Research, 4:61-76, 1996.
24. Tang, Yichuan and Salakhutdinov, Ruslan. Learning stochastic feedforward neural networks. In Advances in Neural Information Processing Systems, 2013.
25. Weaver, Lex and Tao, Nigel. The optimal reward baseline for gradient-based reinforcement learning. In In Proceedings of the Seventeenth Conference on Uncertainty in Artificial Intelligence, 2001.
26. Williams, Ronald J. Simple statistical gradient-following algorithms for connectionist reinforcement learning. Machine learning, 8(3-4):229-256, 1992.