Deep autoregressive networks

31st International Conference on Machine Learning, ICML 2014

Volumn 4, Issue , 2014, Pages 2991-3000

  Gregor, Karol ^a   Danihelka, Ivo ^a   Mnih, Andriy ^a   Blundell, Charles ^a   Wierstra, Daan ^a  

^a DEEPMIND   (United Kingdom)

Author keywords

[No Author keywords available]

Indexed keywords

ARTIFICIAL INTELLIGENCE; FEEDFORWARD NEURAL NETWORKS; STOCHASTIC MODELS; STOCHASTIC SYSTEMS;

APPROXIMATE INFERENCE; AUTO ENCODERS; AUTO-REGRESSIVE; HIDDEN LAYERS; LOG LIKELIHOOD; LOWER BOUNDS; MINIMUM DESCRIPTION LENGTH PRINCIPLE; PARAMETER ESTIMATION METHOD;

LEARNING SYSTEMS;

EID: 84919796355     PISSN: None     EISSN: None     Source Type: Conference Proceeding    
DOI: None     Document Type: Conference Paper

References (28)

1. Andradottir, Sigrun. A review of simulation optimization techniques. In Simulation Conference Proceedings, 1998. Winter, volume 1, pp. 151-158. IEEE, 1998.
2. Bache, K. and Lichman, M. UCI machine learning repository, 2013.URL http://archive.ics.uci.edu/ml.
3. Bellemare, M. G., Naddaf, Y., Veness, J., and Bowling, M. The arcade learning environment: An evaluation platform for general agents. Journal of Artificial Intelligence Research, 47:253-279, 06 2013.
4. Bengio, Yoshua, Leonard, Nicholas, and Courville, Aaron. Estimating or propagating gradients through stochastic neurons for conditional computation. arXiv preprint arXiv: 1308.3432, 2013a.
5. Bengio, Yoshua, Yao, Li, Alain, Guillaume, and Vincent, Pascal. Generalized denoising autoencoders as generative models. arXiv preprint arXiv: 1305.6663, 2013b.
6. Frey, Brendam J. Graphical models for machine learning and digital communication. The MIT press, 1998.
7. Graves, Alex. Practical variational inference for neural networks. In Advances in Neural Information Processing Systems, pp. 2348-2356, 2011.
8. Graves, Alex. Generating sequences with recurrent neural networks. arXiv preprint arXiv: 1308.0850, 2013.
9. Gregor, Karol and LeCun, Yann. Emergence of complexlike cells in a temporal product network with local receptive fields. arXiv preprint arXiv: 1006.0448, 2010.
10. Gregor, Karol and LeCun, Yann. Learning representations by maximizing compression. arXiv preprint arXiv: 1108.1169, 2011.
11. Hinton, Geoffrey. Neural networks for machine learning. Coursera video lectures, 2012.
12. Hinton, Geoffrey E and Van Camp, Drew. Keeping the neural networks simple by minimizing the description length of the weights. In Proceedings of the sixth annual conference on Computational learning theory, pp. 5-13, 1993.
13. Hinton, Geoffrey E and Zemel, Richard S. Autoencoders, minimum description length, and helmholtz free energy. Advances in neural information processing systems, pp. 3-3, 1994.
14. Larochelle, Hugo and Murray, Iain. The neural autoregres- sive distribution estimator. Journal of Machine Learning Research, 15:29-37, 2011.
15. Le Roux, Nicolas and Bengio, Yoshua. Representational power of restricted boltzmann machines and deep belief networks. Neural Computation, 20(6):1631-1649, 2008.
16. LeCun, Yann, Bottou, Leon, Bengio, Yoshua, and Haffner, Patrick. Gradient-based learning applied to document recognition. Proceedings of the IEEE, 86(11):2278-2324, 1998.
17. MacKay, David IC. Information theory, inference and learning algorithms, Ch 6. Cambridge university press, 2003.
18. Murray, Iain and Salakhutdinov, Ruslan. Evaluating probabilities under high-dimensional latent variable models. In Koller, D., Schuurmans, D., Bengio, Y., and Bottou, L. (eds.), Advances in Neural Information Processing Systems 21, pp. 1137-1144. 2009.
19. Paisley, John, Blei, David, and Jordan, Michael. Variational bayesian inference with stochastic search. arXiv preprint arXiv:1206.6430, 2012.
20. Ranzato, Marc'Aurelio, Boureau, Y-lan, and Cun, Yann L. Sparse feature learning for deep belief networks. In Advances in neural information processing systems, pp. 1185-1192, 2007.
21. Rifai, Salah, Bengio, Yoshua, Dauphin, Yann, and Vincent, Pascal. A generative process for sampling contractive auto-encoders. 2012.
22. Rissanen, Jorma. Modeling by shortest data description. Automatica, 14(5):465-471, 1978.
23. Salakhutdinov, Ruslan and Hinton, Geoffrey E. Deep boltzmann machines. In International Conference on Artificial Intelligence and Statistics, pp. 448-455, 2009.
24. Salakhutdinov, Ruslan and Murray, Iain. On the quantitative analysis of deep belief networks. In Proceedings of the 25th international conference on Machine learning, pp. 872-879. ACM, 2008.
25. Uria, Benigno, Murray, Iain, and Larochelle, Hugo. A deep and tractable density estimator. arXiv preprint arXiv:1310.1757, 2013.
26. Vincent, Pascal. A connection between score matching and denoising autoencoders. Neural computation, 23(7): 1661-1674, 2011.
27. Vincent, Pascal, Larochelle, Hugo, Bengio, Yoshua, and Manzagol, Pierre-Antoine. Extracting and composing robust features with denoising autoencoders. In Proceedings of the 25th international conference on Machine learning, pp. 1096-1103, 2008.
28. Williams, Ronald J. Simple statistical gradient-following algorithms for connectionist reinforcement learning. Machine learning, 8(3-4):229-256, 1992.