Pixel recurrent neural networks

33rd International Conference on Machine Learning, ICML 2016

Volumn 4, Issue , 2016, Pages 2611-2620

  Van Den Oord, Aäron ^a   Kalchbrenner, Nal ^a   Kavukcuoglu, Koray ^a  

^a DEEPMIND   (United Kingdom)

Author keywords

[No Author keywords available]

Indexed keywords

ARTIFICIAL INTELLIGENCE; LEARNING SYSTEMS; PIXELS;

DEEP NEURAL NETWORKS; IMAGE MODELING; LOG LIKELIHOOD; NATURAL IMAGES; RECURRENT LAYERS; RECURRENT NETWORKS; SPATIAL DIMENSION; STATE OF THE ART;

RECURRENT NEURAL NETWORKS;

EID: 84999048365     PISSN: None     EISSN: None     Source Type: Conference Proceeding    
DOI: None     Document Type: Conference Paper

References (32)

1. Dinh, Laurent, Krueger, David, and Bengio, Yoshua. NICE: Non-linear independent components estimation. arXiv preprint arXiv:1410.8516, 2014.
2. Germain, Mathieu, Gregor, Karol, Murray, Iain, and Larochelle, Hugo. MADE: Masked autoencoder for distribution estimation. arXiv preprint arXiv:1502.03509, 2015.
3. Graves, Alex. Generating sequences with recurrent neural networks. arXiv preprint arXiv:1308.0850, 2013.
4. Graves, Alex and Schmidhuber, Jiirgen. Offline handwriting recognition with multidimensional recurrent neural networks. In Advances in Neural Information Processing Systems, 2009.
5. Gregor, Karol, Danihelka, Ivo, Mnih, Andriy, Blundell, Charles, and Wierstra, Daan. Deep autoregressive networks. In Proceedings of the 31st International Conference on Machine Learning, 2014.
6. Gregor, Karol, Danihelka, Ivo, Graves, Alex, and Wierstra, Daan. DRAW: A recurrent neural network for image generation. Proceedings of the 32nd International Conference on Machine Learning, 2015.
7. He, Kaiming, Zhang, Xiangyu, Ren, Shaoqing, and Sun, Jian. Deep residual learning for image recognition. arXiv preprint arXiv: 1512.03385, 2015.
8. Hochreiter, Sepp and Schmidhuber, Jiirgen. Long short- term memory. Neural computation, 1997.
9. Kalchbrenner, Nal and Blunsom, Phil. Recurrent continuous translation models. In Proceedings of the 2013 Conference on Empirical Methods in Natural Language Processing, 2013.
10. Kalchbrenner, Nal, Danihelka, Ivo, and Graves, Alex. Grid long short-term memory. arXiv preprint arXiv: 1507.01526, 2015.
11. Kingma, Diederik P and Welling, Max. Auto-encoding variational bayes. arXiv preprint arXiv: 1312.6114, 2013.
12. Krizhevsky, Alex. Learning multiple layers of features from tiny images. 2009.
13. Larochelle, Hugo and Murray, Iain. The neural autoregressive distribution estimator. The Journal of Machine Learning Research, 2011.
14. LeCun, Yann, Bottou, Leon, Bengio, Yoshua, and Haffner, Patrick. Gradient-based learning applied to document recognition. Proceedings of the IEEE, 1998.
15. Murray, Iain and Salakhutdinov, Ruslan R. Evaluating probabilities under high-dimensional latent variable models. In Advances in Neural Information Processing Systems, 2009.
16. Neal, Radford M. Connectionist learning of belief networks. Artificial intelligence, 1992.
17. Raiko, Tapani, Li, Yao, Cho, Kyunghyun, and Bengio, Yoshua. Iterative neural autoregressive distribution estimator NADE-k. In Advances in Neural Information Processing Systems, 2014.
18. Rezende, Danilo J, Mohamed, Shakir, and Wierstra, Daan. Stochastic backpropagation and approximate inference in deep generative models. In Proceedings of the 31st International Conference on Machine Learning, 2014.
19. Russakovsky, Olga, Deng, Jia, Su, Hao, Krause, Jonathan, Satheesh, Sanjeev, Ma, Sean, Huang, Zhiheng, Karpathy, Andrej, Khosla, Aditya, Bernstein, Michael, Berg, Alexander C., and Fei-Fei, Li. ImageNet Large Scale Visual Recognition Challenge. International Journal of Computer Vision (IJCV), 2015.
20. Salakhutdinov, Ruslan and Hinton, Geoffrey E. Deep boltz- mann machines. In International Conference on Artificial Intelligence and Statistics, 2009.
21. Salakhutdinov, Ruslan and Murray, Iain. On the quantitative analysis of deep belief networks. In Proceedings of the 25th international conference on Machine learning, 2008.
22. Salimans, Tim, Kingma, Diederik P, and Welling, Max. Markov chain monte carlo and variational inference: Bridging the gap. Proceedings of the 32nd International Conference on Machine Learning, 2015.
23. Sohl-Dickstein, Jascha, Weiss, Eric A., Maheswaranathan, Niru, and Ganguli, Surya. Deep unsupervised learning using nonequilibrium thermodynamics. Proceedings of the 32nd International Conference on Machine Learning, 2015.
24. Stollenga, Marijn F, Byeon, Wonmin, Liwicki, Marcus, and Schmidhuber, Juergen. Parallel multi-dimensional lstm, with application to fast biomedical volumetric image segmentation. In Advances in Neural Information Processing Systems 28. 2015.
25. Sutskever, Ilya, Martens, James, and Hinton, Geoffrey E. Generating text with recurrent neural networks. In Proceedings of the 28th International Conference on Machine Learning, 2011.
26. Theis, Lucas and Bethge, Matthias. Generative image modeling using spatial LSTMs. In Advances in Neural Information Processing Systems, 2015.
27. Theis, Lucas, van den Oord, Aaron, and Bethge, Matthias. A note on the evaluation of generative models. arXiv preprint arXiv:1511.01844, 2015.
28. Uria, Benigno, Murray, Iain, and Larochelle, Hugo. RNADE: The real-valued neural autoregressive density- estimator. In Advances in Neural Information Processing Systems, 2013.
29. Uria, Benigno, Murray, Iain, and Larochelle, Hugo. A deep and tractable density estimator. In Proceedings of the 31st International Conference on Machine Learning, 2014.
30. van den Oord, Aaron and Schrauwen, Benjamin. Factoring variations in natural images with deep Gaussian mixture models. In Advances in Neural Information Processing Systems, 2014a.
31. van den Oord, Aaron and Schrauwen, Benjamin. The student-t mixture as a natural image patch prior with application to image compression. The Journal of Machine Learning Research, 2014b.
32. Zhang, Yu, Chen, Guoguo, Yu, Dong, Yao, Kaisheng, Khu- danpur, Sanjeev, and Glass, James. Highway long short- term memory RNNs for distant speech recognition. In Proceedings of the International Conference on Acoustics, Speech and Signal Processing, 2016.