MADE: Masked autoencoder for distribution estimation

32nd International Conference on Machine Learning, ICML 2015

Volumn 2, Issue , 2015, Pages 881-889

  Germain, Mathieu ^a   Gregor, Karol ^b   Murray, Iain ^c   Larochelle, Hugo ^a  

^a UNIVERSITÉ DE SHERBROOKE   (Canada)

^b DEEPMIND   (United Kingdom)

^c UNIVERSITY OF EDINBURGH   (United Kingdom)

Author keywords

[No Author keywords available]

Indexed keywords

ARTIFICIAL INTELLIGENCE; PROBABILITY; PROGRAM PROCESSORS;

AUTO-REGRESSIVE; CONDITIONAL PROBABILITIES; DISTRIBUTION ESTIMATION; GENERATIVE MODEL; JOINT PROBABILITY; NEURAL NETWORK MODEL; SIMPLE MODIFICATIONS; STATE OF THE ART;

LEARNING SYSTEMS;

EID: 84969749373     PISSN: None     EISSN: None     Source Type: Conference Proceeding    
DOI: None     Document Type: Conference Paper

References (21)

1. Bastien, Frédéric, Lamblin, Pascal, Pascanu, Razvan, Bergstra, James, Goodfellow, Ian J., Bergeron, Arnaud, Bouchard, Nicolas, and Bengio, Yoshua. Theano: new features and speed improvements. Deep Learning and Unsupervised Feature Learning NIPS 2012 Workshop, 2012.
2. Bengio, Yoshua and Bengio, Samy. Modeling high-dimensional discrete data with multi-layer neural networks. In Advances in Neural Information Processing Systems 12 (NIPS 1999), pp. 400-406. MIT Press, 2000.
3. Bengio, Yoshua, Laufer, Eric, Alain, Guillaume, and Yosin-ski, Jason. Deep generative stochastic networks trainable by backprop. In Proceedings of the 31th Annual International Conference on Machine Learning (ICML 2014), pp. 226-234. JMLR.org, 2014.
4. Bergstra, James, Breuleux, Olivier, Bastien, Frédéric, Lamblin, Pascal, Pascanu, Razvan, Desjardins, Guillaume, Turian, Joseph, Warde-Farley, David, and Bengio, Yoshua. Theano: a CPU and GPU math expression compiler. In Proceedings of the Python for Scientific Computing Conference (SciPy), June 2010. Oral Presentation.
5. Dinh, Laurent, Krueger, David, and Bengio, Yoshua. NICE: non-linear independent components estimation, 2014. arXiv:1410.8516v2.
6. Duchi, John, Hazan, Elad, and Singer, Yoram. Adaptive subgradient methods for online learning and stochastic optimization. Technical report, EECS Department, University of California, Berkeley, Mar 2010.
7. Goodfellow, Ian, Pouget-Abadie, Jean, Mirza, Mehdi, Xu, Bing, Warde-Farley, David, Ozair, Sherjil, Courville, Aaron, and Bengio, Yoshua. Generative adversarial nets. In Advances in Neural Information Processing Systems 27 (NIPS 2014), pp. 2672-2680. Curran Associates, Inc., 2014.
8. Gregor, Karol and Le Cun, Yann. Learning representations by maximizing compression, 2011. arXiv: 1108.1169vl.
9. Gregor, Karol, Danihelka, Ivo, Mnih, Andriy, Blundell, Charles, and Wierstra, Daan. Deep Auto Regressive Networks. In Proceedings of the 31th Annual International Conference on Machine Learning (ICML2014), pp. 1242-1250. JMLR.org, 2014.
10. Kingma, Diederik P. and Welling, Max. Auto-encoding variational bayes. In Proceedings of the 2nd International Conference on Learning Representations (ICLR 2014), 2014.
11. Larochelle, Hugo and Murray, Iain. The neural autoregressive distribution estimator. In Proceedings of the 14th International Conference on Artificial Intelligence and Statistics (AISTATS 2011), volume 15, pp. 29-37, Ft. Lauderdale, USA, 2011. JMLR W&CP.
12. Poon, Hoifung and Domingos, Pedro. Sum-product networks: A new deep architecture. In Proceedings of the 20th Conference on Uncertainty in Artificial Intelligence (UAI2011), pp. 337-346, 2011.
13. Rezende, Danilo Jimenez, Mohamed, Shakir, and Wierstra, Daan. Stochastic backpropagation and approximate inference in deep generative models. In Proceedings of the 31th International Conference on Machine Learning (ICML 2014), pp. 1278-1286, 2014.
14. Salakhutdinov, Ruslan and Murray, Iain. On the quantitative analysis of deep belief networks. In Proceedings of the 25th Annual International Conference on Machine Learning (ICML 2008), pp. 872-879. Omnipress, 2008.
15. Schmah, Tanya, Hinton, Geoffrey E., Zemel, Richard S., Small, Steven L., and Strother, Stephen C. Generative versus discriminative training of RBMs for classification of fMRI images. In Advances in Neural Information Processing Systems 21 (NIPS 2008), pp. 1409-1416, 2009.
16. Srivastava, Nitish, Hinton, Geoffrey, Krizhevsky, Alex, Sutskever, Ilya, and Salakhutdinov, Ruslan. Dropout: A simple way to prevent neural networks from overfitting. Journal of Machine Learning Research, 15:1929-1958, 2014.
17. Uria, Benigno, Murray, Iain, and Larochelle, Hugo. RNADE: The real-valued neural autoregressive density-estimator. In Advances in Neural Information Processing Systems 26 (NIPS 2013), pp. 2175-2183, 2013.
18. Uria, Benigno, Murray, Iain, and Larochelle, Hugo. A deep and tractable density estimator. In Proceedings of the 31th International Conference on Machine Learning, (ICML 2014), pp. 467-475, 2014.
19. Uria, Benigno, Murray, Iain, Renals, Steve, and Valentini-Botinhao, Cassia. Modelling acoustic feature dependencies with artificial neural networks: Trajectory-RNADE. In Proceedings of the 40th IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP 2015). IEEE Signal Processing Society, 2015.
20. Wan, Li, Zeiler, Matthew D., Zhang, Sixin, Le Cun, Yann, and Fergus, Rob. Regularization of neural networks using dropconnect. In Proceedings of the 30th International Conference on Machine Learning, (ICML 2013), pp. 1058-1066, 2013.
21. Zeiler, Matthew D. ADADELTA: an adaptive learning rate method, 2012. arXiv: 1212.5701 vl.