RNADE: The real-valued neural autoregressive density-estimator

Advances in Neural Information Processing Systems

Volumn , Issue , 2013, Pages

  Uria, Benigno ^a   Murray, Iain ^a   Larochellehugo, Hugo ^b  

^a UNIVERSITY OF EDINBURGH   (United Kingdom)

^b UNIVERSITÉ DE SHERBROOKE   (Canada)

Author keywords

[No Author keywords available]

Indexed keywords

AUTO-REGRESSIVE; CALCULATION OF DENSITY; DISTRIBUTED REPRESENTATION; GRADIENT-BASED OPTIMIZERS; JOINT DENSITIES; MIXTURE DENSITY; MIXTURE MODEL;

MIXTURES;

EID: 84898933061     PISSN: 10495258     EISSN: None     Source Type: Conference Proceeding    
DOI: None     Document Type: Conference Paper

References (33)

1. D. Koller and N. Friedman. Probabilistic graphical models: principles and techniques. MIT Press, 2009.
2. T. Cacoullos. Estimation of a multivariate density. Annals of the Institute of Statistical Mathematics, 18 (1):179-189, 1966.
3. D. Zoran and Y. Weiss. From learning models of natural image patches to whole image restoration. In International Conference on Computer Vision, pages 479-486. IEEE, 2011.
4. R. Salakhutdinov and I. Murray. On the quantitative analysis of deep belief networks. In Proceedings of the 25th International Conference on Machine learning, pages 872-879. Omnipress, 2008.
5. H. Larochelle and I. Murray. The neural autoregressive distribution estimator. Journal of Machine Learning Research W&CP, 15:29-37, 2011.
6. C. M. Bishop. Mixture density networks. Technical Report NCRG 4288, Neural Computing Research Group, Aston University, Birmingham, 1994.
7. B. J. Frey, G. E. Hinton, and P. Dayan. Does the wake-sleep algorithm produce good density estimators? In Advances in Neural Information Processing Systems 8, pages 661-670. IT Press, 1996.
8. Y. Bengio and S. Bengio. Modeling high-dimensional discrete data with multi-layer neural networks. Advances in Neural Information Processing Systems, 12:400-406, 2000.
9. H. Larochelle and S. Lauly. A neural autoregressive topic model. In Advances in Neural Information Processing Systems 25, 2012.
10. Y. Bengio. Discussion of the neural autoregressive distribution estimator. Journal of Machine Learning Research W&CP, 15:38-39, 2011.
11. L. Theis, R. Hosseini, and M. Bethge. Mixtures of conditional Gaussian scale mixtures applied to multiscale image representations. PLoS ONE, 7(7), 2012. doi: 10.1371/journal.pone.0039857.
12. N. Friedman and I. Nachman. Gaussian process networks. In Proceedings of the Sixteenth Conference on Uncertainty in Artificial Intelligence, pages 211-219. Morgan Kaufmann Publishers Inc., 2000.
13. I. Murray and R. Salakhutdinov. Evaluating probabilities under high-dimensional latent variable models. In Advances in Neural Information Processing Systems 21, pages 1137-1144, 2009.
14. L. Theis, S. Gerwinn, F. Sinz, and M. Bethge. In all likelihood, deep belief is not enough. Journal of Machine Learning Research, 12:3071-3096, 2011.
15. M. A. Ranzato and G. E. Hinton. Modeling pixel means and covariances using factorized third-order Boltzmann machines. In Computer Vision and Pattern Recognition, pages 2551-2558. IEEE, 2010.
16. A. Courville, J. Bergstra, and Y. Bengio. A spike and slab restricted Boltzmann machine. Journal of Machine Learning Research, W&CP, 15, 2011.
17. V. Nair and G. E. Hinton. Rectified linear units improve restricted Boltzmann machines. In Proceedings of the 27th International Conference on Machine Learning, pages 807-814. Omnipress, 2010.
18. J. S. Bridle. Probabilistic interpretation of feedforward classification network outputs, with relationships to statistical pattern recognition. In Neuro-computing: algorithms, architectures and applications, pages 227-236. Springer-Verlag, 1989.
19. T. Robinson. SHORTEN: simple lossless and near-lossless waveform compression. Technical Report CUED/F-INFENG/TR.156, Engineering Department, Cambridge University, 1994.
20. Y. Tang, R. Salakhutdinov, and G. Hinton. Deep mixtures of factor analysers. In Proceedings of the 29th International Conference on Machine Learning, pages 505-512. Omnipress, 2012.
21. D. Zoran and Y. Weiss. Natural images, Gaussian mixtures and dead leaves. Advances in Neural Information Processing Systems, 25:1745-1753, 2012.
22. K. Bache and M. Lichman. UCI machine learning repository, 2013. http://archive.ics.uci.edu/ml.
23. R. Silva, C. Blundell, and Y. W. Teh. Mixed cumulative distribution networks. Journal of Machine Learning Research W&CP, 15:670-678, 2011.
24. Z. Ghahramani and G. E. Hinton. The EM algorithm for mixtures of factor analyzers. Technical Report CRG-TR-96-1, University of Toronto, 1996.
25. J. Verbeek. Mixture of factor analyzers Matlab implementation, 2005. http://lear.inrialpes.fr/verbeek/code/.
26. D. Martin, C. Fowlkes, D. Tal, and J. Malik. A database of human segmented natural images and its application to evaluating segmentation algorithms and measuring ecological statistics. In International Conference on Computer Vision, volume 2, pages 416-423. IEEE, July 2001.
27. D. Zoran. Personal communication, 2013.
28. B. Frey. Graphical models for machine learning and digital communication. MIT Press, 1998.
29. J. S. Garofolo, L. F. Lamel, W. M. Fisher, J. G. Fiscus, D. S. Pallett, N. L. Dahlgren, and V. Zue. Timitacoustic-phonetic continuous speech corpus. Linguistic Data Consortium, 10(5):0, 1993.
30. T. Schaul, S. Zhang, and Y. LeCun. No More Pesky Learning Rates. In Proceedings of the 30th international conference on Machine learning, 2013.
31. J. Bergstra and Y. Bengio. Random search for hyper-parameter optimization. The Journal of Machine Learning Research, 13:281-305, 2012.
32. J. Snoek, H. Larochelle, and R. Adams. Practical Bayesian optimization of machine learning algorithms. In Advances in Neural Information Processing Systems 25, pages 2960-2968, 2012.
33. G. E. Hinton, N. Srivastava, A. Krizhevsky, I. Sutskever, and R. R. Salakhutdinov. Improving neural networks by preventing co-adaptation of feature detectors. Arxiv preprint arXiv:1207.0580, 2012.