Learning deep energy models

Proceedings of the 28th International Conference on Machine Learning, ICML 2011

Volumn , Issue , 2011, Pages 1105-1112

  Ngiam, Jiquan ^a   Chen, Zhenghao ^a   Koh, Pang Wei ^a   Ng, Andrew Y ^a  

^a Stanford University   (United States)

Author keywords

[No Author keywords available]

Indexed keywords

COMPACT REPRESENTATION; DATA SETS; ENERGY LANDSCAPE; ENERGY MODEL; GENERATIVE MODEL; HIDDEN LAYERS; LAYER-WISE; MULTIPLE LAYERS; NATURAL IMAGES; PROBABILISTIC MODELS; SIGMOIDAL NEURAL NETWORKS; STUDENT-T DISTRIBUTION;

LEARNING SYSTEMS; OBJECT RECOGNITION;

FEEDFORWARD NEURAL NETWORKS;

EID: 80053445973     PISSN: None     EISSN: None     Source Type: Conference Proceeding    
DOI: None     Document Type: Conference Paper

References (31)

1. Bengio, Y. Learning deep architectures for AI. Technical report, Université de Montréal, 2007.
2. Bengio, Y. and LeCun, Y. Scaling learning algorithms towards ai. In Large-Scale Kernel Machines, 2007.
3. Coates, A., Lee, H., and Ng, A. Y. An analysis of single-layer networks in unsupervised feature learning. In AISTATS, 2011.
4. Dahl, G. E., Ranzato, M., Mohamed, A., and Hinton, G. E. Phone recognition with the mean-covariance restricted Boltzmann machine. In NIPS. 2010.
5. Desjardins, G., Courville, A., and Bengio, Y. Parallel tempering for training of restricted boltzmann machines. In AISTATS, 2010.
6. Hinton, G. E., Welling, M., and Mnih, A. Wormholes improve contrastive divergence. In NIPS, 2004.
7. Hinton, G. E., Osindero, S., and Teh, Y. W. A Fast Learning Algorithm for Deep Belief Nets. Neural Computation, 18(7):1527-1554, July 2006a. (Pubitemid 44024729)
8. Hinton, G. E., Osindero, S., Welling, M., and Teh, Y. W. Unsupervised Discovery of Non-linear Structure using Contrastive Backpropagation. Cognitive Science, 30(4): 725-731, 2006b. (Pubitemid 44420054)
9. Huang, F. J. and LeCun, Y. Large-Scale Learning with SVM and Convolutional Nets for Generic Object Categorization. In CVPR, 2006.
10. Kingma, D. P. and LeCun, Y. Regularized estimation of image statistics by score matching. In NIPS, 2010.
11. Köster, U. and Hyvärinen, A. A two-layer model of natural stimuli estimated with score matching. Neural Computation, 22(9):2308-2333, 2010.
12. Larochelle, H. and Bengio, Y. Classification using discriminative restricted boltzmann machines. In ICML, 2008.
13. Le, Q. V., Ngiam, J., Chen, Z., Chia, D., Koh, P. W., and Ng, A. Y. Tiled convolutional neural networks. In NIPS, 2010.
14. LeCun, Y., Huang, F.J., and Bottou, L. Learning methods for generic object recognition with invariance to pose and lighting. In CVPR, 2004.
15. Mnih, A. and Hinton, G. E. Learning nonlinear constraints with contrastive backpropagation. In IJCNN, 2005.
16. Nair, V. and Hinton, G. E. Implicit mixtures of restricted boltzmann machines. In NIPS, 2008.
17. Nair, V. and Hinton, G. E. 3-d object recognition with deep belief nets. In NIPS, 2009.
18. Neal, R. M. Probabilistic inference using markov chain Monte Carlo methods. Technical report, University of Toronto, 1993.
19. Neal, R. M. Annealed Importance Sampling. Statistics and Computing, 11:125-139, 1998. (Pubitemid 33638185)
20. Raina, R., Madhavan, A., and Ng, A. Y. Largescale deep unsupervised learning using graphics processors. In ICML, 2009.
21. Ranzato, M. and Hinton, G. E. Modeling Pixel Means and Covariances Using Factorized Third-Order Boltzmann Machines. In CVPR, 2010.
22. Ranzato, M., Mnih, V., and Hinton, G. E. Generating more realistic images using gated mrf's. In NIPS. 2010.
23. Salakhutdinov, R. and Hinton, G. E. Deep Boltzmann machines. In AISTATS, 2009.
24. Salakhutdinov, R. and Larochelle, H. Efficient learning of deep boltzmann machines. In AISTATS, 2010.
25. Salakhutdinov, R. and Murray, I. On the quantitative analysis of deep belief networks. In ICML, 2008.
26. Saxe, A., Koh, P.W., Chen, Z., Bhand, M., Suresh, B., and Ng, A. On random weights and unsupervised feature learning. In ICML, 2011.
27. Teh, Y., Welling, M., and Osindero, S. Energy-based models for sparse overcomplete representations. JMLR, 4:1235-1260, 2003.
28. Tieleman, T. Training Restricted Boltzmann Machines using Approximations to the Likelihood Gradient. In ICML, 2008.
29. Tieleman, T. and Hinton, G. E. Using fast weights to improve persistent contrastive divergence. In ICML, 2009.
30. van Hateren, J. H. and van der Schaaf, A. Independent component filters of natural images compared with simple cells in primary visual cortex. Proceedings: Biological Sciences, 265(1394):359-366, 1998.
31. Welling, M., Hinton, G. E., and Osindero, S. Learning sparse topographic representations with products of student-t distributions. In NIPS, 2003.