A better way to pretrain Deep Boltzmann Machines

Advances in Neural Information Processing Systems

Volumn 3, Issue , 2012, Pages 2447-2455

  Salakhutdinov, Ruslan ^a   Hinton, Geoffrey ^a  

^a UNIVERSITY OF TORONTO   (Canada)

Author keywords

[No Author keywords available]

Indexed keywords

DEEP BELIEF NETWORKS; DEEP BOLTZMANN MACHINES; GENERATIVE MODEL; HIDDEN LAYERS; LOWER BOUNDS; PRE-TRAINING;

ALGORITHMS;

EID: 84877755914     PISSN: 10495258     EISSN: None     Source Type: Conference Proceeding    
DOI: None     Document Type: Conference Paper

References (15)

1. Y. Bengio. Learning deep architectures for AI. Foundations and Trends in Machine Learning, 2009.
2. G. E. Hinton, S. Osindero, and Y.W. Teh. A fast learning algorithm for deep belief nets. Neural Computation, 18(7):1527-1554, 2006.
3. H. Larochelle, Y. Bengio, J. Louradour, and P. Lamblin. Exploring strategies for training deep neural networks. Journal of Machine Learning Research, 10:1-40, 2009.
4. Y. LeCun, F. J. Huang, and L. Bottou. Learning methods for generic object recognition with invariance to pose and lighting. In CVPR (2), pages 97-104, 2004.
5. V. Nair and G. E. Hinton. Implicit mixtures of restricted Boltzmann machines. In Advances in Neural Information Processing Systems, volume 21, 2009.
6. M. A. Ranzato. Unsupervised learning of feature hierarchies. In Ph.D. New York University, 2009.
7. R. R. Salakhutdinov and G. E. Hinton. Deep Boltzmann machines. In Proceedings of the International Conference on Artificial Intelligence and Statistics, volume 12, 2009.
8. R. R. Salakhutdinov and G. E. Hinton. An efficient learning procedure for Deep Boltzmann Machines. Neural Computation, 24:1967 - 2006, 2012.
9. R. R. Salakhutdinov and H. Larochelle. Efficient learning of deep Boltzmannmachines. In Proceedings of the International Conference on Artificial Intelligence and Statistics, volume 13, 2010.
10. R. R. Salakhutdinov and I. Murray. On the quantitative analysis of deep belief networks. In Proceedings of the International Conference on Machine Learning, volume 25, pages 872 - 879, 2008.
11. T. Tieleman. Training restricted Boltzmann machines using approximations to the likelihood gradient. In ICML. ACM, 2008.
12. M. Welling and G. E. Hinton. A new learning algorithm for mean field Boltzmann machines. Lecture Notes in Computer Science, 2415, 2002.
13. M.Welling and C. Sutton. Learning in markov random fields with contrastive free energies. In International Workshop on AI and Statistics (AISTATS'2005), 2005.
14. L. Younes. On the convergence of Markovian stochastic algorithms with rapidly decreasing ergodicity rates, March 17 2000.
15. A. L. Yuille. The convergence of contrastive divergences. In Advances in Neural Information Processing Systems, 2004.