Training restricted boltzmann machines using approximations to the likelihood gradient

Proceedings of the 25th International Conference on Machine Learning

Volumn , Issue , 2008, Pages 1064-1071

  Tieleman, Tijmen ^a  

^a UNIVERSITY OF TORONTO   (Canada)

Author keywords

[No Author keywords available]

Indexed keywords

LEARNING SYSTEMS; ROBOT LEARNING;

CONTRASTIVE DIVERGENCE; MODELING DISTRIBUTIONS; PSEUDO-LIKELIHOOD; RESTRICTED BOLTZMANN MACHINE;

MACHINE LEARNING; ALGORITHMS;

BOLTZMANN; OTHER ALGORITHMS;

EID: 56449086223     PISSN: None     EISSN: None     Source Type: Conference Proceeding    
DOI: 10.1145/1390156.1390290     Document Type: Conference Paper

References (21)

1. Bengio, Y., & Delalleau, O. (2007). Justifying and generalizing contrastive divergence (Technical Report 1311). Université de Montréal.
2. Bengio, Y., Lamblin, P., Popovici, D., Larochelle, H., & Montreal, Q. (2007). Greedy Layer-Wise Training of Deep Networks. Advances in Neural Information Processing Systems 19: Proceedings of the 2006 Conference.
3. Besag, J. (1986). On the statistical analysis of dirty pictures. Journal of the Royal Statistical Society B, 48, 259-302.
4. Borenstein, E., Sharon, E., & Ullman, S. (2004). Combining Top-Down and Bottom-Up Segmentation. Computer Vision and Pattern Recognition Workshop, 2004 Conference on, 46-46.
5. Carreira-Perpinan, M., & Hinton, G. (2005). On contrastive divergence learning. Artificial Intelligence and Statistics, 2005.
6. Gehler, P., Holub, A., & Welling, M. (2006). The rate adapting poisson model for information retrieval and object recognition. Proceedings of the 23rd international conference on Machine learning, 337-344.
7. Hinton, G. (2002). Training Products of Experts by Minimizing Contrastive Divergence. Neural Computation, 14, 1771-1800.
8. Hinton, G., & Salakhutdinov, R. (2006). Reducing the Dimensionality of Data with Neural Networks. Science, 313, 504-507.
9. Hinton, G. E., Osindero, S., & Teh, Y. W. (2006). A fast learning algorithm for deep belief nets. Neural Computation, 18.
10. Larochelle, H., Erhan, D., Courville, A., Bergstra, J., & Bengio, Y. (2007). An empirical evaluation of deep architectures on problems with many factors of variation. Proceedings of the 24th international conference on Machine learning, 473-480.
11. LeCun, Y., & Cortes, C. The MNIST database of handwritten digits.
12. Neal, R. (1992). Connectionist learning of belief networks. Artificial Intelligence, 56, 71-113.
13. Robbins, H., & Monro, S. (1951). A Stochastic Approximation Method. The Annals of Mathematical Statistics, 22, 400-407.
14. Salakhutdinov, R., Mnih, A., & Hinton, G. (2007). Restricted Boltzmann machines for collaborative filtering. Proceedings of the 24th international conference on Machine learning, 791-798.
15. Salakhutdinov, R., & Murray, I. (2008). On the quantitative analysis of deep belief networks. Proceedings of the International Conference on Machine Learning.
16. Smolensky, P. (1986). Information processing in dynamical systems: foundations of harmony theory. MIT Press Cambridge, MA, USA.
17. Wainwright, M., &. Jordan, M. (2003). Graphical models, exponential families, and variational inference. UC Berkeley, Dept. of Statistics, Technical Report, 649.
18. Welling, M., & Hinton, G. (2002). A New Learning Algorithm for Mean Field Boltzmann Machines. Artificial Neural Networks-Icann 2002: International Conference, Madrid, Spain, August 28-30, 2002: Proceedings.
19. Welling, M., Rosen-Zvi, M., & Hinton, G. (2005). Exponential family harmoniums with an application to information retrieval. Advances in Neural Information Processing Systems, 17, 1481-1488.
20. Younes, L. (1999). On the convergence of markovian stochastic algorithms with rapidly decreasing ergodicity rates. Stochastics An International Journal of Probability and Stochastic Processes, 65, 177-228.
21. Yuille, A. (2004). The Convergence of Contrastive Divergences. Advances in Neural Information Processing Systems, 3, 4.