Using fast weights to improve persistent contrastive divergence

ACM International Conference Proceeding Series

Volumn 382, Issue , 2009, Pages

  Tieleman, Tijmen ^a   Hinton, Geoffrey ^a  

^a UNIVERSITY OF TORONTO   (Canada)

Author keywords

[No Author keywords available]

Indexed keywords

CONTRASTIVE DIVERGENCE; DATA POINTS; ENERGY LANDSCAPE; EQUILIBRIUM DISTRIBUTIONS; MARKOV CHAIN; RESTRICTED BOLTZMANN MACHINE; SUFFICIENT STATISTICS; VARIANCE ESTIMATE; WEIGHT UPDATE;

EDUCATION; MARKOV PROCESSES; ROBOT LEARNING;

LEARNING ALGORITHMS;

EID: 70049098965     PISSN: None     EISSN: None     Source Type: Conference Proceeding    
DOI: 10.1145/1553374.1553506     Document Type: Conference Paper

References (15)

1. Bharath, B., & Borkar, V. (1999). Stochastic approximation algorithms: Overview and recent trends.Sadhana, 24, 425-452.
2. Cvijovi, D., & Klinowski, J. (1995). Taboo Search: An Approach to the Multiple Minima Problem. Science, 267, 664-666.
3. Hinton, G. (2002). Training Products of Experts by Minimizing Contrastive Divergence. Neural Computation, 14, 1771-1800.
4. Hinton, G., Osindero, S., & Teh, Y. (2006). A fast learning algorithm for deep belief nets. Neural Computation, 18, 1527-1554.
5. Jordan, M., Ghahramani, Z., Jaakkola, T., & Saul, L.(1999). An Introduction to Variational Methods for Graphical Models. Machine Learning, 37, 183-233.
6. Larochelle, H., & Bengio, Y. (2008). Classification using discriminative restricted boltzmann machines. Proceedings of the 25th international conference on Machine learning (pp. 536-543).
7. LeCun, Y., & Cortes, C. The MNIST database of handwritten digits. http://yann.lecun.com/ exdb/mnist/.
8. LeCun, Y., Huang, F., & Bottou, L. (2004). Learning methods for generic object recognition with invari ance to pose and lighting. Proceedings of the 2004 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (pp. 97-104).
9. Neal, R. (1992). Connectionist learning of belief networks. Artificial Intelligence, 56, 71-113.
10. Neal, R., & Hinton, G. (1998). A view of the EM algorithm that justifies incremental, sparse, and other variants. Learning in Graphical Models, 89, 355-368.
11. Robbins, H., & Monro, S. (1951). A Stochastic Approximation Method. The Annals of Mathematical Statistics, 22, 400-407.
12. Sminchisescu, C., & Welling, M. (2007). Generalized darting monte carlo (Technical Report CSRG-478). University of Toronto, Department of Computer Science.
13. Smolensky, P. (1986). Information processing in dynamical systems: foundations of harmony theory. MIT Press Cambridge, MA, USA.
14. Ti1eleman, T. (2008). Training restricted Boltzmann machines using approximations to the likelihood gradient. Proceedings of the 25th international conference on Machine learning (pp. 1064-1071).
15. Welling, M., & Hinton, G. (2002). A new learning algorithm for mean field Boltzmann machines. Proceedings of the International Conference on Artificial Neural Networks (pp. 351-357). Springer.