Training RBMs based on the signs of the CD approximation of the log-likelihood derivatives

ESANN 2011 proceedings, 19th European Symposium on Artificial Neural Networks, Computational Intelligence and Machine Learning

Volumn , Issue , 2010, Pages 495-500

  Fischer, Asja ^a   Igel, Christian ^b  

^a RUHR UNIVERSITY BOCHUM   (Germany)

^b UNIVERSITY OF COPENHAGEN   (Denmark)

Author keywords

[No Author keywords available]

Indexed keywords

BIASED APPROXIMATION; BUILDING BLOCKES; CONTRASTIVE DIVERGENCE; LEARNING PROCESS; LOG LIKELIHOOD; OPTIMIZATION TECHNIQUES; RESILIENT BACKPROPAGATION; RESTRICTED BOLTZMANN MACHINE;

NEURAL NETWORKS;

LEARNING SYSTEMS;

EID: 84887100846     PISSN: None     EISSN: None     Source Type: Conference Proceeding    
DOI: None     Document Type: Conference Paper

References (13)

1. P. Smolensky. Information processing in dynamical systems: Foundations of harmony theory. In D. E. Rumelhart and J. L. McClelland, editors, Parallel Distributed Processing: Explorations in the Microstructure of Cognition, vol. 1: Foundations, pages 194-281. MIT Press, 1986.
2. Y. Bengio. Learning deep architectures for AI. Foundations and Trends in Machine Learning, 21(6):1601-1621, 2009.
3. G. E. Hinton. Training products of experts by minimizing contrastive divergence. Neural Computation, 14:1771-1800, 2002.
4. M. Á. Carreira-Perpiñnán and G. E. Hinton. On contrastive divergence learning. In 10th International Workshop on Artificial Intelligence and Statistics (AISTATS 2005), pages 59-66, 2005.
5. Y. Bengio and O. Delalleau. Justifying and generalizing contrastive divergence. Neural Computation, 21(6):1601-1621, 2009.
6. A. Fischer and C. Igel. Bounding the bias of contrastive divergence learning. Neural Computation. In press.
7. A Fischer and C. Igel. Contrastive divergence learning may diverge when training restricted Boltzmann machines. Frontiers in Computational Neuroscience. Bernstein Conference on Computational Neuroscience (BCCN 2009), 2009.
8. G. Desjardins, A. Courville, Y. Bengio, P. Vincent, and O. Dellaleau. Parallel tempering for training of restricted Boltzmann machines. Journal of Machine Learning Research Workshop and Conference Proceedings, 9(AISTATS 2010):145-152, 2010.
9. A. Fischer and C. Igel. Empirical analysis of the divergence of Gibbs sampling based learning algorithms for Restricted Boltzmann Machines. In International Conference on Artificial Neural Networks (ICANN 2010), LNCS. Springer-Verlag, 2010.
10. M. Riedmiller. Advanced supervised learning in multi-layer perceptrons - From backpropagation to adaptive learning algorithms. Computer Standards and Interfaces, 16(5):265-278, 1994.
11. C. Igel and M. Hüsken. Empirical evaluation of the improved Rprop learning algorithm. Neurocomputing, 50(C):105-123, 2003.
12. D. J. C. MacKay. Information Theory, Inference & Learning Algorithms. Cambridge University Press, 2002.
13. A. Fischer and C. Igel. Challenges in training Restricted Boltzmann Machines. In B. Hammer and T. Villmann, editors, New Challenges in Neural Computation (NC2), number 04/2010 in Machine Learning Reports, pages 11-24. 2010.