Improving convergence of restricted Boltzmann machines via a learning adaptive step size

Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)

Volumn 7441 LNCS, Issue , 2012, Pages 511-518

  Lopes, Noel ^a,b   Ribeiro, Bernardete ^a,c  

^a UNIVERSITY OF COIMBRA   (Portugal)

^b POLYTECHNIC INSTITUTE OF GUARDA   (Portugal)

^c UNIVERSITY OF COIMBRA   (Portugal)

Author keywords

Adaptive step size; Deep Belief Networks; Deep learning; Restricted Boltzmann Machines

Indexed keywords

ADAPTIVE STEP SIZE; DEEP BELIEF NETWORKS; DEEP LEARNING; LEARNING RATES; MOMENTUM TERM; RECONSTRUCTION ERROR; RESTRICTED BOLTZMANN MACHINE;

IMAGE ANALYSIS;

COMPUTER VISION;

EID: 84865604539     PISSN: 03029743     EISSN: 16113349     Source Type: Book Series    
DOI: 10.1007/978-3-642-33275-3_63     Document Type: Conference Paper

References (15)

1. Almeida, L.B.: C1.2 Multilayer perceptrons. In: Handbook of Neural Computation, pp. C1.2:1-C1.2:30. IOP Publishing Ltd. and Oxford University Press (1997)
2. Bengio, Y.: Learning deep architectures for AI. Foundations and Trends in Machine Learning 2(1), 1-127 (2009)
3. Hinton, G.E.: Training products of experts by minimizing contrastive divergence. Neural Comput. 14, 1771-1800 (2002)
4. Hinton, G.E., Osindero, S., Teh, Y.W.: A fast learning algorithm for deep belief nets. Neural Comput. 18, 1527-1554 (2006)
5. Hinton, G.: A practical guide to training restricted boltzmann machines. Tech. rep., Dep. of Computer Science, University of Toronto (2010)
6. Larochelle, H., Erhan, D., Courville, A., Bergstra, J., Bengio, Y.: An empirical evaluation of deep architectures on problems with many factors of variation. In: Proc. of the 24th Intl. Conference on Machine Learning, pp. 473-480. ACM (2007)
7. Le Roux, N., Bengio, Y.: Representational power of restricted boltzmann machines and deep belief networks. Neural Comput. 20, 1631-1649 (2008)
8. Lee, H., Grosse, R., Ranganath, R., Ng, A.Y.: Convolutional deep belief networks for scalable unsupervised learning of hierarchical representations. In: Proc. of the 26th Annual Intl. Conference on Machine Learning, pp. 609-616. ACM (2009)
9. Lopes, N., Ribeiro, B.: Restricted boltzmann machines and deep belief networks on multi-core processors. In: IEEE World Congress on Computational Intelligence (2012)
10. Ranzato, M., Boureau, Y., LeCun, Y.: Sparse feature learning for deep belief networks. In: Advances in Neural Information Processing Systems, vol. 20 (2007)
11. Roux, N.L., Bengio, Y.: Deep belief networks are compact universal approximators. Neural Computation 22(8), 2192-2207 (2010)
12. Schulz, H., Müller, A., Behnke, S.: Investigating convergence of restricted boltzmann machine learning. In: NIPS 2010 Workshop on Deep Learning and Unsupervised Feature Learning, Whistler, Canada (2010)
13. Silva, F.M., Almeida, L.B.: Acceleration Techniques for the Backpropagation Algorithm. In: Almeida, L.B., Wellekens, C. (eds.) EURASIP 1990. LNCS, vol. 412, pp. 110-119. Springer, Heidelberg (1990)
14. Swersky, K., Chen, B., Marlin, B., de Freitas, N.: A tutorial on stochastic approximation algorithms for training restricted boltzmann machines and deep belief nets. In: Information Theory and Applications Workshop (2010)
15. Zainuddin, Z., Mahat, N., Hassan, Y.A.: Improving the convergence of the backpropagation algorithm using local adaptive techniques. Intl. Journal of Computational Intelligence, 172-175 (2005)