Parallel tempering is efficient for learning restricted Boltzmann machines

Proceedings of the International Joint Conference on Neural Networks

Volumn , Issue , 2010, Pages

  Cho, Kyunghyun ^a   Raiko, Tapani ^a   Ilin, Alexander ^a  

^a AALTO UNIVERSITY   (Finland)

Author keywords

[No Author keywords available]

Indexed keywords

DEEP NEURAL NETWORKS; TEMPERING;

BIASED APPROXIMATION; CONTRASTIVE DIVERGENCE; GREEDY LEARNING; LEARN+; MONTECARLO METHODS; PARALLEL TEMPERING; RESTRICTED BOLTZMANN MACHINE;

MONTE CARLO METHODS;

EID: 79959388970     PISSN: None     EISSN: None     Source Type: Conference Proceeding    
DOI: 10.1109/IJCNN.2010.5596837     Document Type: Conference Paper

References (14)

1. G. E. Hinton and R. R. Salakhutdinov, "Reducing the dimensionality of data with neural networks," Science, vol. 313, no. 5786, pp. 504-507, July 2006.
2. R. Salakhutdinov and G. Hinton, "Deep Boltzmann machines," in Artificial Intelligence and Statistics, 2009.
3. S. Haykin, Neural Networks: A Comprehensive Foundation (2nd Edition), 2nd ed. Prentice Hall, July 1998.
4. D. J. C. Mackay, Information Theory, Inference & Learning Algorithms, 1st ed. Cambridge University Press, June 2002.
5. R. Salakhutdinov, "Learning in Markov random fields using tempered transitions," NIPS 2009, 2009.
6. P. Smolensky, "Information processing in dynamical systems: foundations of harmony theory," in Parallel distributed processing: explorations in the microstructure of cognition, vol. 1: foundations. Cambridge, MA, USA: MIT Press, 1986, pp. 194-281.
7. G. E. Hinton, "Training products of experts by minimizing contrastive divergence," Neural Comp., vol. 14, no. 8, pp. 1771-1800, August 2002.
8. M. A. Carreira-Perpiñ and G. Hinton, "On contrastive divergence learning," in Proceedings of the Tenth International Workshop on Artificial Intelligence and Statistics, Jan 6-8, 2005, Savannah Hotel, Barbados, R. G. Cowell and Z. Ghahramani, Eds. Society for Artificial Intelligence and Statistics, 2005, pp. 33-40.
9. D. J. Earl and M. W. Deem, "Parallel tempering: Theory, applications, and new perspectives," Phys. Chem. Chem. Phys., vol. 7, no. 23, pp. 3910-3916, 2005.
10. R. M. Neal, "Annealed importance sampling," Statistics and Computing, vol. 11, pp. 125-139, 1998.
11. R. Salakhutdinov, "Learning deep generative models," Ph.D. dissertation, Massachusetts Institute of Technology, 2009.
12. A. Asuncion and D. Newman, "UCI machine learning repository," 2007. [Online]. Available: http://www.ics.uci.edu/~mlearn/MLRepository.html
13. Y. Bengio, "Learning deep architectures for ai," Dept. IRO, Universite de Montreal, Tech. Rep., 2007.
14. G. Desjardins, A. Courville, Y. Bengio, P. Vincent, and O. Delalleau, "Tempered Markov chain Monte Carlo for training of restricted Boltzmann machines," Dept. IRO, Universite de Montreal, Tech. Rep. 1345, 2009.