Training restricted boltzmann machines with multi-tempering: Harnessing parallelization

Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)

Volumn 7553 LNCS, Issue PART 2, 2012, Pages 92-99

  Brakel, Philemon ^a   Dieleman, Sander ^a   Schrauwen, Benjamin ^a  

^a GHENT UNIVERSITY   (Belgium)

Author keywords

Machine Learning; Markov Chain Monte Carlo; Neural Networks; Restricted Boltzmann Machines

Indexed keywords

DATA SETS; DEEP BELIEF NETWORKS; EXCHANGE OF INFORMATION; MARKOV CHAIN MONTE CARLO; MARKOV CHAIN MONTE CARLO METHOD; MAXIMUM LIKELIHOOD LEARNING; PARALLEL COMPUTATION; PARALLEL COMPUTING ARCHITECTURE; PARALLEL TEMPERING ALGORITHMS; PARALLELIZATIONS; PROBABILISTIC NEURAL NETWORKS; RESTRICTED BOLTZMANN MACHINE; WEIGHTED AVERAGING;

LEARNING SYSTEMS; NEURAL NETWORKS; PARALLEL ARCHITECTURES; TEMPERING;

LEARNING ALGORITHMS;

EID: 84867664558     PISSN: 03029743     EISSN: 16113349     Source Type: Book Series    
DOI: 10.1007/978-3-642-33266-1_12     Document Type: Conference Paper

References (13)

1. Athènes, M., Calvo, F.: Multiple-Replica Exchange with Information Retrieval. Chemphyschem. 9(16), 2332-2339 (2008)
2. Bengio, Y.: Learning deep architectures for AI. Foundations and Trends in Machine Learning 2(1), 1-127 (2009), also published as a book. Now Publishers (2009)
3. Brenner, P., Sweet, C.R., VonHandorf, D., Izaguirre, J.A.: Accelerating the Replica Exchange Method through an Efficient All-Pairs Exchange. The Journal of Chemical Physics 126(7), 074103 (2007)
4. Desjardins, G., Courville, A.C., Bengio, Y., Vincent, P., Delalleau, O.: Tempered markov chain monte carlo for training of restricted boltzmann machines. Journal of Machine Learning Research - Proceedings Track 9, 145-152 (2010)
5. Freund, Y., Haussler, D.: Unsupervised Learning of Distributions on Binary Vectors Using Two Layer Networks. Tech. rep., Santa Cruz, CA, USA (1994)
6. Hinton, G.E.: Training Products of Experts by Minimizing Contrastive Divergence. Neural Computation 14(8), 1771-1800 (2002)
7. Hinton, G.E., Osindero, S., Teh, Y.W.: A fast learning algorithm for deep belief nets. Neural Computation 18(7), 1527-1554 (2006)
8. Neal, R.M.: Annealed importance sampling. Statistics and Computing 11, 125-139 (1998)
9. Salakhutdinov, R.: Learning in markov random fields using tempered transitions. In: Bengio, Y., Schuurmans, D., Lafferty, J.D., Williams, C.K.I., Culotta, A. (eds.) NIPS, pp. 1598-1606. Curran Associates, Inc. (2009)
10. Salakhutdinov, R., Murray, I.: On the quantitative analysis of Deep Belief Networks. In: McCallum, A., Roweis, S. (eds.) Proceedings of the 25th Annual International Conference on Machine Learning (ICML 2008), pp. 872-879. Omnipress (2008)
11. Swendsen, R.H., Wang, J.S.: Replica Monte Carlo Simulation of Spin-Glasses. Physical Review Letters 57(21), 2607-2609 (1986)
12. Tieleman, T., Hinton, G.: Using Fast Weights to Improve Persistent Contrastive Divergence. In: Proceedings of the 26th International Conference on Machine Learning, pp. 1033-1040. ACM, New York (2009)
13. Tieleman, T.: Training restricted Boltzmann machines using approximations to the likelihood gradient. In: Proceedings of the International Conference on Machine Learning (2008)