A two-stage pretraining algorithm for deep boltzmann machines

Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)

Volumn 8131 LNCS, Issue , 2013, Pages 106-113

  Cho, Kyunghyun ^a   Raiko, Tapani ^a   Ilin, Alexander ^a   Karhunen, Juha ^a  

^a AALTO UNIVERSITY   (Finland)

Author keywords

Deep Boltzmann Machine; Deep Learning; Pretraining

Indexed keywords

DEEP BOLTZMANN MACHINES; DEEP LEARNING; GENERATIVE MODEL; MARKOV RANDOM FIELD MODEL; POSTERIOR DISTRIBUTIONS; PRE-TRAINING; RESTRICTED BOLTZMANN MACHINE; STOCHASTIC GRADIENT;

MARKOV PROCESSES; NEURAL NETWORKS;

ALGORITHMS;

EID: 84884941662     PISSN: 03029743     EISSN: 16113349     Source Type: Book Series    
DOI: 10.1007/978-3-642-40728-4_14     Document Type: Conference Paper

References (16)

1. Bengio, Y., Courville, A., Vincent, P.: Representation Learning: A Review and New Perspectives. arXiv:1206.5538 [cs.LG] (June 2012)
2. Bishop, C.M.: Pattern Recognition and Machine Learning, corrected 2nd printing edn. Springer (2007)
3. Cho, K.: Improved Learning Algorithms for Restricted Boltzmann Machines.Master's thesis, Aalto University School of Science (2011)
4. Desjardins, G., Courville, A., Bengio, Y.: On training deep Boltzmann machines. arXiv:1203.4416 [cs.NE] (March 2012)
5. Hinton, G., Salakhutdinov, R.: Reducing the dimensionality of data with neural networks. Science 313(5786), 504-507 (2006)
6. Hinton, G.: Training products of experts by minimizing contrastive divergence. Neural Computation 14, 1771-1800 (2002)
7. LeCun, Y., Bottou, L., Bengio, Y., Haffner, P.: Gradient-based learning applied to document recognition. Proceedings of the IEEE (11), 2278-2324
8. Marlin, B.M., Swersky, K., Chen, B., de Freitas, N.: Inductive principles for restricted Boltzmann machine learning. In: Proc. of the 13th Int. Conf. on Artificial Intelligence and Statistics (AISTATS 2010), pp. 509-516 (2010)
9. Montavon, G., Müller, K.-R.: Deep Boltzmann machines and the centering trick. In: Montavon, G., Orr, G.B.,Müller, K.-R. (eds.) NN: Tricks of the Trade, 2nd edn. LNCS, vol. 7700, pp. 621-637. Springer, Heidelberg (2012)
10. Raiko, T., Valpola, H., LeCun, Y.: Deep learning made easier by linear transformations in perceptrons. In: Proc. of the 15th Int. Conf. on Artificial Intelligence and Statistics (AISTATS 2012), La Palma, Canary Islands, Spain (April 2012)
11. Salakhutdinov, R., Hinton, G.E.: A Better Way to Pre-Train Deep Boltzmann Machines. In: Advances in Neural Information Processing Systems (2012)
12. Salakhutdinov, R.: Learning deep Boltzmann machines using adaptive MCMC. In: Fürnkranz, J., Joachims, T. (eds.) Proc. of the 27th Int. Conf. on Machine Learning (ICML 2010), pp. 943-950. Omnipress, Haifa (2010)
13. Salakhutdinov, R., Hinton, G.: An efficient learning procedure for deep Boltzmann machines. Tech. Rep. MIT-CSAIL-TR-2010-037, MIT (August 2010)
14. Salakhutdinov, R., Hinton, G.E.: Deep Boltzmann machines. In: Proc. of the Int. Conf. on Artificial Intelligence and Statistics (AISTATS 2009), pp. 448-455 (2009)
15. Tieleman, T., Hinton, G.E.: Using fast weights to improve persistent contrastive divergence. In: Proceedings of the 26th Annual International Conference on Machine Learning, ICML 2009, pp. 1033-1040. ACM, New York (2009)
16. Vincent, P., Larochelle, H., Lajoie, I., Bengio, Y.,Manzagol, P.A.: Stacked denoising autoencoders: Learning useful representations in a deep network with a local denoising criterion. Journal of Machine Learning Research 11, 3371-3408 (2010)