Gaussian-Bernoulli deep Boltzmann machine

Proceedings of the International Joint Conference on Neural Networks

Volumn , Issue , 2013, Pages

  Cho, Kyung Hyun ^a   Raiko, Tapani ^a   Ilin, Alexander ^a  

^a AALTO UNIVERSITY   (Finland)

Author keywords

[No Author keywords available]

Indexed keywords

ADAPTIVE LEARNING RATES; CONTINUOUS DATA; DEEP BOLTZMANN MACHINES; HIDDEN NEURONS; LEARNING RATES; MULTIPLE LAYERS; PARALLEL TEMPERING; RESTRICTED BOLTZMANN MACHINE;

LEARNING ALGORITHMS; NEURAL NETWORKS;

GAUSSIAN DISTRIBUTION;

EID: 84893549229     PISSN: None     EISSN: None     Source Type: Conference Proceeding    
DOI: 10.1109/IJCNN.2013.6706831     Document Type: Conference Paper

References (30)

1. R. Salakhutdinov and G. E. Hinton, "Deep Boltzmann machines, " in Proceedings of the Internation Conference on Artificial Intelligence and Statistics (AISTATS 2009), 2009, pp. 448-455.
2. G. E. Hinton and R. Salakhutdinov, "Reducing the dimensionality of data with neural networks, " Science, vol. 313, no. 5786, pp. 504-507, July 2006.
3. Y. Bengio, "Learning deep architectures for AI, " Foundations and Trends in Machine Learning, vol. 2, no. 1, pp. 1-127, 2009.
4. B. Lake, R. Salakhudinov, J. Gross, and J. Tenenbaum, "One shot learning of simple visual concepts, " in Proceedings of the 33rd Annual Meeting of the Cognitive Science Society, 2011.
5. R. Salakhutdinov, "Learning deep Boltzmann machines using adaptive MCMC, " in Proceedings of the 27th International Conference on Machine Learning (ICML 2010), J. Fürnkranz and T. Joachims, Eds. Haifa, Israel: Omnipress, June 2010, pp. 943-950.
6. R. Salakhutdinov and H. Larochelle, "Efficient learning of deep boltzmann machines, " in Proceedings of the 27th Conference on Uncertainty in Artificial Intelligence, 2011.
7. A. Krizhevsky, "Learning multiple layers of features from tiny images, " Computer Science Department, University of Toronto, Tech. Rep., 2009.
8. K. Cho, A. Ilin, and T. Raiko, "Improved learning of Gaussian- Bernoulli restricted Boltzmann machines, " in Proceedings of the 20th International Conference on Artificial Neural Networks (ICANN 2010), 2011.
9. K. Cho, T. Raiko, and A. Ilin, "Enhanced gradient and adaptive learning rate for training restricted Boltzmann machines, " in Proceedings of the 28th International Conference on Machine Learning (ICML 2011). New York, NY, USA: ACM, June 2011, pp. 105-112.
10. K. Cho, T. Raiko, and A. Ilin, "Gaussian-Bernoulli deep Boltzmann machine, " in NIPS 2011 Workshop on Deep Learning and Unsupervised Feature Learning, Sierra Nevada, Spain, December 2011.
11. N. Srivastava and R. Salakhutdinov, "Multimodal learning with deep boltzmann machines, " in Advances in Neural Information Processing Systems 25, P. Bartlett, F. Pereira, C. Burges, L. Bottou, and K. Weinberger, Eds., 2012, pp. 2231-2239.
12. K. Cho, "Boltzmann machines and denoising autoencoders for image denoising, " ArXiv e-prints, Jan. 2013.
13. A. S. D.M. Titterington and U. Makov, Statistical Analysis of Finite Mixture Distributions. New York, London, Sydney: John Wiley & Sons, 1985.
14. R. Salakhutdinov, "Learning in Markov random fields using tempered transitions, " in Advances in Neural Information Processing Systems 22, Y. Bengio, D. Schuurmans, J. Lafferty, C. K. I.Williams, and A. Culotta, Eds., 2009, pp. 1598-1606.
15. T. Tieleman, "Training restricted Boltzmann machines using approximations to the likelihood gradient, " in Proceedings of the 25th Internation Conference on Machine Learning (ICML 2008). New York, NY, USA: ACM, 2008, pp. 1064-1071.
16. G. Desjardins, A. Courville, Y. Bengio, P. Vincent, and O. Delalleau, "Parallel tempering for training of restricted Boltzmann machines, " in Proceedings of the Thirteenth International Conference on Artificial Intelligence and Statistics, ser. JMLR Workshop and Conference Proceedings, Y.-W. Teh and M. Titterington, Eds. JMLR W&CP, 2010, pp. 145-152.
17. K. Cho, "Improved Learning Algorithms for Restricted Boltzmann Machines, " Master's thesis, Aalto University School of Science, 2011.
18. K. Cho, T. Raiko, and A. Ilin, "Parallel tempering is efficient for learning restricted boltzmann machines, " in Neural Networks (IJCNN), The 2010 International Joint Conference on, 2010, pp. 1-8.
19. G. Desjardins, A. Courville, and Y. Bengio, "Adaptive parallel tempering for stochastic maximum likelihood learning of RBMs, " in NIPS 2010 Workshop on Deep Learning and Unsupervised Feature Learning, 2010.
20. T. Raiko, K. Cho, and A. Ilin, "Enhanced gradient for learning boltzmann machines (abstract), " in The Learning Workshop, Fort Lauderdale, Florida, April 2011.
21. A. Fischer and C. Igel, "Empirical analysis of the divergence of Gibbs sampling based learning algorithms for restricted Boltzmann machines, " in Proceedings of the 20th international conference on Artificial neural networks: Part III, ser. ICANN'10. Berlin, Heidelberg: Springer- Verlag, 2010, pp. 208-217.
22. H. Schulz, A. Müller, and S. Behnke, "Investigating Convergence of Restricted Boltzmann Machine Learning, " in NIPS 2010 Workshop on Deep Learning and Unsupervised Feature Learning, 2010.
23. D. Erhan, Y. Bengio, A. Courville, P.-A. Manzagol, P. Vincent, and S. Bengio, "Why does unsupervised pre-training help deep learning?" Journal of Machine Learning Research, vol. 11, pp. 625-660, Mar. 2010.
24. F. Samaria and A. Harter, "Parameterisation of a stochastic model for human face identification, " in Proceedings of the Second IEEE Workshop on Applications of Computer Vision, 1994., dec 1994, pp. 138 -142.
25. H. Poon and P. Domingos, "Sum-Product networks: A new deep architecture, " in Proceedings of the 27th Conference on Uncertainty in Artificial Intelligence, 2011.
26. H. Kushner and G. Yin, Stochastic Approximation and Recursive Algorithms and Applications. Springer, 2003.
27. Y. Lecun, F. J. Huang, and L. Bottou, "Learning methods for generic object recognition with invariance to pose and lighting, " vol. 2, 2004.
28. G. Montavon and K.-R. Müller, "Deep Boltzmann machines and the centering trick, " in Neural Networks: Tricks of the trade, Reloaded, 2nd ed., ser. LNCS, G. Montavon, G. B. Orr, and K.-R. Müller, Eds. Springer, 2012, vol. 7700.
29. K. Cho, T. Raiko, A. Ilin, and J. Karhunen, "A Two-Stage Pretraining Algorithm for Deep Boltzmann Machines, " in NIPS 2012 Workshop on Deep Learning and Unsupervised Feature Learning, Lake Tahoe, December 2012.
30. G. Desjardins, R. Pascanu, A. Courville, and Y. Bengio, "Metric-free natural gradient for joint-training of boltzmann machines, " in Proceedings of the First International Conference on Learning Representations (ICLR 2013), 2013.