An improved RBM based on Bayesian Regularization

Proceedings of the International Joint Conference on Neural Networks

Volumn , Issue , 2014, Pages 2935-2939

  Pan, Guangyuan ^a   Qiao, Junfei ^a   Chai, Wei ^a   Dimopoulos, Nikitas ^b  

^a BEIJING UNIVERSITY OF TECHNOLOGY   (China)

^b UNIVERSITY OF VICTORIA   (Canada)

Author keywords

classification; over fitting; regularization; Restricted Boltzmann Machine

Indexed keywords

CLASSIFICATION (OF INFORMATION); DEEP LEARNING;

BAYESIAN REGULARIZATION; GENERALIZATION CAPABILITY; OBJECTIVE FUNCTIONS; OVERFITTING; RECOGNITION ERROR; REGULARIZATION; RESTRICTED BOLTZMANN MACHINE; TRADITIONAL APPROACHES;

LEARNING SYSTEMS;

EID: 84908475951     PISSN: None     EISSN: None     Source Type: Conference Proceeding    
DOI: 10.1109/IJCNN.2014.6889458     Document Type: Conference Paper

References (19)

1. Hinton G E and Salakhutdinov R R, "Reducing the dimensionality of data with neural networks". Science, 313(5786):504-507, 2006.
2. Smolensky P. "Information processing in dynamical systems: Foundations of harmony theory". Rumelhart D E, McClelland J L. parallel distributed processing: Explorations in the microstructure of cognition. Vol. 1: Foundations. Cambridge, MA: MIT Press, 1986.
3. Arel I, Rose D C, Karnowski T P. "Deep machine learning - A new frontier in artificial intelligence research". IEEE Computational Intelligence Magazine, 5(4):13-18, 2010.
4. Hinton G E. "Training products of experts by minimizing contrastive divergence". Neural Computation, 14(8):1771-1800, 2002.
5. Hinton, G. E., Osindero, S. and Teh, Y, "A fast learning algorithm for deep belief nets". Neural Computation, 18:1527-1554, 2006
6. The Y W, Hinton G E. "Pare-coded restricted Boltzmann machines for face recognition". Advances in Neural Information Processing Systems 13(NIPS' 00), MIT Press, pp. 908-914, 2001.
7. Salakhtdinv R, Mnih A, Hinton G E. "Restricted Boltzmann machines for collaborative filtering". Proceedings of the 24th International Conference on Machine Learning, Corvallis, OR, pp. 791-798, 2007.
8. Roux N L, Bengio Y. "Representational power of restricted Boltzmann machines and deep belief networks". Neural Computation, 20(6):1631-1649, 2006.
9. Cho K Y. "Improved learning algorithms for restricted Boltzmann machines". Espoo: Aalto University, 2011.
10. Hinton G E. "A practical guide to training restricted Boltzmann machines". Montreal: Department of Computer Science, University of Toronto, 2010.
11. Tieleman T, Hinton G. E. "Using fast weights to improve persistent contrastive divergence". Proceedings of the 26th International Conference on Machine Learning, Montreal, Canada, pp. 1033-1040, 2009.
12. Lee H, Ekanadham C, Ng A Y. "Sparse deep belief net model for visual area V2". Advances in Neural Information Processing Systems 20(NIPS' 07), Vancouver, Canada: MIT Press, pp. 873-880, 2008.
13. Schulz H, Muller A, Behnke S. "Investigating convergence of restricted Boltzmann machine learning". NIP 2010 Workshop on Deep Learning and Unsupervised Feature Learning, Whistler, Canada, pp. 1-9, 2010.
14. Fischer A, Igel C. "Empirical analysis of the divergence of Gibbs sampling based learning algorithms for restricted Boltzmann machine". Proceedings of the 20th International Conference on Artificial Neural Networks, Part 3, LNCS 6354, Berlin, Springer-Verlag, pp. 208-217, 2010.
15. Marc'Aurelio Ranzato, Christopher Poultney, Sumit Chopra and Yann LeCun, "Efficient Learning of Sparse Representations with an Energy-Based Model". Advances in Neural Information Processing Systems (NIPS 2006) MIT Press, 2007
16. Forsee F D, Hagan F D. "Gauss-Newton Approximation to Bayesian Regularization". Proceeding of the IEEE International Joint Conference on Neural Networks (6):1930-1935, 1997.
17. Kunisch K., Zou J. "Iterative Choices of Regularization Parameter in Linear Inverse Problems". Inverse Problems, 14:1247-1264, 1998.
18. Yann LeCun, Corinna Cortes, Christopher J. C. Burges. THE MNIST DATABASE of handwritten digits. Available: http://yann. lecun. com/exdb/mnist
19. Hinton. Deep learning. Available: http://deeplearning.net/software-links