Sparse hidden units activation in Restricted Boltzmann Machine

Advances in Intelligent Systems and Computing

Volumn 1089, Issue , 2015, Pages 181-185

  Tomczak, Jakub M ^a   Gonczarek, Adam ^a  

^a WROCLAW UNIVERSITY OF TECHNOLOGY   (Poland)

Author keywords

Deep learning; sparse solution; symmetric Kullback Leibler divergence

Indexed keywords

ARTIFICIAL INTELLIGENCE; CHEMICAL ACTIVATION; CLASSIFICATION (OF INFORMATION); INFORMATION RETRIEVAL SYSTEMS; SYSTEMS ENGINEERING;

CLASSIFICATION ERRORS; DEEP LEARNING; DISCRIMINATIVE FEATURES; DOCUMENT CLASSIFICATION; KULLBACK LEIBLER DIVERGENCE; RESTRICTED BOLTZMANN MACHINE; SPARSE SOLUTIONS; SPARSITY REGULARIZATIONS;

IMAGE CLASSIFICATION;

EID: 84906544877     PISSN: 21945357     EISSN: None     Source Type: Book Series    
DOI: 10.1007/978-3-319-08422-0_27     Document Type: Conference Paper

References (18)

1. Bengio, Y.: Learning Deep Architectures for AI. Foundations and Trends® in Machine Learning 2(1): 1-127. (2009).
2. Bishop, C. M.: Pattern Recognition and Machine Learning. Springer New York. (2006).
3. Cho, K., Ilin, A., & Raiko, T.: Tikhonov-Type regularization for Restricted Boltzmann Machines. In Artificial Neural Networks and Machine Learning (ICANN 2012). pp. 81-88. Springer Berlin Heidelberg. (2012).
4. Glorot, X., Bordes, A., & Bengio, Y.: Deep Sparse Rectifier Networks. In Proceedings of the 14th International Conference on Artificial Intelligence and Statistics (AISTATS 2011). Journal of Machine Learning Research Workshop & Conference Proceedings 15: 315-323. (2011).
5. Goh, H., Thome, N., & Cord, M.: Biasing restricted Boltzmann machines to manipulate latent selectivity and sparsity. In NIPS workshop on deep learning and unsupervised feature learning. (2010).
6. Hinton, G. E.: Training products of experts by minimizing contrastive divergence. Neural Comput 14: 1771-1800. (2002)
7. Hinton, G. E.: A practical guide to training Restricted Boltzmann Machines. In Neural Networks: Tricks of the Trade. pp. 599-619. Springer Berlin Heidelberg. (2012).
8. Hinton, G. E., Srivastava, N., Krizhevsky, A., Sutskever, I., & Salakhutdinov, R. R.: Improving neural networks by preventing co-adaptation of feature detectors. arXiv preprint arXiv: 1207. 0580. (2012).
9. Larochelle, H., & Bengio, Y.: Classification using discriminative restricted Boltzmann machines. In Proceedings of the 25th International Conference on Machine learning (ICML 2008). pp. 536-543. (2008, July).
10. Lee, H., Ekanadham, C., & Ng, A.: Sparse deep belief net model for visual area V2. In Advances in Neural Information Processing Systems (NIPS 2007). pp. 873-880. (2007).
11. Le, Q. V., Ngiam, J., Coates, A., Lahiri, A., Prochnow, B., & Ng, A.: On optimization methods for deep learning. In Proceedings of the 28th International Conference on Machine Learning (ICML 2011). pp. 265-272. (2011).
12. Le Roux, N., & Bengio, Y.: Representational power of Restricted Boltzmann Machines and deep belief networks. Neural Computation 20(6): 1631-1649. (2008).
13. Lewicki, M. S., & Sejnowski, T. J.: Learning overcomplete representations. Neural Computation 12(2): 337-365. (2000).
14. Marlin, B. M., Swersky, K., Chen, B., & Freitas, N. D.: Inductive principles for restricted Boltzmann machine learning. In International Conference on Artificial Intelligence and Statistics (ICML 2010). pp. 509-516. (2010).
15. Martens, J., Chattopadhya, A., Pitassi, T., & Zemel, R.: On the Expressive Power of Restricted Boltzmann Machines. In Advances in Neural Information Processing Systems (NIPS 2013). pp. 2877-2885. (2013).
16. Nair, V., & Hinton, G. E.: 3D object recognition with deep belief nets. In Advances in Neural Information Processing Systems (NIPS 2009). pp. 1339-1347. (2009).
17. Nair, V., & Hinton, G. E.: Rectified linear units improve Restricted Boltzmann Machines. In Proceedings of the 27th International Conference on Machine Learning (ICML 2010). pp. 807-814. (2010).
18. Smolensky, P.: Information processing in dynamical systems: foundations of harmony theory. In: Parallel distributed processing: explorations in the microstructure of cognition, Vol. 1: Foundations, pp. 194281. MIT Press, Cambridge, MA, USA. (1986)