A fast learning algorithm for deep belief nets

Neural Computation

Volumn 18, Issue 7, 2006, Pages 1527-1554

  Hinton, Geoffrey E ^a   Osindero, Simon ^a   Teh, Yee Whye ^b  

^a UNIVERSITY OF TORONTO   (Canada)

^b NATIONAL UNIVERSITY OF SINGAPORE   (Singapore)

Author keywords

[No Author keywords available]

Indexed keywords

ALGORITHM; ANIMAL; ARTICLE; ARTIFICIAL NEURAL NETWORK; HUMAN; LEARNING; NERVE CELL; PHYSIOLOGY;

ALGORITHMS; ANIMALS; HUMANS; LEARNING; NEURAL NETWORKS (COMPUTER); NEURONS;

EID: 33745805403     PISSN: 08997667     EISSN: 1530888X     Source Type: Journal    
DOI: 10.1162/neco.2006.18.7.1527     Document Type: Article

References (21)

1. Belongie, S., Malik, J., & Puzicha, J. (2002). Shape matching and object recognition using shape contexts. IEEE Transactions on Pattern Analysis and Machine Intelligence, 24(4), 509-522.
2. Carreira-Perpinan, M. A., & Hinton, G. E. (2005). On contrastive divergence learning. In R. G. Cowell & Z. Ghahramani (Eds.), Artificial Intelligence and Statistics, 2005. (pp. 33-41). Fort Lauderdale, FL: Society for Artificial Intelligence and Statistics.
3. Decoste, D., & Schoelkopf, B. (2002). Training invariant support vector machines, Machine Learning, 46, 161-190.
4. Freund, Y. (1995). Boosting a weak learning algorithm by majority. Information and Computation, 12(2), 256-285.
5. Friedman, J., & Stuetzle, W. (1981). Projection pursuit regression. Journal of the American Statistical Association, 76, 817-823.
6. Hinton, G. E. (2002). Training products of experts by minimizing contrastive divergence, Neural Computation, 14(8), 1711-1800.
7. Hinton, G. E., Dayan, P., Frey, B. J., & Neal, R. (1995). The wake-sleep algorithm for self-organizing neural networks. Science, 268, 1158-1161.
8. LeCun, Y., Bottou, L., & Haffner, P. (1998). Gradient-based learning applied to document recognition. Proceedings of the IEEE, 86(11), 2278-2324.
9. Lee, T. S., & Mumford, D. (2003). Hierarchical Bayesian inference in the visual cortex. Journal of the Optical Society of America, A, 20, 1434-1448.
10. Marks, T. K., & Movellan, J. R. (2001). Diffusion networks, product of experts, and factor analysis. In T. W. Lee, T.-P. Jung, S. Makeig, & T. J. Sejnowski (Eds.), Proc. Int. Conf. on Independent Component Analysis (pp. 481-485). San Diego.
11. Mayraz, G., & Hinton, G. E. (2001). Recognizing hand-written digits using hierarchical products of experts. IEEE Transactions on Pattern Analysis and Machine Intelligence, 24, 189-197.
12. Neal, R. (1992). Connectionist learning of belief networks, Artificial Intelligence, 56, 71-113.
13. Neal, R. M., & Hinton, G. E. (1998). A new view of the EM algorithm that justifies incremental, sparse and other variants. In M. I. Jordan (Ed.), Learning in graphical models (pp. 355-368). Norwell, MA: Kluwer.
14. Ning, F., Delhomme, D., LeCun, Y., Piano, F., Bottou, L., & Barbano, P. (2005). Toward automatic phenotyping of developing embryos from videos. IEEE Transactions on Image Processing, 14(9), 1360-1371.
15. Roth, S., & Black, M. J. (2005). Fields of experts: A framework for learning image priors. In IEEE Conf. on Computer Vision and Pattern Recognition (pp. 860-867). Piscataway, NJ: IEEE.
16. Sanger, T. D. (1989). Optimal unsupervised learning in a single-layer linear feedforward neural networks. Neural Networks, 2(6), 459-473.
17. Simard, P. Y., Steinkraus, D., & Platt, J. (2003). Best practice for convolutional neural networks applied to visual document analysis. In International Conference on Document Analysis and Recogntion (ICDAR) (pp. 958-962). Los Alamitos, CA: IEEE Computer Society.
18. Teh, Y., & Hinton, G. E. (2001). Rate-coded restricted Boltzmann machines for face recognition. In T. K. Leen, T. G. Dietterich, & V. Tresp (Eds.), Advances in neural information processing systems, 13 (pp. 908-914). Cambridge, MA: MIT Press.
19. Teh, Y., Welling, M., Osindero, S., & Hinton, G. E. (2003). Energy-based models for sparse overcomplete representations. Journal of Machine Learning Research, 4, 1235-1260.
20. Welling, M., Hinton, G., & Osindero, S. (2003). Learning sparse topographic representations with products of Student-t distributions. In S. Becker, S. Thrun, & K. Obermayer (Eds.), Advances in neural information processing systems, 15 (pp. 1359-1366). Cambridge, MA: MIT Press.
21. Welling, M., Rosen-Zvi, M., & Hinton, G. E. (2005). Exponential family harmoniums with an application to information retrieval. In L. K. Saul, Y. Weiss, & L. Bottou (Eds.), Advances in neural information processing systems, 17 (pp. 1481-1488). Cambridge, MA: MIT Press.