Learning multiple layers of representation

Trends in Cognitive Sciences

Volumn 11, Issue 10, 2007, Pages 428-434

  Hinton, Geoffrey E ^a  

^a UNIVERSITY OF TORONTO   (Canada)

Author keywords

[No Author keywords available]

Indexed keywords

BRAIN; BRAIN MODELS; COMPUTATIONAL METHODS; LEARNING SYSTEMS; NEURAL NETWORKS; SPEECH INTELLIGIBILITY;

BRAIN EXTRACTS; MULTIPLE LAYERS OF REPRESENTATION; SENSORY INPUT;

SPEECH RECOGNITION;

HUMAN; LEARNING; NERVE CELL NETWORK; REVIEW; SPEECH PERCEPTION; TASK PERFORMANCE; VISUAL MEMORY;

BRAIN; HUMANS; LEARNING; MODELS, PSYCHOLOGICAL; NERVE NET;

EID: 35348818718     PISSN: 13646613     EISSN: None     Source Type: Journal    
DOI: 10.1016/j.tics.2007.09.004     Document Type: Review

References (44)

1. Rumelhart D.E., et al. Learning representations by back-propagating errors. Nature 323 (1986) 533-536
2. Hinton G.E., and Salakhutdinov R.R. Reducing the dimensionality of data with neural networks. Science 313 (2006) 504-507
3. Lee T.S., et al. The role of the primary visual cortex in higher level vision. Vision Res. 38 (1998) 2429-2454
4. Felleman D.J., and Van Essen D.C. Distributed hierarchical processing in the primate cerebral cortex. Cereb. Cortex 1 (1991) 1-47
5. Mumford D. On the computational architecture of the neocortex. II. The role of cortico-cortical loops. Biol. Cybern. 66 (1992) 241-251
6. Dayan P., and Abbott L.F. Theoretical Neuroscience: Computational and Mathematical Modeling of Neural Systems (2001), MIT Press
7. Roweis S., and Ghahramani Z. A unifying review of linear gaussian models. Neural Comput. 11 (1999) 305-345
8. Marks, T.K. and Movellan, J.R. (2001) Diffusion networks, products of experts, and factor analysis. In Proceedings of the International Conference on Independent Component Analysis (Lee, T. W. et al., eds), pp. 481-485, http://citeseer.ist.psu.edu/article/marks01diffusion.html
9. Bell A.J., and Sejnowski T.J. An information-maximization approach to blind separation and blind deconvolution. Neural Comput. 7 (1995) 1129-1159
10. Hyvärinen A., et al. Independent Component Analysis (2001), Wiley
11. Bartlett M.S., et al. Face recognition by independent component analysis. IEEE Trans. Neural Netw. 13 (2002) 1450-1464
12. Olshausen B.A., and Field D. Emergence of simple-cell receptive field properties by learning a sparse code for natural images. Nature 381 (1996) 607-609
13. Pearl J. Probabilistic Inference in Intelligent Systems: Networks of Plausible Inference (1988), Morgan Kaufmann
14. Lewicki M.S., and Sejnowski T.J. Bayesian unsupervised learning of higher order structure. In: Mozer M.C., et al. (Ed). Advances in Neural Information Processing Systems Vol. 9 (1997), MIT Press 529-535
15. Hoyer P.O., and Hyvärinen A. A multi-layer sparse coding network learns contour coding from natural images. Vision Res. 42 (2002) 1593-1605
16. Portilla J., et al. Image denoising using Gaussian scale mixtures in the wavelet domain. IEEE Trans. Image Process. 12 (2004) 1338-1351
17. Schwartz O., et al. Soft mixer assignment in a hierarchical generative model of natural scene statistics. Neural Comput. 18 (2006) 2680-2718
18. Karklin Y., and Lewicki M.S. Learning higher-order structures in natural images. Network 14 (2003) 483-499
19. Cowell R.G., et al. Probabilistic Networks and Expert Systems (2003), Springer
20. O'Reilly R.C. Six principles for biologically based computational models of cortical cognition. Trends Cogn. Sci. 2 (1998) 455-462
21. Hinton G.E., and Zemel R.S. Autoencoders, minimum description length, and Helmholtz free energy. Adv. Neural Inf. Process. Syst. 6 (1994) 3-10
22. Hinton G.E., et al. The wake-sleep algorithm for self-organizing neural networks. Science 268 (1995) 1158-1161
23. Neal R.M., and Hinton G.E. A new view of the EM algorithm that justifies incremental, sparse and other variants. In: Jordan M.I. (Ed). Learning in Graphical Models (1998), Kluwer Academic Publishers 355-368
24. Jordan M.I., et al. An introduction to variational methods for graphical models. Mach. Learn. 37 (1999) 183-233
25. Winn J., and Jojic N. LOCUS: Learning object classes with unsupervised segmentation. Tenth IEEE International Conference on Computer Vision Vol. 1 (2005), IEEE Press 756-763
26. Bishop C.M. Pattern Recognition and Machine Learning (2006), Springer
27. Bishop C.M., et al. VIBES: a variational inference engine for Bayesian networks. Adv. Neural Inf. Process. Syst. 15 (2002) 793-800
28. Hinton G.E. Boltzmann Machines (2007), Scholarpedia
29. Hinton G.E. Training products of experts by minimizing contrastive divergence. Neural Comput. 14 (2002) 1711-1800
30. Hinton G.E., et al. A fast learning algorithm for deep belief nets. Neural Comput. 18 (2006) 1527-1554
31. Decoste D., and Schoelkopf B. Training invariant support vector machines. Mach. Learn. 46 (2002) 161-190
32. Bengio Y., et al. Greedy layerwise training of deep networks. In: Schoelkopf B., et al. (Ed). Advances in Neural Information Processing Systems Vol. 19 (2007), MIT Press 153-160
33. Ranzato M., et al. Efficient learning of sparse representations with an energy-based model. In: Schoelkopf B., et al. (Ed). Advances in Neural Information Processing Systems Vol. 19 (2007), MIT Press 1137-1144
34. Sutskever I., and Hinton G.E. Learning multilevel distributed representations for high-dimensional sequences. In: Meila M., and Shen X. (Eds). Proceeding of the Eleventh International Conference on Artificial Intelligence and Statistics (2007), SAIS 544-551
35. Taylor G.W., et al. Modeling human motion using binary latent variables. In: Schoelkopf B., et al. (Ed). Advances in Neural Information Processing Systems Vol. 19 (2007), MIT Press 1345-1352
36. Roth S., and Black M.J. Fields of experts: a framework for learning image priors. Computer Vision and Pattern Recognition Vol. 2 (2005), IEEE Press 860-867
37. Ranzato M., et al. A unified energy-based framework for unsupervised learning. In: Meila M., and Shen X. (Eds). Proc. Eleventh International Conference on Artificial Intelligence and Statistics (2007), SAIS 368-376
38. Welling M., et al. Exponential family harmoniums with an application to information retrieval. In: Saul L.K., et al. (Ed). Advances in Neural Information Processing Systems Vol. 17 (2005), MIT Press 1481-1488
39. Memisevic R.F., and Hinton G.E. Unsupervised learning of image transformations. Computer Vision and Pattern Recognition (2007), IEEE Press 1-8
40. Mnih A., and Hinton G.E. Three new graphical models for statistical language modelling. In: Ghahramani Z. (Ed). Proceedings of the 24th International Conference on Machine Learning (2007), ACM Press New York 641-648
41. Salakhutdinov, R.R. et al. (2007) Restricted Boltzmann machines for collaborative filtering. In Proceedings of the 24th International Conference on Machine Learning (Ghahramani, Z., ed.), pp. 791-798, ACM Press New York}
42. Bengio, Y. and Le Cun, Y. Scaling learning algorithms towards AI. In Large-Scale Kernel Machines (Bottou, L. et al., eds), MIT Press (in press)
43. Welling, M. and Hinton, G.E. (2002) A new learning algorithm for mean field Boltzmann machines. In Artificial Neural Networks - ICANN 2002: International Conference, Madrid, Spain, August 28-30, 2002 (Lecture Notes in Computer Science Vol. 2415) (Dorronsoro, J.R., ed.), pp.351-357, Springer
44. Osindero, S. and Hinton, G.E. Modelling image patches with a directed hierarchy of Markov random fields. Adv. Neural Inf. Process. Syst. 20 (in press)