To recognize shapes, first learn to generate images

Progress in Brain Research

Volumn 165, Issue , 2007, Pages 535-547

  Hinton, Geoffrey E ^a  

^a UNIVERSITY OF TORONTO   (Canada)

Author keywords

Boltzmann machines; contrastive divergence; feature discovery; generative models; learning algorithms; multilayer neural networks; shape recognition; unsupervised learning; wake sleep algorithm

Indexed keywords

ALGORITHM; GENETIC VARIABILITY; IMAGE ANALYSIS; LEARNING; METHODOLOGY; NERVE CELL NETWORK; PRIORITY JOURNAL; REVIEW; SENSORY STIMULATION; SLEEP; WAKEFULNESS;

ALGORITHMS; CEREBRAL CORTEX; HUMANS; LEARNING; MODELS, NEUROLOGICAL; NEURAL NETWORKS (COMPUTER); PATTERN RECOGNITION, AUTOMATED;

EID: 34848816179     PISSN: 00796123     EISSN: None     Source Type: Book Series    
DOI: 10.1016/S0079-6123(06)65034-6     Document Type: Review

References (26)

1. Bengio, Y., Lamblin, P., Popovici, D. and Larochelle, H. (2007) Greedy layer-wise training of deep networks. In: Advances in Neural Information Processing Systems, 19. MIT Press, Cambridge, MA, pp. 153-160.
2. Bryson A., and Ho Y. Applied optimal control (1975), Wiley, New York
3. Decoste D., and Schoelkopf B. Training invariant support vector machines. Machine Learn. 46 (2002) 161-190
4. Hinton G.E. Training products of experts by minimizing contrastive divergence. Neural Comput. 14 (2002) 1771-1800
5. Hinton G.E., Dayan P., Frey B.J., and Neal R. The wake-sleep algorithm for self-organizing neural networks. Science 268 (1995) 1158-1161
6. Hinton G.E., Osindero S., and Teh Y.W. A fast learning algorithm for deep belief nets. Neural Comput. 18 (2006) 1527-1554
7. Hinton G.E., and Salakhutdinov R.R. Reducing the dimensionality of data with neural networks. Science 313 (2006) 504-507
8. Jabri M., and Flower B. Weight perturbation: an optimal architecture and learning technique for analog VLSI feedforward and recurrent multilayer networks. IEEE Trans. Neural Netw. 3 1 (1992) 154-157
9. Kaganov, A., Osindero, S. and Hinton, G.E. (2007) Learning the relationship between spoken digits and digit images. In: Technical Report, Department of Computer Science, University of Toronto.
10. Karni A., Tanne D., Rubenstein B., Askenasy J., and Sagi D. Dependence on REM sleep of overnight improvement of a perceptual skill. Science 265 5172 (1994) 679
11. LeCun, Y. (1985) Une procédure d'apprentissage pour réseau a seuil asymmetrique (a learning scheme for asymmetric threshold networks). In: Proceedings of Cognitiva 85, Paris, France, pp. 599-604.
12. LeCun Y., Bottou L., Bengio Y., and Haffner P. Gradient based learning applied to document recognition. Proc. IEEE 86 11 (1998) 2278-2324
13. LeCun Y., Huang F.-J., and Bottou L. Learning methods for generic object recognition with invariance to pose and lighting. Proceedings of CVPR'04 (2004), IEEE Press, New York
14. Mazzoni P., Andersen R., and Jordan M. A more biologically plausible learning rule for neural networks. Proc. Natl. Acad. Sci. 88 10 (1991) 4433-4437
15. Merzenich M., Kaas J., Wall J., Nelson R., Sur M., and Felleman D. Topographic reorganization of somatosensory cortical areas 3b and 1 in adult monkeys following restricted deafferentation. Neuroscience 8 1 (1983) 33-55
16. Minsky M., and Papert S. Perceptrons: an introduction to computational geometry (1969), MIT Press, Cambridge, MA
17. Parker D. Learning logic. Technical Report TR-47. Center for Computational Research in Economics and Management Science (1985), Massachusetts Institute of Technology, Cambridge, MA
18. Ranzato M., Poultney C., Chopra S., and LeCun Y. Efficient learning of sparse representations with an energy-based model. Advances in Neural Information Processing Systems 17 (2007), MIT Press, Cambridge, MA
19. Rosenblatt F. Principles of Neurodynamics (1962), Spartan Books, New York
20. Rumelhart D.E., Hinton G.E., and Williams R.J. Learning representations by back-propagating errors. Nature 323 (1986) 533-536
21. Selfridge O.G. Pandemonium: a paradigm for learning. Mechanisation of thought processes: Proceedings of a symposium held at the National Physical Laboratory (1958), HMSO, London
22. Seung H. Learning in spiking neural networks by reinforcement of stochastic synaptic transmission. Neuron 40 6 (2003) 1063-1073
23. Sharma J., Angelucci A., and Sur M. Induction of visual orientation modules in auditory cortex. Nature 404 6780 (2000) 841-847
24. Simard P.Y., Steinkraus D., and Platt J. Best practice for convolutional neural networks applied to visual document analysis. International Conference on Document Analysis and Recogntion (ICDAR) (2003), IEEE Computer Society, Los Alamitos 958-962
25. Vapnik V.N. The Nature of Statistical Learning Theory (2000), Springer, New York
26. Werbos, P. (1974) Beyond regression: new tools for prediction and analysis in the behavioral sciences. PhD thesis, Harvard University.