Non-linear feature extraction by redundancy reduction in an unsupervised stochastic neural network

Neural Networks

Volumn 10, Issue 4, 1997, Pages 683-691

  Deco, G ^a   Parra, L ^a,b  

^a SIEMENS AG   (Germany)

^b UNIVERSITY OF MUNICH   (Germany)

Author keywords

Boltzmann machine; Factorial codes; Redundancy

Indexed keywords

BOLTZMANN MACHINE; REDUNDANCY REDUCTION; STOCHASTIC NEURAL NETWORK;

COMPUTER SIMULATION; DATA TRANSFER; FEATURE EXTRACTION; LEARNING ALGORITHMS; RANDOM PROCESSES; REDUNDANCY;

NEURAL NETWORKS;

ARTICLE; ARTIFICIAL NEURAL NETWORK; COGNITION; LEARNING; PRIORITY JOURNAL; RECEPTIVE FIELD; RETINA; STOCHASTIC MODEL;

EID: 0031171662     PISSN: 08936080     EISSN: None     Source Type: Journal    
DOI: 10.1016/S0893-6080(96)00110-4     Document Type: Article

References (28)

1. Ackley, D., Hinton, G., & Sejnowski (1985). A learning algorithm for Boltzmann machines. Cognitive Science, 9, 147-169.
2. Atick, J., & Redlich, A. (1990). Towards a theory of early visual processing. Neural Computation, 2, 308-320.
3. Atick, J., & Redlich, A. (1992). What does the retina know about natural scenes. Neural Computation, 4, 196-210.
4. Attneave, F. (1954). Informational aspects of visual perception. Psychological Review, 61, 183-193.
5. Barlow, H. (1989). Unsupervised learning. Neural Computation, 1, 295-311.
6. Barlow, H., Kaushal, T., & Mitchison, G. (1989). Finding minimum entropy codes. Neural Computation, 1, 412-423.
7. Becker, S. (1992). An Information-theoretic Unsupervised Learning Algorithm for Neural Networks, unpublished doctoral thesis, University of Toronto.
8. Campbell, F., & Robson, I. (1968). Application of Fourier analysis to the visibility of gratings. J. Physiol. (London), 197, 551-566.
9. Földiak, P. (1989). Adaptive network for optimal linear feature extraction. In Proceedings of the IEEE/INNS International Joint Conference on Neural Networks, Washington DC, Vol. I (pp. 401-405). New York; IEEE Press. Publisher NY.
10. Hentschel, H.G.E., & Barlow, H.B. (1991). Minimum-entropy coding with Hopfield networks. Network, 2, 135-148.
11. Hubel, D., & Wiesel, T. (1962). Receptive Fields, binocular interaction and functional architecture in the cat's visual cortex. Journal of Physiology (London), 160, 106-154.
12. Jones, J.P., & Palmer, L.A. (1987). The two-dimensional spatial structure of simple receptive fields in cat striate cortex. Journal of Neurophysiology, 58, 1187-1211.
13. Jones, J.P., Stepnoskj, A., & Palmer, L.A. (1987). The two-dimensional spectral structure of simple receptive fields in cat striate cortex. Journal of Neurophysiology, 58, 1112-1232.
14. Kuehnel, H., & Tavan, P. (1990), The anti-Hebb rule derived from information theory. In R. Eckmiller, G. Hartmann & G. Hauske (Eds.), Parallel processing in neural systems and computers, (pp. 187-190). North-Holland: Elsevier Science.
15. Linsker, R. (1988). Self-organization in a perceptual network. Computer, 21, 105.
16. Linsker, R. (1989). How to generate ordered maps by maximizing the mutual information between input and output signals. Neural Computation, 1, 402-411.
17. Linsker, R. (1992). Local synaptic learning rules suffice to maximize mutual information in a linear network. Neural Computation, 4, 691-702.
18. MacKay, D. (1981). Strife over visual cortical functions. Nature, 289, 117-118.
19. Pollen, D., Lee, J., & Taylor, I. (1971). How does the striate cortex begin the reconstruction of the visual world? Science, 173, 74-77.
20. Redlich, A.N. (1993). Redundancy reduction as a strategy for unsupervised learning. Neural Computation, 5, 289-304.
21. Redlich, A.N. (1993). Supervised factorial learning. Neural Computation, 5, 750-766.
22. Rubner, J., & Tavan, P. (1989). A self-organization network for principal-component analysis. Europhysics Letters, 10, 693-698.
23. Rubner, J., & Schulten, K. (1990). Development of feature detectors by self-organization. Biological Cybernetics, 62, 193-199.
24. Schmidhuber, J. (1992). Learning factorial codes by predictability minimization. Neural Computation, 4, 863-879.
25. Schürmann, B. (1989). Stability and adaptation in artificial neural systems. Physical Review A, 40 (5), 2681-2688.
26. Shannon, C. (1948). A mathematical theory of communication. Bell System Technical Journal, 7, 379-423.
27. White, R. (1992). Competitive Hebbian learning: algorithm and demonstrations. Neural Networks. 5, 261-275.
28. Zipf, G. (1949). Human behavior and the principle of least effort. Cambridge, MA: Addison-Wesley.