Feedforward Hebbian learning with nonlinear output units: A Lyapunov approach

Neural Networks

Volumn 9, Issue 2, 1996, Pages 321-328

  Troyer, Todd W ^a  

^a UNIVERSITY OF CALIFORNIA   (United States)

Author keywords

competitive learning; correlational learning; high gain sigmoid; Lyapunov functions; principal component analysis; saturation; unsupervised learning

Indexed keywords

LEARNING SYSTEMS; MATHEMATICAL MODELS; NEURAL NETWORKS; NONLINEAR SYSTEMS; TRANSFER FUNCTIONS;

COMPETITIVE LEARNING; CORRELATIONAL LEARNING; FEEDFORWARD HEBBIAN LEARNING; HIGH GAIN SIGMOID; PRINCIPAL COMPONENT ANALYSIS; UNSUPERVISED LEARNING;

LYAPUNOV METHODS;

ARTICLE; ARTIFICIAL NEURAL NETWORK; LEARNING; PRIORITY JOURNAL;

EID: 0030111092     PISSN: 08936080     EISSN: None     Source Type: Journal    
DOI: 10.1016/0893-6080(95)00044-5     Document Type: Article

References (28)

1. Aman, S. (1977). Neural theory of association and concept-formation. Biological Cybernetics, 26, 175-185.
2. Amari, S. (1983). Field theory of self-organizing neural nets. IEEE Transactions on Systems, Man and Cybernetics, SMC-13, 5, 741-748.
3. Amari, S., & Takeuchi, A. (1978). Mathematical theory on formation of category detecting nerve cells. Biological Cybernetics, 29, 127-136.
4. Cohen, M., & Grossberg, S. (1983). Absolute stability of global pattern formation and parallel memory storage by competitive neural networks. IEEE Transactions on Systems, Man and Cybernetics, SMC-13, 815-826.
5. Földiák, P. (1990). Forming sparse representations by local anti-Hebbian learning. Biological Cybernetics, 64, 165-170.
6. Hirsch, M. W. (1989a). Convergent activation dynamics in continuous time networks. Neural Networks, 2, 331-349.
7. Hirsch, M. W. (1989b). On the Amari-Takeuchi theory of category formation. In Proceedings of the International Joint Conference on Neural Networks, Washington DC (Vol. 1, pp. 297-300).
8. Hirsch, M. W. (1991). Saturated outputs for high-gain, self-exciting networks. International Journal of Bifurcation and Chaos, 1, 211-217.
9. Hopfield, J. J. (1982). Neural networks and physical systems with emergent collective computational abilities. Proceedings of the National Academy of Science, USA, 79, 2554-2558.
10. Hopfield, J. J. (1984). Neurons with graded response have collective computational properties like those of two-state neurons. Proceedings of the National Academy of Science, USA, 81, 3088-3092.
11. Horn, R. A., & Johnson, C. R. (1991). Matrix analysis. New York: Cambridge University Press.
12. Hornik, K., & Kuan, C. M. (1992). Convergence analysis of local feature extraction algorithms. Neural Networks, 5(2), 229-240.
13. Intrator, N. (1992). Feature extraction using an unsupervised network. Neural Computation, 4(1), 98-107.
14. Karhunen, J., & Joutsensalo, J. (1994). Representation and separation of signals using nonlinear PCA type learning. Neural Networks, 7(1), 113-128.
15. Kidder, J. (1993). A theory of faulty dynamics. Ph.D. thesis, University of California at Berkeley.
16. Leen, T. K. (1991). Dynamics of learning in linear feature-discovery networks. Network, 2(1), 85-106.
17. Lewis, J. E., & Glass, L. (1992). Nonlinear dynamics and symbolic dynamics of neural networks. Neural Computation, 4(5), 621-642.
18. Linsker, R. (1988). Self-organization in a perceptual network. Computer, 21, 105-107.
19. Malsburg, C. von der (1973). Self-organization of orientation sensitivity cells in the striate cortex. Kybernetik, 14, 85-100.
20. Masuda, T. (1993). Model of competitive learning based upon a generalized energy function. Neural Networks, 6(8), 1095-1103.
21. Matsuoka, K., & Kawamoto, M. (1994). A neural network that self-organizes to perform three operations related to principal component analysis. Neural Networks, 7(5), 753-766.
22. Miller, K. D. (1990). Correlation-based models of neural development. In M. A. Gluck, & D. E. Rummelhart (Eds.), Neuroscience and connectionist theory (Chap. 7, pp. 267-354). Lawrence Erlbaum Associates.
23. Oja, E. (1992). Principal components, minor components, and linear networks. Neural Networks, 5(6), 891-902.
24. Plumbley, M. D. (1995). Lyapunov functions for convergence of principal component algorithms. Neural Networks, 8(1), 11-23.
25. Troyer, T. W. (1991). Dynamics of the Amari-Takeuchi competitive learning model. In Proceedings of the International Joint Conference on Neural Networks, Seattle: IEEE.
26. White, R. H. (1992). Competitive Hebbian learning: Algorithms and demonstrations. Neural Networks, 5(2), 261-276.
27. Whitelaw, V. A., & Cowan, J. D. (1981). Specificity and plasticity of retinotectal connections: a computational model. Journal of Neuroscience, 1(12), 1369-1387.
28. Xu, L. (1993). Least mean squared error reconstruction principal for self-organizing neural-nets. Neural Networks, 6(5), 229-240.