Dynamics of multilayer networks in the vicinity of temporary minima

Neural Networks

Volumn 12, Issue 1, 1999, Pages 43-58

  Ampazis, Nikolaos ^a   Perantonis, S J ^b   Taylor, J G ^a  

^a INSTITUTE OF INFORMATICS AND TELECOMMUNICATIONS   (Greece)

^b KING'S COLLEGE LONDON   (United Kingdom)

Author keywords

Back propagation; Constrained optimization; Dynamical systems; Eigenvalues; Feed forward neural networks; Jacobian matrix; Supervised learning; Temporary minima

Indexed keywords

ALGORITHMS; BACKPROPAGATION; CONSTRAINT THEORY; EIGENVALUES AND EIGENFUNCTIONS; LEARNING SYSTEMS; MATRIX ALGEBRA; MULTILAYER NEURAL NETWORKS; OPTIMIZATION;

JACOBIAN MATRIX; SUPERVISED LEARNING;

FEEDFORWARD NEURAL NETWORKS;

ALGORITHM; ARTIFICIAL NEURAL NETWORK; DYNAMICS; EVALUATION; MATHEMATICAL ANALYSIS; MATHEMATICAL MODEL; PRIORITY JOURNAL; REVIEW; SIGNAL PROCESSING;

EID: 0033055103     PISSN: 08936080     EISSN: None     Source Type: Journal    
DOI: 10.1016/S0893-6080(98)00103-8     Document Type: Article

References (25)

1. Biehl, M., & Schwarze, H. (1995). Learning by on-line gradient descent. Journal of Physics, A28, 643-656.
2. Boyce, W.E., & DiPrima, R.C. (1986). Elementary differential equations and boundary value problems. New York: Wiley.
3. Coddington, E.A., & Levinson, N. (1955). Theory of ordinary differential equations. New York: McGraw-Hill.
4. Fahlman, S.E. (1988). Faster learning variations on back-propagation: an empirical study. In D. Touretzky, G. Hinton & T. Sejnowski (Eds.), Proceedings of the Connectionist Models Summer School (pp. 38-51). San Mateo, CA: Morgan Kaufmann.
5. Gelb, A., & Vander Velde, W.E. (1968). Multiple-input describing functions and nonlinear system design. New York: McGraw-Hill.
6. Gorse, D., Shepherd, A., & Taylor, J.G. (1993). Avoiding local minima using a range expansion algorithm. Neural Network World, 5, 503-510.
7. Gorse, D., Shepherd, A., & Taylor, J.G. (1997). The new ERA in supervised learning. Neural Networks, 10, 343-352.
8. Graham, D., & McRuer, D. (1961). Analysis of nonlinear control systems. New York: Wiley.
9. Guo, H., & Gelfand, S.B. (1991). Analysis of gradient descent learning algorithms for multilayer feedforward neural networks. IEEE Transactions on Circuits and Systems, 38, 883-894.
10. Jacobs, R.A. (1988). Increased rates of convergence through learning rate adaptation. Neural Networks, 1, 295-307.
11. Karras, D.A., & Perantonis, S.J. (1995). An efficient constrained training algorithm for feedforward networks. IEEE Transactions on Neural Networks, 6, 1420-1434.
12. Lee, Y., Oh, S., & Kim, M. (1991). The effect of initial weights on premature saturation in back-propagation learning. In Proceedings of International Joint Conference on Neural Networks (Vol. 1, pp. 765-770) Seattle, WA.
13. Liang, P. (1991). Design artificial neural networks based on the principle of divide-and-conquer. In Proceedings of International Conference on Circuits and Systems (pp. 1319-1322).
14. Lisboa, P.J.G., & Perantonis, S.J. (1991). Complete solution of the local minima in the XOR problem. Network, 2, 119-124.
15. Minsky, M.L., & Papert, S.A. (1988). Perceptrons, expanded ed. Cambridge, MA: MIT Press.
16. Murray, A.F. (1991). Analog VLSI and multi-layer perceptions accuracy, noise and on-chip learning. In Proceedings of Second International Conference on Microelectronics for Neural Networks (pp. 27-34).
17. Parker, D.B. (1985). Learning-logic: casting the cortex of the human brain in silicon. (Tech. Rep. TR-47). Center for Computational Research in Economics and Management Science, Massachussets Institute of Technology.
18. Perantonis, S.J., & Karras, D.A. (1995). An efficient constrained learning algorithm with momentum acceleration. Neural Networks, 8, 237-249.
19. Riedmiller, M., & Braun, H. (1993). A direct adaptive method for faster backpropagation learning: the RPROP algorithm. In Proceedings of the International Conference on Neural Networks (Vol.1, pp. 586-591). San Francisco, CA.
20. Saad, D., & Solla, S.A. (1995a). On-line learning in soft-committee machines. Physical Review, E52, 4225-4243.
21. Saad, D., & Solla, S.A. (1995b). Exact solution for on-line learning in multilayer neural networks. Physical Review Letters, 74, 4337-4340.
22. Sontag, E.D., & Sussmann, H.J. (1991). Backpropagation separates where perceptrons do. Neural Networks, 4, 243-249.
23. Sprinkhuizen-Kuyper, I.G., & Boers, E.J.W. (1996). The error surface of the simplest XOR network has only global minima. Neural Computation, 8, 1301-1320.
24. Sussmann, H.J. (1992). Uniqueness of the weights for minimal feedforward nets with a given input-output map. Neural Networks, 5, 589-593.
25. Woods, D. (1988). Back and counter propagation aberrations. In Proceedings of International Joint Conference on Neural Networks (Vol. 1, pp. 343-353).