Differential-algebraic equations and singular perturbation methods in recurrent neural learning

Dynamical Systems

Volumn 18, Issue 1, 2003, Pages 89-105

  Riaza, Ricardo ^a   Zufiria, Pedro J ^a  

^a UNIVERSIDAD POLITÉCNICA DE MADRID   (Spain)

Author keywords

[No Author keywords available]

Indexed keywords

ALGEBRA; BIFURCATION (MATHEMATICS); COMPUTER SIMULATION; DIFFERENTIAL EQUATIONS; PERTURBATION TECHNIQUES;

RECURRENT NEURAL LEARNING;

LEARNING SYSTEMS;

EID: 0037959724     PISSN: 14689367     EISSN: None     Source Type: Journal    
DOI: 10.1080/1468936031000086843     Document Type: Article

References (40)

1. Allgower, E. L., and Georg, K., 1990, Numerical Continuation Methods. An Introduction (New York: Springer-Verlag).
2. Atiya, A. F., and Parlos, A. G., 2000, New results on recurrent network training: unifying the algorithms and accelerating convergence. IEEE Transactions on Neural Networks, 11: 697-709.
3. Baldi, P., 1995, Gradient descent learning algorithm overview: a general dynamical systems perspective. IEEE Transactions on Neural Networks, 6: 182-195.
4. Brenan, K. E., Campbell S. L., and Petzold, L. R., 1996, Numerical Solution of Initial-value Problems in Differential-algebraic Equations (Philadelphia: SIAM).
5. Chen, J. R., Nambiar, R., and Mars, P., 1997, Learning Algorithms: Theory and Applications in Signal Processing, Control and Communications (Boca Raton, FL: CRC Press).
6. Cohen, M. A., 1992a, The construction of arbitrary stable dynamics in nonlinear neural networks, Neural Networks, 5: 83-103.
7. Cohen, M. A., 1992b, The synthesis of arbitrary stable dynamics in non-linear neural networks II: feed-back and universality. Proc. IJCNN, Baltimore, Vol. 1. pp. 141-146.
8. Cohen, M. A. and Grossberg, S., 1983, Absolute stability of global pattern formation and parallel memory storage by competitive neural networks. IEEE Transactions on Systems, Man and Cybernetics, 13: 815-826.
9. Fernández de Cañete, J., García-Moral, I., and García-Cerezo, A., 1996, Neural identification and control using the dynamic Levenberg-Marquardt algorithm. Proc. EANN'96, London, pp. 147-150.
10. Griñö, R., Cembrano, G., and Torras, C., 2000, Nonlinear system identification using additive dynamic neural networks-two on-line approaches. IEEE Transactions on Circuits and Systems I: Fundamental Theory and Applications, 47: 150-165.
11. Guckenheimer, J., and Holmes, P., 1983, Nonlinear Oscillations, Dynamical Systems and Bifurcations of Vector Fields (Berlin: Springer-Verlag).
12. Hairer, E., and Wanner, G., 1996, Solving Ordinary Differential Equations II: Stiff and Differential-algebraic Problems (Berlin: Springer-Verlag).
13. Hirsch, M. W., 1989, Convergent activation dynamics in continuous time networks. Neural Networks, 2: 331-349.
14. Hopfield, J., 1984, Neurons with graded response have collective computational properties like those of two-state neurons. Proceedings of the National Academy of Science, 81: 3088-3092.
15. Hoppensteadt, F. C., and Izhikevich, E. M., 1997, Weakly Connected Neural Networks (Berlin: Springer-Verlag).
16. Jin, L., and Gupta, M. M., 1999, Stable dynamic back propagation learning in recurrent neural networks. IEEE Transactions on Neural Networks, 10: 1321-1334.
17. Kimura, M., and Nakano, R., 1998, Learning dynamical systems by recurrent neural networks from orbits. Neural Networks, 11: 1589-1599.
18. Knorrenschild, M., 1992, Differential/algebraic equations as stiff ordinary differential equations. SIAM Journal on Numerical Analysis, 29: 1694-1715.
19. Kokotovic, P. V., Khalil H. K., and O'Reilly, J., 1986, Singular Perturbation Methods in Control: Analysis and Design (New York: Academic Press).
20. Narendra, K. S., and Parthasarathy, K., 1990, Identification and control of dynamical systems using neural networks. IEEE Transactions on Neural Networks, 1: 4-27.
21. Narendra, K. S., and Parthasarathy, K., 1991, Gradient methods for the optimization of dynamical systems containing neural networks. IEEE Transactions on Neural Networks, 2: 252-262.
22. O'Malley Jr., R. E., and Kalachev, L. V., 1994, Regularization of nonlinear differential-algebraic equations. SIAM Journal on Mathematical Analysis, 25: 615-629.
23. Pineda, F. J., 1987, Generalization of back-propagation to recurrent neural networks. Physical Review Letters, 59:2229-2232.
24. Rabier, P. J., 1989, Implicit differential equations near a singular point, Journal of Mathematical Analysis and Applications, 144: 425-449.
25. Rabier, P. J., and Rheinboldt, W. C., 1994, On impasse points of quasi-linear differential-algebraic equations. Journal of Mathematical Analysis and Applications, 181: 429-454.
26. Riaza, R., 2002, Singular bifurcations in higher index differential-algebraic equations. Dynamical Systems, 17: 243-261.
27. Riaza, R., Campbell, S. L., and Marszalek, W. 2000, On singular equilibria of index-1 DAEs. Circuits, Systems and Signal Processing, 19: 131-157.
28. Riaza, R., and Zufiria, P. J., 2001, Stability of singular equilibria in quasilinear implicit differential equations. Journal of Differential Equations, 171: 24-53.
29. Riaza, R., and Zufiria, P. J., 2002, Discretization of implicit ODEs for singular root-finding problems, Journal of Computational and Applied Mathematics, 140: 695-712.
30. Saberi, A., and Khalil, H., 1984, Quadratic-type Lyapunov functions for singularly perturbed systems. IEEE Transactions on Automatic Control, 29: 542-550.
31. Schwarz, J., Dangelmayr, G., Stevens, A., and Bräuer, K., 2001, Burst and spike synchronization of coupled neural oscillators. Dynamical Systems, 16: 125-156.
32. Sudharsanan, S. I., and Sundareshan, M. K., 1991a, Equilibrium characterization of dynamical neural networks and a systematic synthesis procedure for associative memories. IEEE Transactions on Neural Networks, 2: 509-521.
33. Sudharsanan, S. I., and Sundareshan, M. K., 1991b, Exponential stability and a systematic synthesis of a neural network for quadratic minimization. Neural Networks, 4: 599-613.
34. Sundareshan, M. K., and Condarcure, T. A., 1998, Recurrent neural-network training by a learning automaton approach for trajectory learning and control system design. IEEE Transactions on Neural Networks, 9: 354-368.
35. Tsutsumi, K., 1999, On redundancy in neural architecture: dynamics of a simple module-based neural network and initial-state independence. Neural Networks, 12: 1075-1085.
36. Vasileva, A. B., Butuzov, V. F., and Kalachev, L. V., 1995, The Boundary Function Method for Singular Perturbation Problems (Philadelphia: SIAM).
37. Wang, Y. J., and Lin, C. T., 1998, Runge-Kutta neural network for identification of dynamical systems in high accuracy. IEEE Transactions on Neural Networks, 9: 294-307.
38. Yang, H., and Dillon, T. S., 1994, Exponential stability and oscillation of Hopfield graded response neural networks. IEEE Transactions on Neural Networks, 5: 719-729.
39. Zufiria, P. J., Fraile-Ardanuy, J., Riaza, R., and Alonso, J. I., 1999, Neural adaptive control of non-linear plants via a multiple inverse model approach. International Journal of Adoptive Control and Signal Processing, 13: 219-239.
40. Zurada, J. M., 1992, Introduction to Artificial Neural Systems (St Paul: West Publishing Company).