On-line stabilization of block-diagonal recurrent neural networks

IEEE Transactions on Neural Networks

Volumn 10, Issue 1, 1999, Pages 167-175

  Sivakumar, S C ^a   Robertson, W ^a   Phillips, W J ^a  

^a DALHOUSIE UNIVERSITY   (Canada)

Author keywords

Nonlinear identification; Recurrent neural networks; Stability of recurrent neural networks

Indexed keywords

ALGORITHMS; ARTIFICIAL INTELLIGENCE; COMPUTER SIMULATION; DISCRETE TIME CONTROL SYSTEMS; FEEDBACK CONTROL; FUNCTIONS; MATRIX ALGEBRA; ONLINE SYSTEMS;

BLOCK-DIAGONAL FEEDBACK WEIGHT MATRIX; CONVENTIONAL BACKPROPAGATION THROUGH TIME (BPTT) ALGORITHMS; DISCRETE-TIME RECURRENT NEURAL NETWORKS (DTRNN);

RECURRENT NEURAL NETWORKS;

EID: 0032762127     PISSN: 10459227     EISSN: None     Source Type: Journal    
DOI: 10.1109/72.737503     Document Type: Article

References (24)

1. R. J. Williams and D. Zipser, "A learning algorithm for continually running fully recurrent neural networks," Neural Comput., vol. 1, pp. 270-280, 1989.
2. L. B. Almeida, "Backpropagation in perceptrons with feedback," NATO ASI series, vol. F41, Neural Comput., R. Eckmiller and Ch. v. d. Malsburg, Eds. New York: Springer-Verlag, 1988, pp. 199-208.
3. F. J. Pineda, "Recurrent backpropagation and the dynamical approach to adaptive neural computation," Neural Comput., vol. 1, pp. 161-172, 1989.
4. B. A. Pearlmutter, "Learning state-space trajectories in recurrent neural networks," Neural Comput., vol. 1, pp. 263-269, 1989.
5. P. J. Werbos, "Backpropagation through time: What it does and how to do it," in Proc. IEEE, vol. 78, no. 10, pp. 1550-1560, Oct. 1990.
6. M. Gori, Y. Bengio, and R. De Mori, "BPS: A learning algorithm for capturing the dynamic nature of speech," in Int. Joint Conf. Neural Networks, 1989, vol. 2, pp. 417-423.
7. A. C. Tsoi and A. D. Black, "Locally recurrent globally feedforward networks: A critical review of architectures," IEEE Trans. Neural Networks, vol. 5, Mar. 1994.
8. B. A. Pearlmutter, "Gradient calculations for dynamic recurrent neural networks: A survey," IEEE Trans. Neural Networks, vol. 6, no. 5, Sept. 1995.
9. L. K. Li, "Fixed point analysis for discrete time recurrent neural networks," in Proc. Int. Joint Conf. Neural Networks, June 1992, vol. 4, pp. 134-139.
10. L. Jin, P. N. Nikiforuk, and M. M. Gupta, "Absolute stability conditions for discrete-time recurrent neural networks," IEEE Trans. Neural Networks, vol. 5, pp. 954-964, Nov. 1994.
11. P. Y. Simard, J. P. Rayzs, and B. Victorri, "Shaping the state space landscape in recurrent networks," Advances in Neural Inform. Processing Syst., vol. 3, pp. 105-112, 1991.
12. L. Jin and M. M. Gupta, "Globally asymptotical stability of discrete-time analog neural networks," IEEE Trans. Neural Networks, vol. 7, pp. 1024-1031, July 1996.
13. D. R. Hush and B. G. Home, "Progress in supervised neural networks - What's new since Lippmann?," IEEE Signal Processing Mag., pp. 8-39, Jan. 1993.
14. K. S. Narendra and K. Parthasarathy, "Identification and control of dynamical systems using neural networks," IEEE Trans. Neural Networks, vol. 1, pp. 4-27, Mar. 1990.
15. S. P. Day and M. R. Davenport, "Continuous-time temporal backpropagation with adaptive time delays," IEEE Trans. Neural Networks, vol. 4, Mar. 1993.
16. R. J. Williams and J. Peng, "An efficient gradient-based algorithm for on-line training of recurrent network trajectories," Neural Comput., vol. 2, pp. 490-501, 1990.
17. F. Beaufays and E. A. Wan, "Relating real-time backpropagation and backpropagation through time: An application of flow graph interreciprocity," Neural Comput., vol. 6, pp. 296-305, 1994.
18. S. Y. Kung, Digital Neural Networks: Deterministic Temporal Neural Networks. Englewood Cliffs, NJ: Prentice-Hall, 1993, pp. 219-224, Ch. 6.
19. C. W. Barnes, "A parametric approach to the realization of second-order digital filter sections," IEEE Trans. Circuits Syst., vol. CAS-32, June 1985.
20. C. C. Ku and K. Y. Lee, "Diagonal recurrent neural networks for dynamic systems control," IEEE Trans. Neural Networks, vol. 6, pp. 144-156, Jan. 1995.
21. Y. Fang and T. J. Sejnowski, "Faster learning for dynamic recurrent backpropagation," Neural Comput., vol. 2, pp. 270-273, 1990.
22. R. Parisi, E. D. Di Claudio, A. Rapagnetta, and G. Orlandi, "Recursive least squares approach to learning in recurrent neural networks," in Proc. Int. Conf. Neural Networks, 1996, vol. 2, pp. 1350-1354.
23. M. C. Mackey and L. Glass, "Oscillation and chaos in physiological control systems," Science, vol. 197, pp. 287-289, July 1987.
24. T. D. Sanger, "A tree structured adaptive network for function approximation in high dimensional spaces," IEEE Trans. Neural Networks, vol. 2, Mar. 1991.