A Layer-by-Layer Least Squares based Recurrent Networks Training Algorithm: Stalling and Escape

Neural Processing Letters

Volumn 7, Issue 1, 1998, Pages 15-25

  Cho, Siu Yeung ^a   Chow, Tommy W S ^a  

^a CITY UNIVERSITY OF HONG KONG   (Hong Kong)

Author keywords

Convergence stalling; Fast convergence speed; Layer by Layer Least Squares algorithm; Recurrent networks

Indexed keywords

CONVERGENCE OF NUMERICAL METHODS; LEAST SQUARES APPROXIMATIONS; MATHEMATICAL MODELS; NEURAL NETWORKS; PROBABILITY;

RECURRENT NETWORKS;

LEARNING ALGORITHMS;

EID: 0031989111     PISSN: 13704621     EISSN: None     Source Type: Journal    
DOI: 10.1023/A:1009672319869     Document Type: Article

References (10)

1. R.E. Rumelhard, G.E. Hinton and R.J. Williams, "Learning internal representation by error baclpropagation", Parallel Distributed Processing: Explorations in Microstructure of Cognition, Vol. 1, 1986.
2. R.J. Williams and D. Ziper, "A learning algorithm for continually running fully recurrent neural networks", Neural Computation, Vol. 1, No. 2, pp. 270-280, 1989.
3. R.A. Jacobs, "Increased rates of convergence through learning rate adaptation", Neural Networks, Vol. 1, pp. 295-307, 1988.
4. C.M. Kuan, "A recurrent Newton algorithm and its convergence properties", IEEE Trans. on Neural Networks, Vol. 6, No. 3, pp. 779-783, 1995.
5. R.J. Willams, "Training Recurrent Networks Using the Extended Kalman Filter", International Joint Conference on Neural Networks, Vol. IV, pp. 241-246, Baltimore, 1992.
6. G.V. Puskorius and L.A. Feldkamp, "Recurrent networks training with the decoupled extended Kalman filter algorithm", in Proceeding of the 1992 SPIE Conference on the Science of Artificial Neural Networks, Vol. 1710, pp. 461-473, Orlando, 1992.
7. G.V. Puskorius and L.A. Feldkamp, "Neurocontrol of Nonlinear Dynamical Systems with Kalman Filter Trained Recurrent Networks", IEEE Trans. on Neural Networks, Vol. 5, No. 2, pp. 279-297, 1994.
8. F. Biegler-König and F. Bärmann, "A learning algorithm for multilayered neural networks based on linear least squares problems", Neural Networks, Vol. 6, pp. 127-131, 1993.
9. Y.F. Yam and T.W.S. Chow, "Accelerated training algorithm for feedforward neural networks based on least squares method", Neural Processing Letter, Vol. 2, No. 4, pp. 20-25, 1995.
10. J.Y.F. Yam and T.W.S. Chow, "Extended Least Squares Based Algorithm for Training Feedforward Networks", IEEE Trans. on Neural Networks, Vol. 8, no. 3, pp. 806-810, May, 1997.