Neural network identification: A survey of gradient based methods

IEE Colloquium (Digest)

Volumn , Issue 521, 1998, Pages

  McLoone, Seán ^a  

^a QUEEN'S UNIVERSITY BELFAST   (United Kingdom)

Author keywords

[No Author keywords available]

Indexed keywords

ALGORITHMS; MATHEMATICAL MODELS; OPTIMIZATION;

GRADIENT BASED OPTIMIZATION METHODS; NEURAL NETWORK IDENTIFICATION;

NEURAL NETWORKS;

EID: 0032255815     PISSN: 09633308     EISSN: None     Source Type: Journal    
DOI: None     Document Type: Review

References (33)

1. An P.E., Brown M., Harris C.J. and Chen S., "Comparative aspects of neural network algorithms for online modelling of dynamic processes", Proc. Instn. Mech. Engrs., Vol. 207, 1993, pp. 223-241.
2. Anguita D., DaCanal A., Da Canal W. Falcone A. and Scapolla A.M., "On the distributed implementation of the back-propagation", Int. Conf. on Artificial Neural Networks, ICANN '94, Sorrento, Italy, pp. 1376-1379, 1994.
3. Battiti R. and Masulli F., "BFGS optimization for faster automated supervised learning", International Neural Network Conference, Vol. 2, pp. 757-760, 1990.
4. Besch M. and Pohl H.W., "Flexible data parallel training of neural networks using MIMD-computers", In: Third Workshop on Parallel and Distributed Processing, Sanremo, Italy, 1995.
5. Bishop C.M., "Improving the generalisation properties of radial basis function neural networks", Neural Computation, Vol.3, 1991, pp. 579-588.
6. Chen S. and Billings S.A., "Recursive prediction error estimator for non-linear models", Int. J. Control, vol. 49, No. 2, 1989, pp. 569-594.
7. Chen S., Billings A. and Grant P.M., "Recursive hybrid algorithm for non-linear system identification using radial basis function networks", Int. J. Control, Vol. 55, No. 5, 1992, 1051-1070.
8. Chen S., Cowan C.F.N, Billings S.A. and Grant P.M., "Parallel recursive prediction error algorithm for training layered neural networks", Int. J. Control, Vol. 51, No. 6, 1990, pp. 1215-1228.
9. Drucker H. and LeCun Y., "Improving generalisation performance using double backpropagation", IEEE Trans, on Neural Networks, Vol.3, No.6, pp. 991-997.
10. Fahlman S.E, "Fast learning variations on backpropagation: An empirical study", Proc. 1988 Connectionist Models Summer School, Touretzky D.S, Hinton G. and Sejnowski T., pp. 38-51, 1988.
11. Lightbody G. and Irwin G.W., "A parallel algorithm for training neural network based nonlinear models", Proc. 2nd IFAC Workshop on Algorithms and Architectures for Real-time Control, pp. 99-104, 1992.
12. Ljung L. and Soderstrom T., Theory and practice of recursive identification, MIT Press, 1983.
13. McLoone S. and Irwin G., "Improving neural network training solutions using regularisation", submitted to Neurocomputing, December 1997.
14. McLoone S., Brown M., Irwin G. and Lightbody G., 'A hybrid linear/nonlinear training algorithm for feedforward neural networks', IEEE Trans, on Neural Networks, Vol. 9, no. 4, 1998, pp. 669-684.
15. McLoone S. and Irwin G., "Nonlinear optimization of RBF networks", International Journal of Systems Science, Vol. 29, No.2, 1998, pp. 179-189.
16. McLoone S. and Irwin G., "A variable memory quasi-newton training algorithm", Neural Processing Letters, Vol. 9, 1998, pp. 1-13.
17. McLoone S. and Irwin G., "Fast parallel off-line training of multilayer perceptrons", IEEE Trans. on Neural Networks, Vol. 8, No. 3, 1997, pp. 646-653.
18. McLoone S. and Irwin G., "Fast gradient based off-line training of multilayer perceptrons", In: Neural Network Engineering in Dynamic Control Systems, Hunt K.J., Irwin G.W. and Warwick K. (Eds.), Springer, 1995, pp. 179-200.
19. Møller M.F., "A scaled conjugate gradient algorithm for fast supervised learning", Neural Networks, Vol. 6, pp. 525-533, 1990
20. Niu S. and Fisher D., "Simultaneous structure identification and parameter estimation of multivariable systems", Int. J. Control, vol. 59, No. 5, 1994, pp. 1127-1141.
21. Orr M.J., "Regularisation in the selection of radial basis function centres", Neural Computation, Vol. 6, 1995, pp. 606-623.
22. Parks P.C. and Militzer J., "A comparison of five algorithms for the training of CMAC memories for learning control systems", Automatica, Vol. 28, No. 5, 1992, pp. 1027-1035.
23. Piché S.W., "Steepest Descent Algorithms for Neural Network Controllers and Filters", IEEE Trans, on Neural Networks", Vol. 5, No. 2, March 1994, pp. 198-210.
24. Reeves C.R., (ed.) Modern Heuristic Techniques for Combinatorial Problems, Wiley, 1993.
25. Rumelhart D.E. and McClelland J.L., Parallel Distributed Processing, Cambridge, MA, The MIT Press, 1986.
26. Schiffmann W., Joost M., Werner R., "Optimization of the backpropagation algorithm for training multilayer perceptrons", University of Koblenz, Institute of Physics, Rheinau 3-4, W-5400 Koblenz, Germany.
27. Sjöberg J. and Viberg M., "Separable non-linear least-squares minimization - Possible improvements for neural networks", Proc. IEEE Workshop in Neural Networks for Signal Processing, Amelia Island Plantation, Florida, Sep. 24-26, 1997.
28. Takumi Ichimura, Takeshi Takano and Eiichiro Tazaki, "Learning of neural networks using hybrid genetic algorithms", 4th European Congress on Intelligent Techniques and Soft Computing (EUFIT'96), 1996.
29. Van Der Smagt P., "Minimisation methods for training feedforward neural networks", Neural Networks, Vol. 7, No.1, pp. 1-11, 1994.
30. Wasserman, P.D.: "Neural Computing, Theory and Practice': Van Nostrand Reinhold, 1989.
31. Webb A.R., Lowe D. and Bedworth M.D., 1988, "A comparison of nonlinear optimization strategies for feed-forward adaptive layered networks", Royal Signals and Radar Establishment, Memorandum 4157, Controller HMSO, London.
32. Werbos P., "Backpropagation through time: what it does and how to do it?", Proc. of IEEE, Vol. 78, 1990, pp. 1550-1560.
33. Zhu Q.M. and Billings S.A., "Fast orthogonal identification of nonlinear stochastic models and radial basis function neural networks", Int. J. control. Vol. 64, No.5, 1996, pp. 871-886.