A homotopy method for training neural networks

Signal Processing

Volumn 64, Issue 3, 1998, Pages 359-370

  Lendl, Markus ^a   Unbehauen, Rolf ^a   Luo, Fa Long ^a  

^a UNIVERSITY OF ERLANGEN NUREMBERG   (Germany)

Author keywords

Gauss Newton method; Homotopy; Multilayer perceptron; Second order learning

Indexed keywords

COMPUTER SIMULATION; CONVERGENCE OF NUMERICAL METHODS; ERROR CORRECTION; LEARNING SYSTEMS; OPTIMIZATION; PROBLEM SOLVING; REGRESSION ANALYSIS;

HOMOTOPY METHOD (HM);

FEEDFORWARD NEURAL NETWORKS;

EID: 0032002106     PISSN: 01651684     EISSN: None     Source Type: Journal    
DOI: 10.1016/S0165-1684(97)00201-6     Document Type: Article

References (20)

1. E.L. Allgower, K. Georg, Numerical Continuation Methods, Springer, Berlin, 1990.
2. R. Battiti, First and second order methods for learning: between steepest descent and Newton's method, Neural Computation 4 (2) (1992) 141-166.
3. C.M. Bishop, Neural Networks for Pattern Recognition, Clarendon Press, Oxford, 1995.
4. A. Cichocki, R. Unbehauen, Neural Networks for Optimization and Signal Processing, Teubner, Stuttgart, 1993.
5. G. Cybenko, Approximation by superposition of a sigmoidal function, Math. Control Signals Systems 2 (1989) 304-314.
6. N. de Villiers, D. Glasser, A continuation method for nonlinear regression, SIAM J. Numer. Anal. 18 (6) (December 1981) 1139-1154.
7. J.E. Dennis, R.B. Schnabel, Numerical Methods for Unconstrained Optimization and Nonlinear Equations, SIAM, Philadelphia, PA, 1996.
8. R.O. Duda, P.E. Hart, Pattern Classification and Scene Analysis, Wiley, New York, 1973.
9. R. Fletcher, Practical Methods of Optimization, Wiley, New York, 1995.
10. K. Funahashi, On the approximate realization of continuous mappings by neural networks, Neural Networks 2 (1989) 183-192.
11. M.T. Hagan, M.B. Menhaj, Training feedforward networks with the Marquardt algorithm, IEEE Transactions on Neural Networks 5 (6) (November 1994) 989-993.
12. S. Haykin, Neural Networks, Macmillan, UK, 1994.
13. K. Hornik, M. Stincombe, H. White, Neural networks are universal approximators, Neural Networks 2 (1990) 359-366.
14. K. Levenberg, A method for the solution of certain problems in least squares, Quart. Appl. Math 2 (1944) 164-168.
15. D. Marquardt, An algorithm for least-squares estimation of nonlinear parameters, SIAM J. Appl. Math 11 (1963) 431-441.
16. D. Nguyen, B. Widrow, Improving the learning speed of 2-layer neural networks by choosing initial values of the adaptive weights, in: IJCNN 1990, 1990, Vol. 3, San Diego, pp. 21-26.
17. A.J. Owens, D.L. Filkin, Efficient training of the back-propagation network by solving a system of stiff ordinary differential equations, in: IJCNN 1989, San Diego, 1989, Vol. 2, pp. 381-386.
18. W.H. Press, Numerical Recipes in C, Cambridge University Press, Cambridge, 1995.
19. W. Schiffmann, M. Joost, R. Werner, Optimization of the backpropagation algorithm for training multilayer perceptrons, Tech. Rep., Institute of Computer Science, University of Koblenz, ftp://128.146.8.52/pub/neuroprose/ schiff.bp_speedup.ps.Z, 1993.
20. C.W. Therrien, Decision, Estimation and Classification, Wiley, New York, 1989.