An Efficient Learning Algorithm with Second-Order Convergence for Multilayer Neural Networks

Proceedings of the International Joint Conference on Neural Networks

Volumn 3, Issue , 2003, Pages 2028-2032

  Ninomiya, Hiroshi ^a   Tomita, Chikahiro ^b   Asai, Hideki ^b  

^a SHONAN INSTITUTE OF TECHNOLOGY   (Japan)

^b SHIZUOKA UNIVERSITY   (Japan)

Author keywords

[No Author keywords available]

Indexed keywords

ALGORITHMS; BACKPROPAGATION; COMPUTER SIMULATION; ITERATIVE METHODS; NUMERICAL ANALYSIS;

STEEPEST DESCENT METHOD;

MULTILAYER NEURAL NETWORKS;

EID: 0141682840     PISSN: None     EISSN: None     Source Type: Conference Proceeding    
DOI: None     Document Type: Conference Paper

References (11)

1. D. E. Rumelhart, G. E. Hinton and R. J. Williams: "Learning representations by back-propagation errors", NATURE, vol.323, 9, pp.533-536, Oct., 1986.
2. nd ed.", Prentice Hall, 1999.
3. H. Ninomiya and N. Kinoshita: "A New learning Algorithm without Explicit Error Back-Propagation", ProcIEEE&INNS/IJCNN'99, July, 1999.
4. H. Ninomiya and A. Sasaki: "3-Layer Recurrent Neural Networks and their Supervised Learning Algorithm", Proc.IEEE&INNS/IJCNN'01, July, 2001.
5. R. Battiti: "First and second-order methods for learning: Between steepest descent and Newton's method", Neural Computa., vol.4, no.2, pp. 141-166, 1992.
6. K. Levenberg: "A method for the solution of certain problems in least squares", Quart. Appl. Math., vol.5, pp. 164-168, 1944.
7. D. Marquardt: "An algorithm for least squares estimation of nonlinear parameters", SIAM J. Appl. Math., vol.11, pp.431-441, 1963.
8. M. T. Hagan and M. Menhaj: "Training feedforward networks with the Marquardt algorithm", IEEE Trans. Neural Networks, vol.5, pp.989-993, Nov., 1994.
9. H. Asai: "Implicit Steepest Descent Method and its Analogy with Charge-Up Method based on virtual Capacitors in Network Analysis", Proc. IEEE/ISCAS'88, pp.1115-1118, 1988.
10. H. Asai: "Equivalent Property between Network Analysis Using Virtual Capacitors and Steepest Descent Method", Trans. IEICE(A), Vol.J71-A, no.5, pp. 1132-1138, May, 1998(in Japanese).
11. W. J. McCalla: "Fundamentals of Computer-Aided Circuit Simulation", Boston, MA, Kluwer Academic Publishers, 1988.