The incremental Gauss-Newton algorithm with adaptive stepsize rule

Computational Optimization and Applications

Volumn 26, Issue 2, 2003, Pages 107-141

  Moriyama, Hiroyuki ^a   Yamashita, Nobuo ^a   Fukushima, Masao ^a  

^a KYOTO UNIVERSITY   (Japan)

Author keywords

Adaptive stepsize; Extended Kalman filter; Incremental algorithm; Neural network

Indexed keywords

ALGORITHMS; BOUNDARY CONDITIONS; CONVERGENCE OF NUMERICAL METHODS; CURVE FITTING; ITERATIVE METHODS; KALMAN FILTERING; MATRIX ALGEBRA; NEURAL NETWORKS; PATTERN RECOGNITION; VECTORS;

ADAPTIVE STEPSIZE RULE; EXTENDED KALMAN FILTERING; GAUSS-NEWTON ALGORITHM; INCREMENTAL ALGORITHM; NONLINEAR LEAST SQUARES PROBLEM;

OPTIMIZATION;

EID: 0242365738     PISSN: 09266003     EISSN: None     Source Type: Journal    
DOI: 10.1023/A:1025703629626     Document Type: Article

References (12)

1. D.P. Bertsekas, "Incremental least squares methods and the extended Kalman filter," SIAM J. Optimization, vol. 3, pp. 807-822, 1996.
2. D.P. Bertsekas, Nonlinear Programming, Athena Scientific: Belmont, MA, 1996.
3. D.P. Bertsekas, "A new class of incremental gradient methods for least squares problems," SIAM J. Optimization, vol. 2, pp. 913-926, 1997.
4. J.E. Dennis, Jr and R.B. Schnabel, Numerical Methods for Unconstrained Optimization and Nonlinear Equations, Prentice-Hall: Englewood Cliffs, NJ, 1983.
5. Z.Q. Luo, "On the convergence of the LMS algorithm with adaptive learning rate for linear feedforward networks," Neural Computation, vol. 3, pp. 226-245, 1991.
6. Z.Q. Luo and P. Tseng, "Analysis of an approximate gradient projection method with applications to the back propagation algorithm," Optimization Methods and Software, vol. 4, pp. 85-101, 1994.
7. O.L. Mangasarian and M.V. Solodov, "Serial and parallel backpropagation convergence via nonmonotone pertubed minimization," Optimization Methods and Software, vol. 4, 103-116, 1994.
8. W.S. McCulloch and W. Pitts, "A logical calculus of the ideas immanent in nervous activity," Bull. Math. Biophys., vol. 5, pp. 115-133, 1943.
9. T.N. Pappas, "Solution of nonlinear equations by Davidon's least squares method," M.S. Thesis, Dept. of Electrical Engineering and Computer Science, Massachusetts Institute of Technology, Cambridge, MA, 1982.
10. B.T. Poljak, Introduction to Optimization, Optimization Software Inc.: New York, NY, 1987.
11. D.E. Rumelhart, G.E. Hinton, and R.J. Williams, "Learning internal representations by error propagation," in Parallel Distributed Processing, Vol.1, D.E. Rumelhart and J.L. MuClelland (Eds.), MIT Press: Cambridge, MA, 1986, pp. 318-362.
12. P. Tseng, "Incremental gradient(-projection) method with momentum term and adaptive stepsize rule," SIAM J. Optimization, vol. 8, pp. 506-531, 1998.