Gradient convergence in gradient methods with errors

SIAM Journal on Optimization

Volumn 10, Issue 3, 2000, Pages 627-642

  Bertsekas, Dimitri P ^a   Tsitsiklis, John N ^a  

^a USA   (United States)

Author keywords

Gradient convergence; Gradient methods; Incremental gradient methods; Stochastic approximation

Indexed keywords

EID: 0034389611     PISSN: 10526234     EISSN: None     Source Type: Journal    
DOI: 10.1137/S1052623497331063     Document Type: Article

References (20)

1. A. BENVENISTE, M. METIVIER, AND P. PRIOURET, Adaptive Algorithms and Stochastic Approximations, Springer-Verlag, New York, 1990.
2. D. P. BERTSEKAS AND J. N. TSITSIKLIS, Parallel and Distributed Computation: Numerical Methods, Prentice-Hall, Englewood Cliffs, NJ, 1989.
3. D. P. BERTSEKAS AND J. N. TSITSIKLIS, Neuro-Dynamic Programming, Athena Scientific, Belmont, MA, 1996.
4. D. P. BERTSEKAS, Nonlinear Programming, Athena Scientific, Belmont, MA, 1995.
5. D. P. BERTSEKAS, A new class of incremental gradient methods for least squares problems, SIAM J. Optim., 7 (1997), pp. 913-926.
6. V. S. BORKAR, Asynchronous stochastic approximations, SIAM J. Control Optim., 36 (1998), pp. 840-851.
7. B. DELYON, General results on the convergence of stochastic algorithms, IEEE Trans. Automat. Control, 41 (1996), pp. 1245-1255.
8. A. A. GAIVORONSKI, Convergence analysis of parallel backpropagation algorithm for neural networks, Optim. Methods Software, 4 (1994), pp. 117-134.
9. L. GRIPPO, A class of unconstrained minimization methods for neural network training, Optim. Methods Software, 4 (1994), pp. 135-150.
10. H. J. KUSHNER AND D. S. CLARK, Stochastic Approximation Methods for Constrained and Unconstrained Systems, Springer-Verlag, New York, 1978.
11. H. J. KUSHNER AND G. YIN, Stochastic Approximation Methods, Springer-Verlag, New York, 1996.
12. L. LJUNG, Analysis of recursive stochastic algorithms, IEEE Trans. Automat. Control, 22 (1977), pp. 551-575.
13. Z. Q. LUO AND P. TSENG, Analysis of an approximate gradient projection method with applications to the backpropagation algorithm, Optim. Methods Software, 4 (1994), pp. 85-101.
14. Z. Q. LUO, On the convergence of the LMS algorithm with adaptive learning rate for linear feedforward networks, Neural Comput., 3 (1991), pp. 226-245.
15. O. L. MANGASARIAN AND M. V. SOLODOV, Serial and parallel backpropagation convergence via nonmonotone perturbed minimization, Optim. Methods Software, 4 (1994), pp. 103-116.
16. G. PFLUG, Optimization of Stochastic Models. The Interface Between Simulation and Optimization, Kluwer, Boston, 1996.
17. B. T. POLJAK AND Y. Z. TSYPKIN, Pseudogradient adaptation and training algorithms, Automat. Remote Control, 12 (1973), pp. 83-94.
18. B. T. POLJAK, Introduction to Optimization, Optimization Software Inc., New York, 1987.
19. J. N. TSITSIKLIS, D. P. BERTSEKAS, AND M. ATHANS, Distributed asynchronous deterministic and stochastic gradient optimization algorithms, IEEE Trans. Automat. Control, 31 (1986), pp. 803-812.
20. H. WALK, Foundations of stochastic approximation, in Stochastic Approximation and Optimization of Random Systems, L. Ljung, G. Pflug, and H. Walk, eds., Birkhauser, Boston, 1992, pp. 53-93.