Distributed asynchronous incremental subgradient methods

Studies in Computational Mathematics

Volumn 8, Issue C, 2001, Pages 381-407

  Nedic A ^a   Bertsekas, D P ^a   Borkar, V S ^b  

^a MASSACHUSETTS INSTITUTE OF TECHNOLOGY   (United States)

^b TATA INSTITUTE OF FUNDAMENTAL RESEARCH   (India)

Author keywords

[No Author keywords available]

Indexed keywords

EID: 77956652909     PISSN: 1570579X     EISSN: None     Source Type: Book Series    
DOI: 10.1016/S1570-579X(01)80023-9     Document Type: Chapter

References (26)

1. A. Ben-Tal, T. Margalit, and A. Nemirovski, The Ordered Subsets Mirror Descent Optimization Method and its Use for the Positron Emission Tomography Reconstruction, SIAM J. Optim., to appear.
2. Benveniste A., Metivier M., and Priouret P. Adaptive Algorithms and Stochastic Approximations (1990), Springer-Verlag, New York
3. Bertsekas D.P. A New Class of Incremental Gradient Methods for Least Squares Problems. SIAM. J. on Optimization 7 (1997) 913-926
4. Bertsekas D.P., and Tsitsiklis J.N. Parallel and Distributed Computation: Numerical Methods (1989), Prentice-Hall, Inc.
5. Bertsekas D.P., and Tsitsiklis J.N. Neuro-Dynamic Programming (1996), Athena Scientific, Belmont, Massachusetts
6. Bertsekas D.P., and Tsitsiklis J.N. Gradient Convergence in Gradient Methods. SIAM. J. on Optimization 3 (2000) 627-642
7. Borkar V.S. Asynchronous Stochastic Approximation. SIAM J. on Optimization 36 (1998) 840-851
8. Gaivoronski A.A. Convergence Analysis of Parallel Backpropagation Algorithm for Neural Networks. Opt. Meth. and Software 4 (1994) 117-134
9. Gallager R.G. Discrete Stochastic Processes (1996), Kluwer Academic Publishers
10. Grippo L. A Class of Unconstrained Minimization Methods for Neural Network Training. Opt. Meth. and Software 4 (1994) 135-150
11. Kaskavelis C.A., and Caramanis M.C. Efficient Lagrangian Relaxation Algorithms for Industry Size Job-Shop Scheduling Problems. IIE Trans. on Scheduling and Logistics 30 (1998) 1085-1097
12. Kibardin V.M. Decomposition into Functions in the Minimization Problem. Automation and Remote Control 40 (1980) 1311-1323
13. K. C. Kiwiel and P. O. Lindberg, Parallel Subgradient Methods for Convex Optimization,submitted to the Proceedings of the March 2000 Haifa Workshop "Inherently Parallel Algorithms in Feasibility and Optimization and Their Applications", D. Butnariu Y. Censor, and S. Reich, Eds., Studies in Computational Mathematics, Elsevier, Amsterdam.
14. K. C. Kiwiel, Convergence of Approximate and Incremental Subgradient Methods forConvex Optimization, submitted to SIAM J. on Optimization.
15. Kushner H.J., and Yin G. It Stochastic Approximation Algorithms and Applications (1997), Springer-Verlag, New York
16. Luo Z.Q. On the Convergence of the LMS Algorithm with Adaptive Learning Rate for Linear Feedforward Networks. Neural Computation 3 (1991) 226-245
17. Luo Z.Q., and Tseng P. Analysis of an Approximate Gradient Projection Method with Applications to the Backpropagation Algorithm. Opt. Meth. and Software 4 (1994) 85-101
18. Mangasarian O.L., and Solodov M.V. Serial and Parallel Backpropagation Convergence Via Nonmonotone Perturbed Minimization. Opt. Meth. and Software 4 (1994) 103-116
19. Nedić A., and Bertsekas D.P. Incremental Subgradient Methods for Nondifferentiable Optimization. Lab. for Info. and Decision Systems Report LIDS-P-2460 (1999), Massachusetts Institute of Technology, Cambridge, MA
20. A. Nedić and D. P. Bertsekas, Incremental Subgradient Methods for Nondifferentiable Optimization, submitted to SIAM J. Optimization.
21. to appear in Stochastic Optimization: Algorithms and Applications. Nedić A., and Bertsekas D.P. Convergence Rate of Incremental Subgradient Algorithms. In: Uryasev S., and Pardalos P.M. (Eds). Lab. for Info. and Decision Systems Report LIDS-P-2475 (2000), Massachusetts Institute of Technology, Cambridge, MA
22. Solodov M.V., and Zavriev S.K. Error Stability Properties of Generalized Gradient-Type Algorithms. J. of Opt. Theory and Applications 98 (1998) 663-680
23. Tseng P. An Incremental Gradient(-Projection) Method with Momentum Term and Adaptive Stepsize Rule. SIAM J. on Optimization 2 (1998) 506-531
24. Tsitsiklis J.N., Bertsekas D.P., and Athans M. Distributed Asynchronous Deterministic and Stochastic Gradient Optimization Algorithms. IEEE Trans. on Automatic Control AC-31 (1986) 803-812
25. Widrow B., and Hoff tiAdaptive Switching Circuits M.E. Institute of Radio Engineers, Western Electronic Show and Convention, Convention Record part 4 (1960) 96-104
26. Zhao X., Luh P.B., and Wang J. Surrogate Gradient Algorithm for Lagrangian Relaxation. J. of Opt. Theory and Applications 100 (1999) 699-712