Error stability properties of generalized gradient-type algorithms

Journal of Optimization Theory and Applications

Volumn 98, Issue 3, 1998, Pages 663-680

  Solodov, M V ^a   Zavriev, S K ^b  

^a IMPA   (Brazil)

^b MOSCOW STATE UNIVERSITY   (Russian Federation)

Author keywords

Approximate solutions; Error stability; Gradient type methods; Incremental algorithms; Perturbation analysis

Indexed keywords

PERTURBATION TECHNIQUES;

APPROXIMATE SOLUTION; ERROR STABILITY; GENERALIZED GRADIENTS; GRADIENT TYPE; GRADIENT-TYPE METHOD; INCREMENTAL ALGORITHM; PERTURBATION ANALYSIS; STABILITY PROPERTIES; TYPE METHODS; UNIFIED FRAMEWORK;

STABILITY;

EID: 0032345479     PISSN: 00223239     EISSN: None     Source Type: Journal    
DOI: 10.1023/A:1022680114518     Document Type: Article

References (29)

1. CLARKE, F. H., Optimization and Nonsmooth Analysis, John Wiley and Sons, New York, New York, 1983.
2. ALBER, YA. I., IUSEM, A. N., and SOLODOV, M. V., On the Projected Subgradient Methods for Nonsmooth Convex Optimization in a Hubert Space, Mathematical Programming, Vol. 81, pp. 23-25, 1998.
3. ZAVRIEV, S. K., and PEREVOZCHIKOV, A. G., Direct Lyapunov Method in Attraction Analysis of Finite-Difference Inclusions, USSR Computational Mathematics and Mathematical Physics, Vol. 30, pp. 22-32, 1990.
4. ZAVRIEV, S. K., Convergence Properties of the Gradient Method under Variable Level Interference, USSR Computational Mathematics and Mathematical Physics, Vol. 30, pp. 997-1007, 1990.
5. SOLODOV, M. V., Convergence Analysis of Perturbed Feasible-Descent Methods, Journal of Optimization Theory and Applications, Vol. 92, pp. 337-353, 1997.
6. SOLODOV, M. V., and SVAITER, B. F., Descent Methods with Line Search in the Presence of Perturbations, Journal of Computational and Applied Mathematics, Vol. 80, pp. 265-275, 1997.
7. BERTSEKAS, D. P., Nonlinear Programming, Athena Scientific, Belmont, Massachusetts, 1995.
8. KHANNA, T., Foundations of Neural Networks, Addison-Wesley, Reading, Massachusetts, 1989.
9. ANDERSON, B. D. O., and MOORE, J. B., Optimal Filtering, Prentice-Hall, Englewood Cliffs, New Jersey, 1979.
10. MAYNE, D. Q., and POLAK, E., Nondifferentiable Optimization via Adaptive Smoothing, Journal of Optimization Theory and Applications, Vol. 43, pp. 601-614, 1984.
11. MANGASARIAN, O. L., Mathematical Programming in Neural Networks, ORSA Journal on Computing, Vol. 5, pp. 349-360, 1993.
12. MANGASARIAN, O. L., and SOLODOV, M. V., Backpropagation Convergence via Deterministic Nonmonotone Perturbed Minimization, Neural Information Processing Systems, Edited by G. Tesauro, J. D. Cowan, and J. Alspector, Morgan Kaufmann Publishers, San Francisco, California, Vol. 6, pp. 383-390, 1994.
13. MANGASARIAN, O. L., and SOLODOV, M. V., Serial and Parallel Backpropagation Convergence via Nonmonotone Perturbed Minimization, Optimization Methods and Software, Vol. 4, pp. 103-116, 1994.
14. LUO, Z. Q., and TSENG, P., Analysis of an Approximate Gradient-Projection Method with Applications to the Backpropagation Algorithm, Optimization Methods and Software, Vol. 4, pp. 85-101, 1994.
15. BERTSEKAS, D. P., Incremental Least-Squares Methods and the Extended Kalman Filter, SIAM Journal on Optimization, Vol. 6, pp. 807-822, 1996.
16. SOLODOV, M. V., Incremental Gradient Algorithms with Stepsizes Bounded Away from Zero, Computational Optimization and Applications (to appear).
17. BERTSEKAS, D. P., A New Class of Incremental Gradient Methods for Least-Squares Problems, SIAM Journal on Optimization, Vol. 7, pp. 913-926, 1997.
18. TSENG, P., Incremental Gradient-Projection Method with Momentum Term and Adaptive Stepsize Rule, SIAM Journal on Optimization, Vol. 8, pp. 506-531, 1998.
19. MIKHALEVITCH, V. S., GUPAL, A. M., and NORKIN, V. I., Methods of Nonconvex Optimization, Nauka, Moscow, Russia, 1987 (in Russian).
20. DOROFEEV, P. A., On Some Properties of the Quasi-Gradient Method, USSR Computational Mathematics and Mathematical Physics, Vol. 25, pp. 181-189, 1985.
21. ZAVRIEV, S. K., and PEREVOZCHIKOV, A. G., Attraction of Trajectories of Finite-Difference Inclusions and Stability of Numerical Methods of Stochastic Nonsmooth Optimization, Soviet Physics Doklady, Vol. 313, pp. 1373-1376, 1990.
22. ROUCHE, N., HABETS, P., and LALOY, M., Stability Theory by Liapunov Direct Method, Springer Verlag, New York, New York, 1977.
23. ZANGWILL, W. I., Nonlinear Programming: A Unified Approach, Prentice-Hall, Englewood Cliffs, New Jersey, 1969.
24. POLYAK, B. T., Introduction to Optimization, Optimization Software, Publications Division, New York, New York, 1987.
25. ZAVRIEV, S. K., Stochastic Subgradient Methods for Minmax Problems, Izdatelstvo MGU, Moscow, Russia, 1984 (in Russian).
26. PAUGAM-MOISY, H., On Parallel Algorithm for Backpropagation by Partitioning the Training Set, Proceedings of the 5th International Conference on Neural Networks and Their Applications, Nimes, France, 1992.
27. LOU, Z. Q., On the Convergence of the LMS Algorithm with Adaptive Learning Rate for Linear Feedforward Networks, Neural Computation, Vol. 3, pp. 226-245, 1991.
28. BURKE, J. V., and FERRIS, M. C., Weak Sharp Minima in Mathematical Programming, SIAM Journal on Control and Optimization, Vol. 31, pp. 1340-1359, 1993.
29. SOLODOV, M. V., New Inexact Parallel Variable Distribution Algorithms, Computational Optimization and Applications, Vol. 7, pp. 165-182, 1997.