Asymptotic behaviour of a family of gradient algorithms in ℝ d and Hilbert spaces

Mathematical Programming

Volumn 107, Issue 3, 2006, Pages 409-438

  Pronzato, Luc ^a   Wynn, Henry P ^b   Zhigljavsky, Anatoly A ^c  

^a UNIVERSITY OF NICE SOPHIA ANTIPOLIS   (France)

^b LONDON SCHOOL OF ECONOMICS   (United Kingdom)

^c CARDIFF UNIVERSITY   (United Kingdom)

Author keywords

[No Author keywords available]

Indexed keywords

ASYMPTOTIC STABILITY; MATHEMATICAL OPERATORS; MATHEMATICAL PROGRAMMING; OPTIMIZATION;

ASYMPTOTIC CONVERGENCE RATES; BOUNDED QUADRATIC OPERATORS; HILBERT SPACES;

ALGORITHMS;

EID: 33645320243     PISSN: 00255610     EISSN: 14364646     Source Type: Journal    
DOI: 10.1007/s10107-005-0602-7     Document Type: Article

References (21)

1. Akaike, H.: On a successive transformation of probability distribution and its application to the analysis of the optimum gradient method. Ann. Inst. Statist. Math. Tokyo 11, 1-16 (1959)
2. Barzilai, J., Borwein, J.M.: Two-point step size gradient methods. IMA J. Numerical Anal. 8, 141-148 (1988)
3. Birgin, E.G., Martínez, J.M.: A spectral conjugate gradient method for unconstrained optimization. Appl. Math. Optim. 43, 117-128 (2001)
4. Fedorov, V.V.: Theory of Optimal Experiments. Academic Press, New York, 1972
5. Forsythe, G.E.: On the asymptotic directions of the s-dimensional optimum gradient method. Numerische Mathematik 11, 57-76 (1968)
6. Hale, J., Koçak, H.: Dynamics and Bifurcations. Springer-Verlag, Heidelberg, 1991
7. Kantorovich, L.V., Akilov, G.P.: Functional Analysis. Pergamon Press, London, 1982. Second edition
8. Kiefer, J., Wolfowitz, J.: The equivalence of two extremum problems. Canadian J. Math. 12, 363-366 (1960)
9. Kozjakin, V.S., Krasnosel'skii, M.A.: Some remarks on the method of minimal residues. Numer. Funct. Anal. Optim. 4 (3), 211-239 (1982)
10. Krasnosel'skii, M.A., Krein, S.G.: An iteration process with minimal residues. Mat. Sb. (in Russian) 31 (4), 315-334 (1952)
11. LaSalle, J.P.: The Stability of Dynamical Systems. SIAM, Philadelphia, 1976
12. Luenberger, D.G.: Introduction to Linear and Nonlinear Programming. Addison-Wesley, Reading, Massachusetts, 1973
13. Nocedal, J., Sartenaer, A., Zhu, C.: On the accuracy of nonlinear optimization algorithms. Technical Report Nov. 1998, ECE Department, Northwestern Univ., Evanston, II 60208, 1998
14. Nocedal, J., Sartenaer, A., Zhu, C.: On the behavior of the gradient norm in the steepest descent method. Computational Optimization and Applications 22, 5-35 (2002)
15. Pronzato, L., Wynn, H.P., Zhigljavsky, A.A.: Dynamical Search. Chapman & Hall/CRC, Boca Raton, 2000
16. Pronzato, L., Wynn, H.P., Zhigljavsky, A.A.: Renormalised steepest descent in Hilbert space converges to a two-point attractor. Acta Applicandae Mathematicae 67, 1-18 (2001)
17. Pronzato, L., Wynn, H.P., Zhigljavsky, A.A.: Kantorovich-type inequalities for operators via D-optimal design theory. Linear Algebra and Its Applications (Special Issue on Linear Algebra and Statistics), 410, 160-169 (2005)
18. Raydan, M.: The Barzilai and Borwein gradient method for the large scale unconstrained minimization problem. SIAM J. Optimi. 7 (1), 26-33 (1997)
19. Raydan, M., Svaiter, B.F.: Relaxed steepest descent and Cauchy-Barzilai-Borwein method. Comput. Optim. Appl. 21, 155-167 (2002)
20. Shiryaev, A.N.: Probability. Springer, Berlin, 1996
21. Silvey, S.D.: Optimal Design. Chapman & Hall, London, 1980