The cyclic Barzilai-Borwein method for unconstrained optimization

IMA Journal of Numerical Analysis

Volumn 26, Issue 3, 2006, Pages 604-627

  Dai, Yu Hong ^a   Hager, William W ^b   Schittkowski, Klaus ^c   Zhang, Hongchao ^b  

^a CHINESE ACADEMY OF SCIENCES   (China)

^b UNIVERSITY OF FLORIDA   (United States)

^c UNIVERSITY OF BAYREUTH   (Germany)

Author keywords

Convex quadratic programming; Gradient method; Non monotone line search; Unconstrained optimization

Indexed keywords

GRADIENT METHODS; NUMERICAL METHODS; QUADRATIC PROGRAMMING;

CONJUGATE GRADIENT ALGORITHMS; CONVERGENCE RATES; CONVEX QUADRATIC PROGRAMMING; GRADIENT ALGORITHM; LINE SEARCHES; NUMERICAL EVIDENCE; NUMERICAL EXPERIMENTS; UNCONSTRAINED OPTIMIZATION;

CONJUGATE GRADIENT METHOD;

EID: 33745923778     PISSN: 02724979     EISSN: 14643642     Source Type: Journal    
DOI: 10.1093/imanum/drl006     Document Type: Article

References (31)

1. AKAIKE, H. (1959) On a successive transformation of probability distribution and its application to the analysis of the optimum gradient method. Ann. Inst. Stat. Math. Tokyo, 11, 1-17.
2. BARZILAI, J. & BORWEIN, J. M. (1988) Two point step size gradient methods. IMA J. Numer. Anal., 8, 141-148.
3. BIRGIN, E. G., CHAMBOULEYRON, I. & MARTÍNEZ, J. M. (1999) Estimation of the optical constants and the thickness of thin films using unconstrained optimization. J. Comput. Phys., 151, 862-880.
4. BIRGIN, E. G., MARTÍNEZ, J. M. & RAYDAN, M. (2000) Nonmonotone spectral projected gradient methods for convex sets. SIAM J. Optim., 10, 1196-1211.
5. BIRGIN, E. G., MARTÍNEZ, J. M. & RAYDAN, M. (2001) Algorithm 813: SPG - software for convex-constrained optimization. ACM Trans. Math. Softw., 27, 340-349.
6. BONGARTZ, I., CONN, A. R., COULD, N. I. M. & TOINT, P. L. (1995) CUTE: constrained and unconstrained testing environments. ACM Trans. Math. Softw., 21, 123-160.
7. CAUCHY, A. (1847) Méthode générale pour la résolution des systems d'équations simultanées. Comp. Rend. Sci. Paris, 25, 46-89.
8. DAI, Y. H. (2003) Alternate stepsize gradient method. Optimization, 52, 395-415.
9. DAI, Y. H. & FLETCHER, R. (2005a) On the asymptotic behaviour of some new gradient methods. Math. Prog., 103, 541-559.
10. DAI, Y. H. & FLETCHER, R. (2005b) Projected Barzilai-Borwein methods for large-scale box-constrained quadratic programming. Numer. Math., 100, 21-47.
11. DAI, Y. H. & FLETCHER, R. (2006) New algorithms for singly linearly constrained quadratic programs subject to lower and upper bounds. Math. Prog., 106, 403-421.
12. DAI, Y. H. & SCHITTKOWSKI, K. (2005) A sequential quadratic programming algorithm with non-monotone line search. Technical Report. Department of Mathematics, University of Bayreuth (submitted).
13. DAI, Y. H. & YANG, X. Q. (2006) A new gradient method with an optimal stepsize property. Comput. Optim. Appl, 33, 73-88.
14. DAI, Y. H. & YUAN, Y. (2003) Alternate minimization gradient method. IMA J. Numer. Anal, 23, 377-393.
15. DAI, Y. H. & ZHANG, H. (2001) An adaptive two-point stepsize gradient algorithm. Numer. Algorithms, 27, 377-385.
16. DOLAN E. D. & MORE, J. J. (2002) Benchmarking optimization software with performance profiles. Math. Program., 91, 201-213.
17. FORSYTHE, G. E. (1968) On the asymptotic directions of the s-dimensional optimum gradient method. Numer. Math., 11, 57-76.
18. FRIEDLANDER, A., MARTÍNEZ, J. M., MOLINA, B. & RAYDAN, M. (1999) Gradient method with retards and generalizations. SIAM J. Numer. Anal, 36, 275-289.
19. GILBERT, J. C. & NOCEDAL, J. (1992) Global convergence properties of conjugate gradient methods for optimization. SIAM J. Optim., 2, 21-42.
20. GLUNT, W., HAYDEN, T. L. & RAYDAN, M. (1993) Molecular conformations from distance matrices. J. Comput. Chem., 14, 114-120.
21. GRIPPO.L., LAMPARIELLO, F. &LUCIDI, S. (1986) A nonmonotone line search technique for Newton's method. SIAM J. Numer. Anal., 23, 707-716.
22. GRIPPO, L. & SCIANDRONE, M. (2002) Nonmonotone globalization techniques for the Barzilai-Borwein gradient method. Comput. Optim. Appl, 23, 143-169.
23. HAGER, W. W. & ZHANG, H. (2005a) A new active set algorithm for box constrained optimization. SIAM J. Optim. (to appear).
24. HAGER, W. W. & ZHANG, H. (2005b) A new conjugate gradient method with guaranteed descent and an efficient line search. SIAM J. Optim., 16, 170-192.
25. HAGER, W. W. & ZHANG, H. (2006) Algorithm 851: CG.DESCENT, a conjugate gradient method with guaranteed descent. ACM Trans. Math. Softw., 32.
26. Liu, W. B. & DAI, Y. H. (2001) Minimization algorithms based on supervisor and searcher cooperation. J. Optim. Theory Appl., 111, 359-379.
27. MORE, J. J. & THUENTE, D. J. (1994) Line search algorithms with guaranteed sufficient decrease. ACM Trans. Math. Softw., 20, 286-307.
28. RAYDAN, M. (1997) The Barzilai and Borwein gradient method for the large scale unconstrained minimization problem. SIAM J. Optim., 7, 26-33.
29. RAYDAN, M. & SVAITER, B. R (2002) Relaxed steepest descent and Cauchy-Barzilai-Borwein method. Cornput. Optim. Appl., 21, 155-167.
30. SERAFINI, T. ZANGHIRATI, O. & ZANNI, L. (2005) Gradient projection methods for quadratic programs and applications in training support vector machines. Optim. Methods Softw., 20, 353-378.
31. TOINT, P. L. (1997) A non-monotone trust region algorithm for nonlinear optimization subject to convex constraints. Math. Prog., 77, 69-94.