A new conjugate gradient method with guaranteed descent and an efficient line search

SIAM Journal on Optimization

Volumn 16, Issue 1, 2006, Pages 170-192

  Hager, William W ^a   Zhang, Hongchao ^a  

^a UNIVERSITY OF FLORIDA   (United States)

Author keywords

Conjugate gradient method; Convergence; Line search; Unconstrained optimization; Wolfe conditions

Indexed keywords

CONJUGATE GRADIENT METHOD; CONVERGENCE; LINE SEARCH; UNCONSTRAINED OPTIMIZATION; WOLFE CONDITIONS;

NONLINEAR SYSTEMS; NUMERICAL ANALYSIS; OPTIMIZATION; PROBLEM SOLVING;

GRADIENT METHODS;

EID: 33144465578     PISSN: 10526234     EISSN: None     Source Type: Journal    
DOI: 10.1137/030601880     Document Type: Article

References (42)

1. M. AL-BAALI, Decent property and global convergence of the Fletcher-Reeves method with in exact line search, IMA J. Numer. Anal., 5 (1985), pp. 121-124.
2. M. AL-BAALI AND R. FLETCHER, An efficient line search for nonlinear least squares, J. Optim. Theory Appl., 48 (1984), pp. 359-377.
3. K. E. ATKINSON, An Introduction to Numerical Analysis, John Wiley, New York, 1978.
4. I. BONGARTZ, A. R. CONN, N. I. M. GOULD, AND P. L. TOINT, CUTE: Constrained and unconstrained testing environments, ACM Trans. Math. Software, 21 (1995), pp. 123-160.
5. A. I. COHEN, Rate of convergence of several conjugate gradient algorithms, SIAM J. Numer. Anal., 9 (1972), pp. 248-259.
6. Y. DAI, J. HAN, G. LIU, D. SUN, H. YIN, AND Y.-X. YUAN, Convergence properties of nonlinear conjugate gradient methods, SIAM J. Optim., 10 (1999), pp. 345-358.
7. Y. H. DAI AND L. Z. LIAO, New conjugate conditions and related nonlinear conjugate gradient methods, Appl. Math. Optim., 43 (2001), pp. 87-101.
8. Y. H. DAI AND Y. YUAN, A nonlinear conjugate gradient method with a strong global convergence property, SIAM J. Optim., 10 (1999), pp. 177-182.
9. Y. H. DAI AND Y. YUAN, Nonlinear Conjugate Gradient Methods, Shang Hai Science and Technology Publisher, Beijing, 2000.
10. Y. H. DAI AND Y. YUAN, An efficient hybrid conjugate gradient method for unconstrained optimization, Ann. Oper. Res., 103 (2001), pp. 33-47.
11. E. D. DOLAN AND J. J. MORE, Benchmarking optimization software with performance profiles, Math. Programming, 91 (2002), pp. 201-213.
12. R. FLETCHER AND C. REEVES, Function minimization by conjugate gradients, Comput. J., 7 (1964), pp. 149-154.
13. J. C. GILBERT AND J. NOCEDAL, Global convergence properties of conjugate gradient methods for optimization, SIAM J. Optim., 2 (1992), pp. 21-42.
14. A. A. GOLDSTEIN, On steepest descent, SIAM J. Control, 3 (1965), pp. 147-151.
15. W. W. HAGER, Applied Numerical Linear Algebra, Prentice-Hall, Englewood Cliffs, NJ, 1988.
16. W. W. HAGER, A derivative-based bracketing scheme for univariate minimization and the conjugate gradient method, Comput. Math. Appl., 18 (1989), pp. 779-795.
17. W. W. HAGER, Numerical Linear Algebra: Practical Insights and Applications, in preparation.
18. W. W. HAGER AND H. ZHANG, CG_DESCENT, A conjugate gradient method with guaranteed descent, ACM Trans. Math. Software, to appear.
19. W. W. HAGER AND H. ZHANG A survey of nonlinear conjugate gradient methods, Pacific J. Optim., submitted.
20. J. HAN, G. LIU, D. SUN, AND H. YIN, Two fundamental convergence theorems for nonlinear conjugate gradient methods and their applications, Acta Math. Appl. Sinica, 17 (2001), pp. 38-46.
21. J. Y. HAN, G. H. LIU, AND H. X. YIN, Convergence of Perry and Shanno's memoryless quasi-Newton method for nonconvex optimization problems, OR Trans., 1 (1997), pp. 22-28.
22. M. R. HESTENES AND E. L. STIEFEL, Methods of conjugate gradients for solving linear systems, J. Res. Nat. Bur. Standards, 49 (1952), pp. 409-436.
23. H. HIRST, N-step Quadratic Convergence in the Conjugate Gradient Method, Ph.D. dissertation, Department of Mathematics, Pennsylvania State University, State College, PA, 1989.
24. C. LEMARECHAL, A view of line-searches, in Optimization and Optimal Control, Lecture Notes in Control and Inform. Sci. 30, A. Auslender, W. Oettli, and J. Stoer, eds., Springer-Verlag, Heidelberg, 1981, pp. 59-79.
25. D. C. LIU AND J. NOCEDAL, On the limited memory BFGS method for large scale optimization, Math. Programming Ser. B, 45 (1989), pp. 503-528.
26. J. J. MORE AND D. C. SORENSEN, Newton's method, in Studies in Numerical Analysis, G. H. Golub, ed., Mathematical Association of America, Washington, DC, 1984, pp. 29-82.
27. J. J. MORÉ AND D. J. THUENTE, Line search algorithms with guaranteed sufficient decrease, ACM Trans. Math. Software, 20 (1994), pp. 286-307.
28. J. NOCEDAL, Updating quasi-Newton matrices with limited storage, Math. Comp., 35 (1980), pp. 773-782.
29. J. M. ORTEGA AND W. C. RHEINBOLDT, Iterative Solution of Nonlinear Equations in Several Variables, Classics in Appl. Math. 30, SIAM, Philadelphia, 2000.
30. J. M. PERRY, A Class of Conjugate Gradient Algorithms with a Two Step Variable Metric Memory, Discussion paper 269, Center for Mathematical Studies in Economics and Management Science, Northwestern University, Chicago, 1977.
31. E. POLAR AND G. RIBIÈRE, Note sur la convergence de méthodes de directions conjuguées, Rev. Française Informat. Recherche Opérationnelle, 3 (1969), pp. 35-43.
32. B. T. POLYAK, The conjugate gradient method in extreme problems, USSR Comp. Math. Math. Phys., 9 (1969), pp. 94-112.
33. M. J. D. POWELL, Nonconvex minimization calculations and the conjugate gradient method, in Numerical Analysis, Lecture Notes in Math. 1066, D. F. Griffiths, ed., Springer, Berlin, 1984, pp. 122-141.
34. M. J. D. POWELL, Restart procedures for the conjugate gradient method, Math. Programming, 12 (1977), pp. 241-254.
35. A. RAMASUBRAMANIAM, Unconstrained Optimization by a Globally Convergent High Precision Conjugate Gradient Method, Master's thesis, Department of Mathematics, University of Florida, Gainesville, FL, 2000.
36. D. F. SHANNO, On the convergence of a new conjugate gradient algorithm, SIAM J. Numer. Anal., 15 (1978), pp. 1247-1257.
37. D. F. SHANNO, Globally convergent conjugate gradient algorithms, Math. Programming, 33 (1985), pp. 61-67.
38. C. WANG, J. HAN, AND L. WANG, Global convergence of the Polak-Ribière and Hestenes-Stiefel conjugate gradient methods for the unconstrained nonlinear optimization, OR Trans., 4 (2000), pp. 1-7.
39. D. F. SHANNO AND K. H. PHUA, Remark on algorithm 500, ACM Trans. Math. Software, 6 (1980), pp. 618-622.
40. P. WOLFE, Convergence conditions for ascent methods, SIAM Rev., 11 (1969), pp. 226-235.
41. P. WOLFE, Convergence conditions for ascent methods. II: Some corrections, SIAM Rev., 13 (1971), pp. 185-188.
42. G. ZOUTENDIJK, Nonlinear programming, computational methods, in Integer and Nonlinear Programming, J. Abadie, ed., North-Holland, Amsterdam, 1970, pp. 37-86.