The limited memory conjugate gradient method

SIAM Journal on Optimization

Volumn 23, Issue 4, 2013, Pages 2150-2168

  Hager, William W ^a   Zhang, Hongchao ^b  

^a UNIVERSITY OF FLORIDA   (United States)

^b LOUISIANA STATE UNIVERSITY   (United States)

Author keywords

Adaptive method; BFGS; CG DESCENT; L BFGS; L RHR; Limited memory; Limited memory BFGS; Nonlinear conjugate gradients; Reduced Hessian method; Unconstrained optimization

Indexed keywords

ADAPTIVE METHODS; BFGS; CG DESCENT; L-BFGS; L-RHR; LIMITED MEMORY; LIMITED MEMORY BFGS; NONLINEAR CONJUGATE GRADIENT; REDUCED HESSIAN; UNCONSTRAINED OPTIMIZATION;

ALGORITHMS; CONJUGATE GRADIENT METHOD; NUMERICAL METHODS;

OPTIMIZATION;

EID: 84892874375     PISSN: 10526234     EISSN: None     Source Type: Journal    
DOI: 10.1137/120898097     Document Type: Article

References (28)

1. R. H. Bartels, G. H. Golub, and M. A. Saunders, Numerical techniques in mathematical programming, in Nonlinear Programming, J. B. Rosen, O. L. Mangasarian, and K. Ritter, eds., Academic Press, New York, 1970, pp. 123-176.
2. J. Barzilai and J. M. Borwein, Two point step size gradient methods, IMA J. Numer. Anal., 8 (1988), pp. 141-148.
3. I. Bongartz, A. R. Conn, N. I. M. Gould, and P. L. Toint, CUTE: Constrained and unconstrained testing environments, ACM Trans. Math. Software, 21 (1995), pp. 123-160.
4. Y. H. Dai and C. X. Kou, A nonlinear conjugate gradient algorithm with an optimal property and an improved Wolfe line search, SIAM J. Optim., 23 (2013), pp. 296-320.
5. Y. H. Dai and Y. Yuan, A nonlinear conjugate gradient method with a strong global convergence property, SIAM J. Optim., 10 (1999), pp. 177-182.
6. Y. H. Dai and Y. Yuan, An efficient hybrid conjugate gradient method for unconstrained optimization, Ann. Oper. Res., 103 (2001), pp. 33-47.
7. E. D. Dolan and J. J. Moré, Benchmarking optimization software with performance profiles, Math. Program., 91 (2002), pp. 201-213.
8. R. Fletcher and C. Reeves, Function minimization by conjugate gradients, Comput. J., 7 (1964), pp. 149-154.
9. J. C. Gilbert and J. Nocedal, Global convergence properties of conjugate gradient methods for optimization, SIAM J. Optim., 2 (1992), pp. 21-42.
10. P. E. Gill and M. W. Leonard, Reduced-Hessian quasi-Newton methods for unconstrained optimization, SIAM J. Optim., 12 (2001), pp. 209-237.
11. P. E. Gill and M. W. Leonard, Limited memory reduced-Hessian methods for large-scale unconstrained optimization, SIAM J. Optim., 14 (2003), pp. 380-401.
12. G. H. Golub and C. Van Loan, Matrix Computations, 2nd ed., Johns Hopkins University Press, Baltimore, MD, 1989.
13. W. W. Hager and H. Zhang, A new conjugate gradient method with guaranteed descent and an efficient line search, SIAM J. Optim., 16 (2005), pp. 170-192.
14. W. W. Hager and H. Zhang, Algorithm 851: CG DESCENT, a conjugate gradient method with guaranteed descent, ACM Trans. Math. Software, 32 (2006), pp. 113-137.
15. W. W. Hager and H. Zhang, A survey of nonlinear conjugate gradient methods, Pacific J. Optim., 2 (2006), pp. 35-58.
16. D. C. Liu and J. Nocedal, On the limited memory BFGS method for large scale optimization, Math. Program., 45 (1989), pp. 503-528.
17. D. G. Luenberger and Y. Ye, Linear and Nonlinear Programming, Springer, Berlin, 2008.
18. J. J. Moré and D. J. Thuente, Line search algorithms with guaranteed sufficient decrease, ACM Trans. Math. Software, 20 (1994), pp. 286-307.
19. J. Nocedal and S. J. Wright, Numerical Optimization, Springer, New York, 1999.
20. J. Nocedal, Updating quasi-Newton matrices with limited storage, Math. Comp., 35 (1980), pp. 773-782.
21. S. S. Oren, Self-scaling Variable Metric Algorithms for Unconstrained Minimization, Ph.D. thesis, Department of Engineering Economic Systems, Stanford University, Stanford, CA, 1972.
22. J. M. Perry, A Class of Conjugate Gradient Algorithms with a Two Step Variable Metric Memory, Technical report 269, Center for Mathematical Studies in Economics and Management Science, Northwestern University, Evanston, IL, 1977.
23. M. J. D. Powell, Convergence properties of algorithms for nonlinear optimization, SIAM Rev., 28 (1986), pp. 487-500.
24. D. F. Shanno, On the convergence of a new conjugate gradient algorithm, SIAM J. Numer. Anal., 15 (1978), pp. 1247-1257.
25. D. Siegel, Modifying the BFGS update by a new column scaling technique, Math. Program., 66 (1993), pp. 48-78.
26. L. N. Trefethen and D. Bau III, Numerical Linear Algebra, SIAM, Philadelphia, 1997.
27. P. Wolfe, Convergence conditions for ascent methods, SIAM Rev., 11 (1969), pp. 226-235.
28. P. Wolfe, Convergence conditions for ascent methods II: Some corrections, SIAM Rev., 13 (1971), pp. 185-188.