Limited-memory reduced-Hessian methods for large-scale unconstrained optimization

SIAM Journal on Optimization

Volumn 14, Issue 2, 2004, Pages 380-401

  Gill, Philip E ^a   Leonard, Michael W ^a  

^a UNIVERSITY OF CALIFORNIA   (United States)

Author keywords

BFGS method; Conjugate direction methods; Quasi Newton methods; Reduced Hessian methods; Unconstrained optimization

Indexed keywords

EID: 2442519470     PISSN: 10526234     EISSN: None     Source Type: Journal    
DOI: 10.1137/S1052623497319973     Document Type: Article

References (34)

1. I. BONGARTZ, A. R. CONN, N. I. M. GOULD, AND P. L. TOINT, CUTE: Constrained and unconstrained testing environment, ACM Trans. Math. Software, 21 (1995), pp. 123-160.
2. A. G. BUCKLEY, A combined conjugate-gradient quasi-Newton minimization algorithm, Math. Programming, 15 (1978), pp. 200-210.
3. A. G. BUCKLEY, Extending the relationship between the conjugate-gradient and BFGS algorithms, Math. Programming, 15 (1978), pp. 343-348.
4. R. H. BYRD, P. LU, J. NOCEDAL, AND C. ZHU, A limited memory algorithm for bound constrained optimization, SIAM J. Sci. Comput., 16 (1995), pp. 1190-1208.
5. R. H. BYRD, J. NOCEDAL, AND R. B. SCHNABEL, Representations of quasi-Newton matrices and their use in limited-memory methods, Math. Programming, 63 (1994), pp. 129-156.
6. J. W. DANIEL, W. B. GRAGG, L. KAUFMAN, AND G. W. STEWART, Reorthogonalization and stable algorithms for updating the Gram-Schmidt QR factorization, Math. Comp., 30 (1976), pp. 772-795.
7. J. E. DENNIS, JR. AND R. B. SCHNABEL, A new derivation of symmetric positive definite secant updates, in Nonlinear Programming 4 (Madison, WI, 1980), Academic Press, New York, 1981, pp. 167-199.
8. M. C. FENELON, Preconditioned Conjugate-Gradient-Type Methods for Large-Scale Unconstrained Optimization, Ph.D. thesis, Department of Operations Research, Stanford University, Stanford, CA, 1981.
9. R. FLETCHER, Practical Methods of Optimization, 2nd ed., John Wiley and Sons, Chichester, New York, 1987.
10. R. FLETCHER, An optimal positive definite update for sparse Hessian matrices, SIAM J. Optim., 5 (1995), pp. 192-218.
11. R. FLETCHER AND M. J. D. POWELL, A rapidly convergent descent method for minimization, Comput. J., 6 (1963), pp. 163-168.
12. R. FLETCHER AND C. M. REEVES, Function minimization by conjugate gradients, Comput. J., 7 (1964), pp. 149-154.
13. P. E. GILL AND M. W. LEONARD, Reduced-Hessian quasi-Newton methods for unconstrained optimization, SIAM J. Optim., 12 (2001), pp. 209-237.
14. P. E. GILL AND W. MURRAY, Conjugate-Gradient Methods for Large-Scale Nonlinear Optimization, Report SOL 79-15, Department of Operations Research, Stanford University, Stanford, CA, 1979.
15. P. E. GILL, W. MURRAY, AND M. A. SAUNDERS, SNOPT: An SQP algorithm for large-scale constrained optimization, SIAM J. Optim., 12 (2002), pp. 979-1006.
16. P. E. GILL, W. MURRAY, M. A. SAUNDERS, AND M. H. WRIGHT, User's Guide for NPSOL (Version 4.0): A Fortran Package for Nonlinear Programming, Report SOL 86-2, Department of Operations Research, Stanford University, Stanford, CA, 1986.
17. P. E. GILL, W. MURRAY, AND M. H. WRIGHT, Practical Optimization, Academic Press, London, New York, 1981.
18. G. H. GOLUB AND C. F. VAN LOAN, Matrix Computations, 2nd ed., The Johns Hopkins University Press, Baltimore, MD, 1989.
19. L. KAUFMAN, Reduced storage, quasi-Newton trust region approaches to function optimization, SIAM J. Optim., 10 (1999), pp. 56-69.
20. M. W. LEONARD, Reduced Hessian Quasi-Newton Methods for Optimization, Ph.D. thesis, Department of Mathematics, University of California, San Diego, 1995.
21. D. C. Liu AND J. NOCEDAL, On the limited memory BFGS method for large scale optimization, Math. Programming, 45 (1989), pp. 503-528.
22. J. L. MORALES AND J. NOCEDAL, Automatic preconditioning by limited memory quasi-Newton updating, SIAM J. Optim., 10 (2000), pp. 1079-1096.
23. J. J. MORÉ AND D. C. SORENSEN, Newton's method, in Studies in Numerical Analysis, MAA Stud. Math. 24, Mathematical Association of America, Washington, DC, 1984, pp. 29-82.
24. J. J. MORE AND D. J. THUENTE, Line search algorithms with guaranteed sufficient decrease, ACM Trans. Math. Software, 20 (1994), pp. 286-307.
25. L. NAZARETH, A relationship between the BFGS and conjugate gradient algorithms and its implications for new algorithms, SIAM J. Numer. Anal., 16 (1979), pp. 794-800.
26. J. NOCEDAL, Updating quasi-Newton matrices with limited storage, Math. Comput., 35 (1980), pp. 773-782.
27. M. J. D. POWELL, Updating conjugate directions by the BFGS formula, Math. Programming, 38 (1987), pp. 693-726.
28. M. J. D. POWELL, TOLMIN: A Fortran Package for Linearly Constrained Optimization Calculations, Report DAMTP/1989/NA2, Department of Applied Mathematics and Theoretical Physics, University of Cambridge, 1989.
29. M. J. D. POWELL AND P. L. TOINT, On the estimation of sparse Hessian matrices, SIAM J. Numer. Anal., 16 (1979), pp. 1060-1074.
30. D. F. SHANNO, Conjugate-gradient methods with inexact searches, Math. Oper. Res., 3 (1978), pp. 244-256.
31. D. SIEGEL, Modifying the BFGS Update by a New Column Scaling Technique, Report DAMTP/1991/NA5, Department of Applied Mathematics and Theoretical Physics, University of Cambridge, 1991.
32. D. SIEGEL, Implementing and Modifying Broyden Class Updates for Large Scale Optimization, Report DAMTP/1992/NA12, Department of Applied Mathematics and Theoretical Physics, University of Cambridge, 1992.
33. D. SIEGEL, Modifying the BFGS update by a new column scaling technique, Math. Programming, 66 (1994), pp. 45-78.
34. C. ZHU, R. H. BYRD, P. LU, AND J. NOCEDAL, Algorithm 778: L-BFGS-B: Fortran subroutines for large-scale bound-constrained optimization, ACM Trans. Math. Software, 23 (1997), pp. 550-560.