A limited memory steepest descent method

Mathematical Programming

Volumn 135, Issue 1-2, 2012, Pages 413-436

  Fletcher, Roger ^a  

^a UNIVERSITY OF DUNDEE   (United Kingdom)

Author keywords

65K05; 90C26; Mathematics Subject Classification (2000): 90C06

Indexed keywords

65K05; 90C26; LANCZOS; LANCZOS PROCESS; LARGE DIMENSIONS; LIMITED MEMORY; LIMITED-MEMORY METHODS; NUMERICAL EVIDENCE; OBJECTIVE FUNCTIONS; RITZ VALUES; STATE OF THE ART; STEP SIZE; SUBJECT CLASSIFICATION; UNCONSTRAINED MINIMIZATION PROBLEM;

CONJUGATE GRADIENT METHOD; STEEPEST DESCENT METHOD;

NUMERICAL METHODS;

EID: 84866235041     PISSN: 00255610     EISSN: 14364646     Source Type: Journal    
DOI: 10.1007/s10107-011-0479-6     Document Type: Article

References (26)

1. Barzilai J., Borwein J.M.: Two-point step size gradient methods. IMA J. Numer. Anal. 8, 141-148 (1988)
2. Byrd R.H., Nocedal J., Schnabel R.B.: Representations of Quasi-Newton matrices and their use in limited memory methods. Math. Progr. 63, 129-156 (1994)
3. Cauchy A.: Méthode générale pour la résolution des systèms d'équations simultanées. Comp. Rend. Sci. Paris 25, 536-538 (1847)
4. Dai Y.H., Fletcher R.: On the asymptotic behaviour of some new gradient methods. Math. Progr. 103, 541-559 (2005)
5. Dai Y.H., Fletcher R.: Projected Barzilai-Borwein methods for large-scale box-constrained quadratic programming. Numer. Math. 100, 21-47 (2005)
6. Dai Y.H., Yuan Y.: Analysis of monotone gradient methods. J. Ind. Manag. Optim. 1, 181-192 (2005)
7. Fletcher R.: Practical Methods of Optimization. 2nd edn. Wiley, Chichester (1987)
8. Fletcher, R.: Low storage methods for unconstrained optimization. In: Allgower, E.L., Georg, K. (eds.) Computational Solution of Nonlinear Systems of Equations. Lectures in Applied Mathematics (AMS), vol. 26, pp. 165-179 (1990)
9. Fletcher, R.: On the Barzilai-Borwein method. In: Qi, L., Teo, K., Yang, X. (eds.) Optimization and Control with Applications, Series in Applied Optimization, vol. 96. Kluwer, pp. 235-256 (2005)
10. Fletcher R., Powell M.J.D.: A rapidly convergent descent method for minimization. Comput. J. 6, 163-168 (1963)
11. Friedlander A., Martínez J.M., Molina B., Raydan M.: Gradient method with retards and generalizations. SIAM J. Numer. Anal. 36, 275-289 (1999)
12. Golub G.H., Van Loan C.F.: Matrix Computations. 3rd edn. The Johns Hopkins Press, Baltimore (1996)
13. Gould N.I.M., Orban D., Toint Ph.L.: CUTEr (and SifDec), a constrained and unconstrained testing environment, revisited. ACM Trans. Math. Softw. 29, 373-394 (2003)
14. Grippo L., Lampariello F., Lucidi S.: A nonmonotone line search technique for Newton's method. SIAM J. Numer. Anal. 23, 707-716 (1986)
15. Lanczos C.: An iteration method for the solution of the eigenvalue problem of linear differential and integral operators. J. Res. Nat. Bur. Stand. 45, 255-282 (1950)
16. Molina B., Raydan M.: Preconditioned Barzilai-Borwein method for the numerical solution of partial differential equations. Numer. Algoritm. 13, 45-60 (1996)
17. Nocedal J.: Updating quasi-Newton matrices with limited storage. Math. Comput. 35, 773-782 (1980)
18. Paige C.C., Parlett B.N., van der Vorst H.: Approximate solutions and eigenvalue bounds from Krylov subspaces. Numer. Linear Algebra Appl. 2, 115-133 (1995)
19. Raydan M.: On the Barzilai and Borwein choice of steplength for the gradient method. IMA J. Numer. Anal. 13, 321-326 (1993)
20. Raydan M.: The Barzilai and Borwein gradient method for the large scale unconstrained minimization problem. SIAM J. Optim. 7, 26-33 (1997)
21. Raydan M., Svaiter B.F.: Relaxed steepest descent and Cauchy-Barzilai-Borwein method. Comput. Optim. Appl. 21, 155-167 (2002)
22. Serafini T., Zanghirati G., Zanni L.: Gradient projection methods for quadratic programs and applications in support vector machines. Optim. Methods Softw. 20, 353-378 (2005)
23. Toint Ph.L.: Some numerical results using a sparse matrix updating formula in unconstrained optimization. Math. Comput. 32, 839-852 (1978)
24. Varadhan R., Gilbert P.D.: BB: an R package for solving a large system of nonlinear equations and for optimizing a high-dimensional nonlinear objective function. J. Stat. Softw. 32, 1-26 (2009)
25. Vargas W.E., Azofeifa D.E., Clark N.: Retrieved optical properties of thin films on absorbing substrates from transmittance measurements by application of a spectral projected gradient method. Thin Solid Films 425, 1-8 (2003)
26. Yuan Y.: A new stepsize for the steepest descent method. J. Comput. Math. 24, 149-156 (2006)