On the complexity of steepest descent, Newton's and regularized Newton's methods for nonconvex unconstrained optimization problems

SIAM Journal on Optimization

Volumn 20, Issue 6, 2010, Pages 2833-2852

  Cartis, C ^a   Gould, N I M ^b   Toint, Ph L ^c  

^a UNIVERSITY OF EDINBURGH   (United Kingdom)

^b RUTHERFORD APPLETON LABORATORY   (United Kingdom)

^c UNIVERSITY OF NAMUR   (Belgium)

Author keywords

Cubic regularization; Global complexity bounds; Global rate of convergence; Newton's method; Nonlinear optimization; Steepest descent method; Trust region methods; Unconstrained optimization

Indexed keywords

COMPLEXITY BOUNDS; CUBIC REGULARIZATION; NEWTON'S METHODS; NON-LINEAR OPTIMIZATION; RATE OF CONVERGENCE; TRUST-REGION METHODS; UNCONSTRAINED OPTIMIZATION;

APPROXIMATION THEORY; FUNCTION EVALUATION; NEWTON-RAPHSON METHOD; OPTIMIZATION;

STEEPEST DESCENT METHOD;

EID: 79251493375     PISSN: 10526234     EISSN: None     Source Type: Journal    
DOI: 10.1137/090774100     Document Type: Article

References (19)

1. H. Behforooz, Thinning out the harmonic series, Math. Mag., 68 (1995), pp. 289-293.
2. C. Cartis, N. I. M. Gould, and Ph. L. Toint, Adaptive cubic regularisation methods for unconstrained optimization. Part I: Motivation, convergence and numerical results, Math. Program., Ser. A, (2009), DOI: 10.1007/s10107-009-0286-5.
3. C. Cartis, N. I. M. Gould, and Ph. L. Toint, Adaptive cubic regularisation methods for unconstrained optimization. Part II: Worst-case function- and derivative-evaluation complexity, Math. Program., Ser. A, (2010a), DOI: 10.1007/s10107-009-0337-y.
4. C. Cartis, N. I. M.Gould, and Ph. L. Toint, Efficiency of Adaptive Cubic Regularisation Methods on Convex Problems, Technical Report (in preparation), Department of Mathematics, FUNDP-University of Namur, Namur, Belgium, 2010b.
5. A. R. Conn, N. I. M. Gould, and Ph. L. Toint, Trust-Region Methods, MPS/SIAM Ser. Optim. 1, SIAM, Philadelphia, PA, 2000.
6. J. E. Dennis and R. B. Schnabel, Numerical Methods for Unconstrained Optimization and Nonlinear Equations, Prentice-Hall, Englewood Cliffs, NJ, 1983.
7. R. Fletcher, Practical Methods of Optimization, 2nd ed., J. Wiley & Sons, Chichester, England, 1987.
8. N. I. M. Gould, C. Sainvitu, and Ph. L. Toint, A filter-trust-region method for unconstrained optimization, SIAM J. Optim., 16 (2005), pp. 341-357.
9. S. Gratton, A. Sartenaer, and Ph. L. Toint, Recursive trust-region methods for multiscale nonlinear optimization, SIAM J. Optim., 19 (2008), pp. 414-444.
10. A. Griewank, The modification of Newton's method for unconstrained optimization by bounding cubic terms, Technical report NA/12, Department of Applied Mathematics and Theoretical Physics, University of Cambridge, Cambridge, United Kingdom, 1981.
11. Yu. Nesterov, Introductory Lectures on Convex Optimization: A Basic Course, Appl. Optim. 87, Kluwer Academic Publishers, Dordrecht, The Netherlands, 2004.
12. Yu. Nesterov, Accelerating the cubic regularization of Newton's method on convex problems, Math. Program., Ser. B, 112 (2008), pp. 159-181.
13. Yu. Nesterov and B. T. Polyak, Cubic regularization of Newton method and its global performance, Math. Program., Ser. A, 108 (2006), pp. 177-205.
14. J. Nocedal and S. J. Wright, Numerical Optimization, Springer Ser. Oper. Res., Springer Verlag, Heidelberg, Berlin, New York, 1999.
15. R. Polyak, Regularized Newton method for unconstrained convex optimization, Math. Program., Ser. A, 120 (2009), pp. 125-145.
16. K. Ueda, Regularized Newton Method without Line Search for Unconstrained Optimization, Ph.D. thesis, Department of Applied Mathematics and Physics, Graduate School of Informatics, Kyoto University, Kyoto, Japan, 2009.
17. K. Ueda and N. Yamashita, A regularized Newton method without line search for unconstrained optimization, Technical report 2009-007, Department of Applied Mathematics and Physics, Graduate School of Informatics, Kyoto University, Kyoto, Japan, 2009.
18. K. Ueda and N. Yamashita, Convergence properties of the regularized Newton method for the unconstrained nonconvex optimization, Appl. Math. Optim., 62 (2010), pp. 27-46.
19. M. Weiser, P. Deuflhard, and B. Erdmann, Affine conjugate adaptive Newton methods for nonlinear elastomechanics, Optim. Methods Softw., 22 (2007), pp. 413-431.