Updating the regularization parameter in the adaptive cubic regularization algorithm

Computational Optimization and Applications

Volumn 53, Issue 1, 2012, Pages 1-22

  Gould, N I M ^a   Porcelli, M ^b   Toint, P L ^b  

^a RUTHERFORD APPLETON LABORATORY   (United Kingdom)

^b University of Namur   (Belgium)

Author keywords

Cubic regularization; Numerical performance; Unconstrained optimization

Indexed keywords

ADAPTIVE REGULARIZATION PARAMETER; CUBIC REGULARIZATION; INTERPOLATION TECHNIQUES; LIPSCHITZ CONSTANT; NONLINEAR LEAST SQUARES; NUMERICAL EXPERIMENTS; NUMERICAL PERFORMANCE; OBJECTIVE FUNCTIONS; REGULARIZATION ALGORITHMS; REGULARIZATION METHODS; REGULARIZATION PARAMETERS; UNCONSTRAINED MINIMIZATION PROBLEM; UNCONSTRAINED OPTIMIZATION; UPDATING STRATEGY;

PARAMETERIZATION;

ADAPTIVE ALGORITHMS;

EID: 84865617925     PISSN: 09266003     EISSN: 15732894     Source Type: Journal    
DOI: 10.1007/s10589-011-9446-7     Document Type: Article

References (26)

1. C. Audet D. Orban 2006 Finding optimal algorithmic parameters using derivative-free optimization SIAM J. Control Optim. 17 3 642 664 2257203 1128.90060 (Pubitemid 47265166)
2. S. Bellavia C. Cartis N.I.M. Gould B. Morini Ph.L. Toint 2010 Convergence of a regularized Euclidean residual algorithm for nonlinear least-squares SIAM J. Numer. Anal. 48 1 29 2608356 1218.90182 10.1137/080732432
3. C. Cartis N.I.M. Gould Ph.L. Toint 2011 Adaptive cubic overestimation methods for unconstrained optimization. Part I: Motivation, convergence and numerical results Math. Program., Ser. A 127 2 245 295 2776701 1229.90192 10.1007/s10107-009-0286-5
4. C. Cartis N.I.M. Gould Ph.L. Toint 2011 Adaptive cubic overestimation methods for unconstrained optimization. Part II: Worst-case function- and derivative-evaluation complexity Math. Program. Ser. A 130 2 295 319 10.1007/s10107-009-0337-y 2855872 1229.90193 10.1007/s10107-009-0337-y
5. C. Cartis N.I.M. Gould Ph.L. Toint 2012 Complexity bounds for second-order optimality in unconstrained optimization J. Complex 28 1 93 108 10.1016/j.jco.2011.06.001 2871787 05996047 10.1016/j.jco.2011.06.001
6. C. Cartis N.I.M. Gould Ph.L. Toint 2010 On the complexity of steepest descent, Newton's and regularized Newton's methods for nonconvex unconstrained optimization SIAM J. Control Optim. 20 6 2833 2852 2721157 1211.90225
7. C. Cartis N.I.M. Gould Ph.L. Toint 2009 Trust-region and other regularisations of linear least-squares problems BIT 49 1 21 53 2486127 1165.65019 10.1007/s10543-008-0206-8
8. Conn, A.R., Gould, N.I.M., Toint, Ph.L.: Trust-Region Methods. SIAM, Philadelphia (2000)
9. E.D. Dolan J.J. Moré 2002 Benchmarking optimization software with performance profiles Math. Program. 91 201 213 1875515 1049.90004 10.1007/s101070100263 (Pubitemid 44744777)
10. Dennis, J.E., Schnabel, R.B.: Numerical Methods for Unconstrained Optimization and Nonlinear Equations. Prentice Hall, Englewood Cliffs (1983)
11. J.B. Erway P.E. Gill 2009 A subspace minimization method for the trust-region step SIAM J. Control Optim. 20 1439 1461 2587735 1195.49042
12. J.B. Erway P.E. Gill J.D. Griffin 2009 Iterative methods for finding a trust-region step SIAM J. Control Optim. 20 1110 1131 2534778 1189.49049
13. G.H. Golub W. Kahan 1965 Calculating the singular values and pseudo-inverse of a matrix SIAM J. Numer. Anal. 2 2 205 224 183105
14. N.I.M. Gould D. Orban A. Sartenaer Ph.L. Toint 2005 Sensitivity of trust-region algorithms to their parameters 4OR 3 3 227 241 2177962 1086.65060 10.1007/s10288-005-0065-y (Pubitemid 41436933)
15. N.I.M. Gould S. Lucidi M. Roma Ph.L. Toint 1999 Solving the trust-region subproblem using the Lanczos method SIAM J. Control Optim. 9 2 504 525 1686795 1047.90510 (Pubitemid 129489293)
16. N.I.M. Gould D. Orban Ph.L. Toint 2003 CUTEr, a constrained and unconstrained testing environment, revisited ACM Trans. Math. Softw. 29 4 373 394 2077337 1068.90526 10.1145/962437.962439
17. N.I.M. Gould D. Orban Ph.L. Toint 2003 GALAHAD-a library of thread-safe Fortran 90 packages for large-scale nonlinear optimization ACM Trans. Math. Softw. 29 4 353 372 2077337 1068.90525 10.1145/962437.962438
18. Griewank, A.: The modification of Newton's method for unconstrained optimization by bounding cubic terms. Technical Report NA/12 (1981), Department of Applied Mathematics and Theoretical Physics, University of Cambridge, United Kingdom (1981)
19. W.W. Hager S.C. Park 2005 Global convergence of SSM for minimizing a quadratic over a sphere Math. Comput. 74 1413 1423 2137009 1059.90112 (Pubitemid 40937970)
20. W.W. Hager 2001 Minimizing a quadratic over a sphere SIAM J. Control Optim. 12 188 208 1870591 1058.90045
21. Yu. Nesterov B.T. Polyak 2006 Cubic regularization of Newton's method and its global performance Math. Program. 108 1 177 205 2229459 1142.90500 10.1007/s10107-006-0706-8
22. C.C. Paige M.A. Saunders 1982 ALGORITHM 583: LSQR: an algorithm for sparse linear equations and sparse least squares ACM Trans. Math. Softw. 8 2 195 209 661121 10.1145/355993.356000
23. A. Sartenaer 1997 Automatic determination of an initial trust region in nonlinear programming SIAM J. Sci. Comput. 18 6 1788 1803 1480635 0891.90151 10.1137/S1064827595286955 (Pubitemid 127639225)
24. T. Steihaug 1983 The conjugate gradient method and trust regions in large scale optimization SIAM J. Numer. Anal. 20 626 637 701102 0518.65042 10.1137/0720042
25. M. Weiser P. Deuflhard B. Erdmann 2007 Affine conjugate adaptive Newton methods for nonlinear elastomechanics Optim. Methods Softw. 22 3 413 431 2319241 1128.74007 10.1080/10556780600605129 (Pubitemid 46479850)
26. Y. Yuan 1999 On the truncated conjugate-gradient method Math. Program., Ser. A 87 3 561 573 10.1007/s101070050012