On the evaluation complexity of cubic regularization methods for potentially rank-deficient nonlinear least-squares problems and its relevance to constrained nonlinear optimization

SIAM Journal on Optimization

Volumn 23, Issue 3, 2013, Pages 1553-1574

  Cartis, Coralia ^a   Gould, Nicholas I M ^b   Toint, Philippe L ^c  

^a UNIVERSITY OF EDINBURGH   (United Kingdom)

^b RUTHERFORD APPLETON LABORATORY   (United Kingdom)

^c University of Namur   (Belgium)

Author keywords

Constrained nonlinear optimization; Cubic regularization methods; Evaluation complexity; Least squares problems; Worst case analysis

Indexed keywords

CONSTRAINED NON-LINEAR OPTIMIZATIONS; EVALUATION COMPLEXITY; LEAST SQUARE; REGULARIZATION METHODS; WORST-CASE ANALYSIS;

COMPUTER SCIENCE; SOFTWARE ENGINEERING;

CONSTRAINED OPTIMIZATION;

EID: 84886236255     PISSN: 10526234     EISSN: None     Source Type: Journal    
DOI: 10.1137/120869687     Document Type: Article

References (28)

1. S. Bellavia, C. Cartis, N. I. M. Gould, B. Morini, and Ph. L. Toint, Convergence of a regularized Euclidean residual algorithm for nonlinear least-squares, SIAM J. Numer. Anal., 48 (2010), pp. 1-29.
2. Å. Björck, Numerical Methods for Least Squares Problems, SIAM, Philadelphia, 1996.
3. R. H. Byrd, R. B. Schnabel, and G. A. Shultz, A trust region algorithm for nonlinearly constrained optimization, SIAM J. Numer. Anal., 24 (1987), pp. 1152-1170.
4. C. Cartis, N. I. M. Gould, and Ph. L. Toint, On the complexity of finding first-order critical points in constrained nonlinear programming, Math. Program., DOI 10.1007/S10107-012-0617-9, 2012 (online).
5. C. Cartis, N. I. M. Gould, and Ph. L. Toint, Optimal Newton-Type Methods for Nonconvex Smooth Optimization Problems, ERGO Technical Report 11-009, School of Mathematics, University of Edinburgh, 2011.
6. C. Cartis, N. I. M. Gould, and Ph. L. Toint, Evaluation complexity of adaptive cubic regularization methods for convex unconstrained optimization, Optim. Methods Software, 27 (2012), pp. 197-219.
7. C. Cartis, N. I. M. Gould, and Ph. L. Toint, An adaptive cubic regularization algorithm for nonconvex optimization with convex constraints and its function-evaluation complexity, IMA J. Numer. Anal., 32 (2012), pp. 1662-1695.
8. C. Cartis, N. I. M. Gould, and Ph. L. Toint, On the oracle complexity of first-order and derivative-free algorithms for smooth nonconvex minimization, SIAM J. Optim., 22 (2012), pp. 66-86.
9. C. Cartis, N. I. M. Gould, and Ph. L. Toint, On the evaluation complexity of composite function minimization with applications to nonconvex nonlinear programming, SIAM J. Optim., 21 (2011), pp. 1721-1739.
10. C. Cartis, N. I. M. Gould, and Ph. L. Toint, Adaptive cubic regularisation methods for unconstrained optimization. Part II: Worst-case function- and derivative-evaluation complexity, Math. Program., 130 (2011), pp. 295-319.
11. C. Cartis, N. I. M. Gould, and Ph. L. Toint, Adaptive cubic regularisation methods for unconstrained optimization. Part I: Motivation, convergence and numerical results, Math. Program., 127 (2011), pp. 245-295.
12. C. Cartis, N. I.M. Gould, and Ph. L. Toint, On the complexity of steepest descent, Newton's and regularized Newton's methods for nonconvex unconstrained optimization problems, SIAM J. Optim., 20 (2010), pp. 2833-2852.
13. M. R. Celis, A Trust Region Strategy for Nonlinear Equality Constrained Optimization, Technical Report 85-4, Department of Computational and Applied Mathematics, Rice University, Houston, TX, 1985.
14. A. R. Conn, N. I. M. Gould, and Ph. L. Toint, Trust-Region Methods, SIAM, Philadelphia, 2000.
15. J. E. Dennis and J. J. Moré, A characterization of superlinear convergence and its application to quasi-Newton methods, Math. Comp., 28 (1974), pp. 549-560.
16. J. E. Dennis and R. B. Schnabel, Numerical Methods for Unconstrained Optimization and Nonlinear Equations, Prentice-Hall, Englewood Cliffs, NJ, 1983.
17. Reprinted as Classics Appl. Math. 16, SIAM, Philadelphia, 1996.
18. R. Fletcher and S. Leyffer, Nonlinear programming without a penalty function, Math. Programming, 91 (2002), pp. 239-270.
19. N. I. M. Gould and Ph. L. Toint, Nonlinear programming without a penalty function or a filter, Math. Program., 122 (2010), pp. 155-196.
20. A. Griewank, The Modification of Newton's Method for Unconstrained Optimization by Bounding Cubic Terms, Technical Report NA/12 (1981), Department of Applied Mathematics and Theoretical Physics, University of Cambridge, UK, 1981.
21. J.-B. Hiriart-Urruty and C. Lemaréchal, Convex Analysis and Minimization Algorithms I, Springer-Verlag, Berlin, Heidelberg, 1993.
22. Yu. Nesterov, Modified Gauss-Newton scheme with worst case guarantees for global performance, Optim. Methods Software, 22 (2007), pp. 469-483.
23. Yu. Nesterov and B. T. Polyak, Cubic regularization of Newton's method and its global performance, Math. Program., 108 (2006), pp. 177-205.
24. J. Nocedal and S. J. Wright, Numerical Optimization, 2nd ed., Springer-Verlag, New York, 2006.
25. E. O. Omojokun, Trust Region Algorithms for Optimization with Nonlinear Equality and Inequality Constraints, Ph.D. Thesis, University of Colorado, Boulder, CO, 1989.
26. M. J. D. Powell and Y. Yuan, A trust region algorithm for equality constrained optimization, Math. Programming, 49 (1990), pp. 189-213.
27. A. Vardi, A trust region algorithm for equality constrained minimization: Convergence properties and implementation, SIAM J. Numer. Anal., 22 (1985), pp. 575-591.
28. M. Weiser, P. Deuflhard, and B. Erdmann, Affine conjugate adaptive Newton methods for nonlinear elastomechanics, Optim. Methods Softw., 22 (2007), pp. 413-431.