On the use of iterative methods in cubic regularization for unconstrained optimization

Computational Optimization and Applications

Volumn 60, Issue 1, 2015, Pages 35-57

  Bianconcini, Tommaso ^a,b   Liuzzi, Giampaolo ^c   Morini, Benedetta ^d   Sciandrone, Marco ^a  

^a Dipartimento di Ingegneria Civile   (Italy)

^b KKT srl   (Italy)

^c CNR   (Italy)

^d UNIVERSITY OF FLORENCE   (Italy)

Author keywords

Cubic regularization; Matrix free subproblem solvers; Unconstrained optimization; Worst case complexity

Indexed keywords

GRADIENT METHODS; MATRIX ALGEBRA; OPTIMIZATION;

CUBIC REGULARIZATION; MATRIX FREE; NUMERICAL EXPERIMENTS; OBJECTIVE FUNCTIONS; SMOOTH FUNCTIONS; STOPPING CRITERIA; UNCONSTRAINED OPTIMIZATION; WORST-CASE COMPLEXITY;

NUMERICAL METHODS;

EID: 84921701625     PISSN: 09266003     EISSN: 15732894     Source Type: Journal    
DOI: 10.1007/s10589-014-9672-x     Document Type: Article

References (19)

1. Bellavia, S., Morini, B.: Strong local convergence properties of adaptive regularized methods for nonlinear least-squares. IMA J. Numer. Anal. doi:10.1093/imanum/dru021
2. Bellavia, S., Cartis, C., Gould, N.I.M., Morini, B., Toint, PhL: Convergence of a regularized euclidean residual algorithm for nonlinear least-squares. SIAM J. Numer. Anal. 48, 1–29 (2010)
3. Benson, H.Y., Shanno, D.F.: Interior-Point methods for nonconvex nonlinear programming: cubic regularization. Comput. Optim. Appl. 58, 323–346 (2014)
4. Bishop, C.M.: Pattern Recognition and Machine Learning. Springer, Berlin (2006)
5. Cartis, C., Gould, N.I.M., Toint, PhL: On the complexity of steepest descent, Newton’s and regularized Newton’s methods for nonconvex unconstrained optimization. SIAM J. Optim. 20, 2833–2852 (2010)
6. Cartis, C., Gould, N.I.M., Toint, PhL: Adaptive cubic overestimation methods for unconstrained optimization. Part I: motivation, convergence and numerical results. Math. Program. A 127, 245–295 (2011)
7. Cartis, C., Gould, N.I.M., Toint, PhL: Adaptive cubic overestimation methods for unconstrained optimization. Part II: worst-case function-evaluation complexity. Math. Program. A 130, 295–319 (2011)
8. Cartis, C., Gould, N.I.M., Toint, PhL: An adaptive cubic regularization algorithm for nonconvex optimization with convex constraints and its function-evaluation complexity. IMA J. Numer. Anal. 32, 1662–1695 (2012)
9. Dolan, E.D., Moré, J.J.: Benchmarking optimization software with performance profiles. Math. Program. Ser. A 91, 201–213 (2002)
10. Gould, N.I.M., Orban, D., Toint, PhL: GALAHAD–a library of thread-safe Fortran 90 packages for large-scale nonlinear optimization. ACM Trans. Math. Softw. 29, 353–372 (2003)
11. Gould, N.I.M., Orban, D., Toint, PhL: CUTEr (and SifDec), a constrained and unconstrained testing environment, revisited. ACM Trans. Math. Softw. 29, 373–394 (2003)
12. Gould, N.I.M., Porcelli, M., Toint, PhL: Updating the regularization parameter in the adaptive cubic regularization algorithm. Comput. Optim. Appl. 53, 1–22 (2012)
13. Gould, N.I.M., Lucidi, S., Roma, M., Toint, PhL: Solving the trust-region subproblem using the Lanczos method. SIAM J. Optim. 9, 504–525 (1999)
14. Griewank, A.: The Modification of Newton’s Method for Unconstrained Optimization by Bounding Cubic Terms. Technical Report NA, 12 (1981). Department of Applied Mathematics and Theoretical Physics, University of Cambridge, UK (1981)
15. Grippo, L., Sciandrone, M.: Nonmonotone globalization techniques for the Barzilai–Borwein gradient method. Comput. Optim. Appl. 23, 143–169 (2002)
16. Nesterov, Yu.: Modified Gauss–Newton scheme with worst-case guarantees for global performance. Optim. Methods Softw. 22, 469–483 (2007)
17. Nesterov, Yu., Polyak, B.T.: Cubic regularization of Newton’s method and its global performance. Math. Program. 108, 177–205 (2006)
18. Toint, L.: Nonlinear stepsize control, trust regions and regularizations for unconstrained optimization. Optim. Methods Softw. 28, 82–95 (2013)
19. Weiser, M., Deuflhard, P., Erdmann, B.: Affine conjugate adaptive Newton methods for nonlinear elastomechanics. Optim. Methods Softw. 22, 413–431 (2007)