Trust-region and other regularisations of linear least-squares problems

BIT Numerical Mathematics

Volumn 49, Issue 1, 2009, Pages 21-53

  Cartis, C ^a   Gould, N I M ^b   Toint, P L ^c  

^a UNIVERSITY OF EDINBURGH   (United Kingdom)

^b RUTHERFORD APPLETON LABORATORY   (United Kingdom)

^c UNIVERSITY OF NAMUR   (Belgium)

Author keywords

Linear least squares; Regularisation; Secular equation; Trust region

Indexed keywords

EID: 61849098346     PISSN: 00063835     EISSN: None     Source Type: Journal    
DOI: 10.1007/s10543-008-0206-8     Document Type: Article

References (45)

1. N. Arora L.T. Biegler 2004 A trust-region SQP algorithm for equality constrained parameter estimation with simple parameter bounds Comput. Optim. Appl. 28 1 51 86
2. Biegler, L.T., Ghattas, O., Heinkenschloss, M., van Bloemen Waanders, B. (eds.): Large-Scale PDE-Constrained Optimization. Lecture Notes in Computational Science and Engineering, vol. 30. Springer, Berlin (2003)
3. Björck, Å.: Numerical Methods for Least Squares Problems. SIAM, Philadelphia (1996)
4. Boyd, S., Vandenberghe, L.: Convex Optimization. Cambridge University Press, Cambridge (2004)
5. Cartis, C., Gould, N.I.M., Toint, Ph.L.: Adaptive cubic overestimation methods for unconstrained optimization. Technical Report RAL-TR-2007-007, Rutherford Appleton Laboratory, Chilton, Oxfordshire, England (2007)
6. T.F. Chan J.A. Olkin D.W. Cooley 1992 Solving quadratically constrained least squares using black box solvers BIT 32 3 481 495
7. Conn, A.R., Gould, N.I.M., Toint, Ph.L.: Trust-Region Methods. SIAM, Philadelphia (2000)
8. Dennis, J.E., Schnabel, R.B.: Numerical Methods for Unconstrained Optimization and Nonlinear Equations. Prentice-Hall, Englewood Cliffs (1983). Reprinted as Classics in Applied Mathematics 16, SIAM, Philadelphia, 1996
9. L. Eldén 1977 Algorithms for the regularization of ill-conditioned least squares problems BIT 17 2 134 145
10. Erway, J.B., Gill, P.E., Griffin, J.D.: Iterative methods for finding a trust-region step. Technical Report NA 07-2, Department of Mathematics, University of California, San Diego (2007)
11. W. Gander 1981 Least squares with a quadratic constraint Numer. Math. 36 3 291 307
12. G.H. Golub W. Kahan 1965 Calculating the singular values and pseudo-inverse of a matrix SIAM J. Numer. Anal. 2 2 205 224
13. G.H. Golub U. von Matt 1991 Quadratically constrained least squares and quadratic problems Numer. Math. 59 1 561 580
14. N.I.M. Gould S. Lucidi M. Roma Ph.L. Toint 1999 Solving the trust-region subproblem using the Lanczos method SIAM J. Optim. 9 2 504 525
15. N.I.M. Gould D. Orban Ph.L. Toint 2003 GALAHAD -a library of thread-safe Fortran 90 packages for large-scale nonlinear optimization ACM Trans. Math. Softw. 29 4 353 372
16. N.I.M. Gould D. Orban Ph.L. Toint 2003 CUTEr (and SifDec), a constrained and unconstrained testing environment, revisited ACM Trans. Math. Softw. 29 4 373 394
17. Gould, N.I.M., Toint, Ph.L.: Nonlinear programming without a penalty function or a filter. Technical Report RAL-TR-2007-016, Rutherford Appleton Laboratory, Chilton, Oxfordshire, England (2007)
18. Gould, N.I.M., Toint, Ph.L.: Regularised-funnel methods for nonlinear programming. Technical Report, Rutherford Appleton Laboratory, Chilton, Oxfordshire, England (2009, in preparation)
19. W.W. Hager 2001 Minimizing a quadratic over a sphere SIAM J. Optim. 12 1 188 208
20. W.W. Hager S. Park 2004 Global convergence of SSM for minimizing a quadratic over a sphere Math. Comput. 74 251 1413 1423
21. Hansen, P.C.: Rank-Deficient and Discrete Ill-Posed Problems: Numerical Aspects of Linear Inversion. Monographs on Mathematical Modeling and Computation, vol. 4. SIAM, Philadelphia (1997)
22. Hebden, M.D.: An algorithm for minimization using exact second derivatives. Technical Report T.P. 515, AERE Harwell Laboratory, Harwell, Oxfordshire, England (1973)
23. Henrici, P.: Elements of Numerical Analysis. Wiley, New York (1964)
24. C.P. Nielsen J R. Bunch D.C. Sorensen 1978 Rank-one modification of the symmetric eigenproblem Numer. Math. 31 1 31 48
25. M. Lalee J. Nocedal T. Plantenga 1998 On the implementation of an algorithm for large-scale equality constrained optimization SIAM J. Optim. 8 3 682 706
26. K. Levenberg 1944 A method for the solution of certain problems in least squares Q. Appl. Math. 2 2 164 168
27. L. Lukšan 1993 Inexact trust region method for large sparse nonlinear least-squares Kybernetica 29 4 305 324
28. D. Marquardt 1963 An algorithm for least-squares estimation of nonlinear parameters SIAM J. Appl. Math. 11 2 431 441
29. A. Melman 1995 Numerical solution of a secular equation Numer. Math. 69 4 483 493
30. Moré, J.J.: The Levenberg-Marquardt algorithm: implementation and theory. In: Watson, G.A. (ed.) Numerical Analysis, Dundee 1977. Lecture Notes in Mathematics, vol. 630, pp. 105-116. Springer, Berlin (1978)
31. Morrison, D.D.: Methods for nonlinear least squares problems and convergence proofs. In: Lorell, J., Yagi, F. (eds.) Proceedings of the Seminar on Tracking Programs and Orbit Determination, pp. 1-9. Jet Propulsion Laboratory, Pasadena (1960)
32. Yu. Nesterov 2007 Modified Gauss-Newton scheme with worst-case guarantees for global performance Optim. Methods Softw. 22 3 469 483
33. Yu. Nesterov B.T. Polyak 2006 Cubic regularization of Newton method and its global performance Math. Program. 108 1 177 205
34. A. Neumaier 1998 Solving ill-conditioned and singular linear systems: a tutorial on regularization SIAM Rev. 40 3 636 666
35. Nocedal, J., Wright, S.J.: Numerical Optimization, 2nd edn. Series in Operations Research. Springer, Heidelberg (2006)
36. Omojokun, E.O.: Trust region algorithms for optimization with nonlinear equality and inequality constraints. Ph.D. Thesis, University of Colorado, Boulder, Colorado, USA (1989)
37. Ortega, J.M., Rheinboldt, W.C.: Iterative Solution of Nonlinear Equations in Several Variables. Academic Press, San Diego (1970)
38. C.C. Paige M.A. Saunders 1982 ALGORITHM 583: LSQR: an algorithm for sparse linear equations and sparse least squares ACM Trans. Math. Softw. 8 2 195 209
39. C.C. Paige M.A. Saunders 1982 LSQR: an algorithm for sparse linear equations and sparse least squares ACM Trans. Math. Softw. 8 1 43 71
40. C. Reinsch 1971 Smoothing by spline functions II Numer. Math. 16 5 451 454
41. Rojas, M.: A large-scale trust-region approach to the regularization of discrete ill-posed problems. Ph.D. Thesis, Rice University, Houston, Texas, USA (1998)
42. M. Rojas D.C. Sorensen 2002 A trust-region approach to the regularization of large-scale discrete forms of ill-posed problems SIAM J. Sci. Stat. Comput. 23 6 1842 1860
43. T. Steihaug 1983 The conjugate gradient method and trust regions in large scale optimization SIAM J. Numer. Anal. 20 3 626 637
44. Toint, Ph.L.: Towards an efficient sparsity exploiting Newton method for minimization. In: Duff, I.S. (ed.) Sparse Matrices and Their Uses, pp. 57-88.1 Academic Press, London (1981)
45. Y. Yuan 2000 On the truncated conjugate-gradient method Math. Program. Ser. A 87 3 561 573