MINRES-QLP: A Krylov subspace method for indefinite or singular symmetric systems

SIAM Journal on Scientific Computing

Volumn 33, Issue 4, 2011, Pages 1810-1836

  Chop, Sou Cheng T ^a   Paige, Christopher C ^b   Saunders, Michael A ^a  

^a Stanford University   (United States)

^b MCGILL UNIVERSITY   (Canada)

Author keywords

Conjugate gradient method; Krylov subspace method; Lanczos process; Minimum residual method; MINRES; Singular least squares problem; Sparse matrix

Indexed keywords

LEAST SQUARES APPROXIMATIONS; MATRIX ALGEBRA; NUMBER THEORY;

CONJUGATE-GRADIENT METHOD; KRYLOV-SUBSPACE METHODS; LANCZOS PROCESS; LEAST SQUARE PROBLEMS; MINIMUM-RESIDUAL METHOD; MINRES; RESIDUAL METHOD; SINGULAR LEAST-SQUARE PROBLEM; SPARSE MATRICES; SYMMETRIC SYSTEMS;

CONJUGATE GRADIENT METHOD;

EID: 80052698424     PISSN: 10648275     EISSN: None     Source Type: Journal    
DOI: 10.1137/100787921     Document Type: Article

References (55)

1. M. ARIOLI AND S. GRATTON, Least-Squares Problems, Normal Equations, and Stopping Criteria for the Conjugate Gradient Method, Technical Report RAL-TR-2008-008, Rutherford Appleton Laboratory, Oxfordshire, UK, 2008.
2. Z.-Z. BAI AND G.-Q. LI, Restrictively preconditioned conjugate gradient methods for systems of linear equations, IMA J. Numer. Anal., 23(2003), pp. 561-580.
3. M. BENZI, Preconditioning techniques for large linear systems: A survey, J. Comput. Phys., 182(2002), pp. 418-477. (Pubitemid 36010064)
4. D. S. BERNSTEIN, Matrix Mathematics: Theory, Facts, and Formulas, 2nd ed., Princeton University Press, Princeton, NJ, 2009.
5. E. Y. BOBROVNIKOVA AND S. A. VAVASIS, Accurate solution of weighted least squares by iterative methods, SIAM J. Matrix Anal. Appl., 22(2001), pp. 1153-1174.
6. A. M. BRADLEY, Algorithms for the Equilibration of Matrices and Their Application to Limited-Memory Quasi-Newton Methods, Ph. D. thesis, ICME, Stanford University, 2010.
7. X.-W. CHANG, C. C. PAIGE, AND D. TITLEY-PELOQUIN, Stopping criteria for the iterative solution of linear least squares problems, SIAM J. Matrix Anal. Appl., 31(2009), pp. 831-852.
8. T.-Y. CHEN AND J. W. DEMMEL, Balancing sparse matrices for computing eigenvalues, Linear Algebra Appl., 309(2000), pp. 261-287.
9. S.-C. T. CHOI, Iterative Methods for Singular Linear Equations and Least-Squares Problems, Ph. D. thesis, ICME, Stanford University, 2006.
10. S.-C. CHOI, D. L. DONOHO, A. G. FLESIA, X. HUO, O. LEVI, AND D. SHI, About Beamlab-A Toolbox for New Multiscale Methodologies, http://www-stat. stanford.edu/~beamlab/, 2002.
11. J. CLAERBOUT, Hypertext documents about reproducible research, http://sepwww.stanford.edu/doku.php?id=sep:research:reproducible.
12. F. A. DUL, MINRES and MINERR are better than SYMMLQ in eigenpair computations, SIAM J. Sci. Comput., 19(1998), pp. 1767-1782. (Pubitemid 128614418)
13. L. ELDÉN, Algorithms for the regularization of ill-conditioned least squares problems, Nordisk Tidskr. Informationsbehandling (BIT), 17(1977), pp. 134-145.
14. B. FISCHER, A. RAMAGE, D. J. SILVESTER, AND A. J. WATHEN, Minimum residual methods for augmented systems, BIT, 38(1998), pp. 527-543. (Pubitemid 128437154)
15. R. W. FREUND AND N. M. NACHTIGAL, A new Krylov-subspace method for symmetric indefinite linear systems, in Proceedings of the 14th IMACS World Congress on Computational and Applied Mathematics, W. F. Ames, ed., IMACS, 1994, pp. 1253-1256.
16. K. A. GALLIVAN, S. THIRUMALAI, P. VAN DOOREN, AND V. VERMAUT, High performance algorithms for Toeplitz and block Toeplitz matrices, Linear Algebra Appl., 241/243(1996), pp. 343-388.
17. P. E. GILL, W. MURRAY, D. B. PONCELEÓN, AND M. A. SAUNDERS, Preconditioners for indefinite systems arising in optimization, SIAM J. Matrix Anal. Appl., 13(1992), pp. 292-311.
18. G. H. GOLUB AND C. F. VAN LOAN, Matrix Computations, 3rd ed., Johns Hopkins University Press, Baltimore, MD, 1996.
19. J. HADAMARD, Sur les problèmes aux dérivées partielles et leur signification physique, Princeton Univ. Bull., 13(1902), pp. 49-52.
20. W. W. HAGER, Iterative methods for nearly singular linear systems, SIAM J. Sci. Comput., 22(2000), pp. 747-766.
21. M. HANKE AND J. G. NAGY, Restoration of atmospherically blurred images by symmetric indefinite conjugate gradient techniques, Inverse Problems, 12(1996), pp. 157-173. (Pubitemid 126636637)
22. P. C. HANSEN, Truncated singular value decomposition solutions to discrete ill-posed problems with ill-determined numerical rank, SIAM J. Sci. Statist. Comput., 11(1990), pp. 503-518.
23. P. C. HANSEN AND D. P. O'LEARY, The use of the L-curve in the regularization of discrete ill-posed problems, SIAM J. Sci. Comput., 14(1993), pp. 1487-1503.
24. M. R. HESTENES AND E. STIEFEL, Methods of conjugate gradients for solving linear systems, J. Res. Nat. Bur. Standards, 49(1952), pp. 409-436.
25. N. J. HIGHAM, Accuracy and Stability of Numerical Algorithms, 2nd ed., SIAM, Philadelphia, PA, 2002.
26. P. JIRÁNEK AND D. TITLEY-PELOQUIN, Estimating the backward error in LSQR, SIAM J. Matrix Anal. Appl., 31(2010), pp. 2055-2074.
27. E. F. KAASSCHIETER, Preconditioned conjugate gradients for solving singular systems, J. Comput. Appl. Math., 24(1988), pp. 265-275.
28. KAUSTUV, IPSOL: An Interior Point Solver for Nonconvex Optimization Problems, Ph. D. thesis, SCCM Program, Stanford University, Stanford, CA, 2008.
29. M. KILMER AND G. W. STEWART, Iterative regularization and MINRES, SIAM J. Matrix Anal. Appl., 21(1999), pp. 613-628.
30. C. LANCZOS, An iteration method for the solution of the eigenvalue problem of linear differential and integral operators, J. Res. Nat. Bur. Standards, 45(1950), pp. 255-282.
31. R. M. LARSEN, Efficient Algorithms for Helioseismic Inversion, Ph. D. thesis, Department of Computer Science, University of Aarhus, 1998.
32. J. G. LEWIS, Algorithms for Sparse Matrix Eigenvalue Problems, Ph. D. thesis, Department of Computer Science, Stanford University, 1976.
33. O. E. LIVNE AND G. H. GOLUB, Scaling by binormalization, Numer. Algorithms, 35(2004), pp. 97-120. (Pubitemid 39072772)
34. M. F. MURPHY, G. H. GOLUB, AND A. J. WATHEN, A note on preconditioning for indefinite linear systems, SIAM J. Sci. Comput., 21(2000), pp. 1969-1972. (Pubitemid 32076050)
35. M. G. NEYTCHEVA AND P. S. VASSILEVSKI, Preconditioning of indefinite and almost singular finite element elliptic equations, SIAM J. Sci. Comput., 19(1998), pp. 1471-1485. (Pubitemid 128623558)
36. J. NOCEDAL AND S. J. WRIGHT, Numerical Optimization, 2nd ed., Springer, New York, 2006.
37. Y. NOTAY, Solving positive (semi) definite linear systems by preconditioned iterative methods, in Preconditioned Conjugate Gradient Methods (Nijmegen, 1989), Lecture Notes in Math. 1457, Springer, Berlin, 1990, pp. 105-125.
38. C. C. PAIGE, Error analysis of the Lanczos algorithm for tridiagonalizing a symmetric matrix, J. Inst. Math. Appl., 18(1976), pp. 341-349.
39. C. C. PAIGE AND M. A. SAUNDERS, Solution of sparse indefinite systems of linear equations, SIAM J. Numer. Anal., 12(1975), pp. 617-629.
40. C. C. PAIGE AND M. A. SAUNDERS, LSQR: An algorithm for sparse linear equations and sparse least squares, ACM Trans. Math. Software, 8(1982), pp. 43-71.
41. C. C. PAIGE AND M. A. SAUNDERS, Algorithm 583; LSQR: Sparse linear equations and leastsquares prob lems, ACM Trans. Math. Software, 8(1982), pp. 195-209.
42. W.-Q. REN AND J.-X. ZHAO, Iterative methods with preconditioners for indefinite systems, J. Comput. Math., 17(1999), pp. 89-96.
43. M. ROZLOŽNÍK AND V. SIMONCINI, Krylov subspace methods for saddle point problems with indefinite preconditioning, SIAM J. Matrix Anal. Appl., 24(2002), pp. 368-391. (Pubitemid 36446414)
44. Y. SAAD AND M. H. SCHULTZ, GMRES: A generalized minimal residual algorithm for solving nonsymmetric linear systems, SIAM J. Sci. Statist. Comput., 7(1986), pp. 856-869.
45. M. A. SAUNDERS, Solution of sparse rectangular systems using LSQR and Craig, BIT, 35(1995), pp. 588-604.
46. M. A. SAUNDERS, Computing projections with LSQR, BIT, 37(1997), pp. 96-104. (Pubitemid 127485204)
47. G. L. G. SLEIJPEN, H. A. VAN der VORST, AND J. MODERSITZKI, Differences in the effects of rounding errors in Krylov solvers for symmetric indefinite linear systems, SIAM J. Matrix Anal. Appl., 22(2000), pp. 726-751.
48. G. W. STEWART, On the continuity of the generalized inverse, SIAM J. Appl. Math., 17(1969), pp. 33-45.
49. G. W. STEWART, Research, development and LINPACK, in Mathematical Software III, J. R. Rice, ed., Academic Press, New York, 1977, pp. 1-14.
50. G. W. STEWART, The QLP approximation to the singular value decomposition, SIAM J. Sci. Comput., 20(1999), pp. 1336-1348. (Pubitemid 32251220)
51. Systems Optimization Laboratory (SOL), Stanford University, downloadable software, http://www.stanford.edu/group/SOL/software.html.
52. K.-C. TOH, K.-K. PHOON, AND S.-H. CHAN, Block preconditioners for symmetric indefinite linear systems, Internat. J. Numer. Methods Engrg., 60(2004), pp. 1361-1381. (Pubitemid 38904857)
53. L. N. TREFETHEN AND D. BAU, III, Numerical Linear Algebra, SIAM, Philadelphia, PA, 1997.
54. University of Florida Sparse Matrix Collection, http://www.cise.ufl.edu/ research/sparse/matrices/.
55. Y. ZHANG, Solving large-scale linear programs by interior-point methods under the MATLAB environment, Optim. Methods Softw., 10(1998), pp. 1-31.