The Lanczos and conjugate gradient algorithms in finite precision arithmetic

Acta Numerica

Volumn 15, Issue , 2006, Pages 471-542

  Meurant, Gérard ^a   Strakoš, Zdeněk ^b  

^a CEA DIF   (France)

^b INSTITUTE OF COMPUTER SCIENCE   (Czech Republic)

Author keywords

[No Author keywords available]

Indexed keywords

EID: 33646773628     PISSN: 09624929     EISSN: 14740508     Source Type: Journal    
DOI: 10.1017/S096249290626001X     Document Type: Review

References (140)

1. M. Arioli (2004), A stopping criterion for the conjugate gradient algorithms in a finite element method framework, Numer. Math. 97, 1-24.
2. M. Arioli, E. Noulard and A. Russo (2001), Stopping criteria for iterative methods: applications to PDE's, Calcolo 38, 97-112.
3. W. E. Arnoldi (1951), The principle of minimized iterations in the solution of the matrix eigenvalue problem. Quart. Appl. Math. 9, 17-29.
4. O. Axelsson and I. Kaporin (2001), Error norm estimations and stopping criteria in preconditioned conjugate gradient iterations, Numer. Linear Algebra Appl. 8, 265-286.
5. O. Axelsson and G. Linskog (1986), On the rate of convergence of the preconditioned conjugate gradient methods, Numer. Math. 51, 209-227.
6. B. J. C. Baxter and A. Iserles (2003), On the foundations of computational mathematics, in Vol. XI of Handbook of Numerical Analysis, North-Holland, Amsterdam, pp. 3-34.
7. B. Beckermann and A. Kuijlaars (2002), Superlinear CG convergence for special right-hand sides. Electron. Trans. Numer. Anal. 14. 1-19.
8. A. Björck, T. Elfving and Z. Strakos (1998), Stability of conjugate gradients and Lanczos methods for linear least squares problems, SIAM J. Matrix Anal. Appl. 19, 720-736.
9. J. A. M. Bollen (1980). Round-off error analysis of descent methods for solving linear equations, PhD thesis. Technische Hogeschool Eindhoven, the Netherlands.
10. J. A. M. Bollen (1984), Numerical stability of descent methods for solving linear equations, Numer. Math. 43, 361-377.
11. A. Bouras and V. Frayssé (2005), Inexact matrix-vector products in Krylov methods for solving linear systems: A relaxation strategy, SIAM J. Matrix Anal. Appl. 26, 660-678.
12. D. Calvetti. S. Morigi, L. Reichel and F. Sgallari (2000), Computable error bounds and estimates for the conjugate gradient. Numer. Algorithms 25, 79-88.
13. E. B. Christoffel (1877), Sur une classe particulière de fonctions entières et de fractions continues, Ann. Mat. Pura Appl. 8, 1-10.
14. J. K. Cullum and R. A. Willoughby (1985). Lanczos Algorithms for Large Symmetric Eigenvalue Computations. Vol. I. Theory, Vol. II, Programs, Birkhäuser.
15. Reprinted by SIAM in the series Classics in Applied Mathematics.
16. G. Dahlquist. S. C. Eisenstat and G. H. Golub (1972), Bounds for the error of linear systems of equations using the theory of moments, J. Math. Anal. Appl. 37, 151-166.
17. G. Dahlquist, G. H. Golub and S. G. Nash (1978), Bounds for the error in linear systems, in Proc. Workshop on Semi-Infinite Programming (R. Hettich. ed.). Springer, pp. 154-172.
18. J. W. Daniel (1967). The conjugate gradient method for linear and nonlinear operator equations, SIAM J. Numer. Anal. 4, 10-26.
19. G. Darboux (1878), Mémoire sur l'approximation des fonctions de très grand nombres et sur une classe étendue de développements en série. J. Mat. Pures Appl. 4, 5-56, 377-416.
20. P. Davis and P. Rabinowitz (1984), Methods of Numerical Integration, second edition. Academic Press.
21. P. Deuflhard (1994), Cascadic conjugate gradient methods for elliptic partial differential equations: Algorithm and numerical results, in Domain Decomposition Methods in Scientific and Engineering Computing (University Park PA, 1993). Vol. 180 of Contemporary Mathematics, AMS, Providence, RI, pp. 2942.
22. I. Dhillon and B. N. Parlett (2003), Orthogonal eigenvectors and relative gaps, SIAM J. Matrix Anal. Appl. 25, 858-899.
23. I. Dhillon and B. N. Parlett (2004). Multiple representations to compute orthogonal eigenvectors of symmetric tridiagonal matrices, Linear Algebra Appl. 387, 1-28.
24. V. Druskin, A. Greenbaum and L. Knizhnerman (1998). Using nonorthogonal Lanczos vectors in the computation of matrix functions, SIAM J. Sci. Comput. 19, 38-54.
25. V. Druskin and L. Knizhnerman (1991), Error bounds in the simple Lanczos procedure for computing functions of symmetric matrices and eigenvalues. Comput. Math. Math. Phys. 31, 20-30.
26. V. Druskin, L. Borcea and L. Knizhnerman (2005). On the sensitivity of Lanczos recursions to the spectrum, Linear Algebra Appl. 396, 103-125.
27. S. Elhay, G. M. L. Gladwell. G. H. Golub and Y. M. Ram (1999), On some eigenvector-eigenvalue relations. SIAM J. Matrix Anal. Appl. 20. 563-574.
28. J. van den Eshof (2003), Nested iteration methods for nonlinear matrix problems. PhD thesis, University of Utrecht.
29. B. Fischer (1996), Polynomial Based Iteration Methods for Symmetric Linear Systems, Wiley, Chichester.
30. R. Fletcher (1976). Conjugate gradient methods for indefinite systems, in Numerical Analysis: Proc. 6th Biennial Dundee Conf. Univ. Dundee (Dundee, 1975). Vol. 506 of Lecture Notes in Mathematics, Springer, Berlin, pp. 73-89.
31. V. M. Fridman (1963), The method of minimum iterations with minimum errors for a system of linear algebraic equations with a symmetrical matrix, USSR Comput. Math. Math. Phys. 2, 362-363.
32. F. R. Gantmacher (1959), The Theory of Matrices. Vol. 1 and 2, Chelsea Publishing Co., New York.
33. W. Gautschi (1968), Construction of Gauss-Christoffel quadrature formulas, Math. Comp. 22, 251-270.
34. W. Gautschi (1981), A survey of Gauss-Christoffel quadrature formulae, in E. B. Christoffel: The Influence of His Work on Mathematics and the Physical Sciences (P. L. Bultzer and F. Fehér, eds), Birkhauser, Boston, pp. 73-157.
35. W. Gautschi (1982), On generating orthogonal polynomials, SIAM J. Sci. Statist. Comput. 3. 289-317.
36. W. Gautschi (1985), Orthogonal polynomials: Constructive theory and applications, J. Comput. Appl. Math. 12 & 13, 61-76.
37. W. Gautschi (2002). The interplay between classical analysis and (numerical) linear algebra: A tribute to Gene H. Golub, Electron. Trans. Numer. Anal. 13. 119-147.
38. W. Gautschi (2004), Orthogonal Polynomials. Computation and Approximation. Oxford University Press, Oxford.
39. G. H. Golub (1973), Some uses of the Lanczos algorithm in numerical linear algebra, in Topics in Numerical Analysis (J. H. H. Miller, ed.), Springer, Heidelberg/New York. pp. 23-31.
40. G. H. Golub and U. von Matt (1991). Quadratically constrained least squares and quadratic problem, Numer. Math. 59, 561-580.
41. G. H. Golub and G. Meurant (1994), Matrices, moments and quadrature, in Numerical Analysis 1993 (D. F. Griffiths and G. A. Watson, eds), Pitman Research Notes in Mathematics, pp. 105-156.
42. G. H. Golub and G. Meurant (1997). Matrices, moments and quadrature II: How to compute the norm of the error in iterative methods, BIT 37. 687-705.
43. G. H. Golub and D. P. O'Leary (1989), Some history of the conjugate gradient and Lanczos algorithms: 1948-1976, SIAM Rev. 31, 50-102.
44. G. H. Golub and Z. Strakoš (1994), Estimates in quadratic formulas, Numer. Algorithms 8, 241-268.
45. G. H. Golub and J. H. Welsch (1969), Calculation of Gauss quadrature rules, Math. Comp. 23, 221-230.
46. W. B. Gragg and W. J. Harrod (1984), The numerically stable reconstruction of Jacob! matrices from spectral data, Numer. Math. 44, 317-335.
47. J. Grcar (1981), Analysis of the Lanczos algorithm and of the approximation problem in Richardson's method, PhD thesis, University of Illinois at UrbanaChampaign.
48. A. Greenbaum (1979), Comparison of splittings used with the conjugate gradient algorithm. Numer. Math. 33, 181-194.
49. A. Greenbaum (1981), Convergence properties of the conjugate gradient algorithm in exact and finite precision arithmetic, PhD thesis, University of California, Berkeley.
50. A. Greenbaum (1989), Behavior of slightly perturbed Lanczos and conjugate gradient recurrences, Linear Algebra Appl. 113, 7-63.
51. A. Greenbaum (1994), The Lanczos and conjugate gradient algorithms in finite precision arithmetic, in Proc. Cornelius Lanczos International Centenary Conference. 1993 (J. D. Brown, M. T. Chu. D. C. Ellison and R. J. Plemmons, eds), SIAM, pp. 49-60.
52. A. Greenbaum (1997o), Iterative, Methods for Solving Linear Systems, SIAM.
53. A. Greenbaum (1997e), Estimating the attainable accuracy of recursively computed residual methods, SIAM J. Matrix Anal. Appl. 18, 535-551.
54. A. Greenbaum and Z. Strakos (1992), Predicting the behavior of finite precision Lanczos and conjugate gradient computations. SIAM J. Matrix Anal. Appl. 13, 121-137.
55. M. Gutknecht and Z. Strakos (2000). Accuracy of two three-term and three twoterm recurrences for Krylov space solvers, SIAM J. Matrix Anal. Appl. 22, 213-229.
56. W. Hackbusch (1994), Iterative Solution of Large Sparse Systems of Equations, Vol. 95 of Applied Mathematical Sciences. Springer. New York. Translated and revised from the 1991 German original.
57. L. Hageman and D. Young (1981), Applied Iterative Methods, Academic Press, Orlando.
58. M. R. Hestenes and E. Stiefel (1952). Methods of conjugate gradients for solving linear systems, J. Nat. Bur. Standards 49, 409-436.
59. M. R. Hestenes and J. Todd (1991). Mathematicians Learning to Use Computers, National Institute of Standards and Technology Special Publication 730. US department of Commerce, National Institute of Standards and Technology, Washington. DC.
60. N. J. Higham (2002). Accuracy and Stability of Numerical Algorithms, second edition. SIAM.
61. A. S. Householder (1975), The Theory of Matrices in Numerical Analysis, Dover, New York. Reprint of 1964 edition.
62. S. Kaniel (1966), Estimates of some computational techniques in linear algebra, Math. Cornp. 20, 369-378.
63. J. Kautsky and G. H. Golub, (1983), On the calculation of Jacobi matrices. Linear Algebra Appl. 53/53. 439-455.
64. L. Knizhnerman (1995a), The quality of approximations to an isolated eigenvalue and the distribution of 'Ritz numbers' in the simple Lanczos procedure, Cornput. Math. Math. Phys. 35, 1175-1187.
65. L. Knizhnerman (1995b), On adaptation of the Lanczos method to the spectrum. Report EMG-001-95-12, Schlumberger-Doll-Research.
66. L. Knizhnerman (1996), The simple Lanczos procedure: estimates of the error of the Gauss quadrature formula and their applications, Comput. Math. Math. Phys. 36, 1481-1492.
67. A. N. Krylov (1931), O Čislemnon rešenii uravnenija, kotorym v techničeskih voprasah opredeljajutsja častoy malyh kolebaniǐ material'nyh., Izv. Adad. Nauk SSSR old. Mat. Estet., pp. 491-539.
68. A. B. J. Kuijlaars (2006), Convergence analysis of Krylov subspace iterations with methods from potential theory, SIAM Review 48, 3-40.
69. C. Lanczos (1950), An iteration method for the solution of the eigenvalue problem of linear differential and integral operators, J. Res. Nat. Bur. Standards 45. 255-282.
70. C. Lanczos (1952), Solution of systems of linear equations by minimized iterations, J. Res. Nat. Bur. Standards 49, 33-53.
71. D. P. Laurie (1999), Accurate recovery of recursion coefficients from Gaussian quadrature formulas. J. Comput. Appl. Math. 112, 165-180.
72. D. P. Laurie (2001), Computation of Gauss-type quadrature formulas, J. Comput. Appl. Math. 127, 201-217.
73. J. G. Lewis (1977), Algorithms for sparse matrix eigenvalue problems. PhD thesis. Report STAN-CS-77-595, Computer Science Department, Stanford University, Stanford, CA.
74. Ren-Cang Li (2005), On Meinardus' examples for the conjugate gradient method. Technical Report 2005-06, Department of Mathematics, University of Kentucky.
75. J. Liesen and P. Tichy (2005), On the worst case convergence of MR and CG for symmetric positive definite tridiagonal Toeplitz matrices, Electron. Trans. Numer. Anal. 20, 180-197.
76. D. G. Luenberger (1969), Hyperbolic pairs in the method of conjugate gradients, SIAM J. Appl. Math. 17, 1263-1267.
77. D. G. Luenberger (1970), The conjugate residual method for constrained minimization problems, SIAM J. Numer. Anal 7, 390-398.
78. G. Meinardus (1963), Über eine Verallgemeinerung einer Ungleichung von L. V. Kantorowitsch. Numer. Math. 5, 14-23.
79. G. Meurant (1997), The computation of bounds for the norm of the error in the conjugate gradient algorithm, Numer. Algorithms 16, 77-87.
80. G. Meurant (1999a), Numerical experiments in computing bounds for the norm of the error in the preconditioned conjugate gradient algorithm, Numer. Algorithms 22, 353-365.
81. G. Meurant (1999b). Computer Solution of Large Linear Systems. North-Holland.
82. 2 norm of the error in the conjugate gradient algorithm, Nume.r. Algorithms 40, 157-169.
83. G. Meurant (2006). The Lanczos and Conjugate Gradient Algorithms: From Theory to Finite Precision Computations, SIAM, book to appear.
84. Y. Notay (1993), On the convergence rate of the conjugate gradients in the presence of rounding errors, Numer. Math. 65, 301-317.
85. D. P. O'Leary and Z. Strakoš (2004), On sensitivity of Gauss-Christoffel quadrature estimates. Computer Science Department Report CS-TR-4622, Institute for Advanced Computer Studies Report UMIACS-2004-64, University of Maryland.
86. M. M. Overton (2001), Numerical Computing with IEEE Floating Point Arithmetic, SI AM.
87. C. C. Paige (1969a), Error analysis of the generalized Hessenberg processes. Technical Note ICSI 179, London University Institute of Computer Science.
88. C. C. Paige (1969b), Eigenvalues of perturbed Hermitian matrices. Technical Note ICSI 179. London University Institute of Computer Science.
89. C. C. Paige (1970a). Error analysis of the symmetric Lanczos process for the eigenproblem. Technical Note ICSI 248. London University Institute of Computer Science.
90. C. C. Paige (1970b), Practical use of the symmetric Lanczos process with reorthogonalization, BIT 10, 183-195.
91. C. C. Paige (1971). The computation of eigenvalues and eigenvectors of very large sparse matrices, PhD thesis, University of London.
92. C. C. Paige (1972). Computational variants of the Lanczos method for the eigenproblem, J. Inst. Math. Appl. 10. 373-381.
93. C. C. Paige (1976), Error analysis of the Lanczos algorithm for tridiagonalizing a symmetric matrix, J. Inst. Math. Appl. 18, 341-349.
94. C. C. Paige (1980), Accuracy and effectiveness of the Lanczos algorithm for the symmetric eigenproblem, Linear Algebra Appl. 34, 235-258.
95. C. C. Paige and M. Saunders (1975). Solution of sparse indefinite systems of linear equations, SIAM J. Numer. Anal. 12, 617-629.
96. C. C. Paige and Z. Strakoš (1999), Correspondence between exact arithmetic and finite precision behavior of Krylov space methods, in XIV Householder Symposium (J. Varah. ed.), University of British Columbia, pp. 250-253.
97. C. C. Paige, B. N. Parlett and H. van der Vorst (1995), Approximate solutions and eigenvalue bounds from Krylov subspaces, Numer. Linear Algebra, Appl. 2, 115-133.
98. B. N. Parlett (1980). The Symmetric Eigenvalue Problem, Prentice-Hall.
99. B. N. Parlett (1990), The contribution of J. H. Wilkinson to numerical analysis, in A History of Scientific Computing (Princeton, NJ, 1987), ACM Press Hist. Ser., ACM. New York, pp. 17-30.
100. B. N. Parlett (1992), The rewards for maintaining semi-orthogonality among Lanczos vectors, Numer. Linear Algebra Appl. 1, 234-267.
101. B. N. Parlett (1994), Do we fully understand the symmetric Lanczos algorithms yet?, in Proc. Cornelius Lanczos International Centenary Conference. 1993 (J. D. Brown, M. T. Chu, D. C. Ellison and R. J. Plemmons, eds), SIAM, pp. 93-108.
102. B. N. Parlett (1996), Invariant subspaces for tightly clustered eigenvalues of tridiagonals, BIT 36. 542-562.
103. B. N. Parlett and I. Dhillon (2000), Relatively robust representations of symmetric tridiagonals. Linear Algebra Appl. 309, 121-151.
104. B. N. Parlett and D. S. Scott (1979), The Lanczos algorithm with selective orthogonalization, Math. Comp. 33, 217-238.
105. H. Rutishauser (1959), Theory of gradient methods, in Refined Iterative Alehods for Computation of the Solution and the. Eigenvalues of Self-Adjoint Boundary Value Problems. Mitt. Inst. Angew. Math. ETH Zürich. Birkhäuser. Basel, pp. 24-49.
106. Y. Saad (1980), On the rates of convergence of the Lanczos and the block Lanezos methods, SIAM J. Numer. Anal. 17, 687-706.
107. Y. Saad (1992), Numerical Methods for Large Eigenvalue Problems, Wiley.
108. D. S. Scott (1978), Analysis of the symmetric Lanczos algorithm, PhD thesis. University of California, Berkeley.
109. D. S. Scott (1979), How to make the Lanczos algorithm converge slowly, Math. Comp. 33, 239-247.
110. D. S. Scott (1981), The Lanczos algorithm, in Sparse Matrices and Their Use (I. S. Duff, ed.), Academic Press, pp. 139-159.
111. H. D. Simon (1982), The Lanczos algorithm for solving symmetric linear systems, PhD thesis, University of California, Berkeley.
112. H. D. Simon (1984a), The Lanczos algorithm with partial reorthogonalization, Math. Comp. 42, 115-142.
113. H. D. Simon (1984b), Analysis of the symmetric Lanczos algorithm with reorthogonalization methods, Linear Algebra Appl. 61, 101-131.
114. V. Simoncini and D. Szyld (2005), Recent developments in Krylov subspace methods for linear systems. Research Report 05-9-25. Department of Mathematics, Temple University.
115. G. L. G. Sleijpen, H. A. van der Vorst and D. R. Fokkema (1994), BiCGstab(l) and other hybrid Bi-CG methods, Numer. Algorithms 7, 75-109.
116. G. L. G. Sleijpen, H. A. van der Vorst and J. Modersitzki (2001), Difference in the effects of rounding errors in Krylov solvers for symmetric indefinite systems. SIAM J. Matrix Anal. Appl. 22, 726-751.
117. A. van der Sluis and H. A. van der Vorst (1986), The rate of convergence of conjugate gradients, Numer. Math. 48, 543-560.
118. A. van der Sluis and H. A. van der Vorst (1987), The convergence behavior of Ritz values in the presence of close eigenvalues, Linear Algebra Appl. 88, 651-694.
119. G. W. Stewart (2001), Matrix Algorithms, Vol. II: Eigensystems, SIAM.
120. T. J. Stieltjes (1884), Quelques recherches sur la théorie des quadratures mécaniques. Ann. Sci. Ecole Norm. Paris 1, 409-426.
121. [Oeuvres I, 377-396.]
122. J. Stoer (1983), Solution of large linear systems of equations by conjugate gradient type methods, in Mathematical Programming: The. State of the Art (A. Bachern, M. Grötschel and B. Körte, eds), Springer, pp. 540-565.
123. J. Stoer and R. Bulirsch (1983), Introduction to Numerical Analysis, second edition. Springer.
124. J. Stoer and R. W. Freund (1982), On the solution of large linear systems of equations by conjugate gradient type methods, in Computer Methods in Applied Science and Engineering V (R. Glowinski and J. L. Lions, eds), NorthHolland, pp. 35-53.
125. Z. Strakos (1991), On the real convergence rate of the conjugate gradient method, Lmear Algebra Appl. 154-156, 535-549.
126. Z. Strakos (1998), Convergence and numerical behavior of the Krylov space methods, NATO ASI Institute Algorithms for Large, Sparse, Linear Algebraic Systems: The State, of the Art and Applications in Science and Engineering (G. Winter Althaus and E, Spedicato, eds), Kluwer Academic, pp. 175-197,
127. Z. Strakos and A. Greenbaum (1992), Open questions in the convergence analysis of the Lanczos process for the real symmetric eigenvalue problem, IMA Preprint Series 934, University of Minnesota.
128. Z. Strakos and J. Liesen (2005), On numerical stability in large scale linear algebraic computations, ZAMM Z. Angew. Math. Mech. 85, No. 5, 307-325.
129. Z. Strakos and P. Tichy (2002), On error estimation in the conjugate gradient method and why it works in finite precision computation. Electron. Trans. Numer. Anal. 13, 56-80,
130. Z, Strakos and P. Tichy (2005), Error estimation in preconditioned conjugate gradients, BIT Numerical Mathematics 45, 789-817.
131. R. C. Thompson and P. McEnteggert (1968). Principal submatrices II: The upper and lower quadratic inequalities, Linear Algebra Appl. 1, 211-243.
132. H. A. van der Vorst (1982), Preconditioning by incomplete decompositions, PhD thesis, Academic Computer Centrum Utrecht.
133. H. A. van der Vorst (2003), Iterative Krylov Methods for Large Linear Systems, Cambridge University Press.
134. J. H. Wilkinson (1965), The Algebraic Eigenvalue, Problem. Oxford University Press,
135. H, Wozniakowski (1978), Round-off error analysis of iterations for large linear systems, Numer. Math. 30, 301-314,
136. H, Wozniakowski (1980), Round-off error analysis of a new class of conjugate gradient algorithms, Linear Algebra Appl. 29, 507-529.
137. W. Wülling (2005), The stabilization of weights in the Lanczos and conjugate gradient methods, BIT Numerical Mathematics 45, 395-414.
138. W. Wülling (2006), On stabilization and convergence of clustered Ritz values in the Lanczos method, SIAM J. Matrix Anal. Appl 27, 891-908.
139. Q. Ye (1995), On close eigenvalues of tridiagonal matrices, Numer. Math. 70, 507-514.
140. J.-P. M. Zemke (2003), Krylov subspace methods in finite precision: a unified approach. PhD thesis. Technical University of Hamburg-Harburg.