On error estimation in the conjugate gradient method and why it works in finite precision computations

Electronic Transactions on Numerical Analysis

Volumn 13, Issue , 2002, Pages 56-80

  Strakoš, Zdeněk ^a   Tichý, Petr ^a  

^a INSTITUTE OF COMPUTER SCIENCE   (Czech Republic)

Author keywords

Conjugate gradient method; Error bounds; Evaluation of convergence; Finite precision arithmetic; Gauss quadrature; Loss of orthogonality; Rounding errors

Indexed keywords

EID: 0347094766     PISSN: 10689613     EISSN: 10689613     Source Type: Journal    
DOI: None     Document Type: Article

References (47)

1. M. ARIOLI, Stopping criterion for the Conjugate Gradient algorithm in a Finite Element method framework, submitted to Numer. Math., (2001).
2. M. ARIOLI AND L. BALDINI, Backward error analysis of a null space algorithm in sparse quadratic programming, SIAM J. Matrix Anal. Appl., 23 (2001), pp. 425-442.
3. O. AXELSSON AND I. KAPORIN, Error norm estimation and stopping criteria in preconditioned Conjugate Gradient iterations, Numer. Linear Algebra Appl., 8 (2001), pp. 265-286.
4. D. CALVETTI, G. H. GOLUB, AND L. REICHEL, Estimation of the L-curve via Lanczos bidiagonalization, BIT, 39 (1999), pp. 603-609.
5. D. CALVETTI, S. MORIGI, L. REICHEL, AND F. SGALLARI, Computable error bounds and estimates for the Conjugate Gradient Method, Numer. Algorithms, 25 (2000), pp. 79-88.
6. _, An iterative method with error estimators, J. Comput. Appl. Math., 127 (2001), pp. 93-119.
7. G. DAHLQUIST, S. EISENSTAT, AND G. H. GOLUB, Bounds for the error of linear systems of equations using the theory of moments, J. Math. Anal. Appl., 37 (1972), pp. 151-166.
8. G. DAHLQUIST, G. H. GOLUB, AND S. G. NASH, Bounds for the error in linear systems, in Proc. Workshop on Semi-Infinite Programming, R. Hettich, ed., Springer, Berlin, 1978, pp. 154-172.
9. P. DEUFLHARD, Cascadic conjugate gradient methods for elliptic partial differential equations I: Algorithm and numerical results, preprint SC 93-23, Konrad-Zuse-Zentrum für Informationstechnik Berlin, Heilbronnen Str., D-10711 Derlin, October 1993.
10. _, Cascadic conjugate gradient methods for elliptic partial differential equations: Algorithm and numerical results, Contemp. Math., 180 (1994), pp. 29-42.
11. B. FISCHER, Polynomial Based Iteration Methods for Symmetric Linear Systems, Wiley Teubner Advances in Numerical Mathematics, Wiley Teubner, 1996.
12. B. FISCHER AND G. H. GOLUB, On the error computation for polynomial based iteration methods, in Recent Advances in Iterative Methods, G. H. Golub, A. Greenbaum, and M. Luskin, eds., Springer, N.Y., 1994, pp. 59-67.
13. A. FROMMER AND A. WEINBERG, Verified error bounds for linear systems through the Lanczos process, Reliable Computing, 5 (1999), pp. 255-267.
14. W. GAUTSCHI, A survey of Gauss-Christoffel quadrature formulae, in E.B. Christoffel. The Influence of His Work on Mathematics and the Physical Sciences, P. Bultzer and F. Fehér, eds., Birkhauser, Boston, 1981, pp. 73-157.
15. G. H. GOLUB AND G. MEURANT, Matrices, moments and quadrature, in Numerical Analysis 1993, vol 303, Pitman research notes in mathematics series, D. Griffiths and G. Watson, eds., Longman Sci. Tech. Publ., 1994, pp. 105-156.
16. _, Matrices, moments and quadrature II: How to compute the norm of the error in iterative methods, BIT, 37 (1997), pp. 687-705.
17. G. H. GOLUB AND Z. STRAKOŠ, Estimates in quadratic formulas, Numer. Algorithms, 8 (1994), pp. 241-268.
18. G. H. GOLUB AND C. VAN LOAN, Matrix Computation, The Johns Hopkins University Press, Baltimore MD, third ed., 1996.
19. A. GREENBAUM, Behavior of slightly perturbed Lanczos and Conjugate Gradient recurrences, Linear Algebra Appl., 113 (1989), pp. 7-63.
20. _, Estimating the attainable accuracy of recursively computed Residual methods, SIAM J. Matrix Anal. Appl., 18 (3) (1997), pp. 535-551.
21. _, Iterative Methods for Solving Linear Systems, SIAM, Philadelphia, 1997.
22. A. GREENBAUM AND Z. STRAKOŠ, Predicting the behavior of finite precision Lanczos and Conjugate Gradient computations, SIAM J. Matrix Anal. Appl., 18 (1992), pp. 121-137.
23. M. H. GUTKNECHT AND Z. STRAKOŠ, Accuracy of two three-term and three two-term recurrences for Krylov space solvers, SIAM J. Matrix Anal. Appl., 22 (2001), pp. 213-229.
24. M. R. HESTENES AND E. STIEFEL, Methods of conjugate gradients for solving linear systems, J. Research Nat. Bur. Standards, 49 (1952), pp. 409-435.
25. N. J. HIGHAM, Accuracy and stability of numerical algorithms, SIAM, Philadelphia, PA, 1996.
26. S. KARLIN AND L. S. SHAPLEY, Geometry of moment spaces, Memoirs of the Americam Mathematical Society 12, Providence, (1953).
27. C. LANCZOS, Solution of systems of linear equations by minimized iterations, J. Research Nat. Bur. Standards, 49 (1952), pp. 33-53.
28. G. MEURANT, The computation of bounds for the norm of the error in the Conjugate Gradient algorithm, Numer. Algorithms, 16 (1997), pp. 77-87.
29. _, Computer Solution of Large Linear Systems, vol. 28 of Studies in Mathematics and Its Applications, Elsevier, 1999.
30. _, Numerical experiments in computing bounds for the norm of the error in the preconditioned Conjugate Gradient algorithm, Numer. Algorithms 22, 3-4 (1999), pp. 353-365.
31. _, Towards a reliable implementation of the conjugate gradient method. Invited plenary lecture at the Latsis Symposium: Iterative Solvers for Large Linear Systems, Zurich, February 2002.
32. Y. NOTAY, On the convergence rate of the Conjugate Gradients in the presence of rounding errors, Numer. Math., 65 (1993), pp. 301-317.
33. C. C. PAIGE, The Computation of Eigenvalues and Eigenvectors of Very Large Sparse Matrices, PhD thesis, Intitute of Computer Science, University of London, London, U.K., 1971.
34. _, Computational variants of the Lanczos method for the eigenproblem, J. Inst. Math. Appl., 10 (1972), pp. 373-381.
35. _, Error analysis of the Lanczos algorithm for tridiagonalizing a symmetric matrix, J. Inst. Math. Appl., 18 (1976), pp. 341-349.
36. _, Accuracy and effectiveness of the Lanczos algorithm for the symmetric eigenproblem, Linear Algebra Appl., 34 (1980), pp. 235-258.
37. C. C. PAIGE AND Z. STRAKOŠ, Correspondence between exact arithmetic and finite precision behaviour of Krylov space methods, in XIV. Householder Symposium, J. Varah, ed., University of British Columbia, 1999, pp. 250-253.
38. B.N. PARLETT, The Symmetric Eigenvalue Problem, Prentice Hall, Englewood Cliffs, 1980.
39. _, Do we fully understand the symmetric Lanczos algorithm yet?, in Proceedins of the Lanczos Centennary Conference, Philadelphia, 1994, SIAM, pp. 93-107.
40. P. E. SAYLOR AND D. C. SMOLARSKI, Addendum to: Why Gaussian quadrature in the complex plane?, Numer. Algorithms, 27 (2001), pp. 215-217.
41. _, Why Gaussian quadrature in the complex plane?, Numer. Algorithms, 26 (2001), pp. 251-280.
42. G. L. SLEIJPEN, H. A. VAN DER VORST, AND D. R. FOKKEMA, BiCGstab(l) and other hybrid BiCG methods, Numer. Algorithms, 7 (1994), pp. 75-109.
43. Z. STRAKOŠ, On the real convergence rate of the Conjugate Gradient method, Linear Algebra Appl., 154-156 (1991), pp. 535-549.
44. _, Convergence and numerical behaviour of the Krylov space methods, in Algorithms for Large Sparse Linear Algebraic Systems: The State of the Art and Applications in Science and Engineering, G. W. Althaus and E. Spedicato, eds., NATO ASI Institute, Kluwer Academic, 1998, pp. 175-197.
45. Z. STRAKOŠ AND A. GREENBAUM, Open questions in the convergence analysis of the Lanczos process for the real symmetric eigenvalue problem, IMA preprint series 934, University of Minnesota, 1992.
46. G. SZEGÖ, Orthogonal Polynomials, AMS Colloq. Publ. 23, AMS, Providence, 1939.
47. K. F. WARNICK, Nonincreasing error bound for the Biconjugate Gradient method, report, University of Illinois, 2000.