Choosing regularization parameters in iterative methods for ill-posed problems

SIAM Journal on Matrix Analysis and Applications

Volumn 22, Issue 4, 2001, Pages 1204-1221

  Kilmer, Misha E ^a   O'Leary, Dianne P ^b  

^a TUFTS UNIVERSITY   (United States)

^b University of Maryland ^*  (United States)

Author keywords

Discrepancy principle; Ill posed problems; Iterative methods; Krylov subspace; L curve; Projection; Regularization; Tikhonov; Truncated singular value decomposition

Indexed keywords

EID: 0035537408     PISSN: 08954798     EISSN: None     Source Type: Journal    
DOI: 10.1137/S0895479899345960     Document Type: Article

References (38)

1. A. BJÖRCK, A bidiagonalization algorithm for solving large and sparse ill-posed systems of linear equations, BIT, 28 (1988), pp. 659-670.
2. A. BJÖRCK, E. GRIMME, AND P. V. DOOREN, An implicit shift bidiagonalization algorithm for ill-posed systems, BIT, 34 (1994), pp. 510-534.
3. P. BLOMGREN AND T. F. CHAN, Modular Solvers for Constrained Image Restoration Problems, Tech. Report, Mathematics Department, UCLA, Los Angeles, 1999.
4. D. CALVETTI, G. GOLUB, AND L. REICHEL, Estimation of the L-curve via Lanczos bidiagonalization, BIT, 39 (1999), pp. 603-619.
5. T. CHAN AND M. NG, Galerkin projection method for solving multiple linear systems, SIAM J. Sci. Comput., 21 (1999), pp. 836-850.
6. J. CULLUM AND R. A. WILLOUGHBY, Lanczos and the computation in specified intervals of the spectrum of large, sparse real symmetric matrices, in Sparse Matrix Proceedings 1978, I. S. Duff and G. W. Stewart, eds., SIAM, Philadelphia, 1979, pp. 220-255.
7. L. DESBAT AND D. GIRARD, The "minimum reconstruction error" choice of regularization parameters: Some more efficient methods and their application to deconvolution problems, SIAM J. Sci. Comput., 16 (1995), pp. 1387-1403.
8. H. W. ENGL AND W. GREVER, Using the L-curve for determining optimal regularization parameters, Numer. Math., 69 (1994), pp. 25-31.
9. H. E. FLEMING, Equivalence of regularization and truncated iteration in the solution of ill-posed image reconstruction problems, Linear Algebra Appl., 130 (1990), pp. 133-150.
10. A. FROMMER AND P. MAASS, Fast CG-based methods for Tikhonov-Phillips regularization, SIAM J. Sci. Comput., 20 (1999), pp. 1831-1850.
11. G. GOLUB, M. HEATH, AND G. WAHBA, Generalized cross-validation as a method for choosing a good ridge parameter, Technometrics, 21 (1979), pp. 215-223.
12. G. GOLUB AND W. KAHAN, Calculating the singular values and pseudo-inverse of a matrix, J. Soc. Indust. Appl. Math. Ser. B Numer. Anal., 2 (1965), pp. 205-224.
13. W. GROETSCH, Theory of Tikhonov Regularization for Fredholm Equations of the First Kind, Pitman, Boston, 1984.
14. M. GU AND S. EISENSTAT, A Stable and Fast Algorithm for Updating the Singular Value Decomposition, Tech. Report RR-939, Department of Computer Science, Yale University, New Haven, 1993.
15. M. HANKE AND P. C. HANSEN, Regularization methods for large-scale problems, Surveys Math. Indust., 3 (1993), pp. 253-315.
16. M. HANKE, J. NAGY, AND R. PLEMMONS, Preconditioned iterative regularization for ill-posed problems, in Numerical Linear Algebra and Scientific Computing, L. Reichel, A. Ruttan, and R. S. Varga, eds. 1993, pp. 141-163.
17. P. C. HANSEN, Analysis of discrete ill-posed problems by means of the L-curve, SIAM Rev., 34 (1992), pp. 561-580.
18. P. C. HANSEN, Regularization tools: A Matlab package for analysis and solution of discrete ill-posed problems, Numer. Algorithms, 6 (1994), pp. 1-35.
19. P. C. HANSEN, Rank-Deficient and Discrete Ill-Posed Problems. Numerical Aspects of Linear Inversion, SIAM Monogr. Math. Model Comput., SIAM, Philadelphia, 1998.
20. P. C. HANSEN AND D. P. O'LEARY, The use of the L-curve in the regularization of discrete ill-posed problems, SIAM J. Sci. Comput., 14 (1993), pp. 1487-1503.
21. B. HOFMANN, Regularization for Applied Inverse and Ill-Posed Problems, Teubner-Texte Mathe. 85, Teubner, Leipzig, 1986.
22. L. KAUFMAN AND A. NEUMAIER, Regularization of ill-posed problems by envelope guided conjugate gradients, J. Comput. Graph. Statist., 6 (1997), pp. 451-463.
23. M. KILMER, Regularization of ill-posed problems using (symmetric) Cauchy-like preconditioners, in Proceedings of the SPIE Annual Meeting, Advanced Signal Processing Algorithms, Architectures, and Implementations VIII, 1998, SPIE, San Diego, CA, pp. 381-392.
24. M. KILMER AND D. P. O'LEARY, Pivoted Cauchy-like preconditioners for regularized solution of ill-posed problems, SIAM J. Sci. Stat. Comput., 21 (1999), pp. 88-110.
25. C. L. LAWSON AND R. J. HANSON, Solving Least Squares Problems, Prentice-Hall, Englewood Cliffs, NJ, 1974.
26. V. A. MOROZOV, On the solution of functional equations by the method of regularization, Soviet Math. Dokl., 7 (1966), pp. 414-417.
27. J. NAGY, R. PLEMMONS, AND T. TORGERSEN, Iterative image restoration using approximate inverse preconditioning, IEEE Trans. Image Process., 5 (96), pp. 1151-1163.
28. D. P. O'LEARY AND J. A. SIMMONS, A bidiagonalization-regularization procedure for large scale discretization of ill-posed problems, SIAM J. Sci. Statist. Comput., 2 (1981), pp. 474-489.
29. C. C. PAIGE AND M. A. SAUNDERS, Algorithm 583, LSQR: Sparse linear equations and least squares problems, ACM Trans. Math. Software, 8 (1982), pp. 195-209.
30. C. C. PAIGE AND M. A. SAUNDERS, LSQR: An algorithm for sparse linear equations and sparse least squares, ACM Trans. Math. Software, 8 (1982), pp. 43-71.
31. B. N. PARLETT, The Symmetric Eigenvalue Problem, Prentice-Hall, Englewood Cliffs, NJ, 1980.
32. B. N. PARLETT AND D. S. SCOTT, The Lanczos algorithm with selective orthogonalization, Math. Comp., 33 (1979), pp. 217-238.
33. B. W. RUST, Truncating the Singular Value Decomposition for Ill-Posed Problems, Tech. Report NISTIR 6131, Mathematical and Computational Sciences Division, National Institute of Standards and Technology, Gaithersburg, MD, 1998.
34. Y. SAAD, Iterative Methods for Sparse Linear Systems, PWS Publishing Company, Boston, 1996.
35. Y. SAAD AND M. H. SCHULTZ, GMRES: A generalized minimal residual algorithm for solving nonsymmetric linear systems, SIAM J. Sci. Statist. Comput., 7 (1986), pp. 856-869.
36. H. D. SIMON, Analysis of the symmetric Lanczos algorithm with reorthogonalization methods, Linear Algebra Appl., 61 (1984), pp. 101-131.
37. J. M. VARAH, Pitfalls in the numerical solution of linear ill-posed problems, SIAM J. Sci. Statist. Comput., 4 (1983), pp. 164-176.
38. 1993, pp. 141-163. C. R. VOGEL, Non-convergence of the L-curve regularization parameter selection method, Inverse Problems, 12 (1996), pp. 535-547.