Solving ill-conditioned and singular linear systems: A tutorial on regularization

SIAM Review

Volumn 40, Issue 3, 1998, Pages 636-666

  Neumaier, Arnold ^a  

^a UNIVERSITY OF VIENNA   (Austria)

Author keywords

Error bounds; Generalized cross validation; Generalized maximum likelihood; Ill conditioned; Ill posed; Regularization; Tikhonov regularization

Indexed keywords

ERROR ANALYSIS; FUNCTIONS; LINEAR ALGEBRA; MAXIMUM LIKELIHOOD ESTIMATION; NUMERICAL ANALYSIS;

GENERALIZED CROSS VALIDATION (GCV); GENERALIZED MAXIMUM LIKELIHOOD (GML); TIKHONOV REGULARIZATION;

APPROXIMATION THEORY;

EID: 0032163862     PISSN: 00361445     EISSN: None     Source Type: Journal    
DOI: 10.1137/S0036144597321909     Document Type: Article

References (38)

1. A. B. BAKUSHINSKII, Remarks on choosing a regularization parameter using the quasioptimality and ratio criterion, USSR Comput. Math. Math. Phys., 24 (1984), pp. 181-182.
2. O. E. BARNDORFF-NIELSEN AND D. R. Cox, Inference and Asymptotics, Chapman and Hall, London, 1994.
3. M. BERTERO, C. DE MOL, AND G. A. VIANO, The stability of inverse problems, in Inverse Scattering in Optics, H. P. Baltes, ed., Springer-Verlag, New York, 1980, pp. 161-214.
4. A. BJÖRCK, Numerical Methods for Least Squares Problems, SIAM, Philadelphia, PA, 1996.
5. H. BRAKHAGE, On ill-posed problems and the method of conjugate gradients, in Inverse and Ill-Posed Problems, H. W. Engl and C. W. Groetsch, eds., Academic Press, Boston, 1987, pp. 165-175.
6. P. CRAVEN AND G. WAHBA, Smoothing noisy data with spline functions: Estimating the correct degree of smoothing by the method of generalized cross-validation, Numer. Math., 31 (1979), pp. 377-403.
7. L. ELDÉN, Algorithms for the regularization of ill-conditioned least squares problems, BIT, 17 (1977), pp. 134-145.
8. H. W. ENGL, Regularization methods for the stable solution of inverse problems, Surveys Math. Indust., 3 (1993), pp. 71-143.
9. H. W. ENGL, M. HANKE, AND A. NEUBAUER, Regularization of Inverse. Problems, Kluwer, Dordrecht, 1996.
10. P. E. GILL, W. MURRAY, AND M. H. WRIGHT, Practical Optimization, Academic Press, London, 1981.
11. G. H. GOLUB AND C. F. VAN LOAN, Matrix Computations, John Hopkins University Press, Baltimore, MD, 1989.
12. C. W. GROETSCH, Generalized Inverses of Linear Operators, Marcel Dekker, New York, 1977.
13. CHONG GU AND G. WAHBA, Minimizing GCV/GML scores with multiple smoothing parameters via the Newton method, SIAM J. Sci. Stat. Comput., 12 (1991), pp. 383-398.
14. M. HANKE, Conjugate Gradient Type Methods for Ill-Posed Problems, Pitman Res. Notes Math. Ser., Longman, Harlow, UK, 1995.
15. M. HANKE AND M. P. C. HANSEN, Regularization methods for large-scale problems, Surveys Math. Indust., 3 (1993), pp. 253-315.
16. M. HANKE AND T. RAUS, A general heuristic for choosing the regularization parameter in ill-posed problems, SIAM J. Sci. Comput., 17 (1996), pp. 956-972.
17. P. C. HANSEN, Analysis of discrete ill-posed problems by means of the L-curve, SIAM Rev., 34 (1992), pp. 561-580.
18. P. C. HANSEN, Rank-Deficient and Discrete Ill-Posed Problems. Numerical Aspects of Linear Inversion, SIAM, Philadelphia, PA, 1997.
19. A. K. KATSAGGELOS, Digital Image Restoration, Springer-Verlag, Berlin, 1991.
20. L. KAUFMAN AND A. NEUMAIER, PET regularization by envelope guided conjugate gradients, IEEE Trans. Medical Imag., 15 (1996), pp. 385-389.
21. L. KAUFMAN AND A. NEUMAIER, Regularization of ill-posed problems by envelope guided conjugate gradients, J. Comput. Graph. Stat., 6 (1997), pp. 451-463.
22. C. F. VAN LOAN, Generalizing the singular value decomposition, SIAM J. Numer. Anal., 13 (1976), pp. 76-83.
23. C. F. VAN LOAN, Computing the CS and generalized singular value decomposition, Numer. Math., 46 (1985), pp. 479-492.
24. K. MILLER, Least squares methods for ill-posed problems with a prescribed bound, SIAM, J. Math. Anal., 1 (1970), pp. 52-74.
25. F. NATTERER, Error bounds for Tikhonov regularization in Hilbert scales, Appl. Anal., 18 (1984), pp. 29-37.
26. A. S. NEMIROVSKI, The regularization properties of the adjoint method in ill-posed problems, USSR Comput. Math. Math. Phys., 26 (1986), pp. 7-16.
27. C. C. PAIGE, Computing the generalized singular value decomposition, SIAM J. Sci. Stat. Comput., 7 (1986), pp. 1126-1146.
28. D. L. PHILLIPS, A technique for the numerical solution of certain integral equations of the first kind, J. Assoc. Comput. Mach., 9 (1962), pp. 84-97.
29. J. D. RILEY, Solving systems of linear equations with a positive definite symmetric but possibly ill-conditioned matrix, Math. Tables Aids Comput., 9 (1956), pp. 96-101.
30. G. W. STEWART, A method for computing the generalized singular value decomposition, in Matrix Pencils, B. Kagström and A. Ruhe, eds., Springer-Verlag, New York, 1983, pp. 207-220.
31. A. N. TIKHONOV, Solution of incorrectly formulated problems and the regularization method, Soviet Math. Dokl., 4 (1963), pp. 1035-1038.
32. C. R. VOGEL AND M. E. OMAN, Iterative methods for total variation denoising, SIAM J. Sci. Comput., 17 (1996), pp. 227-238.
33. G. WAHBA, A comparison of GCV and GML for choosing the smoothing parameter in the generalized spline smoothing problem, Ann. Statist., 13 (1985), pp. 1378-1402.
34. G. WAHBA, Spline Models for Observational Data, SIAM, Philadelphia, PA, 1990.
35. A. H. WELSH. On M-processes and M-estimation. Ann. Statist., 17 (1990), pp. 337-361. (Correction, Ann. Statist., 18 (1990), p. 1500.)
36. N. WIENER, Cybernetics, MIT Press, Cambridge, MA, 1948.
37. G. M. WING AND J. D. ZAHRT, A Primer on Integral Equations of the First Kind, SIAM, Philadelphia, PA, 1991.
38. E. T. WHITTAKER, On a new method of graduation, Proc. Edinburgh Math. Soc., 41 (1923), pp. 63-75.