Comparingparameter choice methods for regularization of ill-posed problems

Mathematics and Computers in Simulation

Volumn 81, Issue 9, 2011, Pages 1795-1841

  Bauer, Frank ^a   Lukas, Mark A ^b  

^a JOHANNES KEPLER UNIVERSITY LINZ   (Austria)

^b MURDOCH UNIVERSITY   (Australia)

Author keywords

Ill posed problem; Inverse problem; Regularization parameter; Spectral cut off regularization; Tikhonov regularization

Indexed keywords

CUT-OFF; ILL POSED PROBLEM; KNOW-HOW; LINEAR INVERSE PROBLEMS; PARAMETER CHOICE; REGULARIZATION PARAMETERS; SIMULATION RESULT; SIMULATION STUDIES; STOCHASTIC NOISE; STOCHASTIC SETTINGS; TEST CASE; TIKHONOV REGULARIZATION;

EULER EQUATIONS; PARAMETERIZATION; PROBLEM SOLVING; STOCHASTIC SYSTEMS;

INVERSE PROBLEMS;

EID: 79955564733     PISSN: 03784754     EISSN: None     Source Type: Journal    
DOI: 10.1016/j.matcom.2011.01.016     Document Type: Article

References (146)

1. J.F.P.J. Abascal, S.R. Arridge, R.H. Bayford, and D.S. Holder Comparison of methods for optimal choice of the regularization parameter for linear electrical impedance tomography of brain function Physiol. Meas. 29 2008 1319 1334
2. H. Akaike Information theory and an extension of the maximum likelihood principle Second International Symposium on Information Theory (Tsahkadsor, 1971) 1973 Akadémiai Kiadó Budapest 267 281
3. E.O. kesson, and K.J. Daun Parameter selection methods for axisymmetric flame tomography through Tikhonov regularization Appl. Opt. 47 2008 407 416 (Pubitemid 351462481)
4. O. Alifanov, and S. Rumyantsev On the stability of iterative methods for the solution of linear ill-posed problems Sov. Math., Dokl. 20 1979 1133 1136
5. R.S. Anderssen, and P. Bloomfield Numerical differentiation procedures for non-exact data Numer. Math. 22 1974 157 182
6. R. Arcangeli Pseudo-solution de l'équation Ax = y C.R. Acad. Sci. Paris, Ser. A 263 8 1966 282 285
7. A.B. Bakushinskii Remarks on choosing a regularization parameter using the quasi-optimality and ratio criterion U.S.S.R. Comp. Math. Math. Phys. 24 1984 181 182
8. A. Bakushinsky, and A. Smirnova On application of generalized discrepancy principle to iterative methods for nonlinear ill-posed problems Numer. Funct. Anal. Optim. 26 2005 35 48 (Pubitemid 40494000)
9. F. Bauer Some considerations concerning regularization and parameter choice algorithms Inverse Probl. 23 2007 837 858 (Pubitemid 46453837)
10. F. Bauer, Applying Lepskij-balancing in practice, http://arxiv.org/abs/ 1008.0657 (2010).
11. F. Bauer, Parameter choice by fast balancing, http://arxiv.org/abs/1008. 0620 (2010).
12. F. Bauer, and T. Hohage A Lepskij-type stopping rule for regularized Newton methods Inverse Probl. 21 2005 1975 1991 (Pubitemid 41744510)
13. F. Bauer, T. Hohage, and A. Munk Iteratively regularized Gauss-Newton method for nonlinear inverse problems with random noise SIAM J. Numer. Anal. 47 2009 1827 1846
14. F. Bauer, and S. Kindermann The quasi-optimality criterion for classical inverse problems Inverse Probl. 24 2008 035002
15. F. Bauer, and S. Kindermann Recent results on the quasi-optimality principle J. Inverse Ill-Posed Probl. 17 2009 5 18
16. F. Bauer, and P. Mathé Parameter choice methods using minimization schemes J. Complexity 2010 doi:10.1016/j.jco.2010.10.001
17. F. Bauer, and A. Munk Optimal regularization for ill-posed problems in metric spaces J. Inverse Ill-Posed Probl. 15 2007 137 148 (Pubitemid 46900005)
18. F. Bauer, and S. Pereverzev Regularization without preliminary knowledge of smoothness and error behavior Eur. J. Appl. Math. 16 2005 303 317 (Pubitemid 41742826)
19. F. Bauer, and M. Reiß Regularization independent of the noise level: an analysis of quasi-optimality Inverse Probl. 24 2008 055009
20. C. Brezinski, G. Rodriguez, and S. Seatzu Error estimates for linear systems with applications to regularization Numer. Algorithms 49 2008 85 104
21. C. Brezinski, G. Rodriguez, and S. Seatzu Error estimates for the regularization of least squares problems Numer. Algorithms 51 2009 61 76
22. L.D. Brown, and M.G. Low Asymptotic equivalence of nonparametric regression and white noise Ann. Stat. 24 1996 2384 2398
23. D. Calvetti, P.C. Hansen, and L. Reichel L-curve curvature bounds via Lanczos bidiagonalization Elect. Trans. Numer. Anal. 14 2002 20 35 (Pubitemid 38816943)
24. E.J. Cands Modern statistical estimation via oracle inequalities Acta Numer. 15 2006 257 325
25. L. Cavalier Nonparametric statistical inverse problems Inverse Probl. 24 2008 034004 19
26. L. Cavalier, G.K. Golubev, D. Picard, and A.B. Tsybakov Oracle inequalities for inverse problems Ann. Stat. 30 2002 843 874 (Pubitemid 37095353)
27. L. Cavalier, and Y. Golubev Risk hull method and regularization by projections of ill-posed inverse problems Ann. Stat. 34 2006 1653 1677 (Pubitemid 44875076)
28. D. Colton, and R. Kress Inverse Acoustic and Electromagnetic Scattering Theory 2nd ed. 1998 Springer Berlin
29. T. Correia, A. Gibson, M. Schweiger, and J. Hebden Selection of regularization parameter for optical topography J. Biomed. Opt. 14 2009 034044
30. D.D. Cox Approximation of method of regularization estimators Ann. Stat. 16 1988 694 712
31. P. Craven, and G. Wahba Smoothing noisy data with spline functions Numer. Math. 31 1979 377 403
32. D.J. Cummins, T.G. Filloon, and D. Nychka Confidence intervals for nonparametric curve estimates: toward more uniform pointwise coverage J. Am. Stat. Assoc. 96 2001 233 246
33. A.R. Davies On the maximum likelihood regularization of Fredholm convolution equations of the first kind C. Baker, G. Miller, Treatment of Integral Equations by Numerical Methods 1982 Academic Press London 95 105
34. A.R. Davies, and R.S. Anderssen Improved estimates of statistical regularization parameters in Fourier differentiation and smoothing Numer. Math. 48 1986 671 697
35. P. Ditmar, R. Klees, and X. Liu Frequency-dependent data weighting in global gravity field modeling from satellite data contaminated by non-stationary noise J. Geod. 81 2007 81 96 (Pubitemid 46023970)
36. P. Ditmar, J. Kusche, and R. Klees Computation of spherical harmonic coefficients from gravity gradiometry data to be acquired by the GOCE satellite: regularization issues J. Geod. 77 2003 465 477 (Pubitemid 37458424)
37. B. Efron Selection criteria for scatterplot smoothers Ann. Stat. 29 2001 470 504
38. L. Eldén A note on the computation of the generalized cross-validation function for ill-conditioned least squares problems BIT 24 1984 467 472
39. H.W. Engl Regularization methods for the stable solution of inverse problems Surv. Math. Ind. 3 1993 71 143
40. H.W. Engl, and H. Gfrerer A posteriori parameter choice for general regularization methods for solving linear ill-posed problems Appl. Numer. Math. 4 1988 395 417
41. H.W. Engl, H. Hanke, and A. Neubauer Regularization of Inverse Problems 1996 Kluwer Dordrecht
42. H.W. Engl, and O. Scherzer Convergence rate results for iterative methods for solving nonlinear ill-posed problems D. Colton, H. Engl, A. Louis, J. McLaughlin, W. Rundell, Survey on Solution Methods for Inverse Problems 2000 Springer New York 7 34
43. R.L. Eubank Spline Smoothing and Nonparametric Regression 1988 Marcel Dekker New York
44. S.N. Evans, and P.B. Stark Inverse problems as statistics Inverse Probl. 18 2002 55 R97
45. C.G. Farquharson, and D.W. Oldenburg A comparison of automatic techniques for estimating the regularization parameter in non-linear inverse problems Geophys. J. Int. 156 2004 411 425 (Pubitemid 38398606)
46. B.G. Fitzpatrick Bayesian analysis in inverse problems Inverse Probl. 7 1991 675 702
47. H. Gfrerer An a posteriori parameter choice for ordinary and iterated Tikhonov regularization of ill-posed problems leading to optimal convergence rates Math. Comput. 49 1987 507 522
48. D. Girard A fast "Monte-Carlo cross-validation" procedure for large least squares problems with noisy data Numer. Math. 56 1989 1 23
49. V. Glasko, and Y. Kriksin On the quasioptimality principle for linear ill-posed problems in Hilbert space Vychisl. Mat. Mat. Fiz. 24 1984 1603 1613
50. A. Goldenshluger, and S. Pereverzev Adaptive estimation of linear functionals in Hilbert scales from indirect white noise observations Probl. Th. Rel. Fields 118 2000 169 186 (Pubitemid 33228650)
51. G.H. Golub, M. Heath, G. Wahba, Generalized cross-validation as a method for choosing a good ridge parameter, Technometrics 21 (1979) 215-223.
52. G.H. Golub, and U. von Matt Generalized cross-validation for large scale problems J. Comput. Graph. Stat. 6 1997 1 34
53. J. Grad, and E. Zakrajšek LR algorithm with Laguerre shift for symmetric tridiagonal matrices Comput. J. 15 1972 268 270
54. C.W. Groetsch The Theory of Tikhonov Regularization for Fredholm Equations of the First Kind 1984 Pitman Boston
55. C.W. Groetsch, and A. Neubauer Regularization of ill-posed problems: optimal parameter choice in finite dimensions J. Approx. Th. 58 1989 184 200
56. C.W. Groetsch, and E. Schock Asymptotic convergence rate of Arcangeli's method for ill-posed problems Appl. Anal. 18 1984 175 182
57. C. Gu Smoothing Spline ANOVA Models 2002 Springer New York
58. C. Gu, D.M. Bates, Z. Chen, and G. Wahba The computation of generalized cross-validation functions through Householder tridiagonalization with applications to the fitting of interaction spline models SIAM J. Matrix Anal. Appl. 10 1989 457 480
59. E. Haber, and D.W. Oldenburg A GCV based method for nonlinear ill-posed problems Comput. Geosci. 4 2000 41 63
60. U. Hämarik, R. Palm, and T. Raus On minimization strategies for choice of the regularization parameter in ill-posed problems Numer. Funct. Anal. Optim. 30 2009 924 950
61. U. Hämarik, and T. Raus On the a posteriori parameter choice in regularization methods Proc. Estonian Acad. Sci. Phys. Math. 48 1999 133 145
62. U. Hämarik, and T. Raus On the choice of the regularization parameter in ill-posed problems with approximately given noise level of data J. Inverse Ill-Posed Probl. 14 2006 251 266 (Pubitemid 44017419)
63. U. Hämarik, and T. Raus About the balancing principle for choice of the regularization parameter Numer. Funct. Anal. Optim. 30 2009 951 970
64. U. Hämarik, and U. Tautenhahn On the monotone error rule for parameter choice in iterative and continuous regularization methods BIT 41 2001 1029 1038 (Pubitemid 33752965)
65. U. Hämarik, and U. Tautenhahn On the monotone error rule for choosing the regularization parameter in ill-posed problems M.M. Lavrent'ev, Ill-posed and Non-classical Problems of Mathematical Physics and Analysis 2003 VSP Utrecht 27 55
66. M. Hanke Limitations of the L-curve method in ill-posed problems BIT 36 1996 287 301 (Pubitemid 126419071)
67. M. Hanke, and P.C. Hansen Regularization methods for large-scale problems Surv. Math. Ind. 3 1993 253 315
68. M. Hanke, and T. Raus A general heuristic for choosing the regularization parameter in ill-posed problems SIAM J. Sci. Comput. 17 1996 956 972 (Pubitemid 126621187)
69. P.C. Hansen Analysis of discrete ill-posed problems by means of the L-curve SIAM Rev. 34 1992 561 580
70. P.C. Hansen Regularization tools: A Matlab package for analysis and solution of discrete ill-posed problems Numer. Algorithms 6 1994 1 35
71. P.C. Hansen Rank-Deficient and Discrete Ill-Posed Problems 1998 SIAM Philadelphia
72. P.C. Hansen The L-curve and its use in the numerical treatment of inverse problems P.R. Johnstone, Computational Inverse Problems in Electrocardiography 2001 WIT Press Southampton 119 142
73. P.C. Hansen, T.K. Jensen, and G. Rodriguez An adaptive pruning algorithm for the discrete L-curve criterion J. Comput. Appl. Math. 198 2007 483 492 (Pubitemid 44402949)
74. P.C. Hansen, M.E. Kilmer, and R.H. Kjeldsen Exploiting residual information in the parameter choice for discrete ill-posed problems BIT 46 2006 41 59 (Pubitemid 43417398)
75. P.C. Hansen, and D.P. O'Leary The use of the L-curve in the regularization of discrete ill-posed problems SIAM J. Sci. Comput. 14 1993 1487 1503
76. A. Hofinger, and H.K. Pikkarainen Convergence rate for the Bayesian approach to linear inverse problems Inverse Probl. 23 2007 2469 2484 (Pubitemid 350212310)
77. B. Hofmann Regularization of Applied Inverse and Ill-Posed Problems 1986 Teubner Leipzig
78. M. Hutchinson A stochastic estimator of the trace of the influence matrix for Laplacian smoothing splines Commun. Stat. Simul. Comput. 18 1989 1059 1076
79. M.F. Hutchinson, and F.R. de Hoog Smoothing noisy data with spline functions Numer. Math. 47 1985 99 106
80. M. Jansen, M. Malfait, and A. Bultheel Generalized cross validation for wavelet thresholding Signal Process. 56 1997 33 44 (Pubitemid 127398556)
81. Q. Jin, and U. Tautenhahn On the discrepancy principle for some Newton type methods for solving nonlinear inverse problems Numer. Math. 111 2009 509 558
82. Q.n. Jin On the iteratively regularized Gauss-Newton method for solving nonlinear ill-posed problems Math. Comput. 69 2000 1603 1623
83. Q.n. Jin, and Z.y. Hou On an a posteriori parameter choice strategy for Tikhonov regularization of nonlinear ill-posed problems Numer. Math. 83 1999 139 159
84. P.R. Johnstone, and R.M. Gulrajani Selecting the corner in the L-curve approach to Tikhonov regularization IEEE Trans. Biomed. Eng. 47 2000 1293 1296
85. J. Kaipio, and E. Somersalo Statistical and Computational Inverse Problems 2005 Springer New York
86. B. Kaltenbacher, A. Neubauer, and O. Scherzer Iterative Regularization Methods for Nonlinear Ill-Posed Problems 2008 Walter de Gruyter Berlin
87. M.E. Kilmer, and D.P. O'Leary Choosing regularization parameters in iterative methods for ill-posed problems SIAM J. Matrix Anal. Appl. 22 2001 1204 1221
88. S. Kindermann, and A. Neubauer On the convergence of the quasioptimality criterion for (iterated) Tikhonov regularization Inverse Probl. Imaging 2 2008 291 299
89. R. Kohn, C.F. Ansley, and D. Tharm The performance of cross-validation and maximum likelihood estimators of spline smoothing parameters J. Am. Stat. Assoc. 86 1991 1042 1050
90. p, generalized maximum likelihood, and extended exponential criteria: a geometric approach J. Am. Stat. Assoc. 97 2002 766 782
91. F.M. Larkin Gaussian measure in Hilbert space and applications in numerical analysis Rocky Mt. J. Math. 2 1972 379 421
92. C.L. Lawson, and R.J. Hanson Solving Least Squares Problems 1974 Prentice-Hall Englewood Cliffs, NJ
93. A. Leonov Justification of the choice of regularization parameter according to quasi-optimality and quotient criteria U.S.S.R. Comput. Math. Math. Phys. 18 1979 1 15
94. O. Lepskij On a problem of adaptive estimation in Gaussian white noise Theor. Probab. Appl. 35 1990 454 466
95. L and generalized cross-validation in ridge regression with application to spline smoothing Ann. Stat. 14 1986 1101 1112
96. L, cross-validation and generalized cross-validation: Discrete index set Ann. Stat. 15 1987 958 975
97. M.A. Lukas Convergence rates for regularized solutions Math. Comput. 51 1988 107 131
98. M.A. Lukas Asymptotic optimality of generalized cross-validation for choosing the regularization parameter Numer. Math. 66 1993 41 66
99. M.A. Lukas On the discrepancy principle and generalised maximum likelihood for regularisation Bull. Aust. Math. Soc. 52 1995 399 424
100. M.A. Lukas Asymptotic behaviour of the minimum bound method for choosing the regularization parameter Inverse Probl. 14 1998 149 159 (Pubitemid 128555638)
101. M.A. Lukas Comparisons of parameter choice methods for regularization with discrete noisy data Inverse Probl. 14 1998 161 184 (Pubitemid 128555639)
102. M.A. Lukas Robust generalized cross-validation for choosing the regularization parameter Inverse Probl. 22 2006 1883 1902 (Pubitemid 44444683)
103. M.A. Lukas Strong robust generalized cross-validation for choosing the regularization parameter Inverse Probl. 24 2008 034006 16
104. M.A. Lukas Robust GCV choice of the regularization parameter for correlated data J. Integral Equations Appl. 22 2010 519 547
105. P. Mathé, and S.V. Pereverzev Geometry of linear ill-posed problems in variable Hilbert spaces Inverse Probl. 19 2003 789 803
106. P. Mathé, and S.V. Pereverzev Regularization of some linear ill-posed problems with discretized random noisy data Math. Comput. 75 2006 1913 1929 (Pubitemid 44719738)
107. V.A. Morozov On the solution of functional equations by the method of regularization Soviet Math. Dokl. 7 1966 414 417
108. V.A. Morozov Methods for Solving Incorrectly Posed Problems 1984 Springer-Verlag New York
109. M.T. Nair, and M.P. Rajan Generalized Arcangeli's discrepancy principles for a class of regularization methods for solving ill-posed problems J. Inverse Ill-posed Probl. 10 2002 281 294 (Pubitemid 35178947)
110. A. Neubauer An a posteriori parameter choice for Tikhonov regularization in the presence of modeling error Appl. Numer. Math. 4 1988 507 519
111. A. Neubauer The convergence of a new heuristic parameter selection criterion for general regularization methods Inverse Probl. 24 2008 055005 10
112. J. Opsomer, Y. Wang, and Y. Yang Nonparametric regression with correlated errors Stat. Sci. 16 2001 134 153 (Pubitemid 33636345)
113. R. Palm, Numerical comparison of regularization algorithms for solving ill-posed problems, PhD Thesis, University of Tartu, Estonia, 2010.
114. D. Phillips A technique for the numerical solution of certain integral equations of the first kind J. Assoc. Comput. Mach. 9 1962 84 97
115. S. Pohl, B. Hofmann, R. Neubert, T. Otto, and C. Radehaus A regularization approach for the determination of remission curves Inverse Probl. Eng. 9 2001 157 174 (Pubitemid 33678963)
116. T. Raus On the discrepancy principle for the solution of ill-posed problems Uch. Zap. Tartu. Gos. Univ. 672 1984 16 26
117. T. Raus The principle of the residual in the solution of ill-posed problems with nonselfadjoint operator Tartu Riikl. Ül. Toimetised 1985 12 20
118. T. Raus An a posteriori choice of the regularization parameter in case of approximately given error bound of data A. Pedas, Collocation and Projection Methods for Integral Equations and Boundary Value Problems 1990 Tartu University Tartu 73 87
119. T. Raus About regularization parameter choice in case of approximately given error bounds of data G. Vainikko, Methods for Solution of Integral Equations and Ill-posed Problems 1992 Tartu University Tartu
120. T. Raus, and U. Hämarik On the quasioptimal regularization parameter choices for solving ill-posed problems J. Inverse Ill-posed Probl. 15 2007 419 439 (Pubitemid 47213847)
121. T. Raus, and U. Hämarik New rule for choice of the regularization parameter in (iterated) Tikhonov method Math. Model. Anal. 14 2009 187 198
122. T. Reginska A regularization parameter in discrete ill-posed problems SIAM J. Sci. Comput. 17 1996 740 749
123. T. Robinson, and R. Moyeed Making robust the cross-validatory choice of smoothing parameter in spline smoothing regression Commun. Stat. Theory Methods 18 1989 523 539
124. B.W. Rust Parameter selection for constrained solutions to ill-posed problems Comput. Sci. Stat. 32 2000 333 347
125. B.W. Rust, and D.P. O'Leary Residual periodograms for choosing regularization parameters for ill-posed problems Inverse Probl. 24 2008 034005 30
126. R. Santos, and A. De Pierro A cheaper way to compute generalized cross-validation as a stopping rule for linear stationary iterative methods J. Comput. Graph. Stat. 12 2003 417 433 (Pubitemid 36802939)
127. O. Scherzer, H. Engl, and K. Kunisch Optimal a posteriori parameter choice for Tikhonov regularization for solving nonlinear ill-posed problems SIAM J. Numer. Anal. 30 1993 1796 1838
128. V. Spokoiny, and C. Vial Parameter tuning in pointwise adaptation using a propagation approach Ann. Stat. 37 2009 2783 2807
129. A. Tarantola Inverse Problem Theory: Methods for Data Fitting and Model Parameter Estimation 1987 Elsevier Amsterdam
130. U. Tautenhahn, and U. Hämarik The use of monotonicity for choosing the regularization parameter in ill-posed problems Inverse Probl. 15 1999 1487 1505 (Pubitemid 129655864)
131. A.M. Thompson, J.C. Brown, J.W. Kay, and D.M. Titterington A study of methods for choosing the smoothing parameter in image restoration by regularization IEEE Trans. Pattern Anal. Mach. Intell. 13 1991 3326 3339
132. A.M. Thompson, J.W. Kay, and D.M. Titterington A cautionary note about crossvalidatory choice J. Stat. Comput. Simul. 33 1989 199 216
133. A. Tikhonov, and V. Arsenin Solutions of Ill-posed Problems 1977 Wiley New York
134. A. Tikhonov, and V. Glasko Use of the regularization method in non-linear problems U.S.S.R. Comput. Math. Math. Phys. 5 1965 93 107
135. A. Tsybakov On the best rate of adaptive estimation in some inverse problems C.R. Acad. Sci., Paris, Ser. I, Math. 330 2000 835 840
136. R. Vio, P. Ma, W. Zhong, J. Nagy, L. Tenorio, and W. Wamsteker Estimation of regularization parameters in multiple-image deblurring Astron. Astrophys. 423 2004 1179 1186
137. C.R. Vogel Optimal choice of a truncation level for the truncated SVD solution of linear first kind integral equations when data are noisy SIAM J. Numer. Anal. 23 1986 109 117 (Pubitemid 16474842)
138. C.R. Vogel Non-convergence of the L-curve regularization parameter selection method Inverse Probl. 12 1996 535 547 (Pubitemid 126655737)
139. C.R. Vogel Computational Methods for Inverse Problems 2002 SIAM Philadelphia
140. G. Wahba Practical approximate solutions to linear operator equations when the data are noisy SIAM J. Numer. Anal. 14 1977 651 667
141. G. Wahba A comparison of GCV and GML for choosing the smoothing parameter in the generalized spline smoothing problem Ann. Stat. 13 1985 1378 1402
142. G. Wahba Spline Models for Observational Data 1990 SIAM Philadelphia
143. G. Wahba, and Y.H. Wang When is the optimal regularization parameter insensitive to the choice of the loss function? Commun. Stat. Theory Methods 19 1990 1685 1700
144. Y. Wang Smoothing spline models with correlated random errors J. Am. Stat. Assoc. 93 1998 341 348
145. W.E. Wecker, and C.F. Ansley The signal extraction approach to nonlinear regression and spline smoothing J. Am. Stat. Assoc. 78 1983 81 89
146. A.G. Yagola, A.S. Leonov, and V.N. Titarenko Data errors and an error estimation for ill-posed problems Inverse Probl. Eng. 10 2002 117 129