An information approach to regularization parameter selection under model misspecification

Inverse Problems

Volumn 18, Issue 5, 2002, Pages 1207-1228

  Urmanov, A M ^a   Gribok, A V ^a   Hines, J W ^a   Uhrig, R E ^a  

^a University of Tennessee   (United States)

Author keywords

[No Author keywords available]

Indexed keywords

COMPUTATIONAL COMPLEXITY; GAUSSIAN NOISE (ELECTRONIC); MAXIMUM LIKELIHOOD ESTIMATION; PARAMETER ESTIMATION; STATISTICAL METHODS; VECTORS;

MEAN EXPECTED LOG LIKELIHOOD (MELL);

INVERSE PROBLEMS;

EID: 0036799010     PISSN: 02665611     EISSN: None     Source Type: Journal    
DOI: 10.1088/0266-5611/18/5/301     Document Type: Review

References (37)

1. Akaike H. 1973 Information theory and an extension of the maximum likelihood principle 2nd Int. Symp. on Information Theory ed B.N. Petrov and F. Csaki (Budapest: Akademiai Kiado) pp 267-81.
2. Bozdogan H. 1987 Model selection and Akaike's information criterion (AIC): the general theory and its analytical extensions Psychometrika 52 345-70.
3. Bozdogan H. 1996 Informational complexity criteria for regression models Information Theory and Statistics Section on Bayesian Stat. Science (ASA Ann. Meeting) (Chicago, IL, Aug. 1996).
4. Bozdogan H. 2000 Akaike's information criterion and recent developments in information complexity J. Math. Psychol. 44 62-91.
5. Cox D.D. and O'Sullivan F. 1990 Asymptotic analysis of penalized likelihood and related estimators Ann. Stat. 18 1676-95.
6. Fisher R.A. 1921 On the mathematical foundations of theoretical statistics Phil. Trans. A 222 309.
7. Good I.J. and Gaskins R.A. 1971 Nonparametric roughness penalties for probability densities Biometrika 58 255-77.
8. Green P.J. 1987 Penalized likelihood for general semi-parametric regression models Int. Stat. Rev. 55 255-9.
9. Hansen P.C. 1994 Regularization tools: a Matlab package for analysis and solution of discrete ill-posed problems Numer. Algorithms 6 1-35.
10. Hansen P.C. 1998 Rank-deficient and discrete ill-posed problems SIAM Monographs on Mathematical Modeling and Computation (Philadelphia, PA: SLAM).
11. Hoerl A.E. and Kennard R.W. 1970 Ridge regression: biased estimation for nonorthogonal problems Technometrics 12 55-82.
12. Huber P.J. 1981 Robust Statistics (New York: Wiley).
13. Konishi S. and Kitagawa G. 1996 Generalized information criteria in model selection Biometrika 83 875-90.
14. Kullback S. and Leibler R.A. 1951 On information and sufficiency Ann. Math. Stat. 22 79-86.
15. Leonard T. 1978 Density estimation, stochastic processes and prior information J. R. Stat. Soc. B 40 113-46.
16. Leonov A.S. and Yagola A.G. 1997 The L-curve method always introduces a nonremovable systematic error Moscow Univ. Phys. Bull. 52 20-3.
17. Lukas M.A. 1998 Comparison of parameter choice methods for regularization with discrete noisy data Inverse Problems 14 161-84.
18. Mallows C.L. 1973 Some comments on CP Technometrics 15 661-75.
19. Morozov V.A. 1984 Methods for Solving Incorrectly Posed Problems (New York: Springer).
20. Murata N., Yoshizawa S. and Amari S. 1994 Network information criterion-determining the number of hidden units for an artificial neural network model IEEE Trans. Neural Networks 5 865-72.
21. Phillips D.L. 1962 A technique for the numerical solution of certain integral equations of the first kind J. Assoc. Comput. Mach. 9 84-97.
22. Sakamoto Y. 1986 Akaike Information Criterion Statistics (Tokyo: KTK).
23. Shaw C.B. Jr. 1972 Improvements of the resolution of an instrument by numerical solution of an integral equation J. Math. Anal. Appl. 37 83-112.
24. Shibata R. 1989 Statistical aspects of model selection From Data to Model ed J.C. Willems (New York: Springer) pp 215-40.
25. Silverman B.W. 1985 Penalized maximum likelihood estimation Encyclopedia of Statistical Sciences vol 6 pp 664-7.
26. Takeuchi K. 1976 Distribution of information statistics and criteria for adequacy of models Math. Sci. 153 12-8 (in Japanese).
27. Tibshirani R. 1996 Regression shrinkage and selection via the lasso J. R. Stat. Soc. B 58 267-88.
28. Tikhonov A.N. 1963 Solution of incorrectly formulated problems and regularization method Sov. Math. Dokl. 4 1035-8.
29. Thompson A.M., Kay J.W. and Titterington D.M. 1989 A cautionary note about crossvalidatory choice J. Stat. Comput. Simul. 33 199-216.
30. Urmanov A.M., Gribok A.V., Bozdogan H., Hines J.W. and Uhrig R.E. 2002 Information complexity-based regularization parameter selection for solution of ill-conditioned inverse problems Inverse Problems 18 L1-9.
31. van Emden M.H. 1971 An analysis of complexity Mathematical Centre Tracts vol 35 (Amsterdam: Mathematisch Centrum).
32. Vogel C.R. 1996 Non-convergence of the L-curve regularization parameter selection method Inverse Problems 12 535-47.
33. Wahba G. 1990 Spline Models for Observational Data (Philadelphia, PA: Society for Industrial and Applied Mathematics).
34. Wahba G. and Wang Y. 1993 Behavior near zero of the distribution of GCV smoothing parameter estimates Technical Report No 910, Department of Statistics, University of Wisconsin, Madison.
35. White H. 1980 Using least squares to approximate unknown regression functions Int. Econ. Rev. 21 149-70.
36. White H. 1981 Consequences and detection of misspecified nonlinear regression models JASA 76 419-33.
37. White H. 1994 Estimation, Inference and Specification Analysis (Cambridge: Cambridge University Press).