Minimum variance in biased estimation: Bounds and asymptotically optimal estimators

IEEE Transactions on Signal Processing

Volumn 52, Issue 7, 2004, Pages 1915-1929

  Eldar, Yonina ^a  

^a TECHNION ISRAEL INSTITUTE OF TECHNOLOGY   (Israel)

Author keywords

[No Author keywords available]

Indexed keywords

ASYMPTOTIC STABILITY; MATHEMATICAL MODELS; MAXIMUM LIKELIHOOD ESTIMATION; OPTIMIZATION; PARAMETER ESTIMATION; PROBABILITY DISTRIBUTIONS;

ASYMPTOTIC OPTIMALITY; BIAS GRADIENT NORM; BIASED ESTIMATION; CRAMER-RAO LOWER BOUND; PENALIZED MAXIMUM LIKELIHOOD; TIKHONOV REGULARIZATION;

SIGNAL FILTERING AND PREDICTION;

EID: 3142775716     PISSN: 1053587X     EISSN: None     Source Type: Journal    
DOI: 10.1109/TSP.2004.828929     Document Type: Article

References (35)

1. A. N. Tikhonov, "Solution of incorrectly formulated problems and the regularization method," Soviet. Math. Dokl., vol. 5, pp. 1035-1038, 1963.
2. A. N. Tikhonov and V. Y. Arsenin, Solution of Ill-Posed Problems. Washington, DC: V.H. Winston, 1977.
3. G. Demoment, "Image reconstruction and restoration: Overview of common estimation structures and problems," IEEE Trans. Acoust., Speech, Signal Processing, vol. 37, pp. 2024-2036, Dec. 1989.
4. D. M. Titterington, "Common structure of smoothing techniques in statistics," Int. Statist. Rev., vol. 53, pp. 141-170, 1985.
5. F. O'Sullivan, "A statistical perspective on ill-posed inverse problems," Statist. Sci., vol. 1, no. 4, pp. 502-527, 1986.
6. S. M. Kay, Modern Spectral Estimation: Theory and Application. Englewood Cliffs, NJ: Prentice-Hall, 1988.
7. K. S. Riedel and A. Sidorenko, "Minimum biased multitaper spectral estimation," IEEE Trans. Signal Proc., vol. 43, pp. 188-195, Jan. 1995.
8. H. Cramer, Mathematical Methods of Statistics. Princeton, NJ: Princeton Univ. Press, 1946.
9. C. R. Rao, "Minimum variance and the estimation of several parameters," in Proc. Cambridge Phil. Soc., 1946, pp. 280-283.
10. C. R. Rao, Linear Statistical Inference and its Applications, Second ed. New York: Wiley, 1973.
11. H. V. Poor, An Introduction to Signal Detection and Estimation, Second ed. New York: Springer-Verlag, 1994.
12. E. L. Lehmann and G. Casella, Theory of Point Estimation, Second ed. New York: Springer-Verlag, 1998.
13. H. L. Van Trees, Detection, Estimation, and Modulation Theory. New York: Wiley, 1968.
14. A. O. Hero, "A Cramer-Rao Type Lower Bound for Essentially Unbiased Parameter Estimation," Lincoln Lab., Mass. Inst. Technol., Lexington, MA, Tech. Rep. 890, DTIC AD-A246 666, 1992.
15. A. O. Hero, J. A. Fessler, and M. Usman, "Exploring estimator bias-variance tradeoffs using the uniform CR bound," IEEE Trans. Signal Processing, vol. 44, pp. 2026-2041, Aug. 1996.
16. A. E. Hoerl and R. W. Kennard, "Ridge regression: Biased estimation for nonorthogonal problems," Technometrics, vol. 12, pp. 55-67, Feb. 1970.
17. L. S. Mayer and T. A. Willke, "On biased estimation in linear models," Technometrics, vol. 15, pp. 497-508, Aug. 1973.
18. I. J. Good and R. A. Gaskins, "Nonparametric roughness penalties for probability densities," Biometrika, vol. 58, pp. 255-277, 1971.
19. I. J. Good and R. A. Gaskins, "Density estimation and bump hunting by the penalized likelihood method exemplified by scattering and meteorite data (with discussion and rejoinder)," J. Amer. Statist., vol. 10, pp. 811-824, 1980.
20. L. Kaufman, "Maximum likelihood, least squares, and penalized least squares for PET," IEEE Trans. Med. Imag., vol. 12, pp. 200-214, June 1993.
21. J. A. Fessler and A. O. Hero, "Penalizes maximum-likelihood image reconstruction using space alternating EM algorithms," IEEE Trans. Image Processing, vol. 4, pp. 1417-1425, Oct. 1995.
22. J. A. Fessler, "Mean and variance of implicity defined biased estimators (such as penalized maximum likelihood): Applications to tomography," IEEE Trans. Image Processing, vol. 5, pp. 493-506, Mar. 1996.
23. T. J. Schulz, "Penalized maximum-likelihood estimation of covariance matrices with linear structure," IEEE Trans. Neural Networks, vol. 45, pp. 3027-3038, Dec. 1997.
24. D. Ormoneit and V. Tresp, "Averaging, maximum penalized likelihood and Bayesian estimation for improving Gaussian mixture probability density estimates," IEEE Trans. Neural Networks, vol. 9, pp. 639-650, July 1998.
25. S. M. Kay, Fundamentals of Statistical Signal Processing: Estimation Theory. Upper Saddle River, NJ: Prentice-Hall, 1993.
26. R. A. Horn and C. R. Johnson, Matrix Analysis. Cambridge, U.K.: Cambridge Univ. Press, 1985.
27. J. A. Fessler and A. O. Hero, "Cramer-Rao bounds for biased estimators in image restoration," in Proc. 36th IEEE Midwest Symp. Circuits Syst., Detroit, MI, Aug. 1993.
28. D. P. Bertsekas, Nonlinear Programming, Second ed. Belmont, MA: Athena Scientific, 1999.
29. A. Ben-Tal and A. Nemirovski, Lectures on Modern Convex Optimization, ser. Optimization. Philadelphia, PA: MPS-SIAM, 2001.
30. L. Vandenberghe and S. Boyd, "Semidefinite programming," SIAM Rev., vol. 38, no. 1, pp. 40-95, Mar. 1996.
31. Y. Nesterov and A. Nemirovski, Interior-Point Polynomial Algorithms in Convex Programming. Philadelphia, PEA: SIAM, 1994.
32. F. Alizadeh, Combinatorial Optimization With Interior Point Methods and Semi-Definite Matrices," Ph.D. dissertation, Univ. Minnesota, Minneapolis, MN, 1991.
33. V. A. Morozov, Methods for Solving Incorrectly Posed Problems. New York: Springer-Verlag, 1984.
34. M. H. J. Gruber, Regression Estimators: A Comparative Study. San Diego, CA: Academic, 1990.
35. R. J. Serfling, Approximation Theorems of Mathematical Statistics. New York: Wiley, 1980.