Exploring estimator bias-variance tradeoffs using the uniform CR bound

IEEE Transactions on Signal Processing

Volumn 44, Issue 8, 1996, Pages 2026-2041

  Hero III, Alfred O ^a   Fessler, Jeffrey A ^a   Usman, Mohammad ^b  

^a UNIVERSITY OF MICHIGAN   (United States)

^b FAST Inst of Computer Science   (Pakistan)

Author keywords

[No Author keywords available]

Indexed keywords

INVERSE PROBLEMS; MATHEMATICAL MODELS; MATHEMATICAL TRANSFORMATIONS; OPTIMIZATION; PARAMETER ESTIMATION; SIGNAL PROCESSING; STATISTICAL METHODS;

BIAS GRADIENT; DELTA SIGMA TRADEOFF CURVE; ESTIMATOR BIAS VARIANCE TRADEOFFS; PARAMETER TRANSFORMATIONS; UNIFORM CRAMER RAN BOUND;

SIGNAL THEORY;

EID: 0030215043     PISSN: 1053587X     EISSN: None     Source Type: Journal    
DOI: 10.1109/78.533723     Document Type: Article

References (51)

1. S. M. Kay, Modern Spectral Estimation: Theory and Application. Englewood Cliffs, NJ: Prentice Hall, 1988.
2. P. J. Daniell, Discussion of paper by M. S. Bartlett, J. Royal Stat. Soc., Ser. B, vol. 8, p. 27, 1946.
3. R. B. Blackman and J. W. Tukey, The measurement of power spectra from the point of view of communication engineering, Bell Syst. Tech. J., vol. 37, pp. 183-282, 1958.
4. R. Lagendijk and J. Biemond, Iterative Identification and Restoration of Images. Boston: Kluwer, 1991.
5. C. M. Stein, Multiple regression, in Contributions to Probability and Statistics: Essays in Honor of Harold Hotelling, I. Olkin, S. Ghurge, W. Hoeffding, W. Nadow, and H. Mann, Eds. Stanford, CA: Stanford Univ. Press, 1960, pp. 424-443.
6. A. E. Hoerl and R. W. Kennard, Ridge regression: Biased estimation of nonorthogonal components, Technometrics, vol. 12, pp. 55-67, 1970.
7. J.-S. Liow and S. C. Strother, Practical tradeoffs between noise, quantitation, and number of iterations for maximum likelihood-based reconstructions, IEEE Trans. Med. Imaging, vol. 10, no. 4, pp. 563-571, 1991.
8. S. J. Lee, G. R. Gindi, I. G. Zubal, and A. Rangarajan, Using ground-truth data to design priors in Bayesian SPECT reconstruction, in Information Processing in Medical Imaging, Y. Bizais, C. Barillot, and R. D. Paola, Eds. Boston, MA: Kluwer, 1995.
9. J. A. Fessler, Penalized weighted least-squares image reconstruction for positron emission tomography, IEEE Trans. Med. Imaging, vol. 13, pp. 290-300, June 1994.
10. R. E. Carson, Y. Yan, B. Chodkowski, T. K. Yap, and M. E. Daube-Witherspoon, Precision and accuracy of regional radioactivity quantitation using the maximum likelihood EM reconstruction algorithm, IEEE Trans. Med. Imaging, vol. 13, pp. 526-537, Sept. 1994.
11. D. Wang, V. Haese-Coat, A. Bruno, and J. Ronsin, Some statistical properties of mathematical morphology, IEEE Trans. Signal Processing, vol. 43, pp. 1955-1965, Aug. 1995.
12. N. Himayat and S. A, Kassam, Approximate performance analysis of edge preserving filters, IEEE Trans. Signal Processing, vol. 41, pp. 2764-2777, Sept. 1993.
13. M. B. Woodroofe and J. W. Van Ness, The maximum deviation of sample spectral densities, Ann. Math. Stat., vol. 38, pp. 1558-1569, 1967.
14. P. Bloomfield, Fourier Analysis of Time Series. New York: Wiley, 1976.
15. A. O. Hero, A Cramer-Rao type lower bound for essentially unbiased parameter estimation, MIT Lincoln Lab., Lexington, MA, Jan. 1992, Tech. Rep. 890, DTIC AD-A246666.
16. J. Fessler and A. Hero, Cramer-Rao bounds for biased estimators in image restoration, in Proc. 36th IEEE Midwest Symp. Circuits Syst., Detroit, MI, Aug. 1993.
17. M. Usman, A. Hero, J. A. Fessler, and W. Rogers, Bias-variance tradeoffs analysis using uniform CR bound for a SPECT system, in Proc. IEEE Nuclear Sci. Symp. Med. Imaging Conf., San Francisco, CA, Nov. 1993, pp. 1463-1467.
18. M. Usman, A. Hero, and J. A. Fessler, Bias-variance tradeoffs analysis using uniform CR bound for images, in Proc. IEEE Image Processing Conf., Austin, TX, Nov. 1994, pp. 835-839.
19. C. R. Rao, Linear Statistical Inference and Its Applications. New York: Wiley, 1973.
20. J. A. Fessler and A. O. Hero, Space-alternating generalized EM algorithm, IEEE Trans. Signal Processing, vol. 42, no. 10, pp. 2664-2677, Oct. 1994.
21. G. E. P. Box and N. R. Draper, A basis for the selection of a response surface design, J. Am. Statist. Assoc., vol. 54, pp. 622-654, 1959.
22. M. A. Ali, On a class of shrinkage estimators of the vector of shrinkage coefficients, Commun. Statist.-Theory Meth., vol. 18, no. 12, pp. 4491-4500, 1981.
23. T. Bednarski, On minimum bias and variance estimation for parametric models with shrinking contamination, Prob. Math. Stat., vol. 6, no. 2, pp. 121-129, 1985.
24. M. Quenouille, Approximate tests of correlation in time series, J. Royal Stat. Soc., Ser. B, vol. 11, pp. 18-84, 1949.
25. J. Tukey, Bias and confidence in not quite large samples, Ann. Math. Stat., vol. 29, pp. 614, 1956.
26. B. Efron, Bootstrap methods: Another look at the jacknife, Ann. Stat., vol. 7, pp. 1-26, 1979.
27. I. A. Ibragimov and R. Z. Has'minskii, Statistical Estimation: Asymptotic Theory. New York: Springer-Verlag, 1981.
28. M. Fréchet, Sur l'extension de certaines évaluations statistiques de petits echantillons, Rev. Inst. M. Stat., vol. 11, pp. 128-205, 1943.
29. G. Darmois, Sur les lois limites de la dispersion de certaines estimations, Rev. Inst. Int. Stat., vol. 13, pp. 9-15, 1945.
30. H. Cramér, A contribution to the theory of statistical estimation, Skand. Aktuaries Tidskrift, vol. 29, pp. 458-463, 1946.
31. C. R. Rao, Minimum variance and the estimation of several parameters, Proc. Cambridge Phil. Soc., vol. 43, pp. 280-283, 1946.
32. H. L. Van-Trees, Detection, Estimation, and Modulation Theory: Part I. New York: Wiley, 1968.
33. J. D. Gorman and A. O. Hero, Lower bounds for parametric estimation with constraints, IEEE Trans. Inform. Theory, vol. 36, pp. 1285-1301, Nov. 1990.
34. M. Usman, Biased and unbiased Cramer-Rao bounds: Computational issues and applications, Ph.D. Thesis, Dept. Elec. Eng. Comput. Sci., Univ. Michigan, Aug. 1994.
35. W. Press, S. Teukolsky, W. Vetterling, and B. Flannery, Numerical Recipes in C: The Art of Scientific Computing. Cambridge, UK: Cambridge Univ. Press, 1992, 2nd ed.
36. A. O. Hero, M. Usman, A. C. Sauve, and J. A. Fessler, Recursive algorithms for computing the Cramer-Rao bound, IEEE Trans. Signal Processing, submitted for publication.
37. B. E. Efron, The Jackknife, the Bootstrap, and Other Resampling Plans. Philadelphia, PA: SIAM, 1982.
38. V. A. Morozov, Methods for Solving Incorrectly Posed Problems. New York: Springer-Verlag, 1984.
39. K. Lange, Convergence of EM image reconstruction algorithms with Gibbs smoothing, IEEE Trans. Med. Imaging, vol. 9, pp. 439-446, June 1991.
40. T. Hebert and R. Leahy, A generalized EM algorithm for 3-D Bayesian reconstruction from Poisson data using Gibbs priors, IEEE Trans. Med. Imaging, vol. 8, pp. 194-203, 1989.
41. V. Johnson, W. Wong, X. Hu, and C. Chen, Image restoration using Gibbs priors: boundary modeling, treatment of blurring, and selection of hyperparameters, IEEE Trans. Patt. Anal. Machine Intell., vol. PAMI-13, pp. 413-425, 1991.
42. R. C. Liu and L. D. Brown, Nonexistence of informative unbiased estimators in singular problems, Ann. Stat., vol. 21, no. 1, pp. 1-13, 1993.
43. G. H. Golub and C. F. Van Loan, Matrix Computations. Baltimore, MA: The Johns Hopkins Univ. Press, 1989, 2nd ed.
44. D. L. Snyder and M. I. Miller, Random Point Processes in Time and Space. New York: Springer-Verlag, 1991.
45. A. O. Hero and J. A. Fessler, A recursive algorithm for computing CR-type bounds on estimator covariance, IEEE Trans. Inform. Theory, vol. 40, pp. 1205-1210 July 1994.
46. J. A. Fessler and A. O. Hero, Penalized maximum likelihood image reconstruction using space alternating generalized EM algorithms, IEEE Trans. Image Processing, vol. 4, pp. 1417-1429, Oct. 1995.
47. J. A. Fessler and W. L. Rogers, Uniform quadratic penalties cause nonuniform image resolution (and sometimes vice versa), in Proc. IEEE Nuclear Sci. Symp., vol. 4, 1994, pp. 1915-1919.
48. J. A. Fessler, Resolution properties of regularized image reconstruction methods, Technical Report 297, Comm. Signal Processing Lab. (CSPL), Dept. Elec. Eng. Comput. Sci., Univ. of Michigan, Ann Arbor, Aug. 1995.
49. E. B. Manoukian, Modern Concepts and Theorems of Mathematical Statistics. New York: Springer-Verlag, 1986.
50. P. J. Huber, Robust Statistics. New York: Wiley, 1981.
51. R. J. Serfling, Approximation Theorems of Mathematical Statistics. New York: Wiley, 1980.