Information-theoretic image formation

IEEE Transactions on Information Theory

Volumn 44, Issue 6, 1998, Pages 2094-2123

  O'Sullivan, Joseph A ^a,b   Blahut, Richard E ^a,c   Snyder, Donald L ^a,b  

^a IEEE   (United States)

^b WASHINGTON UNIVERSITY IN ST LOUIS   (United States)

^c University of Illinois at Urbana Champaign   (United States)

Author keywords

Image analysis; Image formation; Image processing; Image reconstruction; Image restoration; Imaging; Inverse problems; Maximum likelihood estimation; Pattern recognition

Indexed keywords

ALGORITHMS; IMAGE ANALYSIS; IMAGE PROCESSING; IMAGE RECONSTRUCTION; IMAGING TECHNIQUES; INVERSE PROBLEMS; MAXIMUM LIKELIHOOD ESTIMATION; PATTERN RECOGNITION;

FISHER INFORMATION;

INFORMATION THEORY;

EID: 0032184237     PISSN: 00189448     EISSN: None     Source Type: Journal    
DOI: 10.1109/18.720533     Document Type: Article

References (108)

1. S. M. Au and S. D. Silvey, "A general class of coefficients of divergence of one distribution from another," J. Roy. Statist. Soc. Ser. B, vol. 28, pp. 131-142, 1966.
2. S. Amari, Differential-Geometrical Methods in Statistics (Lecture Notes in Statistics, vol. 28). Berlin, Germany: Springer-Verlag, 1985.
3. S. Arimoto, "An algorithm for computing the capacity of an arbitrary discrete memoryless channel," IEEE Trans. Inform. Theory, vol. IT-18, pp. 14-20, 1972.
4. J. Besag, "Spatial interaction and the statistical analysis of lattice systems," J. Roy. Statist. Soc., vol. 36, pp. 192-236, 1974.
5. A. Blake and A. Zisserman, Visual Reconstruction. Cambridge, MA: MIT Press, 1987.
6. R. E. Blahut, "Computation of channel capacity and rate distortion functions," IEEE Trans. Inform. Theory, vol. IT-18, pp. 460-473, 1972.
7. L. M. Bregmen, "The relaxation method of finding the common point of convex sets and its application to the solution of problems in convex programming," U.S.S.R. Comp. Math, and Math. Phys., vol. 7, pp. 200-217, 1967.
8. J. Browne and A. R. De Pierro, "A row-action alternative to the EM algorithm for maximizing likelihoods in emission tomography," IEEE Trans. Med. Imag., vol. 15, pp. 687-699, Oct. 1996.
9. J. A. Bucklew, Large Deviation Techniques in Decision, Simulation, and Estimation. New York: Wiley, 1990.
10. J. P. Burg, D. G. Luenberger, and D. L. Wenger, "Estimation of structured covariance matrices," Proc. IEEE, vol. 70, pp. 963-974, Sept. 1982.
11. C. S. Butler and M. I. Miller, "Maximum a posteriori estimation for single photon emission computed tomography using regularization techniques on a massively parallel computer," IEEE Trans. Med. Imag., vol. 12, pp. 84-89, Mar. 1993.
12. C. L. Byrne, "Iterative image reconstruction algorithms based on cross-entropy minimization," IEEE Trans. Image Processing, vol. 2, pp. 96-103, Jan. 1993.
13. R. Chellappa, Markov Random Fields: Theory and Applications. New York: Academic, 1993.
14. P. A. Chou, M. Effros, and R. M. Gray, "A vector quantization approach to universal noiseless coding and quantization," IEEE Trans. Inform. Theory, vol. 42, pp. 1109-1138, July 1996.
15. Y. Chow and U. Grenander, "A sieve method for the spectral density," Ann. Stat., vol. 13, no. 3, pp. 998-1010, 1985.
16. C. K. Chui, Muttivariate Splines. Philadelphia, PA: SIAM, 1988.
17. P. L. Combettes, "The foundation of set theoretic estimation," Proc. IEEE, vol. 81, pp. 182-208, 1993.
18. T. M. Cover, "An algorithm for maximizing expected log investment return," IEEE Trans. Inform. Theory, vol. IT-30, pp. 369-373, 1984.
19. T. M. Cover and J. A. Thomas, Elements of Information Theory. New York: Wiley, 1991.
20. I. Csisza"circ;r, "Information-type measures of difference of probability distributions and indirect observations," Stadia Sci. Math. Hangar., vol. 2, pp. 299-318, 1967.
21. "I-divergence geometry of probability distributions and minimization problems," Ann. Prob., vol. 3, no. 1, pp. 146-158, 1975.
22. "A geometric interpretation of Darroch and Ratcliffs gener-alized iterative scaling," Ann. Stat., vol. 17, no. 3, pp. 1409-1413, 1989.
23. "Why least squares and maximum entropy? An axiomatic approach to inference for linear inverse problems," Ann. Stat., vol. 19, pp. 2032-2066, 1991.
24. I. Csisza"circ;r and G. Tusnady, "Information geometry and alternating decisions," Stat. Decisions, Suppl. issue no. 1, pp. 205-207, 1984.
25. J. N. Darroch and D. Ratcliff, "Generalized iterative scaling for log-linear models," Ann. Math. Stat., vol. 43, no. 5, pp. 1470-1480, 1972.
26. I. Daubechies, Ten Lectures on Wavelets. Philadelphia, PA: SIAM, 1992.
27. A. P. Dempster, N. M. Laird, and D. B. Rubin, "Maximum likelihood from incomplete data via the EM algorithm," J. Roy. Stat. Soc. Ser. B, vol. 39, pp. 1-37, 1977.
28. D. L. Donoho and I. M. Johnstone, "Ideal spatial adaptation via wavelet shrinkage," Biometrika, vol. 81, pp. 425-455, 1994.
29. D. L. Donoho, "De-noising by soft-thresholding," IEEE Trans. Inform. Theory, vol. 41, pp. 613-627, May 1995.
30. D. L. Donoho etal. "Wavelet shrinkage: Asymptotia?" (with discussion), J. Roy. Stat. Soc. Ser. B, vol. 57, no. 2, pp. 301-369, 1995.
31. R. S. Ellis, Entropy, Large Deviations, and Statistical Mechanics. New York: Springer-Verlag, 1985
32. J. A. Fessier and A. O. Hero, "Space alternating generalized expectation-maximization algorithm," IEEE Trans. Signal Processing, vol. 42, pp. 2664-2677, 1994.
33. "Penalized maximum likelihood image reconstruction using space alternating generalized EM algorithms," IEEE Trans. Image Pro-cessing, vol. 4, pp. 1417-1429, Oct. 1995.
34. S. Geman and D. Geman, "Stochastic relaxation, Gibbs' distributions, and Bayesian restoration of images," IEEE Trans. Pattern Anal. Mach. Intell., vol. PAMI-6, pp. 721-741, Nov. 1984.
35. S. Geman and C.-R. Hwang, "Diffusions for global optimization," SIAM J. Contr. Optimiz.. vol. 24, pp. 1031-1043, 1987.
36. A. Gersho and R. M. Gray, Vector Quantization and Signal Compression. Boston, MA: Kluwer, 1992.
37. G. H. Golub and C. F. Van Loan, Matrix Computations, 3rd ed. Baltimore, MD: Johns Hopkins Univ. Press, 1996.
38. I. J. Good and R. A. Gaskins, "Nonparametric roughness penalties for probability densities," Biometrika, vol. 58, pp. 255-277, 1971.
39. J. W. Goodman, Statistical Optics. New York: Wiley-Interscience, 1986
40. Fourier Optics. New York: McGraw-Hill, 1985.
41. J. D. Gorman and A. O. Hero, "Lower bounds for parameter estimation with constraints," IEEE Trans. Inform. Theory, vol. 36, pp. 1285-1301, Nov. 1990
42. U. Grenander, Abstract Inference. New York: Wiley, 1981.
43. U. Grenander and M. I. Miller, "Representations of knowledge in complex systems," J. Roy. Stat. Soc., Ser. B, vol. 56, pp. 549-603, 1994.
44. U. Grenander, M. I. Miller, and A. Srivastava, "Hubert-Schmidt lower bounds for estimators on matrix Lie groups," IEEE Trans. Pattern Anal. Mach. Intell., to be published.
45. R. M. Haralick and L. G. Shapiro, Computer and Robot Vision, vols. 1 and 2. Reading, MA: Addison-Wesley, 1993.
46. C. Heil, "Wavelets and frames," in Signal Processing Part I: Signal Processing Theory, L. Auslander et al., Eds. New York: Springer-Verlag, 1990.
47. A. Hero and J. A. Fessier, "A recursive algorithm for computing Cramer-Rao type bounds on estimator covariance," IEEE Trans. Inform. Theory, vol. 40, pp. 1205-1210, July 1994.
48. A. O. Hero, M. Usman, A. C. Sauve, and J. A. Fessier, "Recursive algorithms for computing the Cramer-Rao bound," IEEE Trans. Signal Processing, vol. 45, pp. 803-807, Mar. 1997.
49. H. M. Hudson and R. S. Larkin, "Accelerated image reconstruction using ordered subsets of projection data," IEEE Trans. Med. Imag., vol. 13, pp. 601-609, Aug. 1994.
50. F. Itakura and S. Saito, "Analysis synthesis telephony based on the maximum likelihood method," in Repts. 6th Int. Congr. Acoustics, Y. Kohasi, Ed. (Tokyo, Japan, 1968), pp. 17-20.
51. E. T. Jaynes, "On the rationale of maximum entropy methods," Proc. IEEE, vol. 70, pp. 939-952, 1982.
52. L. K. Jones and C. L. Byrne, "General entropy criteria for inverse problems, with applications to data compression, pattern classification and cluster analysis," IEEE Trans. Inform. Theory, vol. 36, pp. 23-30, 1990.
53. L. S. Joyce and W. L. Root, "Precision bounds in superresolution processing," J. Opt. Soc. Amer. A, vol. 1, pp. 149-168, 1984.
54. A. Kirsch, "An introduction to the mathematical theory of inverse prob-lems," Applied Mathematical Sciences, vol. 120. Berlin, Germany: Springer-Verlag, 1996.
55. S. Kullback, Information Theory and Statistics. New York: Wiley, 1959; Dover, 1968.
56. S. Kullback and M. A. Khairat, "A note on minimum discrimination information," Ann. Math. Stat., vol. 37, pp. 279-280, 1966.
57. S. Kullback and R. A. Leibler, "On information and sufficiency," Ann. Math. Stat., vol. 22, pp. 79-86, 1951.
58. K. Lange and R. Carson, "EM reconstruction algorithms for emission and transmission tomography," J. Comp. Assist. Tomogr., vol. 8, no. 2, pp. 306-316, 1984.
59. A. D. Lanterman, M. I. Miller, and D. L. Snyder, "General metropolishastings jump diffusions for automatic target recognition in infrared scenes," Opt. Eng., vol. 36, pp. 1123-1137, 1997.
60. M. I. Miller and C. S. Butler, "3-D maximum a posteriori estimation for single photon emission computed tomography on massively parallel computers," IEEE Trans. Med. Imag., vol. 12, pp. 560-565, Sept. 1993.
61. M. I. Miller and B. Roysam, "Bayesian image reconstruction for emission tomography: Implementation of the EM algorithm and good's roughness prior on massively parallel processors," Proc. Nat. Acad. Sci., vol. 88, pp. 3223-3227, 1991.
62. M. I. Miller and D. L. Snyder, "The role of likelihood and entropy in incomplete-data problems: Applications to estimating point-process intensities and toeplitz and constrained covariances," Proc. IEEE, vol. 75, pp. 892-907, 1987.
63. M. I. Miller, A. Srivastava, and U. Grenander, "Conditional-mean estimation via jump-diffusion processes in multiple target tracking and recognition," IEEE Trans. Signal Processing, vol. 43, pp. 2678-2690, 1995.
64. P. Moulin, "A method of sieves for radar imaging and spectrum estimation," D.Sc. dissertation, Dept. Elec. Eng. Washington Univ., St. Louis, MO, 1990.
65. "A multiscale relaxation technique for SNR maximization in nonorthogonal subband coding," IEEE Trans. Image Processing, vol. 4, pp. 1269-1281, Sept. 1995.
66. P. Moulin and J. Liu, "Analysis of multiresolution image denoising schemes using generalized-gaussian and complexity priors," preprint, submitted for publication.
67. P. Moulin, J. A. O'Sullivan, and D. L. Snyder, "A method of sieves for multiresolution spectrum estimation and radar imaging," IEEE Trans. Inform. Theory, vol. 38, pp. 801-813, 1992
68. M. Nikolova, J. Idler, and A. Mohammad-Djafari, "Inversion of large-support Ill-posed linear operators using a piecewise Gaussian MRF," IEEE Trans. Image Processing, vol. 7, pp. 571-585, Apr. 1998.
69. J. A. O'Sullivan, "Roughness penalties on finite domains," IEEE Trans. Image Processing, vol. 4, pp. 1258-1268, Sept. 1995.
70. "Alternating minimization algorithms: From Blahut-Arimoto to expectation-maximization," in Codes, Curves, and Signals: Common Threads in Communications, A. Vardy, Ed. Norwell, MA: Kluwer, 1998.
71. J. A. O'Sullivan, D. L. Snyder, D. G. Porter, and P. Moulin, "An application of splines to maximum likelihood radar imaging," J. Imaging Syst. Technol, vol. 4, pp. 256-264, 1992.
72. R. G. Paxman, T. J. Schulz, and J. R. Fienup, "Joint estimation of object and aberrations by using phase diversity," J. Opt. Soc. Amer. A, vol. 9, pp. 1072-11085, July 1992.
73. D. G. Politte, "Reconstruction algorithms for time-of-flight assisted positron-emission tomography," M.S.E.E. thesis, Seh. Eng. Appl. Sci., Washington Univ., St. Louis, MO, 1983.
74. D. G. Politte and D. L. Snyder, "Corrections for accidental coincidences in maximum-likelihood image reconstruction for position-emission to-mography," IEEE Trans. Med. Imag., vol. 10, pp. 82-89, 1991.
75. D. D. Robertson, J. Yuan, G. Wang, and M. W. Vannier, "Total hip prosthesis metal-artifact suppression using iterative deblurring recon-struction," J. Comp. Assist. Tomogr., vol. 21, pp. 293-298, 1997.
76. A. Rockmore and A. Macovski, "A maximum likelihood approach to emission image reconstruction from projections," IEEE Trans. Nucl. Sci., vol. NS-23, pp. 1428-1432, 1976.
77. M. C. Roggemann and B. Welsh, Imaging Through Turbulence. New York: CRC Press, 1996.
78. I. N. Sanov, "On the probability of large deviations of random variance," Matem. Sbornik, vol. 42, pp. 1144, 1957.
79. T. J. Schulz and D. L. Snyder, "Imaging a randomly moving object from quantum-limited data: Applications to image recovery from second- and third-order autocorrelations," J. Opt. Soc. Amer. A, vol. 8, pp. 801-807, May 1991.
80. "Image recovery from correlations," J. Opt. Soc. Amer. A, vol. 9, pp. 1266-1272, Aug. 1992.
81. J. H. Seldin and R. G. Paxman, "Phase-diverse speckle reconstruction of solar data," in Proc. SPIE Conf. 2302, July 1994.
82. M. I. Sezan and H. Stark, "Image restoration by the method of convex projections: Part 2-Applications and numerical results," IEEE Trans. Med. Imag., vol. MI-1, pp. 95-101, 1982.
83. C. E. Shannon, "A mathematical theory of communication," Bell Syst. Tech. J., vol. 27, pp. 37923, 1948.
84. L. A. Shepp and Y. Vardi, "Maximum likelihood reconstruction for emission tomography," IEEE Trans. Med. Imag., vol. MI-1, pp. 113-122, 1982.
85. J. E. Shore and R. W. Johnson, "Axiomatic derivation of the principle of maximum entropy and the principle of minimum cros-entropy," IEEE Trans, Inform. Theory, vol. IT-26, pp. 26-37, Jan. 1980.
86. D. L. Snyder and T. J. Schulz, "High-resolution imaging at low-light levels through weak turbulence," J. Opt. Soc. Amer. A, vol. 7, pp. 1251-1265, July 1990.
87. D. L. Snyder, C. W. Helstrom, A. D. Lanterman, M. Faisal, and R. L. White, "Compensation for read-out noise in CCD images," J. Opt. Soc. Amer. A, vol. 12, pp. 272-283, 1995.
88. D. L. Snyder, A. M. Hammoud, and R. L. White, "Image recovery from data acquired with a charge-coupled-device camera," J. Opt. Soc. Amer. A, vol. 10, pp. 1014-1023, 1993.
89. D. L. Snyder and M. I. Miller, "The use of sieves to stabilize images produced with the EM algorithm for emission tomography," IEEE Trans. Nucl. Sei, vol. NS-32, pp. 3864-3872, 1985.
90. Random Point Processes in Time and Space. New York: Springer-Verlag, 1991, ch. 3.
91. D. L. Snyder, J. A. O'Sullivan, and M. I. Miller, "The use of maximum likelihood estimation for forming images of diffuse radar targets from delay-doppler data," IEEE Trans. Inform. Theory, vol. 35, pp. 536-548, 1989.
92. D. L. Snyder, T. J. Schulz, and J. A. O'Sullivan, "Deblurring subject to nonnegativity constraints," IEEE Trans. Signal Processing, vol. 40, pp. 1143-1150, 1992.
93. D. L. Snyder, L. J. Thomas Jr., and M. M. TerPogossian, "A mathematical model for positron-emission tomography systems having time-of-flight measurements," IEEE Trans. Nucl. Sci., vol. NS-28, pp. 3575-3583, 1981.
94. A. Srivastava, M. I. Miller, and U. Grenander, "Multiple target direction of arrival tracking," IEEE Trans. Signal Processing, vol. 43, pp. 1282-1285, 1995.
95. R. A. Tapia and J. R. Thompson, Nonparametric Probability Density Estimation. Baltimore, MD: Johns Hopkins Univ. Press, 1978.
96. A. N. Tikhonov and V. Y. Arsenin, Solutions of Ill-Posed Problems. Washington, DC: Winston, 1977.
97. R. Y. Tsai, "A versatile camera calibration technique for high-accuracy 3D machine vision metrology using off-the-shelf TV cameras and lenses," IEEE J. Robot. Automat., vol. RA-3, no. 4, pp. 323-344, 1987.
98. M. W. Vannier, M. I. Miller, and U. Grenander, "Modeling and data structure for registration to a brain atlas of multimodality images," in Functional Neuroimaging-Technical Foundations, R. W. Thatcher et ai, Eds. New York: Academic, 1994, pp. 217-221.
99. H. L. Van Trees, Detection, Estimation, and Modulation Theory, pt. I. New York: Wiley, 1968.
100. Y. Vardi and D. Lee, "From image deblurring to optimal investments: Maximum-likelihood solutions to positive linear inverse problems," J. Roy. Stat. Soc. Ser. B, pp. 569-612, 1993.
101. Y. Vardi, L. A. Shepp, and L. Kaufmann, "A statistical model for positron emission tomography," J. Amer. Stat. Soc., vol. 80, pp. 8-35, 1985.
102. G. Wahba, Spline Models for Observational Data. Philadelphia, PA: SIAM, 1990.
103. G. Wang, D. L. Snyder, J. A. O'Sullivan, and M. W. Vannier, "Iterative deblurring for CT metal artifact reduction," IEEE Trans. Med. Imag., vol. 15, pp. 657-664, 1996.
104. J. M. Wozencraft and I. M. Jacobs, Principles of Communication Engineering. New York: Wiley, 1965.
105. C. F. J. Wu, "On the convergence properties of the EM algorithm," Ann. Stat., vol. 11, pp. 95-103, 1983.
106. D. C. Youla, "Generalized image restoration by the method of alternating orthogonal protections," IEEE Trans. Circuits Syst., vol. CAS-25, pp. 694-702, 1978.
107. "Mathematical theory of image restoration by the method of convex projections," in Image Recovery, Theory and Applications, H. Stark, Ed. New York: Academic, 1987, ch. 2.
108. D. C. Youla and H. Webb, "Image restoration by the method of convex projections: Part 1-Theory," IEEE Trans. Med. Imag., vol. MI-1, no. 2, pp. 81-94, 1982.