The Role of Likelihood and Entropy in Incomplete-Data Problems: Applications to Estimating Point-Process Intensities and Toeplitz Constrained Covariances

Proceedings of the IEEE

Volumn 75, Issue 7, 1987, Pages 892-907

  Miller, Michael I ^a   Snyder, Donald L ^a  

^a WASHINGTON UNIVERSITY   (United States)

Author keywords

[No Author keywords available]

Indexed keywords

INCOMPLETE DATA PROBLEMS; LIKELIHOOD/ENTROPY; POINT-PROCESS INTENSITIES; TOEPLITZ CONSTRAINED COVARIANCES;

MATHEMATICAL TECHNIQUES - ESTIMATION; PROBABILITY - RANDOM PROCESSES;

IMAGE PROCESSING;

EID: 0023383312     PISSN: 00189219     EISSN: 15582256     Source Type: Journal    
DOI: 10.1109/PROC.1987.13825     Document Type: Article

References (46)

1. [1] A. D. Dempster, N. M. Laird, and D. B. Rubin, “Maximum likelihood from incomplete data via the EM algorithm,” J. Roy. Stat. Soc., vol. B39, pp. 1–37, 1977.
2. [2] S. F. Gull and G. J. Daniell, “Image reconstruction from incomplete and noisy data,” Nature, vol. 272, p. 686, 1978.
3. [3] S. F. Gull and J. Skilling, “The maximum entropy method,” in Indirect Imaging, J. A. Roberts, Ed. London, UK: Cambridge Univ. Press, 1984.
4. [4] J. Skilling and R. K. Bryan, “Maximum entropy image reconstruction,” MNRAS, submitted 1982.
5. [5] J. P. Burg, “Maximum entropy spectral analysis,” Univ. Microfilms no. 75–25, Stanford Univ., Stanford, CA, 1975.
6. [6] J. E. Shore, “Minimum cross-entropy spectral analysis,” IEEE Trans. Acoust., Speech, Signal Processing, vol. ASSP-29, no. 2, pp. 230–237, Apr. 1981.
7. [7] E. T. Jaynes, “Information theory and statistical mechanics,” Phys. Rev., vol. 106, pp. 620–630, 1957.
8. [8] E. T. Jaynes, “On the rationale of maximum-entropy methods,” Proc. IEEE, vol. 70, pp. 939–952, Sept. 1982.
9. [9] D. L. Snyder and D. G. Politte, “Image reconstruction from list-mode data in an emission tomography system having time-of-flight measurements,” IEEE Trans. Nucl. Sci., vol. NS-30, pp. 1843–1849, 1983.
10. [10] D. L. Snyder and M. I. Miller, “The use of sieves to stabilize images produced with the EM algorithm for emission tomography,” IEEE Trans. Nucl. Sci., vol. NS-32, pp. 3864–3872, Oct. 1985.
11. [11] M. I. Miller, D. L. Snyder, and T. Miller, “Maximum likelihood reconstruction for single photon emission computed tomography,” IEEE Trans. Nucl. Sci., vol. NS-32, no. 1, pp. 769–778, Feb. 1985.
12. [12] M. I. Miller, “Algorithms for removing recovery related distortion from auditory-nerve discharge patterns,” J. Acoust. Soc. Amer., vol. 77, pp. 1452–1464, 1985.
13. [13] M. I. Miller, N. Karamanos, and W. E. Bosch, “EM algorithms for estimating parameters from single-memory Markov point-processes processes having a multiplicative intensity,” presented at the 23rd Annual Allerton Conf., Univ. of Illinois, Urbana, IL, Oct. 1985.
14. [14] M. I. Miller, K. B. Larson, J. E. Saffitz, D. L. Snyder, and L. J. Thomas, Jr., “Maximum-likelihood estimation applied to electron-microscope autoradiography,” J. Electron Microscopy Tech., 1985.
15. [15] J. P. Burg, D. G. Luenberger, and D. L. Wenger, “Estimation of structured covariance matrices,” Proc. IEEE, vol. 70, no. 9, pp. 963–974, Sept. 1982.
16. [16] L. A. Shepp and Y. Vardi, “Maximum-likelihood reconstruction for emission tomography,” IEEE Trans. Med. Imag., vol. MI-1, pp. 113–121, 1982.
17. [17] S. Kullback, in Information Theory and Statistics. New York, NY: Wiley, Dover, 1959, 1968.
18. [18] I. Csiszar, “I-divergence geometry of probability distributions and minimization problems,” Ann. Prob., vol. 3, pp. 146–158, 158, 1975.
19. [19] J. M. Van Campenhout and T. M. Cov e r, “Maximum entropy and conditional probability,” IEEE Trans. Informat. Theory, vol. IT-27, no. 4, pp. 483–489, July 1981.
20. [20] I. Csiszar and G. Tusnady, “Information geometry and alternating minimization procedures,” Statistics and Decisions (Supplement Issue no. 1). Munchen, West Germany: R. Old-enbourg Verlag, 1984, pp. 205–237.
21. [21] Physics, R. D. Rosenkrantz, Ed. Dordrecht: The Netherlands/ Boston: USA/London, England: D. Reidel, 1983.
22. [22] B. R. Musicus, “Iterative algorithms for optimal signal reconstruction and parameter identification given noisy and incomplete data,” MIT thesis, Cambridge, MA, 1982.
23. [23] R. D. Evans, in The Atomic Nucleus. New York, NY: McGraw-Hill, Hill, 1955.
24. [24] D. L. Snyder, L. J. Thomas, Jr., and M. M. Ter-Pogossian, “A mathematical model for positron emission tomography systems having time-of-flight measurements,” IEEE Trans. Nucl. Sci., vol. NS-28, pp. 3575–3583, 1981.
25. [25] D. L. Snyder, Random Point Processes. New York, NY: Wiley, 1975.
26. [26] K. Lange and R. Carson, “EM reconstruction algorithms for emission and transmission tomography,” J. Comput. Assisted Tomography, vol. 8, no. 2. New York, NY: Raven Press, Apr. 1984, pp. 306–316.
27. [27] B. R. Fried e n, “Restoring with maximum likelihood and maximum entropy,” J. Opt. Soc. Amer., vol. 62, no. 4, pp. 511–518, 1972.
28. [28] B. R. Frieden and D. C. Wells, “Restoring with maximum entropy. III. Poisson sources and backgrounds,” J. Opt. Soc. Amer., vol. 68, no. 1, pp. 93–103, 1978.
29. [29] J. Skilling, A. W. Strong, and K. Bennett, “Maximum-entropy image processing in gammaray astronomy,” Mon. Not. R. Astr. Soc., vol. 187, pp. 145–152, 1979.
30. [30] L. Scarsi, in Proc. 12th ESLAB Symp., vol. ESA-124, p. 3, 1977.
31. [31] Y. Vardi, L. A. Shepp, and L. Kaufman, “A statistical model for positron emission tomography,” J. Amer. Statist. Assoc., vol. 80, pp. 8–35, Mar. 1985.
32. [32] D. Politte, “Reconstruction algorithms for time-of-flight assisted positron-emission tomographs,” M.S. thesis, Sever Institute of Technology, Washington Univ., St. Louis, MO, Dec. 1983 (supervised by D. L. Snyder).
33. [33] M. Mintun, J. Gorman, and D. L. Snyder, “Evaluation of the maximum-likelihood method for reconstruction of images in positron emission tomography,” in Proc. Soc. Nuclear Med-icine 32nd Annual Meeting (Houston, TX, June 1985).
34. [34] J. M. Ollinger and D. L. Snyder, “An evaluation of an improved method for computing histograms in dynamic tracer studies using positron-emission tomography,” IEEE Trans. Nucl. Sci., vol. NS-33, pp. 435–438, Feb. 1986.
35. [35] M. I. Miller, D. L. Snyder, and S. M. Moore, “An evaluation of the use of sieves for producing estimates of radioactivity distributions with the EM algorithm for PET,” IEEE Trans. Nucl. Sci., vol. NS-33, pp. 492–495, Feb. 1986.
36. [36] L. A. Shepp, Y. Vardi, J. B. Ra, S. K. Hilal, and Z. H. Cho, “Max-imum-likelihood with real data,” IEEE Trans. Nucl. Sci., vol. NS-31, pp. 910–913, 1984.
37. [37] R. Carson, personal communication, 1985.
38. [38] R. A. Tapia and J. R. Thompson, Nonparametric Probability Density Estimation. Baltimore, MD: Johns Hopkins Univ. Press, 1978.
39. [39] U. Grenander, Abstract Inference. New York, NY: Wiley, 1981.
40. [40] S. Geman, “Sieves for nonparametric estimation of densities and regressions,” Repts. Pattern Anal., vol. 99, D.A.M. Brown Univ., 1981.
41. [41] I. J. Good and R. A. Gaskins, “Nonparametric roughness penalties for probability densities,” Biometrika, vol. 58, no. 2, pp. 255–277, 1971.
42. [42] A. van den Bos, IEEE Trans. Inform. Theory, vol. IT-17, pp. 493–494, 494, 1971.
43. [43] Modern Spectrum Analysis, D. G. Childers, Ed. New York, NY: IEEE Press, 1978.
44. [44] I. B. Rhodes, “A tutorial introduction to estimation and fil-tering,” IEEE Trans. Automat. Contr., vol. AC-16, no. 6, pp. 688–706, 706, 1971.
45. [45] D. R. Fuhrmann and M. I. Miller, On the Singularity of Max-imum Likelihood Estimates of Structured Covariance Matri-ces, Monograph of Electronic Systems and Signals Research Lab., Washington Univ., St. Louis, MO, 1986.
46. [46] T. M. Cover, “An algorithm for maximizing expected log investment return,” IEEE Trans. Inform. Theory, vol. IT-30, pp. 369–373, Mar. 1984.