On the equivalence of set-theoretic and maxent MAP estimation

IEEE Transactions on Signal Processing

Volumn 51, Issue 3, 2003, Pages 698-713

  Ishwar, Prakash ^a,b   Moulin, Pierre ^b  

^a UNIVERSITY OF CALIFORNIA   (United States)

^b University of Illinois at Urbana Champaign   (United States)

Author keywords

Estimation theory; Incomplete statistics; Inverse problems; Maximum entropy methods; Modeling; Set theory; Signal restoration

Indexed keywords

COMPUTER SIMULATION; INVERSE PROBLEMS; PARAMETER ESTIMATION; PROBABILITY DISTRIBUTIONS; SET THEORY; SPURIOUS SIGNAL NOISE;

SIGNAL RESTORATION;

SIGNAL DETECTION;

EID: 0037333182     PISSN: 1053587X     EISSN: None     Source Type: Journal    
DOI: 10.1109/TSP.2002.808111     Document Type: Article

References (51)

1. P. L. Combettes, "The foundations of set theoretic estimation," Proc. IEEE, vol. 81, pp. 182-208, Feb. 1993.
2. J. M. Borwein and M. A. Limber. On entropy maximization via convex programming. [Online]. Available: http://www.cecm.sfu.ca/̃malimber/IEEE/IEEE.html.
3. P. Ishwar, "A unified framework for image modeling and estimation using measurement constraints," Ph.D. dissertation, Univ. Illinois at Urbana-Champaign, Urbana, IL, 2002, [Online] Available: http://www.ifp.uiuc.edu/̃moulin/students/ishwar/PhDthesis.ps.
4. H. V. Poor, An Introduction to Signal Detection and Estimation. New York: Springer-Verlag, 1994.
5. P. Billingsley, Probability and Measure. New York: Wiley-Interscience, 1995.
6. S. G. Mallat, "A Theory of Multiresolution Signal Decomposition: The Wavelet Representation," IEEE Trans. Pattern Anal. Machine Intell., vol. 11, pp. 674-693, July 1989.
7. S. M. LoPresto, K. Ramchandran, and M. T. Orchard, "Image coding based on mixture modeling of wavelet coefficients and a fast estimation-quantization framework," in Proc. IEEE Data Compression Conf., Snowbird, UT, Mar. 1997, pp. 221-230.
8. D. G. Luenberger, Optimization by Vector Space Methods. New York: Wiley, 1969.
9. J. Rissanen, Stochastic Complexity in Statistical Inquiry. Singapore: World Scientific, 1989.
10. F. Topsøe, "Information theoretical optimization techniques," Kybernetika, vol. 15, pp. 7-17, 1979.
11. P. Grunwald, "The minimum description length principle and reasoning under uncertainty," Ph.D. dissertation, Univ. van Amsterdam, Amsterdam, Netherlands, 1998.
12. P. H. A. Vogel, "On the rate distortion function of sources with incomplete statistics," IEEE Trans. Inform. Theory, vol. 38, pp. 131-135, Jan. 1992.
13. T. M. Cover and J. A. Thomas, Elements of Information Theory. New York: Wiley, 1991.
14. E. T. Jaynes, "On the rationale of maximum entropy methods," Proc. IEEE, vol. 70, pp. 939-952, Sept. 1982.
15. J. Shore and R. Johnson, "Axiomatic derivation of the principle of maximum entropy and the principle of minimum cross entropy," IEEE Trans. Inform. Theory, vol. IT-26, pp. 26-37, Jan. 1980.
16. I. Csiszár, "Why least squares and maximum entropy? An axiomatic approach to inference for linear inverse problems," Ann. Statist., vol. 19, pp. 2303-2066, 1991.
17. D. P. Bertsekas, Nonlinear Programming. Belmont, MA: Athena Scientific, 1995.
18. L. G. Gubin, B. T. Ployak, and E. V. Raik, "The method of projections for finding the common point of convex sets," U.S.S.R. Comput. Math. Math. Phys., vol. 7, no. 6, pp. 1-24, 1967.
19. B. Javidi and J. L. Homer, Eds., Real-Time Optical Information Processing. Boston, MA: Academic, 1994, ch. 5, pp. 185-232.
20. I. Csiszár, "Generalized projections for nonnegative functions," in Proc. IEEE Int. Symp. Inform. Theory, Whistler, BC, Canada, Sept. 1995, pp. 17-22.
21. C. Byrne, "Iterative projection onto convex sets using multiple Bregman distances," Inverse Problems, vol. 15, pp. 1295-1313, Oct. 1999.
22. P. L. Combettes and H. J. Trussell, "The use of noise properties in set theoretic estimation," IEEE Trans. Signal Processing, vol. 39, pp. 1630-1641, July 1991.
23. P. L. Combettes and T. J. Chaussalet, "Combining statistical information in set theoretic estimation," IEEE Signal Processing Lett., vol. 3, pp. 61-62, Mar. 1996.
24. C. J. Kuo, J. R. Deller Jr., C. Y. Lin, and Y. C. Tsai, "Set-theoretic estimation based on a priori knowledge of the noise distribution," IEEE Trans. Signal Procesing, vol. 48, pp. 2150-2156, July 2000.
25. S. Nagaraj and Y.-F. Huang, "A statistical approach to set-membership identification," in Proc. IEEE Int. Symp. Inform. Theory, Washington, DC, May 2001, pp. 42-42.
26. C. D. Aliprantis and O. Burkinshaw, Principles of Real Analysis. San Diego, CA: Academic, 1998.
27. H. L. Royden, Real Analysis. New York: Prentice-Hall, 1988.
28. W. Feller, An Introduction to Probability Theory and its Applications, 3rd ed. New York: Wiley, 1968, vol. 1.
29. R. Durrett, Probability: Theory and Examples, 2nd ed. Belmont, CA: Duxbury, 1996.
30. A. K. Katsaggelos, J. Biemond, R. W. Schafer, and R. M. Mersereau, "A regularized iterative image restoration algorithm," IEEE Trans. Signal Processing, vol. 39, pp. 914-929, Apr. 1991.
31. P. Ishwar and P. Moulin, "Multiple-domain image modeling and restoration," in Proc. IEEE Int. Conf. Image Process., vol. 1, Kobe, Japan, Oct. 1999, pp. 362-366.
32. _, "Fundamental equivalences between set-theoretic and maximum-entropy methods in multiple-domain image restoration," in Proc. IEEE Int, Conf. Acoust. Speech, Signal Process., vol. 1, Istanbul, Turkey, June 2000, pp. 161-164.
33. H. L. V. Trees, Detection, Estimation, and Modulation Theory. New York: Wiley, 1971.
34. R. M. Gray, "On the asymptotic eigenvalue distribution of Toeplitz matrices," IEEE Trans. Inform. Theory, vol. IT-18, pp. 725-730, Nov. 1972.
35. H. Choi and R. G. Baraniuk, "Multiple basis wavelet denoising using besov projections," in Proc. IEEE Int. Conf. Image Process., vol. 1, Kobe, Japan, Oct. 1999, pp. 595-599.
36. P. Moulin and J. Liu, "Analysis of multiresolution image denoising schemes using generalized-gaussian and complexity priors," IEEE Trans. Inform. Theory, vol. 45, pp. 909-919, Apr. 1999.
37. D. L. Donoho and I. M. Johnstone, "Ideal spatial adaptation by wavelet shrinkage," Biometrika, vol. 81, pp. 425-455, 1994.
38. H. Akaike, "A new look at the statistical model identification," IEEE Trans. Automat. Contr., vol. AC-19, pp. 716-723, Dec. 1974.
39. I. Csiszár, "I-divergence geometry of probability distributions and minimization problems," Ann. Prob., vol. 3, pp. 147-158, 1975.
40. A. Gersho and R. M. Gray, Vector Quantization and Signal Compression. Boston, MA: Kluwer, 1992.
41. P. P. Vaidyanathan, Multirate Systems and Filter Banks. Englewood Cliffs, NJ: Prentice-Hall, 1993.
42. A. Tikhonov and V. Arsenin, Solution of III-Posed Problems. New York: Wiley, 1977.
43. N. P. Galatsanos and A. K. Katsaggelos, "Methods for choosing the regularization parameter and estimating the noise variance in image restoration and their relation," IEEE Trans. Image Processing, vol. 1, pp. 322-336, July 1992.
44. G. Archer and D. M. Titterington, "On some Bayesian/regularization methods for image restoration," IEEE Trans. Image Processing, vol. 4, pp. 989-995, July 1995.
45. M. R. Banham and A. K. Katsaggelos, "Digital image restoration," IEEE Signal Processing Mag., pp. 24-41, Mar. 1997.
46. M. Belge, M. Kilmer, and E. L. Miller, "Simultaneous multiple regularization parameter selection by means of 1-hypersurface with applications to linear inverse problems posed in the wavelet transform domain," in Proc. SPIE Conf. Bayesian Inference Inverse Problems, July 1998.
47. M. Belge, M. E. Kilmer, and E. L. Miller, "Wavelet domain restoration with adaptive edge-preserving regularization," IEEE Trans. Image Processing, vol. 9, pp. 597-608, Apr. 2000.
48. P. Hansen, "Analysis of discrete ill-posed problems by means of the L-curve," SIAM Rev., vol. 34, pp. 561-680, 1992.
49. A. Mohammad-Djafari, "On the estimation of hyperparameters in Bayesian approach of solving inverse problems," in Proc. IEEE Int. Conf. Image Process., Minneapolis, MN, Apr. 1993, pp. 567-571.
50. S. C. Fang, J. R. Rajasekera, and H. S. J. Tsao, Entropy Optimization and Mathematical Programming. Boston, MA: Kluwer, 1997.
51. C. P. Robert and G. Casella, Monte Carlo Statistical Methods. New York: Springer-Verlag, 1999.