A recursive model-reduction method for approximate inference in Gaussian Markov random fields

IEEE Transactions on Image Processing

Volumn 17, Issue 1, 2008, Pages 70-83

  Johnson, Jason K ^a   Willsky, Alan S ^a  

^a USA   (United States)

Author keywords

Approximate inference; Gaussian Markov random fields; Graphical models; Information projection; Maximum entropy; Model reduction

Indexed keywords

APPROXIMATE INFERENCE; GAUSSIAN MARKOV RANDOM FIELDS; GRAPHICAL MODELS; INFORMATION PROJECTION; MAXIMUM ENTROPY; MODEL REDUCTION;

ENTROPY; FISHER INFORMATION MATRIX; GAUSSIAN DISTRIBUTION; GAUSSIAN NOISE (ELECTRONIC); GRAPHIC METHODS; HIDDEN MARKOV MODELS; NEMATIC LIQUID CRYSTALS; OPTIMIZATION; RECURSIVE FUNCTIONS; REMOTE SENSING; STRUCTURED PROGRAMMING; TREES (MATHEMATICS); TRELLIS CODES;

MEDICAL IMAGING;

ALGORITHM; ARTICLE; ARTIFICIAL INTELLIGENCE; AUTOMATED PATTERN RECOGNITION; COMPUTER ASSISTED DIAGNOSIS; COMPUTER SIMULATION; IMAGE ENHANCEMENT; METHODOLOGY; NORMAL DISTRIBUTION; PROBABILITY; REPRODUCIBILITY; SENSITIVITY AND SPECIFICITY; STATISTICAL MODEL;

ALGORITHMS; ARTIFICIAL INTELLIGENCE; COMPUTER SIMULATION; IMAGE ENHANCEMENT; IMAGE INTERPRETATION, COMPUTER-ASSISTED; MARKOV CHAINS; MODELS, STATISTICAL; NORMAL DISTRIBUTION; PATTERN RECOGNITION, AUTOMATED; REPRODUCIBILITY OF RESULTS; SENSITIVITY AND SPECIFICITY;

EID: 39149123561     PISSN: 10577149     EISSN: None     Source Type: Journal    
DOI: 10.1109/TIP.2007.912018     Document Type: Article

References (45)

1. H.-O. Georgii, Gibbs Measures and Phase Transitions. Berlin, Germany: De Gruyter, 1988.
2. B. Simon, The Statistical Mechanics of Lattice Gases. Princeton, NJ: Princeton Univ. Press, 1993.
3. B. Frey, Ed., Graphical Models for Machine Learning and Digital Communication. Cambridge, MA: MIT Press, 1998.
4. J. Woods, "Markov image modeling," IEEE Trans. Autom. Control, vol. 23, no. 5, pp. 846-850, Oct. 1978.
5. J. Besag, "On the statistical analysis of dirty pictures," J. Roy. Statist. Soc., vol. 48, no. 3, pp. 259-302, 1986.
6. P. Pérez, A. Chardin, and J.-M. Laferté, "Noniterative manipulation of discrete energy-based models for image analysis," Pattern Recognit., vol. 33, pp. 573-586, 2000.
7. J.-M. Laferté, P. Pérez, and F. Heitz, "Discrete Markov image modeling and inference on the quadtree," IEEE Trans. Image Process. vol. 9, no. 3, pp. 390-404, Mar. 2000.
8. E. Monfrini, J. Lecomte, F. Desbouvries, and W. Pieczynski, "Image and signal restoration using pairwise Markov trees," in Proc. IEEE Workshop on Signal Processing, Oct. 2003, pp. 174-177.
9. W. Pieczynski, "Hidden evidential Markov trees and image segmentation," in Proc. Int. Conf. Image Processing, 1999, pp. 338-342.
10. S. Chevalier, E. Geoffrois, F. Prẽteux, and M. Lemaĩtre, "A generic 2-D approach to handwriting recognition," in Proc. 8th Int. Conf. Document Analysis and Recognition, Sep. 2005, pp. 489-493.
11. P. Fieguth, W. Karl, A. Willsky, and C. Wunsch, "Multiresolution optimal interpolation and statistical analysis of topex/poseidon satellite altimetry," IEEE Trans. Geosci. Remote Sens., vol. 33, no. 2, pp. 280-292, Mar. 1995.
12. M. Daniel and A. Willsky, "A multiresolution methodology for signal-level fusion and data assimilation with applications in remote sensing," Proc. IEEE, vol. 85, no. 7, pp. 164-183, Jul. 1997.
13. X. Descombes, M. Sigelle, and F. Préteux, "Estimating GMRF parameters in a nonstationary framework: Application to remote sensing," IEEE Trans. Image Process., vol. 8, no. 4, pp. 490-503, Apr. 1999.
14. A. Willsky, "Multiresolution Markov models for signal and image processing," Proc. IEEE, vol. 90, no. 8, pp. 1396-1458, Aug. 2002.
15. S. Thrun, D. Koller, Z. Ghahramani, and H. Durrant-Whyte, "Simultaneous localization and mapping with sparse extended information filters," Int. J. Robot. Res., vol. 23, no. 8, pp. 693-716, Aug. 2004.
16. S. Lauritzen, Graphical Models. Oxford, U.K.: Oxford Univ. Press, 1996.
17. R. Cowell, A. Dawid, S. Lauritzen, and D. Spiegelhalter, Probabilistic Networks and Expert Systems. New York: Springer, 1999.
18. G. Grimmett, "A theorem about random fields," Bull. Lond. Math. Soc., vol. 5, pp. 81-84, 1973.
19. L. Saul and M. Jordan, "Exploiting tractable substructures in intractable networks," presented at the Advances in Neural Information Processing Systems, 1995.
20. J. Yedidia, W. Freeman, and Y. Weiss, "Understanding belief propagation and its generalizations," in Proc. Conf. Uncertainty AI, 2003, pp. 239-269.
21. H. Kappen and W. Wiegerinck, "A novel iteration scheme for the cluster variation method," presented at the Advances in Neural Information Processing Systems, 2002.
22. M. Wainwright, T. Jaakkola, and A. Willsky, "Tree-based reparameterization framework for analysis of sum-product and related algorithms," IEEE Trans. Inf. Theory, vol. 49, no. 5, pp. 1120-1130, May 2003.
23. D. Taylor, "Parallel estimation of one and two dimensional systems," Ph.D. dissertation, Massachusetts Inst. Technol., Cambridge, Feb. 1992.
24. M. Luettgen, W. Karl, A. Willsky, and R. Tenney, "Multiscale representations of Markov random fields," IEEE Trans. Signal Process., vol. 41, no. 12, pp. 3377-3395, Dec. 1993.
25. U. Kjaerulff, "Reduction of computational complexity in Bayesian networks through removal of weak dependences," in Proc. Conf. Uncertainty AI, 1994, vol. 10, pp. 374-384.
26. X. Boyen and D. Koller, "Tractable inference for complex stoch. proc," in Proc. Advances in Neural Information Processing Systems, 1998, vol. 14, pp. 33-42.
27. T. Heskes and O. Zoeter, "Expectation propagation for approximate inference in dynamic Bayesian networks," presented at the Proc. Conf. Uncertainty AI, 2002.
28. T. Speed and H. Kiiveri, "Gaussian Markov distributions over finite graphs," Ann. Statist., vol. 14, no. 1, pp. 138-150, Mar. 1986.
29. D. Karger and N. Srebro, "Learning Markov networks: Maximum bounded treewidth graphs," presented at the 12th ACM-SIAM Symp. Discrete Algorihtms, Jan. 2001.
30. J. Pearl, Probabilistic Inference in Intelligent Systems. San Mateo, CA: Morgan Kaufmann, 1988.
31. Y. Weiss and W. Freeman, "Correctness of belief propagation in Gaussian graphical models," Neural Comput., vol. 13, pp. 2173-2200, 2001.
32. D. Malioutov, J. Johnson, and A. Willsky, "Walk-sums and belief propagation in Gaussian graphical models," J. Mach. Learn. Res., vol. 7, pp. 2031-2064, Oct. 2006.
33. W. Irving and A. Willsky, "A canonical correlations approach to multiscale stochastic realization.," IEEE Trans. Autom. Control, vol. 46, no. 10, pp. 1514-1528, Oct. 2001.
34. A. Frakt and A. Willsky, "A scale-recursive method for constructing multiscale stochastic models," Multidim. Signal Process., vol. 12, pp. 109-142, 2001.
35. S. Amari, "Information geometry on hierarchy of probability distributions," IEEE Trans. Inf. Theory, vol. 47, no. 5, pp. 1701-1711, Jul. 2001.
36. F. Bach and M. Jordan, "Thin junction trees," presented at the Advances in Neural Information Processing Systems, 2001.
37. B. Gidas, "A renormalization group approach to image processing problems," IEEE Trans. Image Process., vol. 11, no. 2, pp. 164-180, Feb. 1989.
38. T. Cover and J. Thomas, Elements of Information Theory. New York: Wiley, 1991.
39. O. Barndorff-Nielsen, Information and Exponential Families. New York: Wiley, 1978.
40. M. Wainwright and M. Jordan, "Graphical Models, Exponential Families, and Variational Inference," Tech. Rep. 649, Dept. Statist., Univ. California, Berkeley, 2003.
41. E. Jaynes, "Information theory and statistical mechanics," Phys. Rev., 1957.
42. A. Dempster, "Covariance selection," Biometrics, vol. 28, no. 1, pp. 157-175, Mar. 1972.
43. A. Frakt, H. Lev-Ari, and A. Willsky, "A generalized levinson algorithm for covariance extension with application to multiscale autoregressive modeling," IEEE Trans. Inf. Theory, vol. 49, no. 2, pp. 411-424, Feb. 2003.
44. D. Geman and C. Yang, "Nonlinear image recovery with half-quadratic regularization," IEEE Trans. Image Process., vol. 4, no. 7, pp. 932-946, Jul. 1995.
45. A. Dempster, N. Laird, and D. Rubin, "Maximum likelihood from incomplete date via the EM algorithm," J. Roy. Statist. Soc., vol. 39, no. 1, pp. 1-38, 1977.