Finding non-overlapping clusters for generalized inference over graphical models

IEEE Transactions on Signal Processing

Volumn 60, Issue 12, 2012, Pages 6368-6381

  Vats, Divyanshu ^a   Moura, José M F ^b  

^a UNIVERSITY OF MINNESOTA   (United States)

^b CARNEGIE MELLON UNIVERSITY   (United States)

Author keywords

Approximate inference; belief propagation; block graphs; block trees; generalized belief propagation; graphical models; Markov random fields

Indexed keywords

APPROXIMATE INFERENCE; BELIEF PROPAGATION; BLOCK-GRAPHS; BLOCK-TREES; GENERALIZED BELIEF PROPAGATION; GRAPHICAL MODEL; MARKOV RANDOM FIELDS;

CLUSTERING ALGORITHMS; ESTIMATION; GRAPHIC METHODS; PROBABILITY DISTRIBUTIONS; TREES (MATHEMATICS);

INFERENCE ENGINES;

EID: 84870486301     PISSN: 1053587X     EISSN: None     Source Type: Journal    
DOI: 10.1109/TSP.2012.2214216     Document Type: Article

References (57)

1. H. Rue and L. Held, Gaussian Markov Random Fields: Theory and Applicatioris (Monographs on Statistics and Applied Probability), lst ed. London, U.K.: Chapman & Hall/CRC, Feb. 2005.
2. M. J. Wainwright and M. I. Jordan, GraphicaI Models, ExponentiaI Families, and Variational Inference. Hanover, MA: Now Publishers, 2008.
3. D. Koller and N. Friedman, Probabilistic GraphicaI Models: Princi- ples and Techniques. Cambridge, MA: MIT Press, 2009.
4. S. L. Lauritzen and D. J. Spiegelhalter, "Local computations with probabilities on graphical stractures and their application to expert systems," J. Roy. Stat. Soc. Series B(Methodol.), vol. 50, no. 2, pp. 157-224, 1988.
5. E. Fabre and A. Guyader, "Dealing with short cycles in graphical codes," mProc IEEEInt. Symp. Inf. Theory (ISIT), Jun. 2000, p. 10. (Pubitemid 32393409)
6. E. P. Xing, M. I. Jordan, and S. Russell, "Graph partition strategies for generalized mean fieId inference," in Proc. 20th Conf. Uncertainty in Artif. IntelI. (UAI), Arlington, VA, 2004, pp. 602-610 [Online]. Avail- able:http://portal.acm.org/citation.cfm?id=1036843.1036916
7. J. Pearl, Probabilistic Reasoning in Intelligent Systems: Networks of Plausible Inference. San Mateo, CA: Morgan Kaufmann, 1988.
8. F. Eaton and Z. Ghahramani, "Choosing a variable to clamp: Approx- imate inference using conditioned beliefpropagation," Proc. 12fth Int. Conf.Artif Intell. Stat., 2009, pp. 145-152.
9. A. Montanari and T. Rizzo, "Fiow to compute loop corrections to the Betheapproximation,"J. Stat.Mechan., TheoryandExper.,vol. 2005, no. 10,p. P10011,2005.
10. J. M. Mooij and H. J. Kappen, "Loop corrections for approximate in- ferenceonfactorgraphs," J.Mach. Learn. Res., vol. 8,pp. 1113-1143, May 2007.
11. R. Mateescu, K. Kask, V. Gogate, and R. Dechter, "Join-graph propagation algorithms," J. Artif IntelI. Res., vol. 37, pp. 279-328, 2010.
12. J. S. Yedidia, W. T. Freeman, and Y. Weiss, "Generalized beliefprop- agation," inNIPS, 2001, pp. 689-695.
13. T. Heskes, K. Albers, and B. Kappen, "Approximate inference and con- strained optimization," in UAI, 2003, pp. 313-320.
14. J. S. Yedidia, W. T. Freeman, and Y. Weiss, "Constructing free energy approximations and generalized belief propagation algorithms," IEEE Trans. Inf. Theory, vol. 51, no. 7, pp. 2282-2312, Jul. 2005. (Pubitemid 41136394)
15. A. Pelizzola, "Cluster variation method in statistical physics and prob- abilistic graphical models,"J. Phys. A, Math. Giniral, vol. 38, no. 33, pp. R309-R339, 2005. (Pubitemid 41149047)
16. M. Jordan, Z. Ghahramani, T. Jaakkola, and L. Saul, "An introduction to variational methods for graphical models," Mach. Learn., vol. 37, no. 2, pp. 183-233, Nov. 1999. (Pubitemid 30544678)
17. J. Yedidia, W. Freeman, and Y. Weiss, "Bethe free energy, Kikuchi ap- proximations, and belief propagation algorithms," Mitsubishi Electric Rese. Labs., Tech. Rep. TR2001-16, 2001.
18. R. Kikuchi, "Atheory of cooperative phenomena,"Phys. Rev.,vol. 81, no. 6,pp.988-1003, 1951.
19. M. Welling, "On the choice of regions for generalized belief prop- agation," in Proc. 20th Conf. Uncertainty in Artif Intell., 2004, pp. 585-592.
20. M. Welling, T. Minka, and Y. W. Teh, "Structuredregion graphs: Mor- phing EP into GBP," ìnProc Int. Conf. Uncertainty in Artif Intell., 2005, vol. 21.
21. E. P. Xing, M. I. Jordan, and S. Russell, "A generalized mean fieId algorithm for variational inference in exponential families," in Proc. 20th Conf. Uncertainty in Artif Intell. (UAI), Arlington, VA, 2003, pp. 602-610 [Online]. Available: http://portal.acm.org/cita- tion.cfm?id=1036843.1036916
22. B. M. Marlin and K. P. Murphy, "Sparse Gaussian graphical modeIs with unknown block structure,"inProc. Int. Conf.Mach.Learn., 2009, pp. 89-712.
23. A. Jalali, P. Ravikumar, V. Vasuki, and S. Sanghavi, "On learning dis- crete graphical modeIs using group-sparse regularization," ìnProc. Int. Conf.Mach. Learn., 2011, pp. 89-712.
24. D. Malioutov, "Approximate inference in Gaussian graphical models," Ph.D. dissertation, Electrical Eng. and Comput. Sci. Dept., Massachu- setts Ins. ofTechnol., Cambridge, 2008.
25. P. Lévy, "A special problem of Brownian motion, and a general theory of Gaussian random functions," in Proc. 3rd Berkeley Symp. Math. Stat Probab., 1954-1955, Berkeley and Los Angeles, 1956, vol. II, pp. 133-175.
26. D. Vats and J.M.F. Moura, "Telescoping recursive representations and estimation of Gauss-Markov random fields," Trans. Inf. Theory, vol. 57, no. 3, pp. 1645-1663, 2011.
27. R. E. Kalman and R. Bucy, "New results in linear filtering and predic- tion theory," Trans. ASME\-J. Basic Eng., vol. 83, no. Series D, pp. 95-108, 1960.
28. H. E. Rauch, F. Tung, and C. T. Stribel, "Maximum likelihood estimates of linear dynamical systems," AIAA J., vol. 3, no. 8, pp. 1445-1450, Aug. 1965.
29. S. L. Lauritzen, Graphical Models. New York: OxfordUniv. Press, 1996.
30. J. Besag, "Spatial interaction and the statistical analysis of lattice systems," J. Roy. Sta. Soc., Series B (Methodol.), vol. 36, no. 2, pp. 192-236, 1974.
31. R. Halin, "S-functions for graphs,"J. Geometry, vol. 8, pp. 171-186, 1976, 10.1007/BF01917434.
32. N. Robertson and P. D. Seymour, "Graph minors. ∏. algorithmic as- pects oftree-width," J. Algorithms, vol. 7, no. 3, pp. 309-322, 1986.
33. F. Jensen and F. Jensen, "Optimaljunctiontrees," ìnProc 10thAnnu. Conf. Uncertainty inArtif IntelI. (UAI), San Francisco, CA, 1994, pp. 360-36, Morgan Kaufmann.
34. H. M. Markowitz, "The elimination form of the inverse and its applica- tionto linear programming,"Manage Sci., vol. 3, no. 3, pp. 255-269, 1957.
35. A. Berry, P. Heggernes, and G. Simonet, H. Bodlaender, Ed., "The minimum degree heuristic and the minimal triangulation process," in Graph-Theoretic Concepts in Computer Science, ser. Lecture Notes in Computer Science. Berlin, Germany: Springer, 2003, vol. 2880, pp. 58-70.
36. U. B. Kjaerulff, "Triangulation of graphs\-Algorithms giving small total state space," Math. Comput. Sci. Dept., Aalborg Univ., Aalborg, Denmark, Tech. Rep. Res. Rep. R-90-09, 1990.
37. G. F. Cooper, "Nestor: A computer-based medical diagnostic aid that integrates causal and probabilistic knowledge," Ph.D. dissertation, Comput. Sci. Dept., StanfordUniv., Stanford, CA, 1984.
38. Y. Peng and J. A. Reggia, "Plausibility ofdiagnostic hypotheses," in Proc.Nat Conf. Artif IntelI. (AAAI), 1986,pp. 140-145.
39. J. W. Woods and C. Radewan, "Kalman filtering in two dimensions," IEEE Trans. Inf. Theory, vol. 23, no. 4, pp. 473-182, Jul. 1977.
40. J. M. F. Moura and N. Balram, "Recursive structure of noncausal Gauss-Markov random fields," IEEE Trans. Inf. Theory, vol. IT-38, no. 2, pp. 334-354, Mar. 1992.
41. B. C. Levy, M. B. Adams, and A. S. Willsky, "Solution and linear es- timation of 2-D nearest-neighbor models,"Proc. IEEE, vol. 78, no. 4, pp. 627-641, Apr. 1990.
42. G. Shafer and P. P. Shenoy, "Probability propagation," Ann. Math. Artif Intell., no. 1-4, pp. 327-352, 1990.
43. N. Zhang and D.Poole, "A simple approach to Bayesian network com- putations,"inProc. 10thCan. ConfArtifIntelI, 1994,pp. 171-178.
44. R. Dechter, "Bucketelimination: Aunifying frame work for reasoning," ArtifIntelI, vol. 113, no. l-2, pp. 41-85, Sep. 1999.
45. K. Kask, R. Dechter, J. Larrosa, and F. Cozman, "Bucket-tree elimina- tion for automated reasoning," Univ. of California, Irvine, Tech. Rep. R92, 2001.
46. M. Kolar, L. Song, A. Ahmed, and E. P. Xing, "Estimating time-varying networks," Ann. AppI. Stat., 2009.
47. S. Zhou, J. Lafferty, and L. Wasserman, "Time varying undirected graphs," Mach. Learn., vol. 80, pp. 295-319, 2010, 10.1007/sl0994- 010-5180-0.
48. F. Kschischang, B. Frey, and H.-A. Loeliger, "Factor graphs and the sum-product algorithm," IEEE Trans. Inf. Theory, vol. 47, no. 2, pp. 498-519, Feb. 2001. (Pubitemid 32318087)
49. T. Minka and Y. Qi, "Tree-structured approximations by expectation propagation," in Proc. NeuraI Inf. Process. Syst. Conf. (NIPS), 2003, p. 2003.
50. J. M. Mooij, "libDAI: A free and open source C++ library for discrete approximate inference in graphical models,"J. Mach. Learn. Res., vol. 11, pp. 2169-2173, Aug. 2010.
51. V. Gogate, "Iterative joint graph propagation," [Online]. Available: http://www.hlt.utdallas.edu/~vgogate/?gp.html
52. M. J. Wainwright, T. S. Jaakkola, and A. S. Willsky, "A new class of upper bounds on the log partition function," IEEE Trans. Inf. Theory, vol. 51, no. 7, pp. 2313-2335, Jul. 2005. (Pubitemid 41136395)
53. Q. Liu and A. Ihler, L. Getoor and T. Scheffer, Eds., "Bounding the partition function using Holder's inequality," ìnProc 28th Int. Conf. Mach. Learn., Jun. 2011, pp. 849-856.
54. M. J. Wainwright, T. S. Jaakkola, and A. S. Willsky, "MAP estima- tion via agreement on (hyper)trees: Message-passing and linear pro- gramming approaches," IEEE Trans. Inf. Theory, vol. 51, no. 11, pp. 3697-3717, Nov. 2005. (Pubitemid 41622574)
55. V. Kolmogorov, "Convergenttree-reweighted message passing for en- ergy minimization," IEEE Trans. Pattern AnaI. Mach. Intell., vol. 28, pp. 1568-1583, 2006. (Pubitemid 46405078)
56. D. Sontag and T. Jaakkola, "Tree block coordinate descent for MAP in graphical models,"J. Mach. Learn. Res., vol. 5, pp. 544-551, 2009.
57. T. Jebara, "MAP estimation, message passing, and perfect graphs," in Proc. 25th Conf. Uncertainty inArtif. InteI. (UAI), 2009, pp. 258-267.