Loop corrections for approximate inference on factor graphs

Journal of Machine Learning Research

Volumn 8, Issue , 2007, Pages 1113-1143

  Mooij, Joris M ^a   Kappen, Hilbert J ^a  

^a RADBOUD UNIVERSITY NIJMEGEN   (Netherlands)

Author keywords

Approximate inference; Belief propagation; Factor graphs; Graphical models; Loop corrections

Indexed keywords

APPROXIMATION ALGORITHMS; ERROR ANALYSIS; INFERENCE ENGINES; MARKOV PROCESSES; MATHEMATICAL MODELS;

APPROXIMATE INFERENCES; BELIEF PROPAGATION; FACTOR GRAPHS; GRAPHICAL MODELS; LOOP CORRECTION (LC) METHODS;

GRAPH THEORY;

EID: 34249073686     PISSN: 15324435     EISSN: 15337928     Source Type: Journal    
DOI: None     Document Type: Article

References (27)

1. H. Bethe. Statistical theory of superlattices. Proc. R. Soc. A, 150:552-575, 1935.
2. M. Chertkov and V. Y. Chernyak. Loop calculus helps to improve belief propagation and linear programming decodings of low-density-parity-check codes. arXiv.org preprint, arXiv:cs/0609154v1 [cs.IT], 2006a. URL http://arxiv.org/abs/cs/0609154v1.
3. M. Chertkov and V. Y. Chernyak. Loop series for discrete statistical models on graphs. Journal of Statistical Mechanics: Theory and Experiment, 2006(06):P06009, 2006b. URL http://stacks.iop.org/1742-5468/ 2006/P06009.
4. R. Dechter, K. Kask, and R. Mateescu. Iterative join-graph propagation. In Proceedings of the 18th Annual Conference on Uncertainty in Artificial Intelligence (UAI-02), pages 128-13, San Francisco, CA, 2002. Morgan Kaufmann.
5. G. Elidan, I. McGraw, and D. Koller. Residual belief propagation: Informed scheduling for asynchronous message passing. In Proceedings of the 22nd Annual Conference on Uncertainty in Artificial Intelligence (UAI-06), Boston, Massachussetts, July 2006.
6. V. Gómez, J. M. Mooij, and H. J. Kappen. Truncating the loop series expansion for belief propagation. Journal of Machine Learning Research, forthcoming. URL http://arxiv.org/abs/cs/ 0612030v1.
7. T. Heskes, C. A. Albers, and H. J. Kappen. Approximate inference and constrained optimization. In Proceedings of the 19th Annual Conference on Uncertainty in Artificial Intelligence (UAI-03), pages 313-320, San Francisco, CA, 2003. Morgan Kaufmann Publishers.
8. T. Jaakkola and M. I. Jordan. Variational probabilistic inference and the QMR-DT network. Journal of Artificial Intelligence Research, 10:291-322, 1999. URL http://www. jair.org/papers/ paper583.html.
9. R. Kikuchi. A theory of cooperative phenomena. Phys. Rev., 81:988-1003, 1951.
10. F. R. Kschischang, B. J. Frey, and H.-A. Loeliger. Factor graphs and the sum-product algorithm. IEEE Trans. Inform. Theory, 47(2):498-519, February 2001.
11. M. Mézard, G. Parisi, and M. A. Virasoro. Spin Glass Theory and Beyond. World Scientific, Singapore, 1987.
12. T. Minka. Expectation propagation for approximate Bayesian inference. In Proceedings of the 17th Annual Conference on Uncertainty in Artificial Intelligence (UAI-01), pages 362-369, San Francisco, CA, 2001. Morgan Kaufmann Publishers.
13. T. Minka and Y. Qi. Tree-structured approximations by expectation propagation. In Sebastian Thrun, Lawrence Saul, and Bernhard Schölkopf, editors, Advances in Neural Information Processing Systems 16, Cambridge, MA, 2004. MIT Press.
14. A. Montanari and T. Rizzo. How to compute loop corrections to the Bethe approximation. Journal of Statistical Mechanics: Theory and Experiment, 2005(10):P10011, 2005. URL http://stacks.iop.org/1742-5468/2005/P10011.
15. J. M. Mooij and H. J. Kappen. On the properties of the Bethe approximation and loopy belief propagation on binary networks. Journal of Statistical Mechanics: Theory and Experiment, 2005 (11):P11012, 2005. URL http://stacks.iop.org/1742-5468/2005/P11012.
16. J. M. Mooij and H. J. Kappen. Loop corrections for approximate inference. arXiv.org preprint, arXiv:cs/0612030v1 [cs.AI], 2006. URL http://arxiv.org/abs/cs/0612030v1.
17. J. M. Mooij, B. Wemmenhove, H. J. Kappen, and T. Rizzo. Loop corrected belief propagation. In Proceedings of the Eleventh International Conference on Artificial Intelligence and Statistics (AISTATS-07), 2007.
18. G. Parisi. Statistical Field Theory. Addison-Wesley, Redwood City, Ca, 1988.
19. G. Parisi and F. Slanina. Loop expansion around the Bethe-Peierls approximation for lattice models. Journal of Statistical Mechanics: Theory and Experiment, 2006(02):L02003, 2006. URL http://stacks.iop.org/1742-5468/ 2006/L02003.
20. J. Pearl. Probabilistic Reasoning in Intelligent systems: Networks of Plausible Inference. Morgan Kaufmann, San Francisco, CA, 1988.
21. A. Pelizzola. Cluster variation method in statistical physics and probabilistic graphical models. J. Phys. A: Math. Gen., 38:R309-R339, August 2005.
22. M. A. Shwe, B. Middleton, D. E. Heckerman, M. Henrion, E. J. Horvitz, H. P. Lehmann, and G. F. Cooper. Probabilistic diagnosis using a reformulation of the INTERNIST-1/QMR knowledge base. I. The probabilistic model and inference algorithms. Methods of information in Medicine, 30(4):241-255, October 1991.
23. M. Takikawa and B. D'Ambrosio. Multiplicative factorization of noisy-max. In Proceedings of the 15th Annual Conference on Uncertainty in Artificial Intelligence (UAI-99), pages 622-63, San Francisco, CA, 1999. Morgan Kaufmann.
24. M. Welling, T. Minka, and Y. W. Teh. Structured region graphs: Morphing EP into GBP. In Proceedings of the 21st Annual Conference on Uncertainty in Artificial Intelligence (UAI-05), page 609, Arlington, Virginia, 2005. AUAI Press.
25. W. Wiegerinck, H. J. Kappen, E. W. M. T. ter Braak, W. J. P. P. ter Burg, M. J. Nijman, Y. L. O, and J. P. Neijt. Approximate inference for medical diagnosis. Pattern Recognition Letters, 20: 1231-1239, 1999.
26. J. S. Yedidia, W. T. Freeman, and Y. Weiss. Constructing free-energy approximations and generalized belief propagation algorithms. IEEE Transactions on Information Theory, 51(7):2282-2312, July 2005.
27. A. L. Yuille. CCCP algorithms to minimize the Bethe and Kikuchi free energies: convergent alternatives to belief propagation. Neural Computation, 14(7): 1691-1722, 2002.