Approximate inference techniques with expectation constraints

Journal of Statistical Mechanics: Theory and Experiment

Volumn , Issue 11, 2005, Pages 277-300

  Heskes, Tom ^a   Opper, Manfred ^b   Wiegerinck, Wim ^a   Winther, Ole ^c   Zoeter, Onno ^a  

^a RADBOUD UNIVERSITY NIJMEGEN   (Netherlands)

^b UNIVERSITY OF SOUTHAMPTON   (United Kingdom)

^c TECHNICAL UNIVERSITY OF DENMARK   (Denmark)

Author keywords

Analysis of algorithms; Message passing algorithms

Indexed keywords

EID: 29044438682     PISSN: 17425468     EISSN: None     Source Type: Journal    
DOI: 10.1088/1742-5468/2005/11/P11015     Document Type: Article

References (29)

1. Heskes T 2003 Stable fixed points of loopy belief propagation are minima of the Bethe free energy Advances in Neural Information Processing Systems vol 15 ed S Becker, S Thrun and K Obermayer (Cambridge, MA: MIT Press) pp 359-66
2. Heskes T, Albers K and Kappen H J 2003 Approximate inference and constrained optimization Proc. 19th Ann. Conf. on Uncertainty in AI (UAI-03) (San Francisco, CA: Morgan Kaufmann) pp 313-20
3. Heskes T and Zoeter O 2002 Expectation propagation for approximate inference in dynamic Bayesian networks UAI 2002: Proc. 18th Ann. Conf. on Uncertainty in Artificial Intelligence (San Francisco, CA: Morgan Kaufmann)
4. Ikeda S, Tanaka T and Amari S 2000 Stochastic reasoning, free energy, and information geometry Neural Comput. 16 1779-810
5. Jordan M I, Ghahramani Z, Jaakkola T S and Saul L K 1999 An introduction to variational methods for graphical models Mach. Learn. 37 183-233
6. Kappen H J and Rodríguez F B 1999 Boltzmann machine learning using mean field theory and linear response correction Advances in Neural Information Processing Systems vol 11 ed M S Kearns, S A Solla and D A Cohn (Cambridge, MA: MIT Press) pp 280-6
7. Kschischang F R, Frey B J and Loeliger H-A 2001 Factor graphs and the sum-product algorithm IEEE Trans. Inf. Theory 47 498-519
8. Lauritzen S L 1992 Propagation of probabilities, means, and variances in mixed graphical association models J. Am. Stat. Assoc. 87 1098-108
9. Mézard M, Parisi G and Virasoro M A 1987 Spin Glass Theory and Beyond (Lecture Notes in Physics vol 9) (Singapore: World Scientific)
10. Minka T 2001 The EP energy function and minimization schemes Technical Report
11. Minka T and Qi Y 2004 Tree-structured approximations by expectation propagation Advances in Neural Information Processing Systems vol 16 ed S Thrun et al (Cambridge, MA: MIT Press)
12. Minka T P 2001 A family of algorithms for approximate Bayesian inference PhD Thesis (MIT Media Lab)
13. Murphy K P, Weiss Y and Jordan M I 1999 Loopy belief propagation for approximate inference: an empirical study Proc. 15th Ann. Conf. on Uncertainty in AI (UAI-99) (San Francisco, CA: Morgan Kaufmann) pp 467-75
14. Opper M and Winther O 2001 Adaptive and self-averaging Thouless-Anderson-Palmer mean field theory for probabilistic modeling Phys. Rev. E 64 056131
15. Opper M and Winther O 2001 Tractable approximations for probabilistic models: the adaptive Thouless-Anderson-Palmer mean field approach Phys. Rev. Lett. 86 3695
16. Opper M and Winther O 2005 Expectation consistent free energies for approximate inference Advances in Neural Information Processing Systems vol 17 ed L K Saul et al (Cambridge, MA: MIT Press) pp 1001-8
17. Parisi G and Potters M 1995 Mean-field equations for spin models with orthogonal interaction matrices J. Phys. A: Math. Gen. 28 5267
18. Pearl J 1988 Probabilistic Reasoning in Intelligent Systems (San Francisco, CA: Morgan Kaufmann)
19. Saul L K, Jaakkola T and Jordan M I 1996 Mean field theory for sigmoid belief networks J. Artif. Intell. Res. 4 61-76
20. Teh Y and Welling M 2002 The unified propagation and scaling algorithm Advances in Neural Information Processing Systems vol 14 ed T G Dietterich et al (Cambridge, MA: MIT Press) pp 953-60
21. Thouless D J, Anderson P W and Palmer R G 1977 Solution of a 'solvable model of a spin glass' Phil. Mag. 35 593
22. Wainwright M J and Jordan M I 2003 Graphical models, exponential families, and variational inference Technical Report Department of Statistics, UC Berkeley
23. Weiss Y and Freeman W T 2001 Correctness of belief propagation in Gaussian graphical models of arbitrary topology Neural Comput. 13 2173-200
24. Welling M and Teh Y W 2001 Belief optimization for binary networks: a stable alternative to loopy belief propagation Proc. 17th Ann. Conf. on Uncertainty in Artificial Intelligence (UAI-01) (San Francisco, CA: Morgan Kaufmann) pp 554-61
25. Welling M, Teh Y W and Minka T 2005 Structured region graphs: morphing EP into GBP Proc. 21st Ann. Conf. on Uncertainty in AI (Corvallis, Oregon: AUAI Press)
26. Wiegerinck W 2000 Variational approximations between mean field theory and the junction tree algorithm Proc. 16th Ann. Conf. on Uncertainty in AI (UAI-00) (San Francisco, CA: Morgan Kaufmann) pp 626-33
27. Yedidia J S, Freeman W T and Weiss Y 2001 Generalized belief propagation Advances in Neural Information Processing Systems vol 13 ed T K Leen, T G Dietterich and V Tresp (Cambridge, MA: MIT Press) pp 689-95
28. Yuille A 2002 CCCP algorithms to minimize the Bethe and Kikuchi free energies: convergent alternatives to belief propagation Neural Comput. 14 1691-722
29. Zoeter O and Heskes T 2005 Changepoint problems in linear dynamical systems Technical Report SNN, Radboud University Nijmegen