Estimating the "wrong" graphical model: Benefits in the computation-limited setting

Journal of Machine Learning Research

Volumn 7, Issue , 2006, Pages 1829-1859

  Wainwright, Martin J ^a  

^a UNIVERSITY OF CALIFORNIA   (United States)

Author keywords

Algorithmic stability; Belief propagation; Graphical model; Markov random field; Parameter estimation; Prediction error; Variational method

Indexed keywords

APPROXIMATION THEORY; COMPUTATION THEORY; MARKOV PROCESSES; MATHEMATICAL MODELS; PARAMETER ESTIMATION; PROBLEM SOLVING; VARIATIONAL TECHNIQUES;

ALGORITHMIC STABILITY; BELIEF PROPAGATION; GRAPHICAL MODEL; MARKOV RANDOM FIELDS; PREDICTION ERROR;

GRAPH THEORY;

EID: 33748685798     PISSN: 15337928     EISSN: 15337928     Source Type: Journal    
DOI: None     Document Type: Article

References (42)

1. A. Benveniste, M. Metivier, and P. Priouret. Adaptive Algorithms and Stochastic Approximations. Springer-Verlag, New York, NY, 1990.
2. D.P. Bertsekas. Nonlinear programming. Athena Scientific, Belmont, MA, 1995.
3. J. Besag. Statistical analysis of non-lattice data. The Statistician, 24(3):179-195, 1975.
4. J. Besag. Efficiency of pseudolikelihood estimation for simple Gaussian fields. Biometrika, 64(3): 616-618, 1977.
5. L.D. Brown. Fundamentals of statistical exponential families. Institute of Mathematical Statistics, Hayward, CA, 1986.
6. M.S. Grouse, R.D. Nowak, and R.G. Baraniuk. Wavelet-based statistical signal processing using hidden Markov models. IEEE Trans. Signal Processing, 46:886-902, April 1998.
7. M. Deza and M. Laurent. Geometry of Cuts and Metric Embeddings. Springer-Verlag, New York, 1997.
8. W. T. Freeman, E. C. Pasztor, and O. T. Carmichael. Learning low-level vision. Intl. J. Computer Vision, 40(1):25-47, 2000.
9. W. T. Freeman and Y. Weiss. On the optimality of solutions of the max-product belief propagation algorithm in arbitrary graphs. IEEE Trans. Info. Theory, 47:736-744, 2001.
10. T. Heskes, K. Albers, and B. Kappen. Approximate inference and constrained optimization. In Uncertainty in Artificial Intelligence, volume 13, pages 313-320, July 2003.
11. A. Ihler, J. Fisher, and A. S. Willsky. Loopy belief propagation: Convergence and effects of message errors. Journal of Machine Learning Research, 6:905-936, May 2005.
12. S. L. Lauritzen. Graphical Models. Oxford University Press, Oxford, 1996.
13. M. A. R. Leisink and H. J. Kappen. Learning in higher order Boltzmann machines using linear response. Neural Networks, 13:329-335, 2000.
14. J. S. Liu. Monte Carlo strategies in Scientific Computing. Springer-Verlag, New York, NY, 2001.
15. T. P. Minka. A family of algorithms for approximate Bayesian inference. PhD thesis, MIT, January 2001.
16. J. M. Mooij and H. J. Kappen. On the properties of the Bethe approximation and loopy belief propagation on binary networks. Journal of Statistical Mechanics: Theory and Experiment, P11012: 407-432, 2005a.
17. J. M. Mooij and H. J. Kappen. Sufficient conditions for convergence of loopy belief propagation. Technical Report arxiv:cs.IT:0504030, University of Nijmegen, April 2005b.
18. Submitted to IEEE Trans. Info. Theory.
19. S. Della Pietra, V. Della Pietra, and J. Lafferty. Inducing features of random fields. IEEE Trans. on Pattern Analysis and Machine Intelligence, 19(4):380-393, 1997.
20. T. Richardson and R. Urbanke. The capacity of low-density parity check codes under message-passing decoding. IEEE Trans. Info. Theory, 47:599-618, February 2001.
21. B. D. Ripley. Spatial statistics. Wiley, New York, 1981.
22. C. P. Robert and G. Casella. Monte Carlo statistical methods. Springer-Verlag, New York, NY, 1999.
23. G. Rockafellar. Convex Analysis. Princeton University Press, Princeton, 1970.
24. M. Ross and L. P. Kaebling. Learning static object segmentation from motion segmentation. In 20th National Conference on Artificial Intelligence, 2005.
25. P. Rusmevichientong and B. Van Roy. An analysis of turbo decoding with Gaussian densities. In NIPS 12, pages 575-581. MIT Press, 2000.
26. R. J. Serfling. Approximation Theorems of Mathematical Statistics. Wiley Series in Probability and Statistics. Wiley, 1980.
27. C. Sutton and A. McCallum. Piecewise training of undirected models. In Uncertainty in Artificial Intelligence, July 2005.
28. S. Tatikonda. Convergence of the sum-product algorithm. In Information Theory Workshop, April 2003.
29. S. Tatikonda and M. I. Jordan. Loopy belief propagation and Gibbs measures. In Proc. Uncertainty in Artificial Intelligence, volume 18, pages 493-500, August 2002.
30. Y. W. Teh and M. Welling. On improving the efficiency of the iterative proportional fitting procedure. In Workshop on Artificial Intelligence and Statistics, 2003.
31. D. M. Titterington, A.F.M. Smith, and U.E. Makov. Statistical analysis of finite mixture distributions. Wiley, New York, 1986.
32. M. J. Wainwright, T. S. Jaakkola, and A. S. Willsky. Tree-based reparameterization framework for analysis of sum-product and related algorithms. IEEE Trans. Info. Theory, 49(5):1120-1146, May 2003a.
33. M. J. Wainwright, T. S. Jaakkola, and A. S. Willsky. Tree-reweighted belief propagation algorithms and approximate ML estimation by pseudomoment matching. In Workshop on Artificial Intelligence and Statistics, January 2003b.
34. M. J. Wainwright, T. S. Jaakkola, and A. S. Willsky. A new class of upper bounds on the log partition function. IEEE Trans. Info. Theory, 51(7):2313-2335, July 2005.
35. M. J. Wainwright and M. I. Jordan. Graphical models, exponential families, and variational inference. Technical report, UC Berkeley, Department of Statistics, No. 649, September 2003.
36. M. J. Wainwright and M. I. Jordan. A variational principle for graphical models. In New Directions in Statistical Signal Processing. MIT Press, Cambridge, MA, 2005.
37. M. J. Wainwright and M. I. Jordan. Log-determinant relaxation for approximate inference in discrete Markov random fields. IEEE Trans. Signal Processing, 54(6):2099-2109, June 2006.
38. Y. Weiss. Correctness of local probability propagation in graphical models with loops. Neural Computation, 12:1-41, 2000.
39. W. Wiegerinck. Approximations with reweighted generalized belief propagation. In Workshop on Artificial Intelligence and Statistics, January 2005.
40. J. Yedidia. An idiosyncratic journey beyond mean field theory. In M. Opper and D. Saad, editors, Advanced mean field methods: Theory and practice, pages 21-36. MIT Press, 2001.
41. J.S. Yedidia, W. T. Freeman, and Y. Weiss. Constructing free energy approximations and generalized belief propagation algorithms. IEEE Trans. Info. Theory, 51(7):2282-2312, July 2005.
42. L. Younes. Estimation and annealing for Gibbsian fields. Ann. Inst. Henri Poincare, 24(2):269-294, 1988.