Message-passing for graph-structured linear programs: Proximal methods and rounding schemes

Journal of Machine Learning Research

Volumn 11, Issue , 2010, Pages 1043-1080

  Ravikumar, Pradeep ^a   Agarwal, Alekh ^b   Wainwright, Martin J ^a,c  

^a UNIVERSITY OF CALIFORNIA   (United States)

^b UNIVERSITY OF CALIFORNIA   (United States)

^c NONE   (United States)

Author keywords

Graphical models; Lp relaxation; Map estimation; Proximal minimization; Rounding schemes

Indexed keywords

BREGMAN DIVERGENCES; BREGMAN PROJECTION; COMPUTATIONAL CHALLENGES; CONVERGENT ALGORITHMS; DOUBLE-LOOP; GRAPHICAL MODEL; GRAPHICAL STRUCTURES; INNER LOOPS; LINEAR PROGRAMMING RELAXATION; LINEAR PROGRAMS; LP RELAXATION; MAP ESTIMATION; MARKOV RANDOM FIELDS; MAXIMUM A POSTERIORI; OUTER LOOP; PROXIMAL METHODS; RANDOMIZED ROUNDING; REPARAMETERIZATION; ROUNDING SCHEMES; SOLVING ALGORITHM; TREE-BASED;

ALGORITHMS; GRAPHIC METHODS; LINEARIZATION; MESSAGE PASSING; OPTIMIZATION;

CONVERGENCE OF NUMERICAL METHODS;

EID: 77951167674     PISSN: 15324435     EISSN: 15337928     Source Type: Journal    
DOI: None     Document Type: Article

References (48)

1. H.H.BauschkeandJ.M.Borwein. Legendrefunctionsandthemethodofrandombregmanprojections.JournalofConvexAnalysis, 4(1):27-67,1997.(Pubitemid127714611)
2. M. Bayati, D. Shah, and M. Sharma. Maximum weight matching for max-product belief propagation. In Int. Symp. Info. Theory, Adelaide, Australia, September 2005.
3. M. Bayati, C. Borgs, J. Chayes, and R. Zecchina. Belief-propagation for weighted b-matchings on arbitrary graphs and its relation to linear programs with integer solutions. Technical Report arxiv:0709.1190, Microsoft Research, September 2007.
4. U. Bertele and F. Brioschi. Nonserial Dynamic Programming. Academic Press, New York, 1972.
5. D. P. Bertsekas and J. N. Tsitsiklis. Parallel and Distributed Computation: Numerical Methods. Athena Scientific, Boston, MA, 1997.
6. D.P. Bertsekas. Nonlinear Programming. Athena Scientific, Belmont, MA, 1995.
7. D. Bertsimas and J. Tsitsikilis. Introduction to Linear Optimization. Athena Scientific, Belmont, MA, 1997.
8. J. Besag. On the statistical analysis of dirty pictures. Journal of the Royal Statistical Society, Series B, 48(3):259-279, 1986.
9. S. Boyd and L. Vandenberghe. Convex Optimization. Cambridge University Press, Cambridge, UK, 2004.
10. Y. Boykov, O. Veksler, and R. Zabih. Fast approximate energy minimization via graph cuts. IEEE Trans. Pattern Anal. Mach. Intell., 23(11):1222-1239, 2001.
11. Y Censor and S. A. Zenios. Parallel Optimization - Theory, Algorithms and Applications. Oxford University Press, 1997.
12. C. Chekuri, S. Khanna, J. Naor, and L. Zosin. A linear programming formulation and approximation algorithms for the metric labeling problem. SIAM Journal on Discrete Mathematics, 18(3):608-625, 2005.
13. F. Deutsch and H. Hundal. The rate of convergence for the cyclic projection algorithm I: Angles between convex sets. Journal of Approximation Theory, 142:36 55, 2006.
14. R. L. Dykstra. An iterative procedure for obtaining i-projections onto the intersection of convex sets. Annals of Probability, 13(3):975-984, 1985.
15. J. Feldman, D. R. Karger, and M. J. Wainwright. Linear programming-based decoding of turbo-like codes and its relation to iterative approaches. In Proc. 40th Annual Allerton Conf. on Communication, Control, and Computing, October 2002a.
16. J. Feldman, M. J. Wainwright, and D. R. Karger. Linear programming-based decoding of turbolike codes and its relation to iterative approaches. In Proc. Allerton Conf. Comm., Control and Computing, October 2002b.
17. W. T. Freeman and Y. Weiss. On the optimality of solutions of the max-product belief propagation algorithm in arbitrary graphs. IEEE Trans. Info. Theory, 47:736-744, 2001.
18. S. Geman and D. Geman. Stochastic relaxation, Gibbs distributions, and the Bayesian restoration of images. IEEE Trans. PAMI, 6:721-741, 1984.
19. A. Globerson and T. Jaakkola. Convergent propagation algorithms via oriented trees. In Proc. Uncertainty in Artificial Intelligence, Vancouver, Canada, July 2007a.
20. A. Globerson and T. Jaakkola. Fixing max-product: Convergent message passing algorithms for MAP LP-relaxations. In Neural Information Processing Systems, Vancouver, Canada, December 2007b.
21. D.M. Greig, B.T. Porteous, and A.H. Seheult. Exact maximum a posteriori estimation for binary images. Journal of the Royal Statistical Society, Series B, 51, 1989.
22. S.-P. Han. A successive projection method. Math. Programming, 40:114, 1988.
23. T. Hazan and A. Shashua. Convergent message-passing algorithms for inference over general graphs with convex free energy. In The 24th Conference on Uncertainty in Artificial Intelligence (UAI), Helsinki, 2008.
24. B. Huang and T. Jebara. Loopy belief propagation for bipartite maximum weight b-matching. In AISTATS, San Juan, Puerto Rico, March 2007.
25. A. N. Iusem and M. Teboulle. Convergence rate analysis of nonquadratic proximal methods for convex and linear programming. Mathematics of Operations Research, 20(3):657-677, 1995.
26. J. K. Johnson, D. M. Malioutov, and A. S. Willsky. Lagrangian relaxation for MAP estimation in graphical models. In Proc. 45th Allerton Conf. Comm. Cont. Comp., Urbana-Champaign, IL, September 2007.
27. J. Kleinberg and E. Tardos. Approximation algorithms for classification problems with pairwise relationships: Metric labeling and Markov random fields. In IEEE Symp. Found. Comp. Science, pages 14-24, 1999.
28. V. Kolmogorov. Convergent tree-reweighted message-passing for energy minimization. In International Workshop on Artificial Intelligence and Statistics, January 2005.
29. V. Kolmogorov. Convergent tree-reweighted message-passing for energy minimization. IEEE Trans. PAMI, 28(10): 1568-1583, October 2006.
30. V. Kolmogorov and M. J. Wainwright. On optimality properties of tree-reweighted message-passing. In Uncertainty in Artificial Intelligence, July 2005.
31. N. Komodakis, N. Paragios, and G. Tziritas. MRF optimization via dual decomposition: Messagepassing revisited. In ICCV, Rio di Janeiro, Brazil, 2007.
32. V. K. Koval and M. I. Schlesinger. Two-dimensional programming in image analysis problems. USSR Academy of Science, Automatics and Telemechanics, 8:149-168, 1976. In Russian.
33. P. Kumar, P. H. S. Torr, and A. Zisserman. Solving markov random fields using second order cone programming. IEEE Conference of Computer Vision and Pattern Recognition, 2006.
34. P. Raghavan and C. D. Thompson. Randomized rounding: A technique for provably good algorithms and algorithmic proofs. Combinatorica, 7(4):365-374, 1987.
35. P. Ravikumar and J. Lafferty. Quadratic programming relaxations for metric labeling and markov random field map estimation. In ICML '06: Proceedings of the 23rd international conference on Machine learning, pages 737-744, 2006. (Pubitemid 46966912)
36. G. Rockafellar. Convex Analysis. Princeton University Press, Princeton, 1970.
37. S. Sanghavi, D. Shah, and A. Willsky. Message-passing for max-weight independent set. In Neural Information Processing Systems, Vancouver, Canada, December 2007.
38. M. V. Solodov and B.F. Svaiter. A unified framework for some inexact proximal point algorithms. Numerical Functional Analysis and Optimization, 22:1013-1035, 2001.
39. P. Tseng and D. P. Bertsekas. On the convergence of the exponential multiplier method for convex programming. Math. Programming, 60:1-19, 1993.
40. M. J. Wainwright and M. I. Jordan. Graphical models, exponential families, and variational inference. Technical report, UC Berkeley, Department of Statistics, No.649, September 2003.
41. M. J. Wainwright and M. I. Jordan. Treewidth-based conditions for exactness of the Sherali-Adams and Lasserre relaxations. Technical report, UC Berkeley, Department of Statistics, No.671, September 2004.
42. M. J. Wainwright, T. S. Jaakkola, and A. S. Willsky. A new class of upper bounds on the log partition function. In Uncertainty in Artificial Intelligence, volume 18, August 2002.
43. M. J. Wainwright, T. S. Jaakkola, and A. S. Willsky. Tree-based reparameterization framework for analysis of sum-product and related algorithms. IEEE Trans. Info. Theory, 49(5): 1120-1146, May 2003.
44. M. J. Wainwright, T. S. Jaakkola, and A. S. Willsky. Tree consistency and bounds on the maxproduct algorithm and its generalizations. Statistics and Computing, 14:143-166, April 2004.
45. M. J. Wainwright, T. S. Jaakkola, and A. S. Willsky. Exact MAP estimates via agreement on (hyper)trees: Linear programming and message-passing. IEEE Tram. Information Theory, 51 (11):3697-3717, November 2005.
46. Y. Weiss, C. Yanover, and T. Meltzer. Map estimation, linear programming and belief propagation with convex free energies. In Uncertainty in Artificial Intelligence, 2007.
47. C. Yanover, T. Meltzer, and Y. Weiss. Linear programming relaxations and belief propagation: An empirical study. Journal of Machine Learning Research, 7:1887-1907, September 2006.
48. J. S. Yedidia, W. T. Freeman, and Y. Weiss. Constructing free energy approximations and generalized belief propagation algorithms. IEEE Trans. Info. Theory, 51(7):2282-2312, July 2005.