MRF energy minimization and beyond via dual decomposition

IEEE Transactions on Pattern Analysis and Machine Intelligence

Volumn 33, Issue 3, 2011, Pages 531-552

  Komodakis, Nikos ^a   Paragios, Nikos ^b   Tziritas, Georgios ^a  

^a UNIVERSITY OF CRETE   (Greece)

^b ECOLE CENTRALE PARIS   (France)

Author keywords

Discrete optimization; graph cuts; graphical models; linear programming; Markov random fields; message passing

Indexed keywords

CAN DESIGN; DISCRETE OPTIMIZATION; DUAL DECOMPOSITION; ENERGY MINIMIZATION; GRAPH-CUT; GRAPH-CUTS; GRAPHICAL MODELS; INFERENCE TECHNIQUES; MARKOV RANDOM FIELDS; OPTIMIZATION ALGORITHMS; OPTIMIZATION PROBLEMS; SPECIAL STRUCTURE; SUB-PROBLEMS; SUBGRADIENT; SYNTHETIC AND REAL DATA; THEORETICAL FRAMEWORK;

ALGORITHMS; COMPUTER VISION; GRAPHIC METHODS; INFERENCE ENGINES; MESSAGE PASSING; OPTIMIZATION; STRUCTURAL FRAMES;

LINEAR PROGRAMMING;

ALGORITHM; ARTICLE; ARTIFICIAL INTELLIGENCE; AUTOMATED PATTERN RECOGNITION; COMPUTER ASSISTED DIAGNOSIS; IMAGE ENHANCEMENT; INSTRUMENTATION; METHODOLOGY; MOTION; PROBABILITY; REPRODUCIBILITY; SYSTEM ANALYSIS; THREE DIMENSIONAL IMAGING;

ALGORITHMS; ARTIFICIAL INTELLIGENCE; IMAGE ENHANCEMENT; IMAGE INTERPRETATION, COMPUTER-ASSISTED; IMAGING, THREE-DIMENSIONAL; MARKOV CHAINS; MOTION; PATTERN RECOGNITION, AUTOMATED; PROGRAMMING, LINEAR; REPRODUCIBILITY OF RESULTS;

EID: 79551518880     PISSN: 01628828     EISSN: None     Source Type: Journal    
DOI: 10.1109/TPAMI.2010.108     Document Type: Article

References (57)

1. F. Barahona and R. Anbil, "The Volume Algorithm: Producing Primal Solutions with a Subgradient Method," Math. Programming, vol. 87, pp. 385-399, 2000.
2. D. Bertsekas, Nonlinear Programming. Athena Scientific, 1999.
3. D.P. Bertsekas and S.K. Mitter, "A Descent Numerical Method for Optimization Problems with Nondifferentiable Cost Functionals," SIAM J. Control, vol. 11, pp. 637-652, 1973.
4. Y. Boykov, O. Veksler, and R. Zabih, "Fast Approximate Energy Minimization via Graph Cuts," IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 23, no. 11, pp. 1222-1239, Nov. 2001.
5. A. Braunstein, M. Mezard, and R. Zecchina, "Survey Propagation: An Algorithm for Satisfiability," Random Structures and Algorithms, vol. 27, no. 2, pp. 201-226, 2005.
6. P. Camerini, L. Fratta, and F. Maffioli, "On Improving Relaxation Methods by Modifying Gradient Techniques," Math. Programming Study, vol. 3, pp. 26-34, 1975.
7. C. Chekuri, S. Khanna, J. Naor, and L. Zosin, "Approximation Algorithms for the Metric Labeling Problem via a New Linear Programming Formulation," Proc. 12th Ann. ACM-SIAM Symp. Discrete Algorithms, pp. 109-118, 2001.
8. J. Duchi, D. Tarlow, G. Elidan, and D. Koller, "Using Combinatorial Optimization within Max-Product Belief," Advances in Neural Information Processing Systems, MIT Press, 2006.
9. G. Elidan, I. McGraw, and D. Koller, "Residual Belief Propagation: Informed Scheduling for Asynchronous Message Passing," Proc. 22nd Conf. Uncertainty in AI, 2006.
10. P. Felzenszwalb and D. Huttenlocher, "Pictorial Structures for Object Recognition," Int'l J. Computer Vision, vol. 61, pp. 55-79, 2005.
11. W.T. Freeman, E.C. Pasztor, and O.T. Carmichael, "Learning Low-Level Vision," Int'l J. Computer Vision, vol. 40, no. 1, pp. 25-47, 2000.
12. B. Frey, Graphical Models for Machine Learning and Digital Communication. MIT Press, 1998.
13. B. Frey and D. MacKay, "A Revolution: Belief Propagation in Graphs with Cycles," Advances in Neural Information Processing Systems, MIT Press, 1998.
14. A. Globerson and T. Jaakkola, "Fixing Max-Product: Convergent Message Passing Algorithms for Map LP-Relaxations," Advances in Neural Information Processing Systems, pp. 553-560, MIT Press, 2008.
15. T. Heskes, "Stable Fixed Points of Loopy Belief Propagation Are Minima of the Bethe Free Energy," Advances in Neural Information Processing Systems, pp. 359-366, MIT Press, 2003.
16. T. Heskes, "Convexity Arguments for Efficient Minimization of the Bethe and Kikuchi Free Energies," J. Artificial Intelligence Research, vol. 26, pp. 153-190, 2006.
17. J. Johnson, D. Malioutov, and A. Willsky, "Lagrangian Relaxation for MAP Estimation in Graphical Models," Proc. Allerton Conf. Comm., Control, and Computing, 2007.
18. M. Jordan, Learning in Graphical Models. MIT Press, 1999.
19. J. Kim, V. Kolmogorov, and R. Zabih, "Visual Correspondence Using Energy Minimization and Mutual Information," Proc. IEEE Int'l Conf. Computer Vision, 2003.
20. P. Kohli and P. Torr, "Dynamic Graph Cuts for Efficient Inference in Markov Random Fields," IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 29, no. 12, pp. 2079-2088, Dec. 2007.
21. V. Kolmogorov, "Convergent Tree-Reweighted Message Passing for Energy Minimization," IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 28, no. 10, pp. 1568-1583, Oct. 2006.
22. V. Kolmogorov and M. Wainwright, "On the Optimality of Tree-Reweighted Max-Product Message Passing," Proc. Conf. Uncertainty in Artificial Intelligence, 2005.
23. N. Komodakis and N. Paragios, "Beyond Loose lp-Relaxations: Optimizing MRFs by Repairing Cycles," Proc. European Conf. Computer Vision, 2008.
24. N. Komodakis, N. Paragios, and G. Tziritas, "MRF Optimization via Dual Decomposition: Message-Passing Revisited," Proc. IEEE Int'l Conf. Computer Vision, 2007.
25. N. Komodakis and G. Tziritas, "Image Completion Using Global Optimization," Proc. IEEE Conf. Computer Vision and Pattern Recognition, 2006.
26. N. Komodakis, G. Tziritas, and N. Paragios, "Fast, Approximately Optimal Solutions for Single and Dynamic MRFs," Proc. IEEE Conf. Computer Vision and Pattern Recognition, 2007.
27. M. Kumar, P. Torr, and A. Zisserman, "Obj Cut," Proc. IEEE Conf. Computer Vision and Pattern Recognition, 2005.
28. M.P. Kumar, V. Kolmogorov, and P.H.S. Torr, "An Analysis of Convex Relaxations for MAP Estimation," Advances in Neural Information Processing Systems, MIT Press, 2007.
29. V. Kwatra, A. Schodl, I. Essa, G. Turk, and A. Bobick, "Graphcut Textures: Image and Video Synthesis Using Graph Cuts," Proc. ACM SIGGRAPH, 2003.
30. T. Larsson, M. Patriksson, and A. Stromberg, "Ergodic Primal Convergence in Dual Subgradient Schemes for Convex Programming," Math. Programming, vol. 86, pp. 283-312, 1999.
31. T. Meltzer, C. Yanover, and Y. Weiss, "Globally Optimal Solutions for Energy Minimization in Stereo Vision Using Reweighted Belief Propagation," Proc. IEEE Int'l Conf. Computer Vision, 2005.
32. A. Nedic and D.P. Bertsekas, "Incremental Subgradient Methods for Nondifferentiable Optimization," SIAM J. Optimization, vol. 12, pp. 109-138, 2001.
33. A. Nedic and A. Ozdaglar, "Approximate Primal Solutions and Rate Analysis for Dual Subgradient Methods," SIAM J. Optimization, vol. 19, pp. 1757-1780, 2009.
34. J. Pearl, Probabilistic Reasoning in Intelligent Systems: Networks of Plausible Inference. Morgan Kaufmann, 1988.
35. A. Raj, G. Singh, and R. Zabih, "MRF's for MRI's: Bayesian Reconstruction of MR Images via Graph Cuts," Proc. IEEE Conf. Computer Vision and Pattern Recognition, 2006.
36. P. Ravikumar, A. Agarwal, and M. Wainwright, "Message-Passing for Graph-Structured Linear Programs: Proximal Projections, Convergence and Rounding Schemes," Proc. Int'l Conf. Machine Learning, pp. 800-807, 2008.
37. S. Roth and M. Black, "Fields of Experts: A Framework for Learning Image Priors," Proc. IEEE Conf. Computer Vision and Pattern Recognition, 2005.
38. C. Rother, V. Kolmogorov, V.S. Lempitsky, and M. Szummer, "Optimizing Binary mrfs via Extended Roof Duality," Proc. IEEE Conf. Computer Vision and Pattern Recognition, 2007.
39. M. Schlesinger and V. Giginyak, "Solution to Structural Recognition (MAX,+)-Problems by Their Equivalent Transformations," Control Systems and Computers, vol. 1, pp. 3-15, 2007.
40. M.I. Schlesinger, "Syntactic Analysis of Two-Dimensional Visual Signals in Noisy Conditions," Kybernetica, vol. 4, pp. 113-130, 1976.
41. H. Sherali and G. Choi, "Recovery of Primal Solutions When Using Subgradient Optimization Methods to Solve Lagrangian Duals of Linear Programs," Operations Research Letters, vol. 19, pp. 105-113, 1996.
42. N. Shor, Minimization Methods for Nondifferentiable Functions. Springer, 1985.
43. D. Sontag and T. Jaakkola, "New Outer Bounds on the Marginal Polytope," Advances in Neural Information Processing Systems, MIT Press, 2008.
44. D. Sontag, T. Meltzer, A. Globerson, Y. Weiss, and T. Jaakkola, "Tightening lp Relaxations for Map Using Message Passing," Proc. Conf. Uncertainty in Artificial Intelligence, 2008.
45. E.B. Sudderth, A.T. Ihler, W.T. Freeman, and A.S. Willsky, "Nonparametric Belief Propagation," Proc. IEEE Conf. Computer Vision and Pattern Recognition, 2003.
46. R. Szeliski, R. Zabih, D. Scharstein, O. Veksler, V. Kolmogorov, A. Agarwala, M. Tappen, and C. Rother, "A Comparative Study of Energy Minimization Methods for Markov Random Fields with Smoothness-Based Priors," IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 30, no. 6, pp. 1068-1080, June 2008.
47. M.F. Tappen and W.T. Freeman, "Comparison of Graph Cuts with Belief Propagation for Stereo, Using Identical mrf Parameters," Proc. IEEE Int'l Conf. Computer Vision, pp. 900-907, 2003.
48. L. Torresani, V. Kolmogorov, and C. Rother, "Feature Correspondence via Graph Matching: Models and Global Optimization," Proc. European Conf. Computer Vision, 2008.
49. M. Wainwright, T. Jaakkola, and A. Willsky, "Map Estimation via Agreement on Trees: Message-Passing and Linear Programming," IEEE Trans. Information Theory, vol. 51, no. 11, pp. 3697-3717, 2005.
50. Y. Weiss and W.T. Freeman, "Correctness of Belief Propagation in Gaussian Graphical Models of Arbitrary Topology," Neural Computation, vol. 13, no. 10, pp. 2173-2200, 2001.
51. Y. Weiss and W.T. Freeman, "On the Optimality of Solutions of the Max-Product Belief-Propagation Algorithm in Arbitrary Graphs," IEEE Trans. Information Theory, vol. 47, no. 2, pp. 736-744, 2001.
52. Y. Weiss, C. Yanover, and T. Meltzer, "MAP Estimation, Linear Programming and Belief Propagation with Convex Free Energies," Proc. 22nd Conf. Uncertainty in AI, 2007.
53. T. Werner, "A Linear Programming Approach to Max-Sum Problem: A Review," IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 29, no. 7, pp. 1165-1179, July 2007.
54. T. Werner, "High-Arity Interactions, Polyhedral Relaxations, and Cutting Plane Algorithm for Soft Constraint Optimisation (Map-mrf)," Proc. IEEE Conf. Computer Vision and Pattern Recognition, 2008.
55. W. Wiegerinck and T. Heskes, "Fractional Belief Propagation," Advances in Neural Information Processing Systems, pp. 438-445, MIT Press, 2003.
56. C. Yanover, T. Meltzer, and Y. Weiss, "Linear Programming Relaxations and Belief Propagation-An Empirical Study," J. Machine Learning Research, vol. 7, pp. 1887-1907, 2006.
57. J. Yedidia, W. Freeman, and Y. Weiss, "Constructing Free Energy Approximations and Generalized Belief Propagation Algorithms," IEEE Trans. Information Theory, vol. 51, no. 7, pp. 2282-2312, July 2005.