A linear programming approach to max-sum problem: A review

IEEE Transactions on Pattern Analysis and Machine Intelligence

Volumn 29, Issue 7, 2007, Pages 1165-1179

  Werner, Tomáš ^a  

^a CZECH TECHNICAL UNIVERSITY IN PRAGUE   (Czech Republic)

Author keywords

Belief propagation; Constraint satisfaction; Linear programming relaxation; Markov random fields; Max plus; Max product; Max sum; Supermodular optimization; Undirected graphical models

Indexed keywords

BOOLEAN FUNCTIONS; GRAPHIC METHODS; IMAGE ANALYSIS; MARKOV PROCESSES; OPTIMIZATION;

BINARY VARIABLES; LINEAR PROGRAMMING RELAXATION; MARKOV RANDOM FIELD; SUPERMODULAR OPTIMIZATION;

LINEAR PROGRAMMING;

ALGORITHM; ARTIFICIAL INTELLIGENCE; AUTOMATED PATTERN RECOGNITION; COMPUTER ASSISTED DIAGNOSIS; GAME; IMAGE ENHANCEMENT; METHODOLOGY; REVIEW; SYSTEM ANALYSIS; THREE DIMENSIONAL IMAGING;

ALGORITHMS; ARTIFICIAL INTELLIGENCE; GAME THEORY; IMAGE ENHANCEMENT; IMAGE INTERPRETATION, COMPUTER-ASSISTED; IMAGING, THREE-DIMENSIONAL; PATTERN RECOGNITION, AUTOMATED; PROGRAMMING, LINEAR;

EID: 34249652006     PISSN: 01628828     EISSN: None     Source Type: Journal    
DOI: 10.1109/TPAMI.2007.1036     Document Type: Review

References (79)

1. M.I. Schlesinger, "Sintaksicheskiy analiz dvumernykh zritelnikh signalov v usloviyakh pomekh (Syntactic Analysis of Two-Dimensional Visual Signals in Noisy Conditions)," Kibernetika, vol. 4, pp. 113-130, 1976, in Russian.
2. M.L. Minsky and S.A. Papert, Perceptrons: An Introduction to Computational Geometry, second ed. MIT Press, 1988.
3. A. Mackworth, "Constraint Satisfaction," Encyclopedia of Artificial Intelligence, S. Shapiro, ed., pp. 285-292, Wiley, 1991.
4. V.K. Koval and M.I. Schlesinger, "Dvumernoe programmirovanie v zadachakh analiza izobrazheniy (Two-Dimensional Programming in Image Analysis Problems)," USSR Academy of Science, Automatics, and Telemechanics, vol. 8, pp. 149-168, 1976, in Russian.
5. V.A. Kovalevsky, M.I. Schlesinger, and V.K. Koval, "Ustrojstvo dlya analiza seti," Patent No. 576843, USSR, priority of 4 January 1976, 1977, in Russian.
6. V.A. Kovalevsky and V.K. Koval, "A Diffusion Algorithm for Decreasing Energy of Max-Sum Labeling Problem," Glushkov Inst. of Cybernetics, Kiev, USSR, unpublished, 1975.
7. B. Flach, "A Diffusion Algorithm for Decreasing Energy of Max-Sum Labeling Problem," Fakultät Informatik, Technische Universität Dresden, Germany, unpublished, 1998.
8. M.I. Schlesinger, "False Minima of the Algorithm for Minimizing Energy of Max-Sum Labeling Problem," Glushkov Inst. of Cybernetics, Kiev, USSR, unpublished, 1976.
9. M.I. Schlesinger, Matematicheskie sredstva obrabotki izobrazheniy (Mathematical Tools of Image Processing), Naukova Dumka, 1989, in Russian.
10. M.I. Schlesinger and V. Hlaváč, Ten Lectures on Statistical and Structural Pattern Recognition, M.A. Viergever, ed., Kluwer Academic, 2002.
11. M.I. Schlesinger and B. Flach, "Some Solvable Subclasses of Structural Recognition Problems," Proc. Czech Pattern Recognition Workshop, 2000.
12. A. Koster, C.P.M. van Hoesel, and A.W.J. Kolen, "The Partial Constraint Satisfaction Problem: Facets and Lifting Theorems," Operations Research Letters, vol. 23, nos. 3-5, pp. 89-97, 1998.
13. A. Koster, "Frequency Assignment - Models and Algorithms," PhD dissertation, Universiteit Maastricht, Maastricht, The Netherlands, 1999.
14. C. Chekuri, S. Khanna, J. Naor, and L. Zosin, "Approximation Algorithms for the Metric Labeling Problem via a New Linear Programming Formulation," Proc. Symp. Discrete Algorithms, pp. 109-118, 2001.
15. M. Wainwright, T. Jaakkola, and A. Willsky, "MAP Estimation via Agreement on (Hyper)Trees: Message Passing and Linear Programming Approaches," Proc. Allerton Conf. Comm., Control, and Computing, 2002.
16. C.L. Kingsford, B. Chazelle, and M. Singh, "Solving and Analyzing Side-Chain Positioning Problems Using Linear and Integer Programming," Bioinformatics, vol. 21, no. 7, pp. 1028-1039, 2005.
17. E. Boros and P.L. Hammer, "Pseudo-Boolean Optimization," Discrete Applied Math., vol. 123, nos. 1-3, pp. 155-225, 2002.
18. P.L. Hammer, P. Hansen, and B. Simeone, "Roof Duality, Complementation and Persistency in Quadratic 0-1 Optimization," Math. Programming vol. 28, pp. 121-155, 1984.
19. V.N. Kolmogorov and M.J. Wainwright, "On the Optimality of Tree-Reweighted Max-Product Message-Passing," Proc. Conf. Uncertainty in Artificial Intelligence (UAI), 2005.
20. T. Wierschin and S. Fuchs, "Quadratic Minimization for Labeling Problems," technical report, Technical Univ. Dresden, Germany, 2002.
21. P. Ravikumar and J. Lafferty, "Quadratic Programming Relaxations for Metric Labeling and Markov Random Field MAP Estimation," Proc. Int'l Conf. Machine Learning (ICML), 2006.
22. M.J. Wainwright and M.I. Jordan, "Semidefinite Relaxations for Approximate Inference on Graphs with Cycles," Proc. Conf. Neural Information Processing Systems (NIPS), 2003.
23. D. Schlesinger and B. Flach, "Transforming an Arbitary MinSum Problem into a Binary One," Technical Report TUD-FI06-01, Dresden Univ. of Technology, Germany, Apr. 2006.
24. M.J. Wainwright, T. Jaakkola, and A.S. Willsky, "A New Class of Upper Bounds on the Log Partition Function," IEEE Trans. Information Theory, vol. 51, no. 7, pp. 2313-2335, 2005.
25. M. Wainwright, T. Jaakkola, and A. Willsky, "MAP Estimation via Agreement on (Hyper)Trees: Message Passing and Linear Programming Approaches," IEEE Trans. Information Theory, vol. 51, no. 11, pp. 3697-3717, 2005.
26. M. Wainwright, T. Jaakkola, and A. Willsky, "Tree-Based Reparameterization Framework for Analysis of Sum-Product and Related Algorithms," IEEE Trans. Information Theory, vol. 49, no. 5, pp. 1120-1146, 2003.
27. M. Wainwright, T. Jaakkola, and A. Willsky, "Tree Consistency and Bounds on the Performance of the Max-Product Algorithm and Its Generalizations," Statistics and Computing, vol. 14, pp. 143-166, 2004.
28. V. Kolmogorov, "Convergent Tree-Reweighted Message Passing for Energy Minimization," Technical Report MSR-TR-2004-90, Microsoft Research, 2004.
29. V. Kolmogorov, "Convergent Tree-Reweighted Message Passing for Energy Minimization," Technical Report MSR-TR-2005-38, Microsoft Research, 2005.
30. V. Kolmogorov, "Convergent Tree-Reweighted Message Passing for Energy Minimization," Proc. Int'l Workshop Artificial Intelligence and Statistics (AISTATS), 2005.
31. V. Kolmogorov, "Convergent Tree-Reweighted Message Passing for Energy Minimization," IEEE Trans. Pattern Analysis and Machine Intelligence vol. 28, no. 10, pp. 1568-1583, Oct. 2006.
32. V. Kolmogorov and M. Wainwright, "On the Optimality of Tree-Reweighted Max-Product Message-Passing," Technical Report MSR-TR-2004-37, Microsoft Research, 2005.
33. J. Pearl, Probabilistic Reasoning in Intelligent Systems: Networks of Plausible Inference. Morgan Kaufmann, 1988.
34. J.S. Yedidia, W.T. Freeman, and Y. Weiss, "Constructing Free-Energy Approximations and Generalized Belief Propagation Algorithms," IEEE Trans. Information Theory, vol. 51, no. 7, pp. 2282-2312, 2005.
35. D. Cohen, M. Cooper, P. Jeavons, and A. Krokhin, "Supermodular Functions and the Complexity of Max CSP," Discrete Applied Math., vol. 149, nos. 1-3, pp. 53-72, 2005.
36. S. Bistarelli, U. Montanari, F. Rossi, T. Schiex, G. Verfaillie, and H. Fargier, "Semiring-Based CSPs and Valued CSPs: Frameworks, Properties, and Comparison," Constraints, vol. 4, no. 3, pp. 199-240, 1999.
37. V. Kolmogorov and R. Zabih, "What Energy Functions Can Be Minimized via Graph Cuts?" Proc. European Conf. Computer Vision (ECCV , pp. 65-81, 2002.
38. D. Schlesinger, "Strukturelle Ansätze für die Stereorekonstruktion," PhD dissertation, Technische Universität Dresden, Fakultät Informatik, Institut für Künstliche Intelligenz, July 2005, in German.
39. B. Flach, "Strukturelle Bilderkennung," technical report, Fakultät Informatik, Technische Universität Dresden, Germany, 2002 habilitation thesis, in German.
40. I. Kovtun, "Partial Optimal Labelling Search for a NP-Hard Subclass of (max,+) Problems," Proc. Conf. German Assoc. for Pattern Recognition (DAGM), pp. 402-409, 2003.
41. I. Kovtun, "Segmentaciya zobrazhen na usnovi dostatnikh umov optimalnosti v NP-povnikh klasakh zadach strukturnoi rozmitki (Image Segmentation Based on Sufficient Conditions of Optimality in NP-Complete Classes of Structural Labeling Problems)," PhD dissertation, IRTC ITS Nat'l Academy of Science Ukraine, Kiev, 2004, in Ukrainian.
42. T. Werner, "A Linear Programming Approach to Max-Sum Problem: A Review," Technical Report CTU-CMP-2005-25, Center for Machine Perception, Czech Technical Univ., Dec. 2005.
43. S. Verdú and H.V. Poor, "Abstract Dynamic Programming Models under Commutativity Conditions," SIAM J. Control and Optimization, vol. 25, no. 4, pp. 990-1006, July 1987.
44. S. Bistarelli, U. Montanari, and F. Rossi, "Semiring-Based Constraint Satisfaction and Optimization," J. ACM, vol. 44, no. 2, pp. 201-236, 1997.
45. S. Gaubert, "Methods and Applications of (max,+) Linear Algebra," Technical Report 3088, Institut National de Recherche en Informatique et en Automatique (INRIA), 1997.
46. S.M. Aji and R.J. McEliece, "The Generalized Distributive Law," IEEE Trans. Information Theory, vol. 46, no. 2, pp. 325-343, 2000.
47. D.L. Waltz, "Generating Semantic Descriptions from Drawings of Scenes with Shadows," technical report, Massachusetts Inst. of Technology, Cambridge, Mass., 1972.
48. U. Montanari, "Networks of Constraints: Fundamental Properties and Application to Picture Processing," Information Science, vol. 7, pp. 95-132, 1974.
49. A. Rosenfeld, R.A. Hummel, and S.W. Zucker, "Scene Labeling by Relaxation Operations," IEEE Trans. Systems, Man, and Cybernetics vol. 6, no. 6, pp. 420-433, June 1976.
50. A.K. Mackworth, "Consistency in Networks of Relations," Artificial Intelligence, vol. 8, no. 1, pp. 65-73, 1977.
51. R.M. Haralick and L.G. Shapiro, "The Consistent Labeling Problem," IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 1, no. 2, pp. 173-184, 1979.
52. A. Bulatov, P. Jeavons, and A. Krokhin, "Classifying the Complexity of Constraints Using Finite Algebras," Computing, vol. 34, no. 3, pp. 720-742, 2005.
53. R. Debruyne and C. Bessière, "Domain Filtering Consistencies," J. Artificial Intelligence Research, no. 14, pp. 205-230, May 2001.
54. M.I. Schlesinger, "Lectures on Labeling Problems Attended by the Author of this Article in Kiev, Prague, Dresden," 1996-2006.
55. D.M. Topkis, "Minimizing a Submodular Function on a Lattice," Operations Research, vol. 26, no. 2, pp. 305-321, 1978.
56. M.I. Schlesinger and V. Kovalevsky, "A Hydraulic Model of a Linear Programming Relaxation of Max-Sum Labeling Problem," Glushkov Inst. of Cybernetics, Kiev, USSR, unpublished, 1978.
57. R.J. Vanderbei, Linear Programming: Foundations and Extensions. Kluwer Academic, 1996.
58. M.L. Balinski, "Integer Programming: Methods, Uses, Computation," Management Science, vol. 12, no. 3, pp. 253-313, 1965.
59. M.I. Schlesinger, personal communication, Int'l Research and Training Centre, Kiev, Ukraine, 2000-2005.
60. R.E. Burkard, B. Klinz, and R. Rudolf, "Perspectives of Monge Properties in Optimization," Discrete Applied Math., vol. 70, no. 2, pp. 95-161, 1996.
61. L. Lovász, "Submodular Functions and Convexity," Math. Programming - The State of the Art, A. Bachem, M. Grötschel, and B. Korte, eds., pp. 235-257, Springer-Verlag, 1983.
62. P.L. Hammer, "Some Network Flow Problems Solved with Pseudo-Boolean Programming," Operations Research, vol. 13, pp. 388-399, 1965.
63. D. Greig, B. Porteous, and A. Seheult, "Exact Maximum A Posteriori Estimation for Binary Images," J. Royal Statistical Soc. B, no. 51, pp. 271-279, 1989.
64. H. Ishikawa and D. Geiger, "Segmentation by Grouping Junctions," Proc. IEEE Computer Vision and Pattern Recognition Conf. (CVPR), pp. 125-131, 1998.
65. H. Ishikawa, "Exact Optimization for Markov Random Fields with Convex Priors," IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 25, no. 10, pp. 1333-1336, Oct. 2003.
66. D. Cohen, M. Cooper, P. Jeavons, and A. Krokhin, "A Maximal Tractable Class of Soft Constraints," J. Artificial Intelligence Research, vol. 22, pp. 1-22, 2004.
67. M. Grötschel, L. Lovász, and A. Schrijver, "The Ellipsoid Method and Its Consequences in Combinatorial Optimization," Combinatorica, vol. 1, no. 2, pp. 169-197, 1981.
68. M. Grötschel, L. Lovász, and A. Schrijver, Geometric Algorithms and Combinatorial Optimization, second ed. Springer Verlag, 1993.
69. A. Schrijver, "A Combinatorial Algorithm Minimizing Submodular Functions in Strongly Polynomial Time," Combinatorial Theory, Series B, vol. 80, no. 2, pp. 346-355, 2000.
70. S. Iwata, L. Fleischer, and S. Fujishige, "A Combinatorial Strongly Polynomial-Time Algorithm for Minimizing Submodular Functions," J. ACM, vol. 48, pp. 761-777, 2001.
71. B.A. Davey and H.A. Priestley, Introduction to Lattices and Order. Cambridge Univ. Press, 1990.
72. T. Meltzer, C. Yanover, and Y. Weiss, "Globally Optimal Solutions for Energy Minimization in Stereo Vision Using Reweighted Belief Propagation," Proc. Int'l Conf. Computer Vision (ICCV), pp. 428-435, June 2005.
73. R. Szeliski, R. Zabih, D. Scharstein, O. Veksler, V. Kolmogorov, A. Agarwala, M. Tappen, and C. Rother, "A Comparative Study of Energy Minimization Methods for Markov Random Fields," Proc. European Conf. Computer Vision (ECCV), pp. II:16-29, 2006.
74. Y. Boykov, O. Veksler, and R. Zabih, "Fast Approximate Energy Minimization via Graph Cuts," IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 23, no. 11, pp. 1222-1239, Nov. 2001.
75. N. Komodakis and G. Tziritas, "A New Framework for Approximate Labeling via Graph Cuts," Proc. Int'l Conf. Computer Vision (ICCV), pp. 1018-1025, 2005.
76. V. Kolmogorov and C. Rother, "Comparison of Energy Minimization Algorithms for Highly Connected Graphs," Proc. European Conf. Computer Vision (ECCV), pp. II:1-15, 2006.
77. C. Yanover, T. Meltzer, and Y. Weiss, "Linear Programming Relaxations and Belief Propagation: An Empirical Study," J. Machine Learning Research, vol. 7, pp. 1887-1907, Sept. 2006.
78. T. Werner and A. Shekhovtsov, "Unified Framework for Semiring-Based Arc Consistency and Relaxation Labeling," Proc. 12th Computer Vision Winter Workshop, M. Grabner and H. Grabner, eds., pp. 27-34, Feb. 2007.
79. T. Werner, "What Is Decreased by the Max-Sum Arc Consistency Algorithm?" Proc. Int'l Conf. Machine Learning, June 2007.