Bregman monotone optimization algorithms

SIAM Journal on Control and Optimization

Volumn 42, Issue 2, 2003, Pages 596-636

  Bauschke, Heinz H ^a   Borwein, Jonathan M ^b   Combettes, Patrick L ^c  

^a UNIVERSITY OF GUELPH   (Canada)

^b SIMON FRASER UNIVERSITY   (Canada)

^c UNIVERSITÉ PIERRE ET MARIE CURIE   (France)

Author keywords

class operator; Banach space; Block iterative method; Bregman distance; Bregman monotone; Bregman projection; Convex feasibility problem; Essentially smooth function; Essentially strict convex function; Fej r monotone; Legendre function

Indexed keywords

BANACH SPACES; BLOCK-ITERATIVE METHOD; BREGMAN DISTANCE; BREGMAN MONOTONE; BREGMAN PROJECTION; CONVEX FEASIBILITY PROBLEMS; ESSENTIALLY SMOOTH FUNCTIONS; ESSENTIALLY STRICT CONVEX FUNCTIONS; LEGENDRE FUNCTIONS; MONOTONE OPERATORS;

ALGORITHMS; FUNCTIONS; HARMONIC ANALYSIS; ITERATIVE METHODS; MATHEMATICAL MODELS; MATHEMATICAL OPERATORS; OPTIMIZATION; SET THEORY;

CONTROL SYSTEMS;

EID: 1942509685     PISSN: 03630129     EISSN: None     Source Type: Journal    
DOI: 10.1137/S0363012902407120     Document Type: Article

References (86)

1. Y. ALBER AND D. BUTNARIU, Convergence of Bregman projection methods for solving consistent convex feasibility problems in reflexive Banach spaces, J. Optim. Theory Appl., 92 (1997), pp. 33-61.
2. H. ATTOUCH AND H. BRÉZIS, Duality for the sum of convex functions in general Banach spaces, in Aspects of Mathematics and Its Applications, North-Holland, Amsterdam, 1986, pp. 125-133.
3. J.-P. AUBIN, Viability Theory, Birkhäuser, Boston, 1991.
4. J.-B. BAILLON AND G. HADDAD, Quelques propriétés des opérateurs angle-bornés et ncycliquement monotones, Israel J. Math., 26 (1977), pp. 137-150.
5. H. H. BAUSCHKE, Projection Algorithms and Monotone Operators, Ph.D. thesis, Simon Fraser University, Canada, 1996; available online as preprint 96:080 from http://www.cecm.sfu.ca/preprints.
6. H. H. BAUSCHKE AND J. M. BORWEIN, On projection algorithms for solving convex feasibility problems, SIAM Rev., 38 (1996), pp. 367-426.
7. H. H. BAUSCHKE AND J. M. BORWEIN, Legendre functions and the method of random Bregman projections, J. Convex Anal., 4 (1997), pp. 27-67.
8. H. H. BAUSCHKE, J. M. BORWEIN, AND P. L. COMBETTES, Essential smoothness, essential strict convexity, and Legendre functions in Banach spaces, Commun. Contemp. Math., 3 (2001), pp. 615-647.
9. H. H. BAUSCHKE AND P. L. COMBETTES, A weak-to-strong convergence principle for Fejérmonotone methods in Hilbert spaces, Math. Oper. Res., 26 (2001), pp. 248-264.
10. H. H. BAUSCHKE AND P. L. COMBETTES, Construction of best Bregman approximations in reflexive Banach spaces, Proc. Amer. Math. Soc., published electronically April 24, 2003.
11. H. H. BAUSCHKE AND P. L. COMBETTES, Iterating Bregman retractions, SIAM J. Optim., 13 (2003), pp. 1159-1173.
12. H. H. BAUSCHKE, P. L. COMBETTES, AND D. R. LUKE, Phase retrieval, error reduction algorithm, and Fienup variants: A view from convex optimization, J. Opt. Soc. Amer. A, 19 (2002), pp. 1334-1345.
13. J. M. BORWEIN AND J. D. VANDERWERFF, Convex functions of Legendre type in general Banach spaces, J. Convex Anal., 8 (2001), pp. 569-581.
14. L. M. BREGMAN, The method of successive projection for finding a common point of convex sets, Soviet Math. Dokl., 6 (1965), pp. 688-692.
15. L. M. BREGMAN, The relaxation method of finding the common point of convex sets and its application to the solution of problems in convex programming, USSR Comput. Math. Math. Phys., 7 (1967), pp. 200-217.
16. H. BRÉZIS AND A. HARAUX, Image d'une somme d'opérateurs monotones et applications, Israel J. Math., 23 (1976), pp. 165-186.
17. H. BRÉZIS AND P. L. LIONS, Produits infinis de résolvantes, Israel J. Math., 29 (1978), pp. 329-345.
18. M. BROHE AND P. TOSSINGS, Perturbed proximal point algorithm with nonquadratic kernel, Serdica Math. J., 26 (2000), pp. 177-206.
19. R. E. BRUCK AND S. REICH, Nonexpansive projections and resolvents of accretive operators in Danach spaces, Houston J. Math., 3 (1977), pp. 459-470.
20. R. S. BURACHIK AND A. N. IUSEM, A generalized proximal point algorithm for the variational inequality problem in a Hilbert space, SIAM J. Optim., 8 (1998), pp. 197-216.
21. R. S. BURACHIK AND S. SCHEIMBERG, A proximal point method for the variational inequality problem in Banach spaces, SIAM J. Control Optim., 39 (2001), pp. 1633-1649.
22. D. BUTNARIU, C. BYRNE, AND Y. CENSOR, Redundant axioms in the definition of Bregman functions, J. Convex Anal., to appear in (2003).
23. D. BUTNARIU AND A. N. IUSEM, On a proximal point method for convex optimization in Banach spaces, Numer. Funct. Anal. Optim., 18 (1997), pp. 723-744.
24. D. BUTNARIU AND A. N. IUSEM, Totally Convex Functions for Fixed Points Computation and Infinite Dimensional Optimization, Kluwer Academic Publishers, Boston, MA, 2000.
25. D. BUTNARIU, A. N. IUSEM, AND C. ZǍLINESCU, On uniform convexity, total convexity and convergence of the proximal point and outer Bregman projection algorithms in Banach spaces, J. Convex Anal., to appear in (2003).
26. D. BUTNARIU, S. REICH, AND A. J. ZASLAVSKI, Asymptotic behavior of relatively nonexpansive operators in Banach spaces, J. Appl. Anal., 7 (2001), pp. 151-174.
27. D. BUTNARIU AND E. RESMERITA, The outer Bregman projection method for stochastic feasibility problems in Banach spaces, in Inherently Parallel Algorithms for Feasibility and Optimization, D. Butnariu, Y. Censor, and S. Reich, eds., Elsevier, New York, 2001, pp. 69-86.
28. J. A. CADZOW, Signal enhancement - A composite property mapping algorithm, IEEE Trans. Acoust. Speech Signal Process., 36 (1988), pp. 49-62.
29. Y. CENSOR AND G. T. HERMAN, Block-iterative algorithms with underrelaxed Bregman projections, SIAM J. Optim., 13 (2002), pp. 283-297.
30. Y. CENSOR AND A. LENT, An iterative row-action method for interval convex programming, J. Optim. Theory Appl., 34 (1981), pp. 321-353.
31. Y. CENSOR AND S. REICH, Iterations of paracontractions and firmly nonexpansive operators with applications to feasibility and optimization, Optimization, 37 (1996), pp. 323-339.
32. Y. CENSOR AND S. A. ZENIOS, Proximal minimization algorithm with D-functions, J. Optim. Theory Appl., 73 (1992), pp. 451-464.
33. Y. CENSOR AND S. A. ZENIOS, Parallel Optimization: Theory, Algorithms, and Applications, Oxford University Press, New York, 1997.
34. I. CIORǍNESCU, Geometry of Banach Spaces, Duality Mappings, and Nonlinear Problems, Kluwer Academic Publishers, Boston, MA, 1990.
35. P. L. COMBETTES, Construction d'un point fixe commun à une famille de contractions fermes, C. R. Acad. Sci. Paris Ser. I Math., 320 (1995), pp. 1385-1390.
36. P. L. COMBETTES, The convex feasibility problem in image recovery, in Advances in Imaging and Electron Physics, Vol. 95, P. Hawkes, ed., Academic Press, New York, 1996, pp. 155-270.
37. P. L. COMBETTES, Convex set theoretic image recovery by extrapolated iterations of parallel subgradient projections, IEEE Trans. Image Process., 6 (1997), pp. 493-506.
38. P. L. COMBETTES, Hilbertian convex feasibility problem: Convergence of projection methods, Appl. Math. Optim., 35 (1997), pp. 311-330.
39. P. L. COMBETTES, Strong convergence of block-iterative outer approximation methods for convex optimization, SIAM J. Control Optim., 38 (2000), pp. 538-565.
40. P. L. COMBETTES, Fejér monotonicity in convex optimization, in Encyclopedia of Optimization, Vol. 2, C. A. Floudas and P. M. Pardalos, eds., Kluwer Academic Publishers, Boston, MA, 2001, pp. 106-114.
41. P. L. COMBETTES, Quasi-Fejérian analysis of some optimization algorithms, in Inherently Parallel Algorithms for Feasibility and Optimization, D. Butnariu, Y. Censor, and S. Reich, eds., Elsevier, New York, 2001, pp. 115-152.
42. P. L. COMBETTES AND T. PENNANEN, Generalized Mann iterates for constructing fixed points in Hilbert spaces, J. Math. Anal. Appl., 275 (2002), pp. 521-536.
43. P. L. COMBETTES AND H. J. TRUSSELL, Method of successive projections for finding a common point of sets in metric spaces, J. Optim. Theory Appl., 67 (1990), pp. 487-507.
44. A. R. DE PIERRO AND A. N. IUSEM, A relaxed version of Bregman's method for convex programming, J. Optim. Theory Appl., 51 (1986), pp. 421-440.
45. R. DEVILLE, G. GODEFROY, AND V. ZIZLER, Smoothness and Renormings in Banach Spaces, John Wiley, New York, 1993.
46. J. ECKSTEIN, Nonlinear proximal point algorithms using Bregman functions, with applications to convex programming, Math. Oper. Res., 18 (1993), pp. 202-226.
47. J. ECKSTEIN, Approximate iterations in Bregman-function-based proximal algorithms, Math. Program., 83 (1998), pp. 113-123.
48. I. EKELAND AND R. TÉMAM, Convex Analysis and Variational Problems, Classics Appl. Math. 28, SIAM, Philadelphia, PA, 1999.
49. I. I. EREMIN AND V. D. MAZUROV, Nonstationary Processes of Mathematical Programming, Nauka, Moscow, 1979.
50. U. M. GARCÍA-PALOMARES AND F. J. GONZÁLEZ-CASTAÑO, Incomplete projection algorithms for solving the convex feasibility problem, Numer. Algorithms, 18 (1998), pp. 177-193.
51. K. GOEBEL AND S. REICH, Uniform Convexity, Hyperbolic Geometry, and Nonexpansive Mappings, Marcel Dekker, New York, 1984.
52. K. M. GRIGORIADIS AND R. E. SKELTON, Low-order control design for LMI problems using alternating projection methods, Automatica J. IFAC, 32 (1996), pp. 1117-1125.
53. L. G. GUBIN, B. T. POLYAK, AND E. V. RAIK, The method of projections for finding the common point of convex sets, USSR Comput. Math. Math. Phys., 7 (1967), pp. 1-24.
54. C. D. HA, A generalization of the proximal point algorithm, SIAM J. Control Optim., 28 (1990), pp. 503-512.
55. A. IUSEM AND R. G. OTERO, Inexact versions of proximal point and augmented Lagrangian algorithms in Banach spaces, Numer. Funct. Anal. Optim., 22 (2001), pp. 609-640.
56. A. IUSEM AND R. G. OTERO, Erratum: "Inexact versions of proximal point and augmented Lagrangian algorithms in Banach spaces", Numer. Funct. Anal. Optim., 23 (2002), pp. 227-228.
57. G. KASSAY, The proximal points algorithm for reflexive Banach spaces, Studia Univ. Babeş-Bolyai Math., 30 (1985), pp. 9-17.
58. K. C. KIWIEL, Proximal minimization methods with generalized Bregman functions, SIAM J. Control Optim., 35 (1997), pp. 1142-1168.
59. K. C. KIWIEL, Generalized Bregman projections in convex feasibility problems, J. Optim. Theory Appl., 96 (1998), pp. 139-157.
60. K. C. KIWIEL AND B. LOPUCH, Surrogate projection methods for finding fixed points of firmly nonexpansive mappings, SIAM J. Optim., 7 (1997), pp. 1084-1102.
61. B. LEMAIRE, On the convergence of some iterative methods for convex minimization, in Recent Developments in Optimization, Lect. Notes Econ. Math. Syst. 429, Springer-Verlag, Berlin, 1995, pp. 252-268.
62. A. LEVI AND H. STARK, Image restoration by the method of generalized projections with application to restoration from magnitude, J. Opt. Soc. Amer. A, 1 (1984), pp. 932-943.
63. Y. I. MERZLYAKOV, On a relaxation method of solving systems of linear inequalities, USSR Comput. Math. Math. Phys., 2 (1963), pp. 504-510.
64. J.-J. MOREAU, Fonctions convexes duales et points proximaux dans un espace hilbertien, C. R. Acad. Sci. Paris Sér. A Math., 255 (1962), pp. 2897-2899.
65. J.-J. MOREAU, Propriétés des applications prox, C. R. Acad. Sci. Paris Sér. A Math., 256 (1963), pp. 1069-1071.
66. J.-J. MOREAU, Sur la fonction polaire d'une fonction semi-continue supérieurement, C. R. Acad. Sci. Paris Sér. A Math., 258 (1964), pp. 1128-1130.
67. J.-J. MOREAU, Proximité et dualité dans un espace hilbertien, Bull. Soc. Math. France, 93 (1965), pp. 273-299.
68. Z. OPIAL, Weak convergence of the sequence of successive approximations for nonexpansive mappings, Bull. Amer. Math. Soc., 73 (1967), pp. 591-597.
69. N. OTTAVY, Strong convergence of projection-like methods in Hilbert spaces, J. Optim. Theory Appl., 56 (1988), pp. 433-461.
70. W. V. PETRYSHYN, Construction of fixed points of demicompact mappings in Hilbert space, J. Math. Anal. Appl., 14 (1966), pp. 276-284.
71. R. R. PHELPS, Convex Functions, Monotone Operators and Differentiability, 2nd ed., Lecture Notes in Math. 1364, Springer-Verlag, New York, 1993.
72. G. PIERRA, Decomposition through formalization in a product space, Math. Program., 28 (1984), pp. 96-115.
73. E. RAIK, Fejér type methods in Hilbert space, Esti NSV Tead. Akad. Toimetised Füüs.-Mat., 16 (1967), pp. 286-293.
74. S. REICH, The range of sums of accretive and monotone operators, J. Math. Anal. Appl., 68 (1979), pp. 310-317.
75. S. REICH, A weak convergence theorem for the alternating method with Bregman distances, in Theory and Applications of Nonlinear Operators of Accretive and Monotone Type, A. G. Kartsatos, ed., Marcel Dekker, New York, 1996, pp. 313-318.
76. R. T. ROCKAFELLAR, Level seta and continuity of conjugate convex functions, Trans. Amer. Math. Soc., 123 (1966), pp. 46-63.
77. R. T. ROCKAFELLAR, Convex Analysis, Princeton University Press, Princeton, NJ, 1970.
78. R. T. ROCKAFELLAR, Monotone operators and the proximal point algorithm, SIAM J. Control Optim., 14 (1976), pp. 877-898.
79. S. SIMONS, Minimax and Monotonicity, Lecture Notes in Math. 1693, Springer-Verlag, New York, 1998.
80. I. SINGER, Best Approximation in Normed Linear Spaces by Elements of Linear Subspaces, Springer-Verlag, New York, 1970.
81. I. SINGER, The Theory of Best Approximation and Functional Analysis, CBMS-NSF Regional Conf. Ser. in Appl. Math. 13, SIAM, Philadelphia, PA, 1974.
82. M. V. SOLODOV AND B. F. SVAITER, An inexact hybrid generalized proximal point algorithm and some new results on the theory of Bregman functions, Math. Oper. Res., 25 (2000), pp. 214-230.
83. M. TEBOULLE, Entropic proximal mappings with applications to nonlinear programming, Math. Oper. Res., 17 (1992), pp. 670-690.
84. C. ZǍLINESCU, On uniformly convex functions, J. Math. Anal. Appl., 95 (1983), pp. 344-374.
85. C. ZǍLINESCU, Convex Analysis in General Vector Spaces, World Scientific Publishing, River Edge, NJ, 2002.
86. E. ZEIDLER, Nonlinear Functional Analysis and Its Applications. II/B. Nonlinear Monotone Operators, Springer-Verlag, New York, 1990.