Stochastic quasi-fejér block-coordinate fixed point iterations with random sweeping

SIAM Journal on Optimization

Volumn 25, Issue 2, 2015, Pages 1221-1248

  Combettes, Patrick L ^a   Pesquet, Jean Christophe ^b  

^a Sorbonne Universités UPMC Université Paris 06 UMR 7598 Laboratoire Jacques Louis Lions   (France)

^b UNIVERSITÉ PARIS EST   (France)

Author keywords

Arbitrary sampling; Block coordinate algorithm; Fixed point algorithm; Monotone operator splitting; Primal dual algorithm; Stochastic algorithm; Stochastic quasi Fej r sequence; Structured convex minimization problem

Indexed keywords

ALGORITHMS; ASYMPTOTIC ANALYSIS; ITERATIVE METHODS; NONLINEAR ANALYSIS; RANDOM ERRORS;

CONVEX MINIMIZATION; FIXED-POINT ALGORITHMS; MONOTONE OPERATORS; PRIMAL DUAL ALGORITHMS; R-SEQUENCES; STOCHASTIC ALGORITHMS;

STOCHASTIC SYSTEMS;

EID: 84940391194     PISSN: 10526234     EISSN: None     Source Type: Journal    
DOI: 10.1137/140971233     Document Type: Article

References (67)

1. M. A. Alghamdi, A. Alotaibi, P. L. Combettes, and N. Shahzad, A primal-dual method of partial inverses for composite inclusions, Optim. Lett., 8 (2014), pp. 2271-2284.
2. H. Attouch, L. M. Briceño-Arias, and P. L. Combettes, A parallel splitting method for coupled monotone inclusions, SIAM J. Control Optim., 48 (2010), pp. 3246-3270.
3. A. Auslender, Méthodes numériques pour la décomposition et la minimisation de fonctions non différentiables, Numer. Math., 18 (1971/72), pp. 213-223.
4. K. Barty, J.-S. Roy, and C. Strugarek, Hilbert-valued perturbed subgradient algorithms, Math. Oper. Res., 32 (2007), pp. 551-562.
5. H. H. Bauschke and J. M. Borwein, On projection algorithms for solving convex feasibility problems, SIAM Rev., 38 (1996), pp. 367-426.
6. H. H. Bauschke and P. L. Combettes, A weak-to-strong convergence principle for Fejérmonotone methods in Hilbert spaces, Math. Oper. Res., 26 (2001), pp. 248-264.
7. H. H. Bauschke and P. L. Combettes, Convex Analysis and Monotone Operator Theory in Hilbert Spaces, Springer, New York, 2011.
8. H. H. Bauschke, M. N. Dao, D. Noll, and H. M. Phan, Proximal point algorithm, DouglasRachford algorithm and alternating projections: A case study, J. Convex Anal., to appear.
9. H. H. Bauschke, F. Deutsch, and H. Hundal, Characterizing arbitrarily slow convergence in the method of alternating projections, Int. Trans. Oper. Res., 16 (2009), pp. 413-425.
10. J. Bolte, S. Sabach, and M. Teboulle, Proximal alternating linearized minimization for nonconvex and nonsmooth problems, Math. Program., 146 (2014), pp. 459-494.
11. R. I. Boţ and C. Hendrich, A Douglas-Rachford type primal-dual method for solving inclusions with mixtures of composite and parallel-sum type monotone operators, SIAM J. Optim., 23 (2013), pp. 2541-2565.
12. L. M. Briceño-Arias, Problèmes d'Inclusions Couplées: Éclatement, Algorithmes et Applications, Thèse de doctorat, Université Pierre et Marie Curie, Paris, 2011.
13. L. M. Briceño-Arias, A Douglas-Rachford splitting method for solving equilibrium problems, Nonlinear Anal., 75 (2012), pp. 6053-6059.
14. L. M. Briceño-Arias, Forward-Douglas-Rachford splitting and forward-partial inverse method for solving monotone inclusions, Optimization, 64 (2015), pp. 1239-1261.
15. L. M. Briceño-Arias and P. L. Combettes, Convex variational formulation with smooth coupling for multicomponent signal decomposition and recovery, Numer. Math. Theory Methods Appl., 2 (2009), pp. 485-508.
16. L. M. Briceño-Arias and P. L. Combettes, Monotone operator methods for Nash equilibria in non-potential games, in Computational and Analytical Mathematics, D. Bailey, H. H. Bauschke, P. Borwein, F. Garvan, M. Théra, J. Vanderwerff, and H. Wolkowicz, eds., Springer, New York, 2013, pp. 143-159.
17. C. L. Byrne, Iterative Optimization in Inverse Problems, CRC Press, Boca Raton, FL, 2014.
18. J. Céa, Optimisation: Théorie et Algorithmes, Dunod, Paris, 1971.
19. A. Cegielski, Iterative Methods for Fixed Point Problems in Hilbert Spaces, Lecture Notes in Math. 2057, Springer, Heidelberg, 2012.
20. E. Chouzenoux, J.-C. Pesquet, and A. Repetti, A Block-Coordinate Variable Metric Forward-Backward Algorithm, http://www.optimization-online.org/DB-HTML/2013/12/4178.html.
21. P. L. Combettes, Quasi-Fejérian analysis of some optimization algorithms, in Inherently Parallel Algorithms for Feasibility and Optimization, D. Butnariu, Y. Censor, and S. Reich, eds., Elsevier, New York, 2001, pp. 115-152.
22. P. L. Combettes, Solving monotone inclusions via compositions of nonexpansive averaged operators, Optimization, 53 (2004), pp. 475-504.
23. P. L. Combettes, Fejér monotonicity in convex optimization, in Encyclopedia of Optimization, 2nd ed., C. A. Floudas and P. M. Pardalos, eds., Springer, New York, 2009, pp. 1016-1024.
24. P. L. Combettes, Iterative construction of the resolvent of a sum of maximal monotone operators, J. Convex Anal., 16 (2009), pp. 727-748.
25. P. L. Combettes, Systems of structured monotone inclusions: Duality, algorithms, and applications, SIAM J. Optim., 23 (2013), pp. 2420-2447.
26. P. L. Combettes, Dinh D͡ng, and B. C. V͡, Dualization of signal recovery problems, Set-Valued Var. Anal., 18 (2010), pp. 373-404.
27. P. L. Combettes and J.-C. Pesquet, Proximal splitting methods in signal processing, in Fixed-Point Algorithms for Inverse Problems in Science and Engineering, H. H. Bauschke et al., eds., Springer, New York, 2011, pp. 185-212.
28. P. L. Combettes and B. C. V͡, Variable metric forward-backward splitting with applications to monotone inclusions in duality, Optimization, 63 (2014), pp. 1289-1318.
29. P. L. Combettes and V. R. Wajs, Signal recovery by proximal forward-backward splitting, Multiscale Model. Simul., 4 (2005), pp. 1168-1200.
30. L. Condat, A primal-dual splitting method for convex optimization involving Lipschitzian, proximable and linear composite terms, J. Optim. Theory Appl., 158 (2013), pp. 460-479.
31. E. De Vito, V. Umanità, and S. Villa, A consistent algorithm to solve Lasso, elastic-net and Tikhonov regularization, J. Complexity, 27 (2011), pp. 188-200.
32. J. Eckstein and D. P. Bertsekas, On the Douglas-Rachford splitting method and the proximal point algorithm for maximal monotone operators, Math. Program., 55 (1992), pp. 293-318.
33. I. I. Eremin and L. D. Popov, Fejér processes in theory and practice: Recent results, Russian Math. (Iz. VUZ), 53 (2009), pp. 36-55.
34. Yu. M. Ermol'ev, On the method of generalized stochastic gradients and quasi-Fejér sequences, Cybernetics, 5 (1969), pp. 208-220.
35. Yu. M. Ermol'ev, On convergence of random quasi-Fejér sequences, Cybernetics, 7 (1971), pp. 655-656.
36. Yu. M. Ermol'ev and A. D. Tuniev, Random Fejér and quasi-Fejér sequences, Theory of Optimal Solutions-Akademiya Nauk UkrainskoѤ, SSR Kiev, 2 (1968), pp. 76-83 (in Russian)
37. Amer. Math. Soc. Select. Translat. Math. Statist. Probab., 13 (1973), pp. 143-148 (in English).
38. R. M. Fortet, Vecteurs, Fonctions et Distributions Aléatoires dans les Espaces de Hilbert. Hermès, Paris, 1995.
39. D. Gabay, Applications of the method of multipliers to variational inequalities, in Augmented Lagrangian Methods: Applications to the Numerical Solution of Boundary Value Problems, M. Fortin and R. Glowinski, eds., North-Holland, Amsterdam, 1983, pp. 299-331.
40. R. Glowinski and P. Le Tallec, eds., Augmented Lagrangian and Operator-Splitting Methods in Nonlinear Mechanics, SIAM, Philadelphia, 1989.
41. H. S. Hundal, An alternating projection that does not converge in norm, Nonlinear Anal., 57 (2004), pp. 35-61.
42. F. Iutzeler, P. Bianchi, P. Ciblat, and W. Hachem, Asynchronous distributed optimization using a randomized alternating direction method of multipliers, in Proceedings of the 52nd Conference on Decision Control, Florence, Italy, 2013, pp. 3671-3676.
43. M. Ledoux and M. Talagrand, Probability in Banach Spaces: Isoperimetry and Processes, Springer, New York, 1991.
44. P. L. Lions and B. Mercier, Splitting algorithms for the sum of two nonlinear operators, SIAM J. Numer. Anal., 16 (1979), pp. 964-979.
45. M. Loève, Probability Theory II, 4th ed., Springer, New York, 1978.
46. Z. Lu and L. Xiao, On the complexity analysis of randomized block-coordinate descent methods, Math. Program., to appear.
47. B. Mercier, Topics in Finite Element Solution of Elliptic Problems, Lectures on Math. 63, Tata Institute of Fundamental Research, Bombay, India, 1979.
48. J. J. Moreau, Fonctions convexes duales et points proximaux dans un espace hilbertien, C. R. Acad. Sci. Paris Sér. A, 255 (1962), pp. 2897-2899.
49. I. Necoara and A. Patrascu, A random coordinate descent algorithm for optimization problems with composite objective function and linear coupled constraints, Comput. Optim. Appl., 57 (2014), pp. 307-337.
50. Yu. Nesterov, Efficiency of coordinate descent methods on huge-scale optimization problems, SIAM J. Optim., 22 (2012), pp. 341-362.
51. J. M. Ortega and W. C. Rheinboldt, Iterative Solution of Nonlinear Equations in Several Variables, Academic Press, New York, 1970.
52. N. Papadakis, G. Peyré, and E. Oudet, Optimal transport with proximal splitting, SIAM J. Imaging Sci., 7 (2014), pp. 212-238.
53. T. Pennanen and M. Kallio, A splitting method for stochastic programs, Ann. Oper. Res., 142 (2006), pp. 259-268.
54. J.-C. Pesquet and N. Pustelnik, A parallel inertial proximal optimization method, Pac. J. Optim., 8 (2012), pp. 273-305.
55. W. V. Petryshyn, Construction of fixed points of demicompact mappings in Hilbert space, J. Math. Anal. Appl., 14 (1966), pp. 276-284.
56. B. J. Pettis, On integration in vector spaces, Trans. Amer. Math. Soc., 44 (1938), pp. 277-304.
57. H. Raguet, J. Fadili, and G. Peyré, A generalized forward-backward splitting, SIAM J. Imaging Sci., 6 (2013), pp. 1199-1226.
58. E. Raik, Fejér type methods in Hilbert space, Eesti NSV Tead. Akad. Toimetised Füüs.-Mat., 16 (1967), pp. 286-293.
59. P. Richtárik and M. Takaíc, Iteration complexity of randomized block-coordinate descent methods for minimizing a composite function, Math. Program., 144 (2014), pp. 1-38.
60. H. Robbins and D. Siegmund, A convergence theorem for non negative almost supermartingales and some applications, in Optimizing Methods in Statistics, J. S. Rustagi, ed., Academic Press, New York, 1971, pp. 233-257.
61. S. Sra, S. Nowozin, and S. J. Wright, eds., Optimization for Machine Learning, MIT Press, Cambridge, MA, 2011.
62. P. Tseng, Further applications of a splitting algorithm to decomposition in variational inequalities and convex programming, Math. Program., 48 (1990), pp. 249-263.
63. P. Tseng, Applications of a splitting algorithm to decomposition in convex programming and variational inequalities, SIAM J. Control Optim., 29 (1991), pp. 119-138.
64. S. Villa, S. Salzo, L. Baldassarre, and A. Verri, Accelerated and inexact forward-backward algorithms, SIAM J. Optim., 23 (2013), pp. 1607-1633.
65. B. C. V͡, A splitting algorithm for dual monotone inclusions involving cocoercive operators, Adv. Comput. Math., 38 (2013), pp. 667-681.
66. D. C. Youla, Mathematical theory of image restoration by the method of convex projections, in Image Recovery: Theory and Application, H. Stark, ed., Academic Press, San Diego, CA, 1987, pp. 29-77.
67. E. Zeidler, Nonlinear Functional Analysis and Its Applications I: Fixed-Point Theorems, Springer-Verlag, New York, 1986.