On the convergence of alternating minimization for convex programming with applications to iteratively reweighted least squares and decomposition schemes

SIAM Journal on Optimization

Volumn 25, Issue 1, 2015, Pages 185-209

  Beck, Amir ^a  

^a TECHNION ISRAEL INSTITUTE OF TECHNOLOGY   (Israel)

Author keywords

Alternating minimization; Convex optimization; Iteratively reweighted least squares; Rate of convergence

Indexed keywords

APPROXIMATION THEORY; CONVEX OPTIMIZATION; DECOMPOSITION; FUNCTIONS; LEAST SQUARES APPROXIMATIONS; PROBLEM SOLVING;

ALTERNATING MINIMIZATION; CONVERGENCE PROPERTIES; CONVEX MINIMIZATION; DECOMPOSITION SCHEME; ITERATIVELY REWEIGHTED LEAST SQUARES; MULTIPLICATIVE CONSTANTS; OBJECTIVE FUNCTIONS; RATE OF CONVERGENCE;

ITERATIVE METHODS;

EID: 84925382477     PISSN: 10526234     EISSN: None     Source Type: Journal    
DOI: 10.1137/13094829X     Document Type: Article

References (30)

1. A. AUSLENDER, Optimisation, Méthodes numériques, Mâitrise de Mathématiques et Applications Fondamentales, Masson, Paris, 1976.
2. A. BECK AND M. TEBOULLE, Gradient-based algorithms with applications to signal-recovery problems, in Convex Optimization in Signal Processing and Communications, D. Palomar and Y. Eldar, eds., Cambridge University Press, Cambridge, UK, 2010, pp. 42-88.
3. A. BECK AND M. TEBOULLE, Smoothing and first order methods: A unified framework, SIAM J. Optim., 22 (2012), pp. 557-580.
4. A. BECK AND L. TETRUASHVILI, On the convergence of block coordinate descent type methods, SIAM J. Optim., 23 (2013), pp. 2037-2060.
5. D. P. BERTSEKAS, Nonlinear Programming, 2nd ed., Athena Scientific, Belmont, MA, 1999.
6. D. P. BERTSEKAS AND J. N. TSITSIKLIS, Parallel and Distributed Computation, Prentice-Hall, Englewood Cliffs, NJ, 1989.
7. N. BISSANTZ, L. DÜMBGEN, A. MUNK, AND B. STRATMANN, Convergence analysis of generalized iteratively reweighted least squares algorithms on convex function spaces, SIAM J. Optim., 19 (2008), pp. 1828-1845.
8. R. CHANDRASEKARAN AND A. TAMIR, Open questions concerning Weiszfeld's algorithm for the Fermat-Weber location problem, Math. Programming, 44 (1989), pp. 293-295.
9. P. L. COMBETTES AND V. R. WAJS, Signal recovery by proximal forward-backward splitting, Multiscale Model. Simul., 4 (2005), pp. 1168-1200.
10. I. DAUBECHIES, R. DEVORE, M. FORNASIER, AND C. S. GÜNTÜRK, Iteratively reweighted least squares minimization for sparse recovery, Comm. Pure Appl. Math., 63 (2010), pp. 1-38.
11. Z. DREZNER, K. KLAMROTH, A. SCHOBEL, AND G. O. WESOLOWSKY, The Weber problem, in Facility Location: Applications and Theory, Z. Drezner and H. W. Hamacher, eds., Springer, Berlin, 2002, pp. 1-36.
12. H. A. EISELT AND V. MARIANOV, Pioneering developments in location analysis, in Foundations of Location Analysis, H. A. Eiselt and V. Marianov, eds., International Series in Operations Research & Management Science, Springer, New York, 2011, pp. 3-22.
13. E. ESSER, X. ZHANG, AND T. F. CHAN, A general framework for a class of first order primaldual algorithms for convex optimization in imaging science, SIAM J. Imaging Sci., 3 (2010), pp. 1015-1046.
14. L. GRIPPO AND M. SCIANDRONE, Globally convergent block-coordinate techniques for unconstrained optimization, Optim. Methods Softw., 10 (1999), pp. 587-637.
15. P. C. HANSEN, J. G. NAGY, AND D. P. O'LEARY, Deblurring Images, Matrices, Spectra, and Filtering, Fundamentals of Algorithms 3, SIAM, Philadelphia, 2006.
16. T. HASTIE, R. TIBSHIRANI, AND J. FRIEDMAN, The Elements of Statistical Learning, in Data Mining, Inference, and Prediction, 2nd ed., Springer Ser. Statist., Springer, New York, 2009.
17. B. HE AND X. YUAN, On the O(1/n) convergence rate of the Douglas-Rachford alternating direction method, SIAM J. Numer. Anal., 50 (2012), pp. 700-709.
18. H. W. KUHN, A note on Fermat's problem, Math. Programming, 4 (1973), pp. 98-107.
19. Z.-Q. LUO AND P. TSENG, Error bounds and convergence analysis of feasible descent methods: A general approach, Ann. Oper. Res., 46 (1993), pp. 157-178.
20. P. MCCULLAGH AND J. A. NELDER, Generalized Linear Models, Monogr. Statist. Appl. Probab., Chapman & Hall, London, 1983.
21. R. D. C. MONTEIRO AND B. F. SVAITER, Iteration-complexity of block-decomposition algorithms and the alternating direction method of multipliers, SIAM J. Optim., 23 (2013), pp. 475-507.
22. J.-J. MOREAU, Proximité et dualité dans un espace Hilbertien, Bull. Soc. Math. France, 93 (1965), pp. 273-299.
23. Y. NESTEROV, Introductory Lectures on Convex Optimization, Appl. Optim. 87, Kluwer Academic Publishers, Boston, 2004.
24. J. M. Ortega and W. C. Rheinboldt, Iterative Solution of Nonlinear Equations in Several Variables, Academic Press, New York, 1970.
25. R. SHEFI AND M. TEBOULLE, Rate of convergence analysis of decomposition methods based on the proximal method of multipliers for convex minimization, SIAM J. Optim., 24 (2014), pp. 269-297.
26. P. TSENG, Applications of a splitting algorithm to decomposition in convex programming and variational inequalities, SIAM J. Control Optim., 29 (1991), pp. 119-138.
27. Y. VARDI AND C.-H. ZHANG, A modified Weiszfeld algorithm for the Fermat-Weber location problem, Math. Program., 90 (2001), pp. 559-566.
28. E. WEISZFELD, Sur le point pour lequel la somme des distances de n points donnés est minimum, Tohoku Math. J., 43 (1937), pp. 355-386.
29. E. WEISZFELD, On the point for which the sum of the distances to n given points is minimum, Ann. Oper. Res., 167 (2009), pp. 7-41;
30. [Tohoku Math. J., 43 (1937), pp. 355-386] and annotated by F. Plastria.