A unified convergence analysis of block successive minimization methods for nonsmooth optimization

SIAM Journal on Optimization

Volumn 23, Issue 2, 2013, Pages 1126-1153

  Razaviyayn, Meisam ^a   Hong, Mingyi ^a   Luo, Zhi Quan ^a  

^a University of Minnesota   (United States)

Author keywords

Block coordinate descent; Block successive upper bound minimization; Successive convex approximation; Successive inner approximation

Indexed keywords

BLOCK COORDINATE DESCENTS; DIFFERENCE OF CONVEX FUNCTIONS; EXPECTATION-MAXIMIZATION ALGORITHMS; FORWARD-BACKWARD SPLITTING ALGORITHM; LARGE-SCALE OPTIMIZATION; SUCCESSIVE CONVEX APPROXIMATIONS; SUCCESSIVE INNER APPROXIMATION; UPPER BOUND;

ALGORITHMS; FUNCTIONS; OPTIMIZATION;

ITERATIVE METHODS;

EID: 84880570485     PISSN: 10526234     EISSN: None     Source Type: Journal    
DOI: 10.1137/120891009     Document Type: Article

References (43)

1. H. Attouch, J. Bolte, and B. F. Svaiter, Convergence of descent methods for semialgebraic and tame problems: Proximal algorithms, forwardbackward splitting, and regularized Gauss-Seidel methods, Math. Program., 137 (2013), pp. 91-129.
2. A. Auslender, Asymptotic properties of the Fenchel dual functional and applications to decomposition problems, J. Optim. Theory Appl., 73 (1992), pp. 427-449.
3. D. P. Bertsekas, Nonlinear Programming, 2nd ed., Athena-Scientific, Nashua, NH, 1999.
4. D. P. Bertsekas and J. N. Tsitsiklis, Parallel and Distributed Computation: Numerical Methods, 2nd ed., Athena-Scientific, Nashua, NH, 1999.
5. S. Bonettini, Inexact block coordinate descent methods with application to non-negative matrix factorization, IMA J. Numer. Anal., 31 (2011), pp. 1431-1452.
6. S. Borman, The Expectation Maximization Algorithm: A Short Tutorial, manuscript.
7. S. Boyd and L. Vandenberghe, Convex Optimization, Cambridge University Press, Cambridge, UK,2004.
8. J. D. Carroll and J. J. Chang, Analysis of individual differences in multidimensional scaling via an n-way generalization of "Eckart-Young" decomposition, Psychometrika, 35 (1970), pp. 283-319.
9. Y. Censor and S. A. Zenios, Parallel Optimization: Theory, Algorithm, and Applications, Oxford University Press, Oxford, UK, 1997.
10. B. Chen, S. He, Z. Li, and S. Zhang, Maximum block improvement and polynomial optimization, SIAM J. Optim., 22 (2012), pp. 87-107.
11. M. Chiang, C. W. Tan, D. P. Palomar, D. O'Neill, and D. Julian, Power control by geometric programming, IEEE Trans. Wireless Commun., 6 (2007), pp. 2640-2651.
12. M. Chiang, C. W. Tan, D. P. Palomar, D. O'Neill, and D. Julian, Power control by geometric programming, IEEE Trans. Wireless Commun., 6 (2007), pp. 2640-2651.
13. P. L. Combettes and J.-C Pesquet, Proximal splitting methods in signal processing, Tech. report, http://arxiv.org/abs/0912.3522, 2009.
14. P. L. Combettes and V. R. Wajs, Signal recovery by proximal forward-backward splitting, Multiscale Model. Simul., 4 (2005), pp. 1168-1200.
15. A. P. Dempster, N. M. Laird, and D. B. Rubin, Maximum likelihood from incomplete data via the EM algorithm, J. Roy. Statist. Soc. Ser. B, 39 (1977), pp. 1-38.
16. N. K. M. Faber, R. Bro, and P. K. Hopke, Recent developments in CANDECOMP/PARAFAC algorithms: A critical review, Chemometrics Intell. Lab. Syst., 65 (2003), pp. 119-137.
17. L. Grippo and M. Sciandrone, Globally convergent block-coordinate techniques for unconstrained optimization, Optim. Methods Softw., 10 (1999), pp. 587-637.
18. L. Grippo and M. Sciandrone, On the convergence of the block nonlinear Gauss-Seidel method under convex constraints, Oper. Res. Lett., 26 (2000), pp. 127-136.
19. R. A. Harshman, Foundations of the PARAFAC procedure: Models and conditions for an explanatory" multi-modal factor analysis, UCLA Working Papers in Phonetics, 16 (1970), pp. 1-84.
20. J. A. Hartigan and M. A. Wong, K-means clustering algorithm, J. Roy. Statist. Soc. Ser. C, 28 (1979), pp. 100-108.
21. V. Hasselblad, Estimation of parameters for a mixture of normal distributions, Technometrics, 8 (1966), pp. 431-444.
22. J. Hastad, Tensor rank is NP-complete, J. Algorithms, 11 (1990), pp. 644-654.
23. M. Hong and Z.-Q. Luo, Joint linear precoder optimization and base station selection for an uplink MIMO network: A game theoretic approach, in Proceedings of the IEEE ICASSP, IEEE, Washington, DC, 2012, pp. 2941-2944.
24. H. R. Howson and N. G. F. Sancho, A new algorithm for the solution of multistate dynamic programming problems, Math. Programming, 8 (1975), pp. 104-116.
25. S. J. Kim and G. B. Giannakis, Optimal resource allocation for MIMO ad hoc cognitive radio networks, in Proceedings of the Allerton Conference on Communication, Control, and Computing, Urbana-Champaign, IL, 2008, pp. 39-45.
26. T. G. Kolda and B. W. Bader, Tensor decompositions and applications, SIAM Rev., 51 (2009), pp. 455-500.
27. Z-.Q. Luo and S. Zhang, Dynamic spectrum management: Complexity and duality, IEEE J. Selected Topics Signal Process., 2 (2008), pp. 57-73.
28. B. R. Marks and G. P. Wright, A general inner approximation algorithm for nonconvex mathematical programs, Oper. Res., 26 (1978), pp. 681-683.
29. C. Navasca, L. De Lathauwer, and S. Kindermann, Swamp reducing technique for tensor decomposition, in Proceedings of the 16th European Signal Processing Conference (EUSIPCO), Lausanne, Switzerland, 2008.
30. C. T. K. Ng and H. Huang, Linear precoding in cooperative MIMO cellular networks with limited coordination clusters, IEEE J. Selected Areas Commun., 28 (2010), pp. 1446 -1454.
31. J. Papandriopoulos and J. S. Evans, Low-complexity distributed algorithms for spectrum balancing in multi-user DSL networks, in Proceedings of the IEEE International Conference on Communications (ICC), Volume 7, IEEE, Washington, DC, 2006, pp. 3270-3275.
32. M. J. D. Powell, On search directions for minimization algorithms, Math. Programming, 4 (1973), pp. 193-201.
33. M. Razaviyayn, H. Baligh, A. Callard, and Z.-Q. Luo, Joint transceiver design and user grouping in a MIMO interfering broadcast channel, in Proceedings of the 45th Annual Conference on Information Sciences and Systems (CISS), Baltimore, MD, 2011, pp. 1-6.
34. C. Shi, R. A. Berry, and M. L. Honig, Local interference pricing for distributed beamforming in MIMO networks, in Proceedings of the 28th IEEE Conference on Military Communications (MILCOM), IEEE, Washington, DC, 2009, pp. 1001-1006.
35. C. Shi, R. A. Berry, and M. L. Honig, Monotonic convergence of distributed interference pricing in wireless networks, in Proceedings of the 2009 IEEE International Symposium on Information Theory, Volume 3, IEEE, Washington, DC, 2009, pp. 1619-1623.
36. Q. Shi, M. Razaviyayn, Z.-Q. Luo, and C. He, An iteratively weighted MMSE approach to distributed sum-utility maximization for a MIMO interfering broadcast channel, IEEE Trans. Signal Process., 59 (2011), pp. 4331-4340.
37. G. Tomasi and R. Bro, A comparison of algorithms for fitting the PARAFAC model, Comput. Statist. Data Anal., 50 (2006), pp. 1700-1734.
38. P. Tseng, Convergence of a block coordinate descent method for nondifferentiable minimization, J. Optim. Theory Appl., 109 (2001), pp. 475-494.
39. P. Tseng and S. Yun, A coordinate gradient descent method for nonsmooth separable minimization, Math. Program., 117 (2009), pp. 387-423.
40. Y. Xu and W. Yin, A block coordinate descent method for multi-convex optimization with applications to nonnegative tensor factorization and completion, available online at http://www.caam.rice.edu/∼optimization/BCD/, 2012.
41. A. L. Yuille and A. Rangarajan, The concave-convex procedure, Neural Comput., 15 (2003), pp. 915-936.
42. N. Zadeh, A note on the cyclic coordinate ascent method, Management Sci., 16 (1970), pp. 642-644.
43. Y. Zhang, E. Dall'Anese, and G. B. Giannakis, Distributed robust beamforming for mimo cognitive networks, in Proceedings of the IEEE International Conference in Acoustics, Speech and Signal Processing (ICASSP), IEEE, Washington, DC, 2012, pp. 2953-2956.