A simple parallel algorithm with an O(1=T ) convergence rate for general convex programs

SIAM Journal on Optimization

Volumn 27, Issue 2, 2017, Pages 759-783

  Yu, Hao ^a   Neely, Michael J ^a  

^a University of Southern California ^*  (United States)

Author keywords

Convergence rates; Convex programs; Parallel algorithms

Indexed keywords

CONSTRAINT THEORY; PARALLEL ALGORITHMS;

ALTERNATING DIRECTION METHOD OF MULTIPLIERS; CONVERGENCE RATES; CONVEX PROGRAMS; INEQUALITY CONSTRAINT; NETWORK UTILITY MAXIMIZATION; NON-LINEAR CONSTRAINTS; PARALLEL IMPLEMENTATIONS; SUB-GRADIENT ALGORITHM;

CONVEX OPTIMIZATION;

EID: 85021710810     PISSN: 10526234     EISSN: None     Source Type: Journal    
DOI: 10.1137/16M1059011     Document Type: Article

References (23)

1. M. S. Bazaraa, H. D. Sherali, and C. M. Shetty, Nonlinear Programming: Theory and Algorithms, Wiley-Interscience, Hoboken, NJ, 2006.
2. A. Beck, A. Nedic, A. Ozdaglar, and M. Teboulle, An O(1=k) gradient method for network resource allocation problems, IEEE Trans. Control Netw. Syst., 1 (2014), pp. 64-73.
3. D. P. Bertsekas, Nonlinear Programming, 2nd ed., Athena Scientific, Belmont, MA, 1999.
4. S. Boyd, N. Parikh, E. Chu, B. Peleato, and J. Eckstein, Distributed optimization and statistical learning via the alternating direction method of multipliers, Found. Trends Mach. Learn., 3 (2011), pp. 1-122.
5. S. Boyd and L. Vandenberghe, Convex Optimization, Cambridge University Press, Cam-bridge, 2004.
6. B. He and X. Yuan, On the O(1=n) convergence rate of the Douglas-Rachford alternating direction method, SIAM J. Numer. Anal., 50 (2012), pp. 700-709.
7. J.-B. Hiriart-Urruty and C. Lemarechal, Fundamentals of Convex Analysis, Springer, Berlin, 2001.
8. F. P. Kelly, A. K. Maulloo, and D. K. Tan, Rate control for communication networks: Shadow prices, proportional fairness and stability, J. Oper. Res. Soc., 49 (1998), pp. 237-252.
9. T.-Y. Lin, S.-Q. Ma, and S.-Z. Zhang, On the sublinear convergence rate of multi-block ADMM, J. Oper. Res. Soc. China, 3 (2015), pp. 251-274.
10. S. H. Low, A duality model of TCP and queue management algorithms, IEEE/ACM Trans. Netw., 11 (2003), pp. 525-536.
11. S. H. Low and D. E. Lapsley, Optimization ow control|I: Basic algorithm and convergence, IEEE/ACM Trans. Netw., 7 (1999), pp. 861-874.
12. I. Necoara and V. Nedelcu, Rate analysis of inexact dual first-order methods application to dual decomposition, IEEE Trans. Automat. Control, 59 (2014), pp. 1232-1243.
13. A. Nedific and A. Ozdaglar, Approximate primal solutions and rate analysis for dual subgra-dient methods, SIAM J. Optim., 19 (2009), pp. 1757-1780.
14. A. Nedific and A. Ozdaglar, Subgradient methods for saddle-point problems, J. Optim. Theory Appl., 142 (2009), pp. 205-228.
15. M. J. Neely, Distributed and secure computation of convex programs over a network of con-nected processors, DCDIS International Conference on Engineering Applications and Com-putational Algorithms, Guelph, Canada, 2005.
16. M. J. Neely, Stochastic Network Optimization with Application to Communication and Queue-ing Systems, Morgan & Claypool, 2010.
17. M. J. Neely, A Simple Convergence Time Analysis of Drift-Plus-Penalty for Stochastic Op-Timization and Convex Programs, arXiv:1412.0791, 2014.
18. Y. Nesterov, Introductory Lectures on Convex Optimization: A Basic Course, Kluwer, Boston, 2004.
19. N. Parikh and S. Boyd, Proximal algorithms, Found. Trends Optim., 1 (2013), pp. 123-231.
20. W.-H. Wang, M. Palaniswami, and S. H. Low, Optimal ow control and routing in multi-path networks, Perform. Eval., 52 (2003), pp. 119-132.
21. E. Wei and A. Ozdaglar, On the O(1=k) convergence of asynchronous distributed alternating direction method of multipliers, in Proceedings of the IEEE Global Conference on Signal and Information Processing, Austin, TX, 2013, pp. 551-554.
22. E. Wei, A. Ozdaglar, and A. Jadbabaie, A distributed Newton method for network utility maximization-I: Algorithm, IEEE Trans. Automat. Control, 58 (2013), pp. 2162-2175.
23. H. Yu and M. J. Neely, On the convergence time of the drift-plus-penalty algorithm for strongly convex programs, in Proceedings of IEEE Conference on Decision and Control (CDC), Osaka, Japan, 2015, pp. 2673-2679.