Multi-step gradient methods for networked optimization

IEEE Transactions on Signal Processing

Volumn 61, Issue 21, 2013, Pages 5417-5429

  Ghadimi, Euhanna ^a   Shames, Iman ^b   Johansson, Mikael ^a  

^a ROYAL INSTITUTE OF TECHNOLOGY   (Sweden)

^b UNIVERSITY OF MELBOURNE   (Australia)

Author keywords

accelerated gradient methods; Distributed optimization; fast convergence; primal and dual decomposition; robustness analysis

Indexed keywords

ALGORITHM PARAMETERS; DISTRIBUTED AVERAGING; DISTRIBUTED OPTIMIZATION; DUAL DECOMPOSITION; FAST CONVERGENCE; INTERNET CONGESTION CONTROL; ROBUSTNESS ANALYSIS; RUN-TIME PERFORMANCE;

ALGORITHMS; CONSTRAINED OPTIMIZATION; FUNCTIONS;

GRADIENT METHODS;

EID: 84885164451     PISSN: 1053587X     EISSN: None     Source Type: Journal    
DOI: 10.1109/TSP.2013.2278149     Document Type: Article

References (35)

1. F. Kelly, A. Maulloo, andD. Tan, "Rate control in communication networks: shadow prices, proportional fairness and stability," J. Operat. Res. Soc., vol. 49, pp. 237-252, 1998. (Pubitemid 128610018)
2. R. Olfati-Saber, J. A. Fax, and R. M. Murray, "Consensus and cooperation in networked multi-agent systems," Proc. IEEE, vol. 95, no. 1, pp. 215-233, Jan. 2007.
3. S. Kar, J. M. F. Moura, and K. Ramanan, "Distributed parameter estimation in sensor networks: nonlinear observation models and imperfect communication," IEEE Trans. Inf. Theory, vol. 58, no. 6, pp. 3575-3605, Jun. 2012.
4. S. Boyd,N. Parikh, E. Chu, B. Peleato, and J. Eckstein, "Distributed optimization and statistical learning via the alternating direction method ofmultipliers," Found. Trends inMach. Learn., vol. 3, no. 1, pp. 1-122, 2011.
5. D. Goodman and N. Mandayam, "Power control for wireless data," IEEE Pers. Commun., vol. 7, no. 2, pp. 48-54, 2000.
6. J. Tsitsiklis, D. Bertsekas, and M. Athans, "Distributed asynchronous deterministic and stochastic gradient optimization algorithms," IEEE Trans. Autom. Control, vol. 31, no. 9, pp. 803-812, Sep. 1986. (Pubitemid 16629805)
7. M. Cao, D. A. Spielman, and E. M. Yeh, "Accelerated gossip algorithms for distributed computation," in Proc. 44th Ann. Allerton Conf. Commun., Contr., Computat., 2006, pp. 952-959.
8. B. Johansson, "On distributed optimization in networked systems," Ph. D. dissertation, Royal Inst. Technol., Stokholm, 2008.
9. Y. Nesterov, Introductory Lectures on Convex Optimization: A Basic Course. New York, NY, USA: Springer, 2004.
10. D. Bertsekas, Nonlinear Programming. . New York, NY, USA: Athena Scientific, 1999.
11. F. Zanella, D. Varagnolo, A. Cenedese, G. Pillonetto, and L. Schenato, "Newton-Raphson consensus for distributed convex optimization," presented at the IEEE Conf. Decision Contr. (CDC), 2011.
12. E. Wei, A. Ozdaglar, and A. Jadbabaie, "A distributed Newton method for network utility maximization-I: Algorithm," IEEE Trans. Autom. Control, vol. 58, no. 9, pp. 2162-2175, 2013.
13. B. Polyak, Introduction to Optimization. . New York, NY, USA: Optimization Software, 1987, ISBN 0-911575-14-6.
14. S. Boyd, L. Xiao, A. Mutapcic, and J. Mattingley, Notes on decomposition methods Stanford Univ., CA, 2008.
15. Y. Nesterov, "Smooth minimization of non-smooth functions," Math. Program., 2005.
16. O. Devolder, F. Glineur, and Y. Nesterov, "Double smoothing technique for large-scale linearly constrained convex optimization," SIAM J. Optimiz., vol. 22, no. 2, pp. 702-727, 2012.
17. L. Xiao and S. Boyd, "Optimal scaling of a gradient method for distributed resource allocation," J. Opt. Theory Appl., vol. 129, no. 3, pp. 469-488, 2006. (Pubitemid 44951213)
18. Y. C. Ho, L. Servi, and R. Suri, "A class of center-free resource allocation algorithms," Large Scale Syst., vol. 1, pp. 51-62, 1980.
19. D. Bertsekas and J. Tsitsiklis, Parallel and Distributed Computation: Numerical Methods. New York, NY, USA: Athena Scientific, 1997.
20. P. Maréchal and J. Ye, "Optimizing condition numbers," SIAM J. Optimiz., vol. 20, no. 2, pp. 935-947, 2009.
21. Z. Lu and T. K. Pong, "Minimizing condition number via convex programming," SIAM J. Matrix Anal. Appl., vol. 32, pp. 1193-1211, 2011.
22. L. Vandenberghe, Course notes for optimization methods for largescale systems, ee236c, dual composition chapter, Tech. Rep., 2012.
23. R. A. Horn and C. R. Johnson, Matrix Analysis. Cambridge, U. K.; Cambridge Univ. Press, 1985.
24. A. Jadbabaie, J. Lin, and A. Morse, "Coordination of groups of mobile autonomous agents using nearest neighbor rules," IEEE Trans. Autom. Control, vol. 48, no. 6, pp. 988-1001, Jun. 2003.
25. B. Oreshkin, M. Coates, and M. Rabbat, "Optimization and analysis of distributed averaging with short node memory," IEEE Trans. Signal Process., vol. 58, no. 5, pp. 2850-2865, May 2010.
26. L. Xiao and S. Boyd, "Fast linear iterations for distributed averaging," Syst. Contr. Lett., vol. 53, no. 1, pp. 65-78, 2004.
27. E. Ghadimi, M. Johansson, and I. Shames, "Accelerated gradient methods for networked optimization," in Proc. IEEE Amer. Contr. Conf. (ACC), 2011, pp. 1668-1673.
28. D. M. Young, "Second-degree iterative methods for the solution of large linear systems," J. Approx. Theory, 1972.
29. J. Liu, B. D. O. Anderson, M. Cao, and A. S. Morse, "Analysis of accelerated gossip algorithms," in Proc. 48th IEEE Conf. Decision Contr. CDC), 2009.
30. G. H. Golub and R. S. Varga, "Chebyshev semi-iterative methods, successive overrelaxation iterative methods, and second order richardson iterative methods," Numerische Matematik, vol. 3, pp. 147-156, 1961.
31. F. R. K. Chung, "Spectral Graph Theory," in Proc. CBMS Regional Conf. Ser. Math. No. 92, Amer. Math. Soc., 1997.
32. S. Low and D. Lapsley, "Optimization flow control-I: basic algorithm and convergence," IEEE/ACM Trans. Netw., vol. 7, no. 6, pp. 861-874, 1999. (Pubitemid 30553383)
33. L. Xiao, M. Johansson, and S. Boyd, "Simultaneous routing and resource allocation via dual decomposition," IEEE Trans. Commun., vol. 52, no. 7, pp. 1136-1144, Jul. 2004.
34. M. Chiang, S. Low, A. Calderbank, and J. Doyle, "Layering as optimization decomposition: A mathematical theory of network architectures," Proc. IEEE, vol. 95, no. 1, pp. 255-312, Jan. 2007.
35. J. B. H. Urruty and C. Lemaréchal, Convex Analysis and Minimization Algorithms II. Berlin, Germany: Springer-Berlag, 1996.