Fast distributed gradient methods

IEEE Transactions on Automatic Control

Volumn 59, Issue 5, 2014, Pages 1131-1146

  Jakovetic, Dusan ^a,b,c   Xavier, Joao ^a   Moura, Jose M F ^b,d  

^a INSTITUTO SUPERIOR TÉCNICO   (Portugal)

^b CARNEGIE MELLON UNIVERSITY   (United States)

^c UNIVERSITY OF NOVI SAD   (Serbia)

^d NEW YORK UNIVERSITY   (United States)

Author keywords

Consensus; convergence rate; distributed optimization; Nesterov gradient

Indexed keywords

OPTIMIZATION;

CONSENSUS; CONVERGENCE CONSTANTS; CONVERGENCE RATES; DISTRIBUTED OPTIMIZATION; EXPLICIT DEPENDENCES; GRADIENT ALGORITHM; LIPSCHITZ CONTINUOUS; SIMULATION EXAMPLE;

GRADIENT METHODS;

EID: 84893505377     PISSN: 00189286     EISSN: None     Source Type: Journal    
DOI: 10.1109/TAC.2014.2298712     Document Type: Article

References (38)

1. J. Tsitsiklis, D. Bertsekas, and M. Athans, "Distributed asynchronousdeterministic and stochastic gradient optimization algorithms," IEEETrans. Autom. Control, vol. 31, no. 9, pp. 803-812, Sep. 1986. (Pubitemid 16629805)
2. J. N. Tsitsiklis, "Problems in Decentralized Decision Making and Computation,"Ph.D., Elect. Eng. Comp. Sci., MIT, Cambridge, MA, 1984.
3. M. Rabbat and R. Nowak, "Distributed optimization in sensor networks,"in Proc. 3rd Int. Symp. Inform. Process. Sensor Networks(IPSN'04), Berkeley, CA, USA, Apr. 2004, pp. 20-27.
4. B. Johansson, A. Speranzon, M. Johansson, and K. H. Johansson, "Ondecentralized negotiation of optimal consensus," Autom., vol. 44, no.4, pp. 1175-1179, 2008.
5. G. Mateos, J. A. Bazerque, and G. B. Giannakis, "Distributed sparselinear regression," IEEE Trans. Signal Process., vol. 58, no. 11, pp.5262-5276, Nov. 2010.
6. I. Necoara and J. A. K. Suykens, "Application of a smoothing techniqueto decomposition in convex optimization," IEEE Trans. Autom.Control, vol. 53, no. 11, pp. 2674-2679, Dec. 2008.
7. S. Boyd,N. Parikh, E. Chu, B. Peleato, and J. Eckstein, "Distributed optimizationand statistical learning via the alternating direction methodof multipliers," Found. Trends Machine Learning, vol. 3, no. 1, pp.1-122, 2011.
8. A. Nedic and A. Ozdaglar, "Distributed subgradientmethods formultiagentoptimization," IEEE Trans. Autom. Control, vol. 54, no. 1, pp.48-61, Jan. 2009.
9. S. Ram, A. Nedic, and V. Veeravalli, "Distributed stochastic subgradientprojection algorithms for convex optimization," J. Optim. TheoryAppl., vol. 147, no. 3, pp. 516-545, 2011.
10. I. Lobel and A. Ozdaglar, "Convergence analysis of distributedsubgradient methods over random networks," in Proc. 46th Annu.Allerton Conf. Commun., Control, Comput., Monticello, IL, Sep.2008, pp. 353-360.
11. I. Matei and J. S. Baras, "Performance evaluation of the consensusbaseddistributed subgradient method under random communicationtopologies," IEEE J. Selected Topics Signal Process., vol. 5, no. 4, pp.754-771, 2011.
12. C. Lopes and A. H. Sayed, "Adaptive estimation algorithms over distributednetworks," in Proc. 21st IEICE Signal Process. Symp., Kyoto,Japan, Nov. 2006.
13. J. Chen and A. H. Sayed, "Diffusion adaptation strategies for distributedoptimization and learning over networks," IEEE Trans. Sig.Process., vol. 60, no. 8, pp. 4289-4305, Aug. 2012.
14. J. Duchi, A. Agarwal, and M. Wainwright, "Dual averaging for distributedoptimization: Convergence and network scaling," IEEE Trans.Autom. Control, vol. 57, no. 3, pp. 592-606, Mar. 2012.
15. K. Tsianos and M. Rabbat, "Distributed consensus and optimizationunder communication delays," in Proc. 49th Allerton Conf. Commun.,Control, Comput., Monticello, IL, Sep. 2011, pp. 974-982.
16. M. Zhu and S. Martínez, "On distributed convex optimization underinequality and equality constraints," IEEE Trans. Autom. Control, vol.57, no. 1, pp. 151-164, Jan. 2012.
17. Y. E. Nesterov, "A method for solving the convex programmingproblem with convergence rate ," (in Russian) Dokl. Akad.Nauk SSSR, vol. 269, pp. 543-547, 1983.
18. D. Jakovetic, J. Xavier, and J. M. F. Moura, Fast Distributed GradientMethods [Online]. Available: http://arxiv.org/abs/1112.2972
19. A. Chen and A. Ozdaglar, "A fast distributed proximal gradientmethod," in Proc. 50th Allerton Conf. Commun., Control Comput.,Monticello, IL, Oct. 2012, pp. 601-608.
20. A. Chen, "Fast Distributed First-Order Methods," M.S. thesis, Mass.Inst. Technol. (MIT), Cambridge, 2012.
21. D. Jakovetic, J. Xavier, and J. M. F. Moura, "Cooperative convex optimizationin networked systems: Augmented Lagrangian algorithmswith directed gossip communication," IEEE Trans. Signal Process.,vol. 59, no. 8, pp. 3889-3902, Aug. 2011.
22. J. Mota, J. Xavier, P. Aguiar, and M. Pueschel, "Basis pursuit in sensornetworks," in Proc. IEEE Int. Conf. Acoust., Speech, Signal Process.(ICASSP'11), Prague, Czech Republic, May 2011, pp. 2916-2919.
23. J. Mota, J. Xavier, P. Aguiar, and M. Pueschel, "Distributed basis pursuit,"IEEE Trans. Sig. Process., vol. 60, no. 4, pp. 1942-1956, Apr.2012.
24. U. V. Shanbhag, J. Koshal, and A. Nedic, "Multiuser optimization:distributed algorithms and error analysis," SIAMJ. ControlOptim., vol.21, no. 2, pp. 1046-1081, 2011.
25. H. Terelius, U. Topcu, and R. M. Murray, "Decentralized multi-agentoptimization via dual decomposition," in Proc. 18th World Congr.o Int. Fed. Autom. Control (IFAC), Milano, Italy, Aug. 2011, pp.7391-7397.
26. E. Ghadimi, I. Shames, and M. Johansson, "Accelerated gradientmethods for networked optimization," in Proc. Amer. Control Conf.(ACC'11), San Francisco, CA, Jun. 2011, pp. 1668-1673.
27. E. Ghadimi, I. Shames, and M. Johansson, Accelerated GradientMethods for Networked Optimization 2012 [Online]. Available:http://arxiv.org/abs/1211.2132
28. L. Xiao, S. Boyd, and S. Lall, "A scheme for robust distributed sensorfusion based on average consensus," in Proc. Inform. Process. SensorNetworks (IPSN'05), Los Angeles, CA, 2005, pp. 63-70. (Pubitemid 43841216)
29. D. Blatt, A. Hero, and H. Gauchman, "A convergent incremental gradientmethod with a constant step size," Siam J. Optim., vol. 18, no. 1,pp. 29-51, 2009. (Pubitemid 351271319)
30. P. Tseng, "On accelerated proximal-gradient methods for convex-concaveoptimization," SIAM J. Optim, submitted for publication.
31. L. Vandenberghe, Optimization Methods for Large-Scale Systems2010, Lecture notes [Online]. Available: http://www.ee.ucla.edu/~vandenbe/ee236c.html
32. D. Jakovetic, J. M. F. Moura, and J. Xavier, "Distributed Nesterovlikegradient algorithms," in Proc. 51st IEEE Conf. Decision Control(CDC'12), Dec. 2012, pp. 5459-5464.
33. O. Devolder, F. Glineur, and Y. Nesterov, "First-order methods ofsmooth convex optimization with inexact oracle," Math. Programm.,submitted for publication.
34. G. Shi and K. H. Johansson, "Finite-Time and Asymptotic Convergenceof Distributed Averaging and Maximizing Algorithms," Tech.Rep., 2012 [Online]. Available: http://arxiv.org/pdf/1205.1733.pdf
35. D. Kempe and F. McSherry, "A decentralized algorithm for spectralanalysis," in Proc. 36th Annu. ACM Symp. Theory Comput., Chicago,IL, Aug. 2004, pp. 561-568.
36. S. Boyd, A. Ghosh, B. Prabhakar, andD. Shah, "Randomized gossip algorithms,"IEEE Trans. Inform. Theory, vol. 52, no. 6, pp. 2508-2530,Jun. 2006. (Pubitemid 43898906)
37. M. Zargham, A. Ribeiro, and A. Jadbabaie, "A distributed line searchfor network optimization," in Proc. Amer. Control Conf., Montréal,QC, Canada, Jun. 2012, pp. 472-477.
38. Y. Nesterov, "Gradient Methods for Minimizing Composite ObjectiveFunction," Center for Operations Research and Econometrics (CORE),Catholic University of Louvain (UCL), Tech. Rep. 76, 2007.