Parallel and Distributed Methods for Constrained Nonconvex Optimization - Part I: Theory

IEEE Transactions on Signal Processing

Volumn 65, Issue 8, 2017, Pages 1929-1944

  Scutari, Gesualdo ^a   Facchinei, Francisco ^b   Lampariello, Lorenzo ^c  

^a SCHOOL OF INDUSTRIAL ENGINEERING   (United States)

^b SAPIENZA UNIVERSITY OF ROME   (Italy)

^c ROMA TRE UNIVERSITY   (Italy)

Author keywords

Distributed algorithms; nonconvex optimization; successive convex approximation

Indexed keywords

APPROXIMATION ALGORITHMS; CONSTRAINED OPTIMIZATION; LEARNING SYSTEMS; PARALLEL ALGORITHMS;

ALGORITHMIC FRAMEWORK; CONSTRAINED NONCONVEX OPTIMIZATIONS; DISTRIBUTED METHODS; NONCONVEX OPTIMIZATION; OPTIMIZATION PROBLEMS; SMOOTH CONSTRAINTS; STATIONARY SOLUTIONS; SUCCESSIVE CONVEX APPROXIMATIONS;

ITERATIVE METHODS;

EID: 85046981828     PISSN: 1053587X     EISSN: None     Source Type: Journal    
DOI: 10.1109/TSP.2016.2637317     Document Type: Article

References (58)

1. G. Scutari, F. Facchinei, L. Lampariello, and P. Song, "Parallel and distributed methods for nonconvex optimization," in Proc. IEEE Int. Conf. Acoust., Speech, Signal Process., Florence, Italy,May 4-9, 2014, pp. 840-844.
2. P. Song, G. Scutari, F. Facchinei, and L. Lampariello, "D3m: Distributed multi-cell multigroup multicasting," in Proc. IEEE Int. Conf. Acoust. Speech, Signal Process., Shanghai, China, Mar. 20-25, 2016, pp. 3741-3745.
3. G. Scutari, F. Facchinei, L. Lampariello, and P. Song, "Parallel and distributed methods for nonconvex optimization - Part I: Theory," Oct. 2014, arXiv:1410.4754.
4. R. Byrd, J. Nocedal, and R.Waltz, "Feasible interior methods using slacks for nonlinear optimization," Comput. Opt. Appl., vol. 26, no. 1, pp. 35-61, 2003.
5. A. Fiacco and G. M. Cormick, Nonlinear Programming: Sequential Unconstrained Minimization Techniques. Hoboken, NJ, USA: Wiley, 1968.
6. C. Lawrence and A. Tits, "A computationally efficient feasible sequential quadratic programming algorithm," SIAM J. Optim., vol. 11, no. 4, pp. 1092-1118, 2001.
7. M. Ferris and O. Mangasarian, "Parallel variable distribution," SIAM J. Optim., vol. 4, no. 4, pp. 815-832, 1994.
8. C. Sagastizábal and M. Solodov, "Parallel variable distribution for constrained optimization," Comput. Optim. Appl., vol. 22, no. 1, pp. 111-131, 2002.
9. M. Solodov, "On the convergence of constrained parallel variable distribution algorithms," SIAM J. Optim., vol. 8, no. 1, pp. 187-196, 1998.
10. B. Marks and G. Wright, "A general inner approximation algorithm for nonconvex mathematical programs," Oper. Res., vol. 26, no. 4, pp. 681-683, 1978.
11. A. Beck, A. Ben-Tal, and L. Tetruashvili, "A sequential parametric convex approximation method with applications to nonconvex truss topology design problems," J. Global Optim., vol. 47, no. 1, pp. 29-51, 2010. doi: 10.1007/s10898-009-9456-5.
12. M. Razaviyayn, M. Hong, and Z.-Q. Luo, "A unified convergence analysis of block successive minimization methods for nonsmooth optimization," SIAM J. Optim., vol. 23, no. 2, pp. 1126-1153, 2013.
13. G. Scutari, F. Facchinei, P. Song, D. P. Palomar, and J.-S. Pang, "Decomposition by partial linearization: Parallel optimization of multiuser systems," IEEE Trans. Signal Process., vol. 62, no. 3, pp. 641-656, Feb. 2014.
14. A. Alvarado, G. Scutari, and J.-S. Pang, "A new decomposition method for multiuser dc-programming and its applications," IEEE Trans. Signal Process., vol. 62, no. 11, pp. 2984-2998, Mar. 2014.
15. T. Quoc and M. Diehl, "Sequential convex programming methods for solving nonlinear optimization problems with dc constraints," 2011, arXiv:1107.5841.
16. C. Fleury, "CONLIN: An efficient dual optimizer based on convex approximation concepts," Struct. Opt., vol. 1, no. 2, pp. 81-89, 1989.
17. K. Svanberg, "The method of moving asymptotes-A newmethod for structural optimization," Int. J. Numer. Methods Eng., vol. 24, no. 2, pp. 359-373, 1987.
18. K. Svanberg, "A class of globally convergent optimization methods based on conservative convex separable approximations," SIAM J. Opt., vol. 12, no. 2, pp. 555-573, 2002.
19. G. Scutari, F. Facchinei, L. Lampariello, S. Sardellitti, and P. Song, "Parallel and distributed methods for constrained nonconvex optimization - Part II: Applications in communications and machine learning," IEEE Trans. Signal Process., vol. 65, no. 8, pp. 1945-1960, 2017.
20. D. P. Palomar and M. Chiang, "Alternative distributed algorithms for network utility maximization: Framework and applications," IEEE Trans. Autom. Control, vol. 52, no. 12, pp. 2254-2269, Dec. 2007.
21. D. P. Bertsekas and J. N. Tsitsiklis, Parallel and Distributed Computation: Numerical Methods, 2nd ed. Belmont,MA, USA: Athena Scientific, 1989.
22. K. Phan, S. Vorobyov, C. Telambura, and T. Le-Ngoc, "Power control for wireless cellular systems via D.C. programming," in Proc. IEEE/SP 14th Workshop Stat. Signal Process., 2007, pp. 507-511.
23. H. Al-Shatri and T.Weber, "Achieving the maximum sum rate using D.C. programming in cellular networks," IEEE Trans. Signal Process., vol. 60, no. 3, pp. 1331-1341, Mar. 2012.
24. N. Vucic, S. Shi, and M. Schubert, "DC programming approach for resource allocation in wireless networks," in Proc. 8th Int. Symp. Model. Optim. Mobile, Ad Hoc Wireless Netw., 2010, pp. 380-386.
25. M. Chiang, C. Tan, D. Palomar, D. O. Neill, and D. Julian, "Power control by geometric programming," IEEE Trans.Wireless Commun., vol. 6, no. 7, pp. 2640-2651, Jul. 2007.
26. A. Khabbazibasmenj, F. Roemer, S. Vorobyov, and M. Haardt, "Sum-rate maximization in two-way AF MIMO relaying: Polynomial time solutions to a class of DC programming problems," IEEE Trans. Signal Process., vol. 60, no. 10, pp. 5478-5493, Oct. 2012.
27. Y. Xu, T. Le-Ngoc, and S. Panigrahi, "Global concave minimization for optimal spectrum balancing in multi-user DSL networks," IEEE Trans. Signal Process., vol. 56, no. 7, pp. 2875-2885, Jul. 2008.
28. P. Tsiaflakis, M. Diehl, and M. Moonen, "Distributed spectrum management algorithms for multiuser DSL networks," IEEE Trans. Signal Process., vol. 56, no. 10, pp. 4825-4843, Oct. 2008.
29. D. Schmidt, C. Shi, R. Berry, M. Honig, and W. Utschick, "Distributed resource allocation schemes: Pricing algorithms for power control and beamformer design in interference networks," IEEE Signal Process. Mag., vol. 26, no. 5, pp. 53-63, Sep. 2009.
30. R. Mochaourab, P. Cao, and E. Jorswieck, "Alternating rate profile optimization in single stream MIMO interference channels," IEEE Signal Process. Lett., vol. 21, no. 2, pp. 221-224, Feb. 2014.
31. J. Qiu, R. Zhang, Z.-Q. Luo, and S.Cui, "Optimal distributed beamforming for MISO interference channels," IEEE Trans. Signal Process., vol. 59, no. 11, pp. 5638-5643, Nov. 2011.
32. S.-J. Kim and G. B. Giannakis, "Optimal resource allocation for MIMO ad hoc cognitive radio networks," IEEE Trans. Inf. Theory, vol. 57, no. 5, pp. 3117-3131, May 2011.
33. Y. Zhang, E. Dall'Anese, andG. B.Giannakis, "Distributed optimal beamformers for cognitive radios robust to channel uncertainties," IEEE Trans. Signal Process., vol. 60, no. 12, pp. 6495-6508, Dec. 2012.
34. Y. Yang, G. Scutari, and D. Palomar, "Robust MIMO cognitive radio under interference temperature constraints," IEEE J. Sel. Areas Commun., vol. 31, no. 11, pp. 2465-2482, Nov. 2013.
35. N. D. Sidiropoulos, T. N. Davidson, and Z.-Q. Luo, "Transmit beamforming for physical-layer multicasting," IEEE Trans. Signal Process., vol. 54, no. 6, pp. 2239-2251, Jun. 2006.
36. E. Karipidis, N. Sidiropoulos, and Z.-Q. Luo, "Quality of service andmaxmin fair transmit beamforming to multiple cochannel multicast groups," IEEE Trans. Signal Process., vol. 56, no. 3, pp. 1268-1279, Mar. 2008.
37. M. Chiang, S. Low, A. Calderbank, and J. Doyle, "Layering as optimization decomposition: A mathematical theory of network architectures," Proc. IEEE, vol. 95, no. 1, pp. 255-312, Jan. 2007.
38. M. Chiang, P. Hande, T. Lan, and C. W. Tan, Power Control in Wireless Cellular Networks. Delft, The Netherlands: Now Publishers, Inc., 2008.
39. R. Rockafellar and J. Wets, Variational Analysis. Berlin, Germany: Springer-Verlag, 1998.
40. F. Facchinei and J.-S. Pang, "Exact penalty functions for generalized Nash problems," in Large Scale Nonlinear Optimization, G. D. Pillo and M. Roma, Eds. Berlin, Germany: Springer-Verlag, 2006, pp. 115-126.
41. D. Bertsekas, A. Nedic, and A. Ozdaglar, Convex Analysis and Optimization. Belmont, MA, USA: Athena Scientific, 2003.
42. A. Daneshmand, F. Facchinei, V. Kungurtsev, and G. Scutari, "Hybrid random/deterministic parallel algorithms for convex and nonconvex big data optimization," IEEE Trans. Signal Process., vol. 63, no. 15, pp. 3914-3929, Aug. 2015.
43. M. Su and H.-K. Xu, "Remarks on the gradient-projection algorithm," J. Nonlinear Anal. Optim., vol. 1, pp. 35-43, Jul. 2010.
44. Y. Nesterov, Introductory Lectures on Convex Optimization: A Basic Course (Applied Optimization). Berlin, Germany: Springer-Verlag, 2004.
45. Y. Nesterov, "Smooth minimization of non-smooth functions," Math. Program., vol. 103, no. 1, pp. 127-152, 2005. doi: 10.1007/s10107-004-0552-5.
46. D. Bertsekas, A. Nedic, and A. Ozdaglar, Convex Analysis and Optimization. Belmont, MA, USA: Athena Scientific, 2003.
47. F. Bach, R. Jenatton, J. Mairal, and G. Obozinski, Optimization with Sparsity-inducing Penalties. Foundations and Trends in Machine Learning. Delft, The Netherlands: Now Publishers, Inc., Dec. 2011.
48. A. Beck and M. Teboulle, "A fast iterative shrinkage-thresholding algorithm for linear inverse problems," SIAM J. Imag. Sci., vol. 2, no. 1, pp. 183-202, Jan. 2009.
49. F. Facchinei, S. Sagratella, and G. Scutari, "Flexible parallel algorithms for big data optimization," IEEE Trans. Signal Process., vol. 63, no. 7, pp. 1874-1889, Apr. 2015.
50. R. Tibshirani, "Regression shrinkage and selection via the lasso," J. Roy. Stat. Soc. Series B (Methodological), vol. 58, pp. 267-288, 1996.
51. H. A. Le Thi and T. Pham Dinh, "Recent advances in dc programming and DCA," in Trans. Comput. Intell. XIII, vol. 8342, 2014, pp. 1-37.
52. H. A. Le Thi, V. N. Huynh, and T. Pham Dinh, "DC programming and DCA for general dc programs," in Proc. Adv. Comput. Methods Knowl. Eng., 2014, pp. 15-35.
53. F. Clarke, Optimization and Nonsmooth Analysis. Hoboken, NJ, USA: Wiley, 1983.
54. G. Scutari, F. Facchinei, and L. Lampariello, "Parallel and distributed methods for constrained nonconvex optimization - Part I: Theory," School Ind. Eng., Purdue Univ.,West Lafayette, IN, USA, Tech. Rep., Nov. 2016. Available at arXiv:1410.4754.
55. F. Facchinei and J.-S. Pang, Finite-Dimensional Variational Inequalities and Complementarity Problems. Berlin, Germany: Springer-Verlag, 2003.
56. S. Dafermos, "Sensitivity analysis in variational inequalities," Math. Oper. Res., vol. 13, no. 3, pp. 421-434, 1988.
57. N. Yen, "Hölder continuity of solutions to a parametric variational inequality," Appl. Math. Optim., vol. 31, no. 3, pp. 245-255, 1995.
58. R. Rockafellar, "Lipschitzian properties of multifunctions," Nonlinear Anal.: Theory, Methods Appl., vol. 9, no. 8, pp. 867-885, 1985.