Parallel successive convex approximation for nonsmooth nonconvex optimization

Advances in Neural Information Processing Systems

Volumn 2, Issue January, 2014, Pages 1440-1448

  Razaviyayn, Meisam ^a   Hong, Mingyi ^b   Luo, Zhi Quan ^c   Pang, Jong Shi ^d  

^a STANFORD UNIVERSITY   (United States)

^b Iowa State University of Science and Technology   (United States)

^c UNIVERSITY OF MINNESOTA   (United States)

^d Department of Industrial and Systems Engineering University of Southern California ^*  (United States)

Author keywords

[No Author keywords available]

Indexed keywords

INFORMATION SCIENCE; PROBLEM SOLVING;

BLOCK COORDINATE DESCENTS; CONVEX APPROXIMATION; MINIMIZATION PROBLEMS; MULTI-CORE PARALLELS; NON-CONVEX OBJECTIVE FUNCTIONS; NONSMOOTH NONCONVEX OPTIMIZATION; NUMERICAL EXPERIMENTS; SUCCESSIVE CONVEX APPROXIMATIONS;

ITERATIVE METHODS;

EID: 84937954889     PISSN: 10495258     EISSN: None     Source Type: Conference Proceeding    
DOI: None     Document Type: Conference Paper

References (29)

1. Y. Nesterov. Efficiency of coordinate descent methods on huge-scale optimization problems. SIAM Journal on Optimization, 22(2): 341-362, 2012.
2. P. Richtárik and M. Takáč. Efficient serial and parallel coordinate descent methods for huge-scale truss topology design. In Operations Research Proceedings, pages 27-32. Springer, 2012.
3. Y. T. Lee and A. Sidford. Efficient accelerated coordinate descent methods and faster algorithms for solving linear systems. In 54th Annual Symposium on Foundations of Computer Science (FOCS), pages 147-156. IEEE, 2013.
4. I. Necoara and D. Clipici. Efficient parallel coordinate descent algorithm for convex optimization problems with separable constraints: application to distributed MPC. Journal of Process Control, 23(3): 243-253, 2013.
5. Y. Nesterov. Gradient methods for minimizing composite functions. Mathematical Programming, 140(1): 125-161, 2013.
6. P. Tseng and S. Yun. A coordinate gradient descent method for nonsmooth separable minimization. Mathematical Programming, 117(1-2): 387-423, 2009.
7. A. Beck and M. Teboulle. A fast iterative shrinkage-thresholding algorithm for linear inverse problems. SIAM Journal on Imaging Sciences, 2(1): 183-202, 2009.
8. S. J. Wright, R. D. Nowak, and M. Figueiredo. Sparse reconstruction by separable approximation. IEEE Transactions on Signal Processing, 57(7): 2479-2493, 2009.
9. F. Facchinei, S. Sagratella, and G. Scutari. Flexible parallel algorithms for big data optimization. arXiv preprint arXiv: 1311.2444, 2013.
10. J. K. Bradley, A. Kyrola, D. Bickson, and C. Guestrin. Parallel coordinate descent for l1-regularized loss minimization. arXiv preprint arXiv: 1105.5379, 2011.
11. C. Scherrer, A. Tewari, M. Halappanavar, and D. Haglin. Feature clustering for accelerating parallel coordinate descent. In NIPS, pages 28-36, 2012.
12. C. Scherrer, M. Halappanavar, A. Tewari, and D. Haglin. Scaling up coordinate descent algorithms for large l1 regularization problems. arXiv preprint arXiv: 1206.6409, 2012.
13. Z. Peng, M. Yan, and W. Yin. Parallel and distributed sparse optimization. preprint, 2013.
14. I. Necoara and D. Clipici. Distributed coordinate descent methods for composite minimization. arXiv preprint arXiv: 1312.5302, 2013.
15. P. Richtarik and M. Takáč. Parallel coordinate descent methods for big data optimization. arXiv preprint arXiv: 1212.0873, 2012.
16. P. Richtarik and M. Takáč. On optimal probabilities in stochastic coordinate descent methods. arXiv preprint arXiv: 1310.3438, 2013.
17. O. Fercoq and P. Richtarik. Accelerated, parallel and proximal coordinate descent. arXiv preprint arXiv: 1312.5799, 2013.
18. O. Fercoq, Z. Qu, P. Richtarik, and M. Takáč. Fast distributed coordinate descent for non-strongly convex losses. arXiv preprint arXiv: 1405.5300, 2014.
19. O. Fercoq and P. Richtarik. Smooth minimization of nonsmooth functions with parallel coordinate descent methods. arXiv preprint arXiv: 1309.5885, 2013.
20. A. Patrascu and I. Necoara. A random coordinate descent algorithm for large-scale sparse nonconvex optimization. In European Control Conference (ECC), pages 2789-2794. IEEE, 2013.
21. M. Razaviyayn, M. Hong, and Z.-Q. Luo. A unified convergence analysis of block successive minimization methods for nonsmooth optimization. SIAM Journal on Optimization, 23(2): 1126-1153, 2013.
22. F. Niu, B. Recht, C. Ré, and S. J. Wright. Hogwild!: A lock-free approach to parallelizing stochastic gradient descent. Advances in Neural Information Processing Systems, 24: 693-701, 2011.
23. J. Liu, S. J. Wright, C. Ré, and V. Bittorf. An asynchronous parallel stochastic coordinate descent algorithm. arXiv preprint arXiv: 1311.1873, 2013.
24. J. Mairal. Optimization with first-order surrogate functions. arXiv preprint arXiv: 1305.3120, 2013.
25. J. Mairal. Incremental majorization-minimization optimization with application to large-scale machine learning. arXiv preprint arXiv: 1402.4419, 2014.
26. M. Razaviyayn, M. Sanjabi, and Z.-Q. Luo. A stochastic successive minimization method for nonsmooth nonconvex optimization with applications to transceiver design in wireless communication networks. arXiv preprint arXiv: 1307.4457, 2013.
27. D. P. Bertsekas and J. N. Tsitsiklis. Neuro-dynamic programming. 1996.
28. M. Hong, X. Wang, M. Razaviyayn, and Z.-Q. Luo. Iteration complexity analysis of block coordinate descent methods. arXiv preprint arXiv: 1310.6957, 2013.
29. Y Nesterov. Introductory lectures on convex optimization: A basic course, Volume 87. Springer, 2004.