Flexible selective parallel algorithms for big data optimization

Conference Record - Asilomar Conference on Signals, Systems and Computers

Volumn 2015-April, Issue , 2015, Pages 3-7

  Daneshmand, Amir ^a   Facchinei, Francisco ^b   Kungurtsev, Vyacheslav ^c   Scutari, Gesualdo ^a  

^a the State University of New York   (United States)

^b SAPIENZA UNIVERSITY OF ROME   (Italy)

^c CZECH TECHNICAL UNIVERSITY IN PRAGUE   (Czech Republic)

Author keywords

[No Author keywords available]

Indexed keywords

BIG DATA;

ALMOST SURE CONVERGENCE; CONVEX APPROXIMATION; CURRENT SCHEMES; DATA OPTIMIZATION; DECOMPOSITION SCHEME; NONCONVEX FUNCTIONS; NUMERICAL RESULTS; PARALLEL OPTIMIZATION;

ITERATIVE METHODS;

EID: 84940487684     PISSN: 10586393     EISSN: None     Source Type: Conference Proceeding    
DOI: 10.1109/ACSSC.2014.7094384     Document Type: Conference Paper

References (32)

1. J. K. Bradley, A. Kyrola, D. Bickson, and C. Guestrin, Parallel coordinate descent for l1-regularized loss minimization, in Proc. of the 28th International Conference on Machine Learning, Bellevue, WA, USA, June 28-July 2, 2011
2. P. Tseng and S. Yun, A coordinate gradient descent method for nonsmooth separable minimization, Mathematical Programming, vol. 117, no. 1-2, pp. 387-423, March 2009
3. M. Patriksson, Cost approximation: a unified framework of descent algorithms for nonlinear programs, SIAM Journal on Optimization, vol. 8, no. 2, pp. 561-582, 1998
4. F. Facchinei, S. Sagratella, and G. Scutari, Flexible parallel algorithms for big data optimization, in Proc. of the IEEE 2014 International Conference on Acoustics, Speech, and Signal Processing (ICASSP 2014), Florence, Italy, May 4-9, 2014
5. F. Facchinei, G. Scutari, and S. Sagratella, HyFLEXA: A hybrid random/deterministic parallel algorithm for nonconvex big data optimization, IEEE Trans. on Signal Process., (to appear) 2014. Available: http://arxiv.org/abs/1402.5521
6. O. Fercoq, Z. Qu, P. Richtarik, and M. Takač, Fast distributed coordinate descent for non-strongly convex losses, arXiv preprint arXiv:1405.5300, 2014
7. O. Fercoq and P. Richtarik, Accelerated, parallel and proximal coordinate descent, arXiv preprint arXiv:1312.5799, 2013
8. Z. Lu and L. Xiao, Randomized Block Coordinate Non-Monotone Gradient Method for a Class of Nonlinear Programming, arXiv preprint arXiv:1306.5918v1, 2013
9. I. Necoara and D. Clipici, Efficient parallel coordinate descent algorithm for convex optimization problems with separable constraints: application to distributed MPC, Journal of Process Control, vol. 23, no. 3, pp. 243-253, March 2013
10. Distributed random coordinate descent method for composite minimization, Technical Report, pp. 1-41, Nov. 2013. [Online]. Available: http://arxiv-web.arxiv.org/abs/1312.5302
11. Y. Nesterov, Gradient methods for minimizing composite functions, Mathematical Programming, vol. 140, pp. 125-161, August 2013
12. Efficiency of coordinate descent methods on huge-scale optimization problems, SIAM Journal on Optimization, vol. 22, no. 2, pp. 341-362, 2012
13. Z. Peng, M. Yan, and W. Yin, Parallel and distributed sparse optimization, in Proc. of the 2013 Asilomar Conference on Signals, Systems and Computers, Pacific Grove, CA, USA, 2013, pp. 659-646
14. P. Richtarik and M. Takač, Parallel coordinate descent methods for big data optimization, arXiv preprint arXiv:1212.0873, 2012
15. A. Beck and M. Teboulle, A fast iterative shrinkage-thresholding algorithm for linear inverse problems, SIAM Journal on Imaging Sciences, vol. 2, no. 1, pp. 183-202, Jan. 2009
16. S. J. Wright, R. D. Nowak, and M. A. Figueiredo, Sparse reconstruction by separable approximation, IEEE Trans. on Signal Processing, vol. 57, no. 7, pp. 2479-2493, July 2009
17. S. Shalev-Shwartz and A. Tewari, Stochastic methods for -1-regularized loss minimization, The Journal of Machine Learning Research, pp. 1865-1892, 2011
18. I. Necoara and A. Patrascu, A random coordinate descent algorithm for optimization problems with composite objective function and linear coupled constraints, Computational Optimization and Applications, vol. 57, no. 2, pp. 307-337, 2014
19. A. Patrascu and I.Necoara, Efficient random coordinate descent algorithms for large-scale structured nonconvex optimization, J. of Global Optimization, pp. 1-23, Feb. 2014
20. P. Richtarik and M. Takač, Iteration complexity of randomized blockcoordinate descent methods for minimizing a composite function, Mathematical Programming, vol. 144, no. 1-2, pp. 1-38, 2014
21. I. Dassios, K. Fountoulakis, and J. Gondzio, A second-order method for compressed sensing problems with coherent and redundant dictionaries, arXiv preprint arXiv:1405.4146, 2014
22. K. Fountoulakis and J. Gondzio, A Second-Order Method for Strongly Convex L1-Regularization Problems, arXiv:1306.5386, 2013
23. G.-X. Yuan, C.-H. Ho, and C.-J. Lin, An improved glmnet for -1-regularized logistic regression, The Journal of Machine Learning Research, vol. 13, no. 1, pp. 1999-2030, 2012
24. G. Scutari, F. Facchinei, P. Song, D. Palomar, and J.-S. Pang, Decomposition by Partial linearization: Parallel optimization of multi-agent systems, IEEE Trans. Signal Process., vol. 62, pp. 641-656, Feb. 2014
25. C. Scherrer, A. Tewari, M. Halappanavar, and D. Haglin, Feature clustering for accelerating parallel coordinate descent, in Advances in Neural Information Processing Systems (NIPS2012). Curran Associates, Inc., 2012, pp. 28-36
26. M. Razaviyayn, M. Hong, Z.-Q. Luo, and J.-S. Pang, Parallel successive convex approximation for nonsmooth nonconvex optimization, in Proc. of the Annual Conference on Neural Information Processing Systems (NIPS), Montreal, Quebec, CA, to appear, Dec. 2014
27. R. Tappenden, P. Richtarik, and J. Gondzio, Inexact coordinate descent: complexity and preconditioning, arXiv preprint arXiv:1304.5530, 2013
28. D. P. Bertsekas and J. N. Tsitsiklis, Parallel and Distributed Computation: Numerical Methods, 2nd ed. Athena Scientific Press, 1989
29. A. Daneshmand, F. Facchinei, V. Kungurtsev, and G. Scutari, Hybrid random/deterministic parallel algorithms for nonconvex big data optimization, IEEE Trans. on Signal Process., (submitted), 2014. Available: http://arxiv.org/abs/1407.4504
30. P. Richtarik and M. Takač, On optimal probabilities in stochastic coordinate descent methods, arXiv preprint arXiv:1310.3438, 2013
31. A. Daneshmand, Numerical Comparison of Hybrid Random/ Deterministic Parallel Algorithms for nonconvex big data Optimization, Dept. of Elect. Eng., SUNY Buffalo, Tech. Rep., August 2014. [Online]. Available: http://www.eng.buffalo.edu/amirdane/ DaneshmandTechRepNumCompAug14.pdf
32. P. Richtarik and M. Takač, Distributed coordinate descent method for learning with big data, arXiv preprint arXiv:1310.2059, 2013