Flexible parallel algorithms for big data optimization

ICASSP, IEEE International Conference on Acoustics, Speech and Signal Processing - Proceedings

Volumn , Issue , 2014, Pages 7208-7212

  Facchinei, Francisco ^a   Sagratella, Simone ^a   Scutari, Gesualdo ^b  

^a SAPIENZA UNIVERSITY OF ROME   (Italy)

^b State University of New York   (United States)

Author keywords

Jacobi method; LASSO; Parallel optimization; Sparse solution

Indexed keywords

ALGORITHMS; ITERATIVE METHODS; SIGNAL PROCESSING;

CONVERGENCE RESULTS; DATA OPTIMIZATION; DIFFERENTIABLE FUNCTIONS; JACOBI METHODS; LASSO; NEWTON-TYPE METHODS; PARALLEL OPTIMIZATION; SPARSE SOLUTIONS;

CONVERGENCE OF NUMERICAL METHODS;

EID: 84905215270     PISSN: 15206149     EISSN: None     Source Type: Conference Proceeding    
DOI: 10.1109/ICASSP.2014.6854999     Document Type: Conference Paper

References (32)

1. F. Bach, R. Jenatton, J. Mairal, and G. Obozinski, "Optimization with sparsity-inducing penalties," arXiv preprint arXiv:1108. 0775, 2011.
2. J. K. Bradley, A. Kyrola, D. Bickson, and C. Guestrin, "Parallel coordinate descent for l1-regularized loss minimization," arXiv preprint arXiv:1105. 5379, 2011.
3. P. L. Bühlmann, S. A. van de Geer, and S. Van de Geer, Statistics for high-dimensional data. Springer, 2011.
4. R. H. Byrd, J. Nocedal, and F. Oztoprak, "An Inexact Successive Quadratic Approximation Method for Convex L-1 Regularized Optimization," arXiv preprint arXiv:1309. 3529, 2013.
5. K. Fountoulakis and J. Gondzio, "A Second-Order Method for Strongly Convex L1-Regularization Problems," arXiv preprint arXiv:1306. 5386, 2013.
6. I. Necoara and D. Clipici, "Efficient parallel coordinate descent algorithm for convex optimization problems with separable constraints: application to distributed MPC," Journal of Process Control, vol. 23, no. 3, pp. 243-253, 2013.
7. Y. Nesterov, "Gradient methods for minimizing composite functions," Mathematical Programming, pp. 1-37, 2012.
8. -, "Efficiency of coordinate descent methods on hugescale optimization problems," SIAM Journal on Optimization, vol. 22, no. 2, pp. 341-362, 2012.
9. Z. Qin, K. Scheinberg, and D. Goldfarb, "Efficient blockcoordinate descent algorithms for the group lasso," Mathematical Programming Computation, pp. 1-27, 2010.
10. A. Rakotomamonjy, "Surveying and comparing simultaneous sparse approximation (or group-lasso) algorithms," Signal processing, vol. 91, no. 7, pp. 1505-1526, 2011.
11. M. Razaviyayn, M. Hong, and Z.-Q. Luo, "A unified convergence analysis of block successive minimization methods for nonsmooth optimization," SIAM Journal on Optimization, vol. 23, no. 2, pp. 1126-1153, 2013.
12. P. Richtárik and M. Takáč, "Iteration complexity of randomized block-coordinate descent methods for minimizing a composite function," Mathematical Programming, pp. 1-38, 2012.
13. -, "Parallel coordinate descent methods for big data optimization," arXiv preprint arXiv:1212. 0873, 2012.
14. S. Sra, S. Nowozin, and S. J. Wright, Eds., Optimization for Machine Learning, ser. Neural Information Processing. Cambridge, Massachusetts: The MIT Press, Sept. 2011.
15. P. Tseng and S. Yun, "A coordinate gradient descent method for nonsmooth separable minimization," Mathematical Programming, vol. 117, no. 1-2, pp. 387-423, 2009.
16. Y. Xu and W. Yin, "A block coordinate descent method for multi-convex optimization with applications to nonnegative tensor factorization and completion," DTIC Document, Tech. Rep., 2012. [Online]. Available: http://www. caam. rice. edu/~optimization/bcu/multiconvex. html
17. Z. Yin, P. Ming, and Y. Wotao, "Parallel and Distributed Sparse Optimization," 2013. [Online]. Available: http://www. caam. rice. edu/~optimization/disparse
18. G.-X. Yuan, K.-W. Chang, C.-J. Hsieh, and C.-J. Lin, "A comparison of optimization methods and software for large-scale l1-regularized linear classification," The Journal of Machine Learning Research, vol. 9999, pp. 3183-3234, 2010.
19. S. J. Wright, "Accelerated block-coordinate relaxation for regularized optimization," SIAM Journal on Optimization, vol. 22, no. 1, pp. 159-186, 2012.
20. G. Scutari, F. Facchinei, P. Song, D. P. Palomar, and J.-S. Pang, "Decomposition by partial linearization in multiuser systems," in IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP 2013), May 4-9 2013, pp. 4424-4428.
21. G. Scutari, F. Facchinei, P. Song, D. Palomar, and J.-S. Pang, "Decomposition by Partial linearization: Parallel optimization of multi-agent systems," IEEE Trans. Signal Process., vol. 62, pp. 641-656, Feb. 2014.
22. R. Tibshirani, "Regression shrinkage and selection via the lasso," Journal of the Royal Statistical Society. Series B (Methodological), pp. 267-288, 1996.
23. M. Yuan and Y. Lin, "Model selection and estimation in regression with grouped variables," Journal of the Royal Statistical Society: Series B (Statistical Methodology), vol. 68, no. 1, pp. 49-67, 2006.
24. S. K. Shevade and S. S. Keerthi, "A simple and efficient algorithm for gene selection using sparse logistic regression," Bioinformatics, vol. 19, no. 17, pp. 2246-2253, 2003.
25. L. Meier, S. Van De Geer, and P. Bühlmann, "The group lasso for logistic regression," Journal of the Royal Statistical Society: Series B (Statistical Methodology), vol. 70, no. 1, pp. 53-71, 2008.
26. D. Goldfarb, S. Ma, and K. Scheinberg, "Fast alternating linearization methods for minimizing the sum of two convex functions," Mathematical Programming, pp. 1-34, 2012.
27. D. P. Bertsekas and J. N. Tsitsiklis, Parallel and Distributed Computation: Numerical Methods, 2nd ed. Athena Scientific Press, 1989.
28. F. Facchinei, S. Sagratella, and G. Scutari, "Parallel Algorithms for Big Data Optimization," IEEE Trans. Signal Process., (under review). [Online]. Available: http://arxiv. org/pdf/1402. 5521v2. pdf.
29. F. Facchinei and J.-S. Pang, Finite-Dimensional Variational Inequalities and Complementarity Problem. Springer-Verlag, New York, 2003.
30. A. Beck and M. Teboulle, "A fast iterative shrinkagethresholding algorithm for linear inverse problems," SIAM Journal on Imaging Sciences, vol. 2, no. 1, pp. 183-202, 2009.
31. S. Boyd, N. Parikh, E. Chu, B. Peleato, and J. Eckstein, "Distributed optimization and statistical learning via the alternating direction method of multipliers," Foundations and Trends-R in Machine Learning, vol. 3, no. 1, pp. 1-122, 2011.
32. Z.-Q. Luo and M. Hong, "On the linear convergence of the alternating direction method of multipliers," arXiv preprint arXiv:1208. 3922, 2012.