A universal catalyst for first-order optimization

Advances in Neural Information Processing Systems

Volumn 2015-January, Issue , 2015, Pages 3384-3392

  Lin, Hongzhou ^a   Mairal, Julien ^a   Harchaoui, Zaid ^a,b  

^a INRIA   (France)

^b NEW YORK UNIVERSITY   (United States)

Author keywords

[No Author keywords available]

Indexed keywords

ALGORITHMS; INFORMATION SCIENCE;

AUXILIARY PROBLEM; BLOCK COORDINATE DESCENTS; CONVEX OBJECTIVES; FASTER CONVERGENCE; FIRST ORDER OPTIMIZATION METHOD; GRADIENT DESCENT; ILL CONDITIONED PROBLEMS; PROXIMAL POINT ALGORITHM;

OPTIMIZATION;

EID: 84965134988     PISSN: 10495258     EISSN: None     Source Type: Conference Proceeding    
DOI: None     Document Type: Conference Paper

References (28)

1. A. Agarwal and L. Bottou. A lower bound for the optimization of finite sums. In Proc. International Conference on Machine Learning (ICML), 2015.
2. F. Bach, R. Jenatton, J. Mairal, and G. Obozinski. Optimization with sparsity-inducing penalties. Foundations and Trends in Machine Learning, 4(1):1-106, 2012.
3. H. H. Bauschke and P. L. Combettes. Convex Analysis and Monotone Operator Theory in Hilbert Spaces. Springer, 2011.
4. A. Beck and M. Teboulle. A fast iterative shrinkage-thresholding algorithm for linear inverse problems. SIAM Journal on Imaging Sciences, 2(1):183-202, 2009.
5. D. P. Bertsekas. Convex Optimization Algorithms. Athena Scientific, 2015.
6. A. J. Defazio, F. Bach, and S. Lacoste-Julien. SAGA: A fast incremental gradient method with support for non-strongly convex composite objectives. In Adv. Neural Information Processing Systems (NIPS), 2014.
7. A. J. Defazio, T. S. Caetano, and J. Domke. Finito: A faster, permutable incremental gradient method for big data problems. In Proc. International Conference on Machine Learning (ICML), 2014.
8. R. Frostig, R. Ge, S. M. Kakade, and A. Sidford. Un-regularizing: approximate proximal point algorithms for empirical risk minimization. In Proc. International Conference on Machine Learning (ICML), 2015.
9. O. Güler. New proximal point algorithms for convex minimization. SIAM Journal on Optimization, 2(4):649-664, 1992.
10. B. He and X. Yuan. An accelerated inexact proximal point algorithm for convex minimization. Journal of Optimization Theory and Applications, 154(2):536-548, 2012.
11. J.-B. Hiriart-Urruty and C. Lemaréchal. Convex Analysis and Minimization Algorithms I. Springer, 1996.
12. A. Juditsky and A. Nemirovski. First order methods for nonsmooth convex large-scale optimization. Optimization for Machine Learning, MIT Press, 2012.
13. G. Lan. An optimal randomized incremental gradient method. arXiv:1507.02000, 2015.
14. J. Mairal. Incremental majorization-minimization optimization with application to large-scale machine learning. SIAM Journal on Optimization, 25(2):829-855, 2015.
15. A. Nemirovski, A. Juditsky, G. Lan, and A. Shapiro. Robust stochastic approximation approach to stochastic programming. SIAM Journal on Optimization, 19(4):1574-1609, 2009.
16. Y. Nesterov. A method of solving a convex programming problem with convergence rate O (1/k2). Soviet Mathematics Doklady, 27(2):372-376, 1983.
17. Y. Nesterov. Introductory Lectures on Convex Optimization: A Basic Course. Springer, 2004.
18. Y. Nesterov. Efficiency of coordinate descent methods on huge-scale optimization problems. SIAM Journal on Optimization, 22(2):341-362, 2012.
19. Y. Nesterov. Gradient methods for minimizing composite functions. Mathematical Programming, 140(1):125-161, 2013.
20. N. Parikh and S. P. Boyd. Proximal algorithms. Foundations and Trends in Optimization, 1(3):123-231, 2014.
21. P. Richtárik and M. Takáč. Iteration complexity of randomized block-coordinate descent methods for minimizing a composite function. Mathematical Programming, 144(1-2):1-38, 2014.
22. S. Salzo and S. Villa. Inexact and accelerated proximal point algorithms. Journal of Convex Analysis, 19(4):1167-1192, 2012.
23. M. Schmidt, N. Le Roux, and F. Bach. Convergence rates of inexact proximal-gradient methods for convex optimization. In Adv. Neural Information Processing Systems (NIPS), 2011.
24. M. Schmidt, N. Le Roux, and F. Bach. Minimizing finite sums with the stochastic average gradient. arXiv:1309.2388, 2013.
25. S. Shalev-Shwartz and T. Zhang. Proximal stochastic dual coordinate ascent. arXiv:1211.2717, 2012.
26. S. Shalev-Shwartz and T. Zhang. Accelerated proximal stochastic dual coordinate ascent for regularized loss minimization. Mathematical Programming, 2015.
27. L. Xiao and T. Zhang. A proximal stochastic gradient method with progressive variance reduction. SIAM Journal on Optimization, 24(4):2057-2075, 2014.
28. Y. Zhang and L. Xiao. Stochastic primal-dual coordinate method for regularized empirical risk minimization. In Proc. International Conference on Machine Learning (ICML), 2015.