A lower bound for the optimization of finite sums

32nd International Conference on Machine Learning, ICML 2015

Volumn 1, Issue , 2015, Pages 78-86

  Agarwal, Alekh ^a   Bottou, Leon ^b  

^a MICROSOFT RESEARCH   (United States)

^b FACEBOOK AI RESEARCH   (United States)

Author keywords

[No Author keywords available]

Indexed keywords

ARTIFICIAL INTELLIGENCE; NUMBER THEORY; NUMERICAL ANALYSIS;

COMPLEXITY RESULTS; CONDITION NUMBERS; FINITE SUMS; FIRST ORDER METHOD; LOWER BOUNDS; RANDOM DATA; UPPER BOUND;

LEARNING SYSTEMS;

EID: 84969540371     PISSN: None     EISSN: None     Source Type: Conference Proceeding    
DOI: None     Document Type: Conference Paper

References (15)

1. Agarwal, Alekh, Negahban, Sahand, and Wainwright, Martin J. Fast global convergence of gradient methods for high-dimensional statistical recovery. The Annals of Statistics, 40(5):2452-2482, 2012.
2. Arjevani, Y. On Lower and Upper Bounds in Smooth Strongly Convex Optimization-A Unified Approach via Linear Iterative Methods. ArXiv e-prints, 2014.
3. Bertsekas, Dimitri P. Incremental gradient, subgradient, and proximal methods for convex optimization: A survey. In Sra, S., Nowozin, S., and Wright, S. J. (eds.), Optimization for Machine Learning, pp. 85-119. MIT Press, 2012. Extended version: LIDS report LIDS-P2848, MIT, 2010.
4. Bottou, Leon and Bousquet, Olivier. The tradeoffs of large scale learning. In Piatt, J.C., Koller, D., Singer, Y., and Roweis, S. (eds.), Advances in Neural Information Processing Systems, volume 20, pp. 161-168. NIPS Foundation (http://books.nips.cc), 2008. URL http://leon.bottou.org/papers/bottou-bousquet-2008.
5. Defazio, Aaron, Bach, Francis, and Lacoste-Julien, Simon. Saga: A fast incremental gradient method with support for non-strongly convex composite objectives. In Advances in Neural Information Processing Systems, 2014.
6. Johnson, Rie and Zhang, Tong. Accelerating stochastic gradient descent using predictive variance reduction. In Burges, C.J.C., Bottou, L., Welling. M., Ghahramani, Z., and Weinberger, K.Q. (eds.), Advances in Neural Information Processing Systems 26, pp. 315-323. 2013.
7. Le Roux, Nicolas, Schmidt, Mark, and Bach, Francis. A stochastic gradient method with an exponential convergence rate for finite training sets. In Pereira, R, Burges, C.J.C., Bottou, L., and Weinberger, K.Q. (eds.), Advances in Neural Information Processing Systems 25, pp. 2663-2671. 2012.
8. Nemirovsky, Arkadi and Yudin, David B. Problem Complexity and Method Efficiency in Optimization. Inter-science Series in Discrete Mathematics. Wiley, 1983.
9. Nesterov, Yurii. Introductory Lectures on Convex Optimization. Kluwer Academic Publisher, 2004.
10. Nesterov, Yurii. Gradient methods for minimizing composite objective function, 2007.
11. Schmidt, Mark, Le Roux, Nicolas, and Bach, Francis. Minimizing finite sums with the stochastic average gradient. arXiv preprint arXiv:1309.2388, 2013.
12. Shalev-Shwartz, Shai and Zhang, Tong. Stochastic dual coordinate ascent methods for regularized loss. The Journal of Machine Learning Research, 14(1):567-599, 2013.
13. Shalev-Shwartz, Shai and Zhang, Tong. Accelerated proximal stochastic dual coordinate ascent for regularized loss minimization. In Proceedings of the 31th International Conference on Machine Learning, ICML 2014, Beijing, China, 21-26 June 2014, pp. 64-72, 2014.
14. Vershynin, Roman. Introduction to the non-asymptotic analysis of random matrices. In Eldar, Yonina C. and Kutyniok, Gitta (eds.). Compressed Sensing, pp. 210-268. Cambridge University Press, 2012.
15. Zhang, Yuchen and Xiao, Lin. Stochastic primal-dual coordinate method for regularized empirical risk minimization. Technical Report MSR-TR-2014-123, September 2014.