Making gradient descent optimal for strongly convex stochastic optimization

Proceedings of the 29th International Conference on Machine Learning, ICML 2012

Volumn 1, Issue , 2012, Pages 449-456

  Rakhlin, Alexander ^a   Shamir, Ohad ^b   Sridharan, Karthik ^a  

^a UNIVERSITY OF PENNSYLVANIA   (United States)

^b MICROSOFT RESEARCH   (United States)

Author keywords

[No Author keywords available]

Indexed keywords

CONVERGENCE RATES; CONVEX PROBLEMS; GRADIENT DESCENT; NON-SMOOTH; OPTIMALITY; SIMPLE MODIFICATIONS; STOCHASTIC GRADIENT DESCENT; STOCHASTIC OPTIMIZATION PROBLEMS; STOCHASTIC OPTIMIZATIONS; STOCHASTIC SETTINGS;

ALGORITHMS; LEARNING SYSTEMS; OPTIMIZATION; STOCHASTIC SYSTEMS;

PROBLEM SOLVING;

EID: 84867120686     PISSN: None     EISSN: None     Source Type: Conference Proceeding    
DOI: None     Document Type: Conference Paper

References (10)

1. Bach, F. and Moulines, E. Non-asymptotic analysis of stochastic approximation algorithms for machine learning. In NIPS, 2011.
2. Hazan, E. and Kale, S. Beyond the regret minimization barrier: An optimal algorithm for stochastic strongly-convex optimization. In COLT, 2011.
3. Hazan, E., Agarwal, A., and Kale, S. Logarithmic regret algorithms for online convex optimization. Machine Learning, 69(2-3):169-192, 2007.
4. Joachims, T. Training linear SVMs in linear time. In KDD, 2006.
5. Juditsky, A. and Nesterov, Y. Primal-dual subgradient methods for minimizing uniformly convex functions. Technical Report (August 2010), available at http://hal.archives-ouvertes.fr/docs/00/50/89/33/PDF/Strong-hal.pdf, 2010.
6. Kushner, H. and Yin, G. Stochastic Approximation and Recursive Algorithms and Applications. Springer, 2nd edition, 2003.
7. Nemirovski, A., Juditsky, A., Lan, G., and Shapiro, A. Robust stochastic approximation approach to stochastic programming. SIAM J Optim., 19(4):1574-1609, 2009.
8. Rakhlin, A., Shamir, O., and Sridharan, K. Making gradient descent optimal for strongly convex stochastic optimization. ArXiv Technical Report, arXiv:1109.5647, 2011.
9. Shalev-Shwartz, S., Shamir, O., Srebro, N., and Sridharan, K. Stochastic convex optimization. In COLT, 2009.
10. Shalev-Shwartz, S., Singer, Y., Srebro, N., and Cotter, A. Pegasos: primal estimated sub-gradient solver for svm. Mathematical Programming, 127(1):3-30, 2011.