Mixed optimization for smooth functions

Advances in Neural Information Processing Systems

Volumn , Issue , 2013, Pages

  Mahdavi, Mehrdad ^a   Zhang, Lijun ^a   Jin, Rong ^a  

^a Michigan State University   (United States)

Author keywords

[No Author keywords available]

Indexed keywords

ALGORITHMS; FUNCTIONS; STOCHASTIC SYSTEMS;

CONVERGENCE RATES; CONVEX FUNCTIONS; MIXED OPTIMIZATION; OPTIMAL CONVERGENCE; OPTIMIZATION ERRORS; SMOOTH FUNCTIONS; STOCHASTIC OPTIMIZATIONS;

OPTIMIZATION;

EID: 84898988901     PISSN: 10495258     EISSN: None     Source Type: Conference Proceeding    
DOI: None     Document Type: Conference Paper

References (24)

1. A. Agarwal, P. L. Bartlett, P. D. Ravikumar, and M. J. Wainwright. Information-theoretic lower bounds on the oracle complexity of stochastic convex optimization. IEEE Transactions on Information Theory, 58(5):3235-3249, 2012.
2. A. Beck and M. Teboulle. Mirror descent and nonlinear projected subgradient methods for convex optimization. Oper. Res. Lett., 31(3):167-175, 2003.
3. L. Bottou and O. Bousquet. The tradeoffs of large scale learning. In NIPS, pages 161-168, 2008.
4. S. Boucheron, G. Lugosi, and O. Bousquet. Concentration inequalities. In Advanced Lectures on Machine Learning, pages 208-240, 2003.
5. S. Boyd and L. Vandenberghe. Convex Optimization. Cambridge University Press, 2004.
6. R. H. Byrd, G. M. Chin, J. Nocedal, and Y.Wu. Sample size selection in optimization methods for machine learning. Mathematical programming, 134(1):127-155, 2012.
7. A. Cotter, O. Shamir, N. Srebro, and K. Sridharan. Better mini-batch algorithms via accelerated gradient methods. In NIPS, pages 1647-1655, 2011.
8. O. Dekel, R. Gilad-Bachrach, O. Shamir, and L. Xiao. Optimal distributed online prediction using mini-batches. The Journal of Machine Learning Research, 13:165-202, 2012.
9. M. P. Friedlander and M. Schmidt. Hybrid deterministic-stochastic methods for data fitting. SIAM Journal on Scientific Computing, 34(3):A1380-A1405, 2012.
10. E. Hazan, A. Agarwal, and S. Kale. Logarithmic regret algorithms for online convex optimization. Machine Learning, 69(2-3):169-192, 2007.
11. E. Hazan and S. Kale. Beyond the regret minimization barrier: an optimal algorithm for stochastic strongly-convex optimization. Journal of Machine Learning Research-Proceedings Track, 19:421-436, 2011.
12. Q. Lin, X. Chen, and J. Pena. A smoothing stochastic gradient method for composite optimization. arXiv preprint arXiv:1008.5204, 2010.
13. A. Nemirovski, A. Juditsky, G. Lan, and A. Shapiro. Robust stochastic approximation approach to stochastic programming. SIAM J. on Optimization, 19:1574-1609, 2009.
14. A. S. Nemirovsky and D. B. Yudin. Problem complexity and method efficiency in optimization. 1983.
15. Y. Nesterov. A method of solving a convex programming problem with convergence rate o (1/k2). In Soviet Mathematics Doklady, volume 27, pages 372-376, 1983.
16. Y. Nesterov. Introductory Lectures on Convex Optimization: A Basic Course. Kluwer Academic Publishers, 2004.
17. Y. Nesterov. Excessive gap technique in nonsmooth convex minimization. SIAM Journal on Optimization, 16(1):235-249, 2005.
18. Y. Nesterov. Smooth minimization of non-smooth functions. Math. Program., 103(1):127-152, 2005.
19. A. Rakhlin, O. Shamir, and K. Sridharan. Making gradient descent optimal for strongly convex stochastic optimization. In ICML, 2012.
20. N. L. Roux, M. W. Schmidt, and F. Bach. A stochastic gradient method with an exponential convergence rate for finite training sets. In NIPS, pages 2672-2680, 2012.
21. S. Shalev-Shwartz, Y. Singer, and N. Srebro. Pegasos: Primal estimated sub-gradient solver for svm. In ICML, pages 807-814, 2007.
22. S. Shalev-Shwartz and T. Zhang. Stochastic dual coordinate ascent methods for regularized loss minimization. JMLR, 14:567599, 2013.
23. O. Shamir and T. Zhang. Stochastic gradient descent for non-smooth optimization: Convergence results and optimal averaging schemes. ICML, 2013.
24. L. Zhang, T. Yang, R. Jin, and X. He. O(logt) projections for stochastic optimization of smooth and strongly convex functions. ICML, 2013.