Beyond the regret minimization barrier: An optimal algorithm for stochastic strongly-convex optimization

Journal of Machine Learning Research

Volumn 19, Issue , 2011, Pages 421-436

  Hazan, Elad ^a   Kale, Satyen ^b  

^a TECHNION ISRAEL INSTITUTE OF TECHNOLOGY   (Israel)

^b YAHOO RESEARCH   (United States)

Author keywords

Regret Minimization; Stochastic Optimization

Indexed keywords

ALGORITHMS; CONVEX OPTIMIZATION; OPTIMIZATION; STOCHASTIC SYSTEMS;

APPROXIMATE SOLUTION; NOVEL ALGORITHM; ONLINE CONVEX OPTIMIZATIONS; OPTIMAL ALGORITHM; OPTIMIZATION ALGORITHMS; RATE OF CONVERGENCE; REGRET MINIMIZATION; STOCHASTIC OPTIMIZATIONS;

STOCHASTIC MODELS;

EID: 84898471955     PISSN: 15324435     EISSN: 15337928     Source Type: Journal    
DOI: None     Document Type: Conference Paper

References (14)

1. Jacob Abernethy, Alekh Agarwal, Peter L. Bartlett, and Alexander Rakhlin. A stochastic view of optimal regret through minimax duality. In COLT, 2009.
2. Alekh Agarwal, Peter L. Bartlett, Pradeep Ravikumar, and Martin J. Wainwright. Information-theoretic lower bounds on the oracle complexity of convex optimization. In arXiv:1009.0571v1, 2010.
3. Dimitri P. Bertsekas. Nonlinear Programming. Athena Scientific, 2nd edition, September 1999. ISBN 1886529000.
4. Léon Bottou and Olivier Bousquet. The tradeoffs of large scale learning. In NIPS, 2007.
5. Nicolò Cesa-Bianchi and Gabor Lugosi. Prediction, Learning, and Games. Cambridge University Press, 2006.
6. Elad Hazan and Satyen Kale. An optimal algorithm for stochastic strongly-convex optimization. June 2010. URL http://arxiv.org/abs/1006.2425.
7. Elad Hazan, Amit Agarwal, and Satyen Kale. Logarithmic regret algorithms for online convex optimization. Machine Learning, 69(2-3):169-192, 2007.
8. Anatoli Juditsky and Yuri Nesterov. Primal-dual subgradient methods for minimizing uniformly convex functions. August 2010. URL http://hal.archives- ouvertes.fr/docs/ 00/50/89/33/PDF/Strong-hal.pdf.
9. Arkadi S. Nemirovski and David B. Yudin. Problem complexity and method efficiency in optimization. John Wiley UK/USA, 1983.
10. Erik Ordentlich and Thomas M. Cover. The cost of achieving the best portfolio in hindsight. Math. Oper. Res., 23:960-982, November 1998.
11. Shai Shalev-Shwartz and Nathan Srebro. SVM optimization: inverse dependence on training set size. In ICML, pages 928-935, 2008.
12. Shai Shalev-Shwartz, Ohad Shamir, Karthik Sridharan, and Nati Srebro. Stochastic convex optimization. In COLT, 2009.
13. Eiji Takimoto and Manfred K. Warmuth. The minimax strategy for gaussian density estimation. In COLT, pages 100-106, 2000.
14. Martin Zinkevich. Online convex programming and generalized infinitesimal gradient ascent. In ICML, pages 928-936, 2003.