Stochastic convex optimization with bandit feedback

SIAM Journal on Optimization

Volumn 23, Issue 1, 2013, Pages 213-240

  Agarwal, Alekh ^a   Foster, Dean P ^b   Hsu, Daniel ^c   Kakade, Sham M ^c   Rakhlin, Alexander ^b  

^a MICROSOFT RESEARCH   (United States)

^b UNIVERSITY OF PENNSYLVANIA   (United States)

^c MICROSOFT RESEARCH   (United Kingdom)

Author keywords

Bandit optimization; Derivative free optimization; Ellipsoid method

Indexed keywords

BANDIT FEEDBACKS; DERIVATIVE-FREE OPTIMIZATION; ELLIPSOID ALGORITHM; ELLIPSOID METHOD; FUNCTION VALUES; LIPSCHITZ FUNCTIONS; OPTIMAL FUNCTION; QUANTITY OF INTEREST;

CONVEX OPTIMIZATION; OPTIMIZATION; STOCHASTIC MODELS; STOCHASTIC SYSTEMS;

ALGORITHMS;

EID: 84877750537     PISSN: 10526234     EISSN: None     Source Type: Journal    
DOI: 10.1137/110850827     Document Type: Article

References (20)

1. A. Agarwal, O. Dekel, and L. Xiao, Optimal algorithms for online convex optimization with multi-point bandit feedback, in Proceedings of COLT, 2010.
2. R. Agrawal, The continuum-armed bandit problem, SIAM J. Control Optim., 33 (1995), pp. 1926-1951.
3. P. Auer, R. Ortner, and C. Szepesvári, Improved rates for the stochastic continuum-armed bandit problem, in Proceedings of COLT, 2007, pp. 454-468.
4. D. Bertsimas and S. Vempala, Solving convexprograms by random walks, J. ACM, 51 (2004), pp. 540-556.
5. S. Bubeck, R. Munos, G. Stolz, and C. Szepesvári, X-armed bandits, J. Mach. Learn. Res., 12 (2011), pp. 1655-1695.
6. V. V. Buldygin and Yu. V. Kozachenko, Sub-Gaussian random variables, Ukrainian Math. J., 32 (1980), pp. 483-489.
7. A. R. Conn, K. Scheinberg, and L. N. Vicente, Introduction to Derivative-Free Optimization, SIAM, Philadelphia, 2009.
8. E. W. Cope, Regret and convergence bounds for a class of continuum-armed bandit problems, IEEE Trans. Automat. Control, 54 (2009), pp. 1243-1253.
9. V. Dani, T. P. Hayes, and S. M. Kakade, Stochastic linear optimization under bandit feedback, in Proceedings of the 21st Annual Conference on Learning Theory (COLT), 2008.
10. A. D. Flaxman, A. T. Kalai, and B. H. Mcmahan, Online convex optimization in the bandit setting: Gradient descent without a gradient, in Proceedings of the 16th Annual ACM- SIAM Symposium on Discrete Algorithms, 2005, pp. 385-394.
11. D. Goldfarb and M. J. Todd, Modifications and implementation of the ellipsoid algorithm for linear programming, Math. Program., 23 (1982), pp. 1-19.
12. J. Kiefer and J. Wolfowitz, Stochastic estimation of the maximum of a regression function, Ann. Math. Statist., 23 (1952), pp. 462-466.
13. R. Kleinberg, Nearly tight bounds for the continuum-armed bandit problem, Adv. Neural Inf. Process. Syst., 18 (2005).
14. R. Kleinberg, A. Slivkins, and E. Upfal, Multi-armed bandits in metric spaces, in Proceedings of the 40th Annual ACM Symposium on Theory of Computing, 2008, pp. 681-690.
15. L. Lovász, Geometric algorithms and algorithmic geometry, in Proceedings of International Congress of Mathematicians, 1990, pp. 139-154.
16. A. NEMiROVSKI AND D. YuDiN, Problem Complexity and Method Efficiency in Optimization, Wiley, New York, 1983.
17. Y. NESTEROV, Random Gradient-Free Minimization of Convex Functions, Technical report 2011/1, Center for Operations Research and Econometrics, Universit́e catholique de Lou- vain, 2011.
18. M. RAGiNSKY AND A. RAKHLiN, Information-based complexity, feedback and dynamics in convex programming, IEEE Trans. Inform. Theory, 57 (2011), pp. 7036-7056.
19. N. SRiNIVAS, A. KRÄuSE, S.M. KÄKADE, ÄND M. SEeGER, Gaussian Process Optimization in the Bandit Setting: No Regret and Experimental Design, arXiv:0912.3995, 2009.
20. J. Y. Yu AND S. MANNOR, Unimodal bandits, in Proceedings of ICML, 2011.