Sharp dichotomies for regret minimization in metric spaces

Proceedings of the Annual ACM-SIAM Symposium on Discrete Algorithms

Volumn , Issue , 2010, Pages 827-846

  Kleinberg, Robert ^a   Slivkins, Aleksandrs ^b  

^a Department of Computer Science and School of Operations Research and Information Engineering   (United States)

^b MICROSOFT RESEARCH   (United States)

Author keywords

[No Author keywords available]

Indexed keywords

PROBABILITY; SET THEORY;

FINITE METRIC SPACES; MULTI ARMED BANDIT; MULTI-ARMED BANDIT PROBLEM; ONLINE LEARNING; REGRET MINIMIZATION; SIDE INFORMATION; TOPOLOGICAL NOTIONS; UPPER AND LOWER BOUNDS;

TOPOLOGY;

EID: 77951694424     PISSN: None     EISSN: None     Source Type: Conference Proceeding    
DOI: 10.1137/1.9781611973075.68     Document Type: Conference Paper

References (34)

1. Rajeev Agrawal. The continuum-armed bandit problem. SIAM J. Control and Optimization, 33(6):1926-1951, 1995.
2. Peter Auer. Using confidence bounds for exploitation-exploration trade-offs. J. Machine Learning Research, 3:397-422, 2002. Preliminary version in 41st IEEE FOCS, 2000.
3. Peter Auer, Nicolò Cesa-Bianchi, and Paul Fischer. Finite-time analysis of the multiarmed bandit problem. Machine Learning, 47(2-3):235-256, 2002. Preliminary version in 15th ICML, 1998.
4. Peter Auer, Nicolò Cesa-Bianchi, Yoav Freund, and Robert E. Schapire. The nonstochastic multiarmed bandit problem. SIAM J. Comput., 32(1):48-77, 2002. Preliminary version in 36th IEEE FOCS, 1995.
5. Peter Auer, Ronald Ortner, and Csaba Szepesvári. Improved Rates for the Stochastic Continuum-Armed Bandit Problem. In 20th Conference on Learning Theory (COLT), pages 454-468, 2007.
6. Baruch Awerbuch and Robert Kleinberg. Online linear optimization and adaptive routing. Journal of Computer and System Sciences, 74(1):97-114, February 2008. Preliminary version appeared in 36th ACM STOC, 2004.
7. Jeffrey Banks and Rangarajan Sundaram. Denumerable-armed bandits. Econometrica, 60(5):1071-1096, 1992.
8. G. Cantor. Über unendliche, lineare Punktmannichfaltigkeiten, 4. Mathematische Annalen, 21:51-58, 1883.
9. In G. Cantor, Gesammelte Abhandlungen mathematischen und philosophischen Inhalts, Berlin: Teubner, 1932; reprinted in 1980; reprint ed., Hildesheim: Olms, 1966.
10. Nicolò Cesa-Bianchi, Yoav Freund, David Haussler, David P. Helmbold, Robert E. Schapire, and Manfred K. Warmuth. How to use expert advice. J. ACM, 44(3):427-485, 1997.
11. Nicolò Cesa-Bianchi and Gábor Lugosi. Prediction, learning, and games. Cambridge University Press, 2006.
12. Eric Cope. Regret and convergence bounds for immediate-reward reinforcement learning with continuous action spaces, 2004. Unpublished manuscript.
13. Thomas M. Cover and Joy A. Thomas. Elements of Information Theory. John Wiley & Sons, New York, 1991.
14. Varsha Dani and Thomas P. Hayes. Robbing the bandit: less regret in online geometric optimization against an adaptive adversary. In 17th ACM-SIAM Symp. on Discrete Algorithms (SODA), pages 937-943, 2006.
15. Varsha Dani, Thomas P. Hayes, and Sham Kakade. The Price of Bandit Information for Online Optimization. In 20th Advances in Neural Information Processing Systems (NIPS), 2007.
16. Abraham Flaxman, Adam Kalai, and H. Brendan McMahan. Online Convex Optimization in the Bandit Setting: Gradient Descent, without a Gradient. In 16th ACM-SIAM Symp. on Discrete Algorithms (SODA), pages 385-394, 2005.
17. J. C. Gittins. Multi-Armed Bandit Allocation Indices. John Wiley & Sons, 1989.
18. J. C. Gittins and D. M. Jones. A dynamic allocation index for the sequential design of experiments. In J. Gani et al., editor, Progress in Statistics, pages 241-266. North-Holland, 1974.
19. Sudipta Guha and Kamesh Munagala. Approximation algorithms for partial-information based stochastic control with Markovian rewards. In 48th Symp. on Foundations of Computer Science (FOCS), pages 483-493, 2007.
20. Sudipta Guha, Kamesh Munagala, and Peng Shi. Approximation algorithms for restless bandit problems. In 20th ACM-SIAM Symp. on Discrete Algorithms (SODA), pages 28-37, 2009.
21. Anupam Gupta, Mike Dinitz, and Kanat Tangwongsan. Private communication, 2007.
22. Elad Hazan and Satyen Kale. Better algorithms for benign bandits. In 20th ACM-SIAM Symp. on Discrete Algorithms (SODA), pages 38-47, 2009.
23. Elad Hazan and Nimrod Megiddo. Online Learning with Prior Information. In 20th Conference on Learning Theory (COLT), pages 499-513, 2007.
24. Sham M. Kakade, Adam T. Kalai, and Katrina Ligett. Playing Games with Approximation Algorithms. In 39th ACM Symp. on Theory of Computing (STOC), 2007.
25. Robert Kleinberg. Nearly tight bounds for the continuum-armed bandit problem. In 18th Advances in Neural Information Processing Systems (NIPS), 2004. Full version appeared in the author's thesis (MIT, 1995).
26. Robert Kleinberg. Online Decision Problems with Large Strategy Sets. PhD thesis, MIT, Boston, MA, 2005.
27. Robert Kleinberg, Alexandru Niculescu-Mizil, and Yogeshwer Sharma. Regret bounds for sleeping experts and bandits. In 21st Conference on Learning Theory (COLT), pages 425-436, 2008.
28. Robert Kleinberg, Aleksandrs Slivkins, and Eli Upfal. Multi-Armed Bandits in Metric Spaces. In 40th ACM Symp. on Theory of Computing (STOC), pages 681-690, 2008.
29. T.L. Lai and Herbert Robbins. Asymptotically efficient Adaptive Allocation Rules. Advances in Applied Mathematics, 6:4-22, 1985.
30. S. Mazurkiewicz and W. Sierpinski. Contribution à la topologie des ensembles dénombrables. Fund. Math., 1:17-27, 1920.
31. H. Brendan McMahan and Avrim Blum. Online Geometric Optimization in the Bandit Setting Against an Adaptive Adversary. In 17th Conference on Learning Theory (COLT), pages 109-123, 2004.
32. Sandeep Pandey, Deepak Agarwal, Deepayan Chakrabarti, and Vanja Josifovski. Bandits for Taxonomies: A Model-based Approach. In SIAM Intl. Conf. on Data Mining (SDM), 2007.
33. V. Vovk. A game of prediction with expert advice. J. Computer and System Sciences, 56(2):153-173, 1998.
34. P. Whittle. Restless bandits: Activity allocation in a changing world. J. of Appl. Prob., 25A:287-298, 1988.