Adapting to a changing environment: The brownian restless bandits

21st Annual Conference on Learning Theory, COLT 2008

Volumn , Issue , 2008, Pages 343-354

  Slivkins, Aleksandrs ^a   Upfal, Eli ^b  

^a MICROSOFT RESEARCH   (United States)

^b Brown Univerity   (United States)

Author keywords

[No Author keywords available]

Indexed keywords

CHANGING ENVIRONMENT; DISCRETE RANDOM WALK; DYNAMIC SETTINGS; INFINITE TIME HORIZON; MULTI ARMED BANDIT; PRACTICAL ONLINE; RESTLESS BANDITS; STEADY-STATE ANALYSIS;

ALGORITHMS; BROWNIAN MOVEMENT; STOCHASTIC SYSTEMS;

ITERATIVE METHODS;

EID: 84898070003     PISSN: None     EISSN: None     Source Type: Conference Proceeding    
DOI: None     Document Type: Conference Paper

References (33)

1. P. Auer. Using confidence bounds for exploitation-exploration trade-offs. J. Machine Learning Research, 3: 397-422, 2002. Preliminary version in 41st IEEE FOCS, 2000.
2. P. Auer, N. Cesa-Bianchi, and P. Fischer. Finite-time analysis of the multiarmed bandit problem. Machine Learning, 47(2-3): 235-256, 2002. Preliminary version in 15th ICML, 1998.
3. P. Auer, N. Cesa-Bianchi, Y. Freund, and R. E. Schapire. The nonstochastic multiarmed bandit problem. SIAM J. Comput., 32(1): 48-77, 2002. Preliminary version in 36th IEEE FOCS, 1995.
4. B. Awerbuch and R. D. Kleinberg. Adaptive routing with end-to-end feedback: distributed learning and geometric approaches. In 36th ACM Symp. on Theory of Computing (STOC), pages 45-53, 2004.
5. D. A. Berry and B. Fristedt. Bandit problems: sequential allocation of experiments. Chapman and Hall, 1985.
6. D. Bertsimas and J. Nino-Mora. Conservation laws, extended polymatroids and multi-armed bandit problems: A unified polyhedral approach. Math. of Oper. Res, 21(2): 257-306, 1996.
7. D. Bertsimas and J. Nino-Mora. Restless bandits, linear programming relaxations, and a primal-dual index heuristic. Operations Research, 48(1): 80-90, 2000.
8. V. Dani and T. P. Hayes. How to beat the adaptive multiarmed bandit. Technical report. Available from arXiv at http://arxiv.org/cs.DS/0602053, 2006.
9. V. Dani and T. P. Hayes. Robbing the bandit: less regret in online geometric optimization against an adaptive adversary. In 17th ACM-SIAM Symp. on Discrete Algorithms (SODA), pages 937-943, 2006.
10. A. Flaxman, A. Kalai, and H. B. McMahan. Online Convex Optimization in the Bandit Setting: Gradient Descent without a Gradient. In 16th ACM-SIAM Symp. on Discrete Algorithms (SODA), pages 385-394, 2005.
11. J. C. Gittins. Bandit processes and dynamic allocation indices (with discussion). J. Roy. Statist. Soc. Ser. B, 41: 148-177, 1979.
12. J. C. Gittins. Multi-Armed Bandit Allocation Indices. John Wiley & Sons, 1989.
13. J. C. Gittins and D. M. Jones. A dynamic allocation index for the sequential design of experiments. In J. G. et al., editor, Progress in Statistics, pages 241-266. North-Holland, 1974.
14. S. Guha and K. Munagala. Approximation algorithms for partial-information based stochastic control with Markovian rewards. In 48th Symp. on Foundations of Computer Science (FOCS), 2007.
15. S. Guha, K. Munagala, and P. Shi. On Index Policies for Restless Bandit Problems. arXiv: 0711.3861v1 [csDS], 2007.
16. F. Heidari, S. Mannor, and L. Mason. Reinforcement learning-based load shared sequential routing. In IFIP Networking, 2007.
17. R. D. Kleinberg. Nearly tight bounds for the continuum-armed bandit problem. In 18th Advances in Neural Information Processing Systems (NIPS), 2004. Full version appeared as Chapters 4-5 in [18].
18. R. D. Kleinberg. Online Decision Problems with Large Strategy Sets. PhD thesis, MIT, Boston, MA, 2005.
19. R. D. Kleinberg. Anytime algorithms for multi-armed bandit problems. In 17th ACM-SIAM Symp. on Discrete Algorithms (SODA), pages 928-936, 2006. Full version appeared as Chapter 6 in [18].
20. R. D. Kleinberg and F. T. Leighton. The value of knowing a demand curve: Bounds on regret for online posted-price auctions. In 44th Symp. on Foundations of Computer Science (FOCS), pages 594-605, 2003.
21. T. Lai and H. Robbins. Asymptotically efficient Adaptive Allocation Rules. Advances in Applied Mathematics, 6: 4-22, 1985.
22. H. B. McMahan and A. Blum. Online Geometric Optimization in the Bandit Setting Against an Adaptive Adversary. In 17th Conference on Learning Theory (COLT), pages 109-123, 2004.
23. K. Neammanee. On the constant in the nonuniform version of the Berry-Esseen theorem. Intl. J. of Mathematics and Mathematical Sciences, 2005: 12: 1951-1967, 2005.
24. J. Nino-Mora. Restless bandits, partial conservation laws and indexability. Advances in Applied Probability, 33: 76-98, 2001.
25. S. Pandey, D. Agarwal, D. Chakrabarti, and V. Josifovski. Bandits for Taxonomies: A Model-based Approach. In SIAM Intl. Conf. on Data Mining (SDM), 2007.
26. C. H. Papadimitriou and J. N. Tsitsiklis. The complexity of optimal queueing network control. In Structure in Complexity Theory Conference, pages 318-322, 1994.
27. H. Robbins. Some Aspects of the Sequential Design of Experiments. Bull. Amer. Math. Soc., 58: 527-535, 1952.
28. J. S. Rosenthal. Markov chain convergence: From finite to infinite. Stochastic Processes Appl., 62(1): 55-72, 1996.
29. R. K. Sundaram. Generalized Bandit Problems. In D. Austen-Smith and J. Duggan, editors, Social Choice and Strategic Decisions: Essays in Honor of Jeffrey S. Banks (Studies in Choice and Welfare), pages 131-162. Springer, 2005. First appeared as Working Paper, Stern School of Business, 2003.
30. J. N. Tsitsiklis. A short proof of the Gittins index theorem. Annals of Applied Probability, 4(1): 194-199, 1994.
31. G. Weiss. Branching bandit processes. Probab. Engng. Inform. Sci., 2: 269-278, 1988.
32. P. Whittle. Arm acquiring bandits. Ann. Probab., 9: 284-292, 1981.
33. P. Whittle. Restless bandits: Activity allocation in a changing world. J. of Appl. Prob., 25A: 287-298, 1988.