Linearly parameterized bandits

Mathematics of Operations Research

Volumn 35, Issue 2, 2010, Pages 395-411

  Rusmevichientong, Paat ^a   Tsitsiklis, John N ^b  

^a Cornell University   (United States)

^b MASSACHUSETTS INSTITUTE OF TECHNOLOGY   (United States)

Author keywords

Adaptive control; Multi armed bandit; Parametric model

Indexed keywords

ADAPTIVE CONTROL; BANDIT PROBLEMS; BAYES RISK; EXPLORATION AND EXPLOITATION; LINEAR FUNCTIONS; LOWER BOUNDS; MULTI ARMED BANDIT; NEAR-OPTIMAL POLICIES; PARAMETERIZED; PARAMETRIC MODELS; RANDOM VECTORS; UPPER BOUND;

ADAPTIVE CONTROL SYSTEMS;

MODELS;

EID: 77953111834     PISSN: 0364765X     EISSN: 15265471     Source Type: Journal    
DOI: 10.1287/moor.1100.0446     Document Type: Article

References (33)

1. Abe, N., P. M. Long. 1999. Associative reinforcement learning using linear probabilistic concepts. Proc. 16th Internat. Conf. Machine Learn., Morgan Kaufman, San Francisco, 3-11.
2. Agrawal, R. 1995. Sample mean based index policies with 0(log n) regret for the multi-armed bandit problem. Adv. Appl. Probab. 27(4) 1054-1078.
3. Agrawal, R., D. Teneketzis, V. Anantharam. 1989. Asymptotically efficient adaptive allocation schemes for controlled i.i.d. processes: Finite parameter space. IEEE Trans. Automatic Control 34(3) 258-267.
4. Auer, P. 2002. Using confidence bounds for exploitation-exploration trade-offs. J. Machine Learn. Res. 3(3) 397-422.
5. Auer, P., N. Cesa-Bianchi, P. Fischer. 2002. Finite-time analysis of the multi-armed bandit problem. Machine Learn. 47(2) 235-256.
6. Berry, D., B. Fristedt. 1985. Bandit Problems: Sequential Allocation of Experiments. Chapman and Hall, London.
7. Bertsekas, D. 1995. Dynamic Programming and Optimal Controls, Vol.1. Athena Scientific, Belmont, MA.
8. Bertsekas, D., J. N. Tsitsiklis. 1996. Neuro-Dynamic Programming. Athena Scientific, Belmont, MA.
9. Bertsimas, D., J. N. Tsitsiklis. 1997. Introduction to Linear Optimization. Athena Scientific, Belmont, MA.
10. Blum, J. R. 1954. Multidimensional stochastic approximation methods. Ann. Math. Statist. 25(4) 737-744.
11. Cicek, D., M. Broadie, A. Zeevi. 2009. General bounds and finite-time performance improvement for the Kiefer-Wolfowitz stochastic approximation algorithm. Working paper, Columbia Graduate School of Business, New York.
12. Dani, V., T. P. Hayes, S. M. Kakade. 2008a. Stochastic linear optimization under bandit feedback. Proc. 21st Annual Conf. Learn. Theory (COLT 2008), Helsinki, Finland, 355-366.
13. Dani, V., T. P. Hayes, S. M. Kakade. 2008b. Stochastic linear optimization under bandit feedback. Working paper, University of Chicago, Chicago. http://ttic.uchicago.edu/-sham/papers/ml/bandit-linear-long.pdf.
14. Feldman, D. 1962. Contributions to the "two-armed bandit" problem. Ann. Math. Statist. 33(3) 847-856.
15. Fiedler, M., V. Pták. 1997. A new positive definite geometric mean of two positive definite matrices. Linear Algebra Its Appl. 251(1) 1-20.
16. Ginebra, J., M. K. Clayton. 1995. Response surface bandits. J. Roy. Statist. Soc. Ser. B (Methodological) 57(4) 771-784.
17. Goldenshluger, A., A. Zeevi. 2008. Performance limitations in bandit problems with side observations. Working paper, Columbia Graduate School of Business, Columbia University Graduate School of Business, New York.
18. Goldenshluger, A., A. Zeevi. 2009. Woodroofe's one-armed bandit problem revisited. Ann. Appl. Probab. 19(4) 1603-1633.
19. Keener, R. 1985. Further contributions to the "two-armed bandit" problem. Ann. Statist. 13(1) 418-422.
20. Kiefer, J., J. Wolfowitz. 1952. Stochastic estimation of the maximum of a regression function. Ann. Math. Statist. 23(3) 462-466.
21. Lai, T. 2003. Stochastic approximation (invited paper). Ann. Statist. 31(2) 391-406.
22. Lai, T. L. 1987. Adaptive treatment allocation and the multi-armed bandit problem. Ann. Statist. 15(3) 1091-1114.
23. Lai, T. L., H. Robbins. 1985. Asymptotically efficient adaptive allocation rules. Adv. Appl. Math. 6(1) 4-22.
24. Mersereau, A. J., P. Rusmevichientong, J. N. Tsitsiklis. 2009. A structured multi-armed bandit problem and the greedy policy. IEEE Trans. Automatic Control 54(12) 2787-2802.
25. Pandey, S., D. Chakrabarti, D. Agrawal. 2007. Multi-armed bandit problems with dependent arms. Proc. 24th Internat. Conf. Machine Learn., Corvallis, OR, 721-728.
26. Polovinkin, E. S. 1996. Strongly convex analysis. Sbornik: Math. 187(2) 259-286.
27. Pressman, E. L., I. N. Sonin. 1990. Sequential Control with Incomplete Information. Academic Press, London.
28. Robbins, H. 1952. Some aspects of the sequential design of experiments. Bull. Amer. Math. Soc. 58(5) 527-535.
29. Robbins, H., S. Monro. 1951. A stochastic approximation method. Ann. Math. Statist. 22(3) 400-407.
30. Rusmevichientong, P., J. N. Tsitsiklis. 2010. Linearly parameterized bandits (extended version). http://arxiv.org/abs/0812.3465.
31. Thompson, W. R. 1933. On the likelihood that one unknown probability exceeds another in view of the evidence of two samples. Biometrika 25(3) 285-294.
32. Wang, C.-C., S. R. Kulkarni, H. V. Poor. 2005a. Bandit problems with side observations. IEEE Trans. Automatic Control 50(3) 338-355.
33. Wang, C.-C., S. R. Kulkarni, H. V. Poor. 2005b. Arbitrary side observations in bandit problems. Adv. Appl. Math. 34(4) 903-938.