Bandit problems with side observations

IEEE Transactions on Automatic Control

Volumn 50, Issue 3, 2005, Pages 338-355

  Wang, Chih Chun ^a   Kulkarni, Sanjeev R ^a   Poor, H Vincent ^a  

^a Princeton University   (United States)

Author keywords

Adaptive; Allocation rule; Asymptotic; Efficient; Inferior sampling time; Side information; Two armed bandit

Indexed keywords

ALGORITHMS; ASYMPTOTIC STABILITY; DECISION THEORY; OPTIMAL CONTROL SYSTEMS; RANDOM PROCESSES;

ALLOCATION RULE; INFERIOR SAMPLING TIME; SIDE INFORMATION; TWO ARMED BANDIT;

ADAPTIVE CONTROL SYSTEMS;

EID: 15844389867     PISSN: 00189286     EISSN: None     Source Type: Journal    
DOI: 10.1109/TAC.2005.844079     Document Type: Article

References (24)

1. H. Robbins, "Some aspects of the sequential design of experiments," Bull. Amer. Math. Soc., vol. 58, pp. 527-535, 1952.
2. K. Adam, "Learning while searching for the best alternative," J. Econ. Theory, vol. 101, pp. 252-280, 2001.
3. D. A. Berry, "A Bernoulli two-armed bandit," Ann. Math. Stat., vol. 43, no. 3, pp. 871-897, Jun. 1972.
4. H. Chernoff, Sequential Analysis and Optimal Design. Philadelphia, PA: SIAM, 1972.
5. B. Ghosh and P. K. Sen, Handbook of Sequential Analysis. New York: Marcel Dekker, 1991.
6. J. C. Gittins, "Bandit processes and dynamic allocation indices," J. Royal Stat. Soc. B, vol. 41, no. 2, pp. 148-177, 1979.
7. J. C. Gittins, "A dynamic allocation index for the discounted multiarmed bandit problem," Biometrika, vol. 66, no. 3, pp. 561-565, Dec. 1979.
8. T. L. Lai and H. Robbins, "Asymptotically optimal allocation of treatments in sequential experiments," in Design of Experiments: Ranking and Selection, T. J. Santner and A. C. Tamhane, Eds. New York: MarcelDekker, 1984.
9. T. L. Lai and H. Robbins, "Asymptotically efficient allocation rules," Adv. Appl. Math., vol. 6, no. 1, pp. 4-22, 1985.
10. T. L. Lai and S. Yakowitz, "Machine learning and nonparametric bandit theory," IEEE Trans. Autom. Control, vol. 40, no. 7, pp. 1199-1209, Jul. 1995.
11. R. Agrawal, M. V. Hegde, and D. Teneketzis, "Asymptotically efficient adaptive allocation rules for the multiarmed bandit problem with switching cost," IEEE Trans. Autom. Control, vol. 33, no. 10, pp. 899-906, Oct. 1988.
12. R. Agrawal, D. Teneketzis, and V. Anantharam, "Asymptotically efficient adaptive allocation schemes for controlled i.i.d. processes: Finite parameter space," IEEE Trans. Autom. Control, vol. 34, no. 3, pp. 258-267, Mar. 1989.
13. R. Agrawal, D. Teneketzis, and V. Anantharam, "Asymptotically efficient adaptive allocation schemes for controlled Markov chains: Finite parameter space," IEEE Trans. Autom. Control, vol. 34, no. 12, pp. 1249-1259, Dec. 1989.
14. V. Anantharam, P. Varaiya, and J. Walrand, "Asymptotically efficient allocation rules for the multiarmed bandit problem with multiple plays - Part I: I.i.d. rewards," IEEE Trans. Autom. Control, vol. AC-32, no. 11, pp. 968-976, Nov. 1987.
15. V. Anantharam, P. Varaiya, and J. Walrand, "Asymptotically efficient allocation rules for the multiarmed bandit problem with multiple plays - Part II: Markovian rewards," IEEE Trans. Autom. Control, vol. AC-32, no. 11, pp. 977-982, Nov. 1987.
16. M. N. Katehakis and H. Robbins, "Sequential choice from several populations," in Proc. Nat. Acad. Sci., vol. 92, Sep. 1995, pp. 8584-8585.
17. S. R. Kulkarni and G. Lugosi, "Finite-time lower bounds for the two-armed bandit problem," IEEE Trans. Autom. Control, vol. 45, no. 4, pp. 711-714, Apr. 2000.
18. M. K. Clayton, "Covariate models for Bernoulli bandits," Seq. Anal. vol. 8, no. 4, pp. 405-426, 1989.
19. S. R. Kulkarni, "On bandit problems with side observations and learn-ability," in Proc. 31st Allerton Conf. Communications, Control, Computing, Sep. 1993, pp. 83-92.
20. J. Sarkar, "One-armed bandit problems with covariates," Ann. Statist., vol. 19, no. 4, pp. 1978-2002, 1991.
21. M. Woodroofe, "A one-armed bandit problem with a concomitant variable," J. Amer. Stat. Assoc., vol. 74, no. 368, pp. 799-806, Dec. 1979.
22. T. Zoubeidi, "Optimal allocations in sequential tests involving two populations with covariates," Commun. Statist.: Theory Meth., vol. 23, no. 4, pp. 1215-1225, 1994.
23. J. A. Bucklew, Large Deviation Techniques in Decision, Simulation, and Estimation. New York: Wiley, 1990.
24. A. Dembo and O. Zeitouni, Large Deviation Techniques and Applications New York, NY: Springer-Verlag, 1998.