Index policies for discounted bandit problems with availability constraints

Advances in Applied Probability

Volumn 40, Issue 2, 2008, Pages 377-400

  Dayanik, Savas ^a   Powell, Warren ^a   Yamazaki, Kazutoshi ^a  

^a Princeton University   (United States)

Author keywords

Gittins index; Multiarmed bandit problem; Optimal resource allocation; Restart in problem; Whittle index

Indexed keywords

AVAILABILITY CONSTRAINTS; BANDIT PROBLEMS; BERNOULLI; BREAK DOWN; DISCOUNTED REWARD; GITTINS INDEX; INDEX POLICIES; MULTIARMED BANDIT PROBLEM; OPTIMAL RESOURCE ALLOCATION; RESTART-IN PROBLEM; WHITTLE INDEX;

PROBABILITY;

EID: 49349084239     PISSN: 00018678     EISSN: None     Source Type: Journal    
DOI: 10.1239/aap/1214950209     Document Type: Article

References (26)

1. BANKS, J. S. AND SUNDARAM, R. K. (1992). Denumerable-armed bandits. Econometrica 60, 1071-1096.
2. BANKS, J. S. AND SUNDARAM, R. K. (1994). Switching costs and the Gittins index. Econometrica 62, 687-694.
3. BERGEMANN, D. AND VALIMAKI, J. (2006). Efficient dynamic auctions. Cowles Foundation Discussion Paper 1584, Yale University.
4. BREZZI, M. AND LAI, T. L. (2000). Incomplete learning endogenous data in dynamic allocation. Econometrica 68, 1511-1516.
5. DAYANIK, S., POWELL, W. AND YAMAZAKI, K. (2007). Index policies for discounted bandit problems with availability constraints. Tech. Rep., Department of Operations Research and Financial Engineering, Princeton University.
6. GITTINS, J. C. (1979). Bandit processes and dynamic allocation indices. J. R. Statist. Soc. B 41, 148-177.
7. GLAZEBROOK, K. D. (1984). Scheduling stochastic jobs on a single machine subject to breakdowns. Naval Res. Logistics Quart. 31, 251-264.
8. GLAZEBROOK, K. D. (1987). Evaluating the effects of machine breakdowns in stochastic scheduling problems. Naval Res. Logistics 34, 319-335.
9. GLAZEBROOK, K. D. AND MITCHELL, H. M. (2002). An index policy for a stochastic scheduling model with improving/deteriorating jobs. Naval Res. Logistics 49, 706-721.
10. GLAZEBROOK, K. D., NIÑO-MORA, J. AND ANSELL, P. S. (2002). Index policies for a class of discounted restless bandits. Adv. Appl. Prob. 34, 754-774.
11. GLAZERROOK, K. D., RUIZ-HERNANDEZ, D. AND KIRKBRIDE, C. (2006). Some indexable families of restless bandit problems. Adv. Appl. Prob. 38, 643-672.
12. GLAZEBROOK, K. D., ANSELL, P. S., DUNN, R. T. AND LUMLEY, R. R. (2004). On the optimal allocation of service to impatient tasks. J. Appl. Prob. 41, 51-72.
13. JOVANOVIC, B. (1979). Job matching and the theory of turnover. J. Political Econom. 87, 972-990.
14. JUN, T. (2004). A survey on the bandit problem with switching costs. De Economist 1524, 513-541.
15. KATEHAKIS, M. N. AND DERMAN, C. (1986). Computing optimal sequential allocation rules in clinical trials. In Adaptive Statistical Procedures and Related Topics (Upton, NY, 1985; IMS Lecture Notes Monogr. Ser. 8), Institute of Mathematical Statistics, Hayward, CA, pp. 29-39.
16. KATEHAKIS, M. N. AND VEINOTT, A. F., JR. (1987). The multi-armed bandit problem: decomposition and computation. Math. Operat. Res. 12, 262-269.
17. MILLER, R. A. (1984). Job matching and occupational choice. J. Political Econom. 926, 1086-1120.
18. NIÑO-MORA, J. (2001). Restless bandits, partial conservation laws and indexability. Adv. Appl. Prob. 33, 76-98.
19. PAPADIMITRIOU, C. H. AND TSITSIKLIS, J. N. (1999). The complexity of optimal queuing network control. Math. Operat. Res. 24, 293-305.
20. ROSS, S. (1983). Introduction to Stochastic Dynamic Programming. Academic Press, New York.
21. ROTHSCHILD, M. (1974). A two-armed bandit theory of market pricing. J. Econom. Theory 9, 185-202.
22. TSITSIKLIS, J. N. (1994). A short proof of the Gittins index theorem. Ann. Appl. Prob. 4, 194-199.
23. WEBER, R. R. AND WEISS, G. (1990). On an index policy for restless bandits. J. Appl. Prob. 27, 637-648.
24. WEBER, R. R. AND WEISS, G. (1991). Addendum to: 'On an index policy for restless bandits'. Adv. Appl. Prob. 23, 429-430.
25. WHITTLE, P. (1980). Multi-armed bandits and the Gittins index. J. R. Statist. Soc. B 42, 143-149.
26. WHITTLE, P. (1988). Restless bandits: activity allocation in a changing world. In A Celebration of Applied Probability (J. Appl. Prob. Spec. Vol. 25A), Applied Probability Trust, Sheffield, pp. 287-298.