A generalized gittins index for a class of multiarmed bandits with general resource requirements

Mathematics of Operations Research

Volumn 34, Issue 1, 2009, Pages 26-44

  Glazebrook, K D ^a   Minty, R ^a  

^a LANCASTER UNIVERSITY   (United Kingdom)

Author keywords

Asymptotic optimality; Bandit problems; Dynamic programming; Gittins index; Resource allocation

Indexed keywords

ASYMPTOTIC OPTIMALITY; BANDIT PROBLEMS; BANDIT PROCESS; CLASSICAL MODEL; DECISION POINTS; DYNAMIC ALLOCATIONS; GITTINS INDEX; INDEX HEURISTICS; MULTI ARMED BANDIT; NUMERICAL STUDIES; OPTIMALITY; PER UNIT; RESOURCE DISTRIBUTION; RESOURCE REQUIREMENTS; SIMPLE MODEL; WEIGHTED INDEX;

ASYMPTOTIC ANALYSIS; DYNAMIC PROGRAMMING; PLANNING; RESOURCE ALLOCATION; SYSTEMS ENGINEERING;

HEURISTIC METHODS;

EID: 67649922844     PISSN: 0364765X     EISSN: 15265471     Source Type: Journal    
DOI: 10.1287/moor.1080.0342     Document Type: Article

References (31)

1. Agrawal, R., M. Hedge, D. Teneketzis. 1988. Asymptotically efficient adaptive allocation rules for the multiarmed bandit problem with switching cost. IEEE Trans. Aut. Ctrl. 33 899-906.
2. Ansell, P. S., K. D. Glazebrook, J. Niño-Mora, M. O'Keeffe. 2003. Whittle's index policy for a multi-class queueing system with convex holding costs. Math. Methods Oper. Res. 57 21-39.
3. Banks, J. S., D. Sundaram. 1994. Switching costs and the Gittins index. Econometrica 62 687-694.
4. Berry, D. A., B. Fristedt. 2004. Bandit Problems: Sequential Allocation of Experiments. Chapman and Hall, London.
5. Bertsimas, D., J. Niño-Mora. 1996. Conservation laws, extended polymatroids and multi-armed bandit problems: A polyhedral approach to indexable systems. Math. Oper. Res. 21 257-306.
6. Crosbie, J. H., K. D. Glazebrook. 2000. Index policies and a novel performance space structure for a class of generalised branching bandit problems. Math. Oper. Res. 25 281-297.
7. Denardo, E. V., U. G. Rothblum, L. van der Heyden. 2004. Index policies for stochastic search in a forest with an application to R. & D. project management. Math. Oper. Res. 29 162-181.
8. Gittins, J. C. 1979. Bandit processes and dynamic allocation indices (with discussion). J. Roy. Statist. Soc. B41 148-177.
9. Gittins, J. C. 1989. Multi-Armed Bandit Allocation Indices. John Wiley, Chichester, UK.
10. Glazebrook, K. D. 1980. On stochastic scheduling with precedence relations and switching costs. J. Appl. Probab. 17 1016-1024.
11. Glazebrook, K. D., R. Garbe. 1999. Almost optimal policies for stochastic systems which almost satisfy conservation laws. Ann. Oper. Res. 99 19-43.
12. Glazebrook, K. D., C. Kirkbride. 2004. Index policies for the routing of background jobs. Naval Res. Logist. 52 381-398.
13. Glazebrook, K. D., C. Kirkbride. 2007. Dynamic routing to heterogeneous collections of unreliable servers. Queueing Systems 55 9-25.
14. Glazebrook, K. D., C. Kirkbride, D. Ruiz-Hernandez. 2006a. Spinning plates and squad systems - Policies for bi-directional restless bandits. Adv. Appl. Probab. 38 95-115.
15. Glazebrook, K. D., C. Kirkbride, D. Ruiz-Hernandez. 2006b. Some families of indexable restless bandit problems. Adv. Appl. Probab. 38 643-672.
16. Glazebrook, K. D., J. Niño-Mora, P. S. Ansell. 2002. Index policies for a class of discounted restless bandits. Adv. Appl. Probab. 34 754-774.
17. Katehakis, M. N., A. F. Veinott, Jr. 1987. The multi-armed bandit problem: Decomposition and computation. Math. Oper. Res. 12 262-268.
18. Mahajan, A., D. Teneketzis. 2007. Multiarmed bandit problems. A. Hero, D. Castanon, D. Cochran, K. Kastella, eds. Foundations and Applications of Sensor Management.
19. Nash, P. 1980. A generalized bandit problem. J. Roy. Statist. Soc. B42 165-169.
20. Niño-Mora, J. 2001. Restless bandits, partial conservation laws and indexability. Adv. Appl. Probab. 33 76-98.
21. Niño-Mora, J. 2002. Dynamic allocation indices for restless projects and queueing admission control: A polyhedral approach. Math. Program. 93 361-413.
22. 3 fast-pivoting algorithm for the Gittins index and optimal stopping of a Markov chain. INFORMS J. Comput. 19 596-606.
23. Opp, M., K. D. Glazebrook, V. Kulkarni. 2005. Outsourcing warranty repairs - Dynamic allocation. Naval Res. Logist. 52 381-398.
24. Papadimitriou, C. H., J. N. Tsitsiklis. 1999. The complexity of optimal queueing network control. Math. Oper. Res. 24 293-305.
25. Puterman, M. L. 1994. Markov Decision Processes: Discrete Stochastic Dynamic Programming. John Wiley, New York.
26. Reiman, M. I., L. M. Wein. 1998. Dynamic scheduling of a two-class queue with set-ups. Oper. Res. 46 532-547.
27. Robinson, D. R. 1982. Algorithms for evaluating the dynamic allocation index. Oper. Res. Lett. 1 72-74.
28. Van Oyen, M. P., D. Teneketzis. 1994. Optimal stochastic scheduling of forest networks with switching penalties. Adv. Appl. Probab. 26 474-479.
29. Weber, R. R., G. Weiss. 1990. On an index policy for restless bandits. J. Appl. Probab. 27 637-648,
30. (Addendum: Adv. Appl. Probab., 23 1991, 429-430).
31. Whittle, P. 1988. Restless bandits: Activity allocation in a changing world. J. Gani, ed. A Celebration of Applied Probability. J. Appl. Probab. Special Volume 25A 287-298. Probability Trust, Sheffield, UK.