Computing a classic index for finite-horizon bandits

INFORMS Journal on Computing

Volumn 23, Issue 2, 2011, Pages 254-267

  Niño Mora, José ^a  

^a UNIVERSIDAD CARLOS III DE MADRID   (Spain)

Author keywords

Analysis of algorithms; Bandits, finite horizon; Computational complexity; Dynamic programming, Markov; Index policies

Indexed keywords

EID: 79955755016     PISSN: 10919856     EISSN: 15265528     Source Type: Journal    
DOI: 10.1287/ijoc.1100.0398     Document Type: Article

References (17)

1. Bellman, R. 1956. A problem in the sequential design of experiments. Sankhya 16 (3/4) 221-229.
2. Berry, D. A., B. Fristedt. 1985. Bandit Problems: Sequential Allocation of Experiments. Chapman & Hall, London.
3. Bradt, R. N., S. M. Johnson, S. Karlin. 1956. On sequential designs for maximizing the sum of n observations. Ann. Math. Statist. 27 (4) 1060-1074.
4. Caro, F., J. Gallien. 2007. Dynamic assortment with demand learning for seasonal consumer goods. Management Sci. 53 (2) 276-292. (Pubitemid 46326180)
5. Dongarra, J. J., V. Eijkhout. 2000. Numerical linear algebra algorithms and software. J. Comput. Appl. Math. 123 (1-2) 489-514. (Pubitemid 32027699)
6. Ginebra, J., M. K. Clayton. 1999. Small-sample performance of Bernoulli two-armed bandit Bayesian strategies. J. Statist. Plann. Inference 79 (1) 107-122.
7. Gittins, J. C. 1979. Bandit processes and dynamic allocation indices. J. Roy. Statist. Soc. Ser. B 41 (2) 148-177.
8. Gittins, J. C. 1989. Multi-Armed Bandit Allocation Indices. John Wiley & Sons, Chichester, UK.
9. Gittins, J. C., D. M. Jones. 1974. A dynamic allocation index for the sequential design of experiments. J. Gani, K. Sarkadi, I. Vincze, eds. Progress in Statistics (European Meeting of Statisticians, Budapest, 1972). North-Holland, Amsterdam, 241-266.
10. Niño-Mora, J. 2001. Restless bandits, partial conservation laws and indexability Adv. Appl. Probab. 33 (1) 76-98. (Pubitemid 32443970)
11. Niño-Mora, J. 2002. Dynamic allocation indices for restless projects and queueing admission control: A polyhedral approach. Math. Programming 93 (3) 361-413. (Pubitemid 44744737)
12. Niño-Mora, J. 2005. A marginal productivity index policy for the finite-horizon multiarmed bandit problem. CDC-ECC'05: Proc. 44th IEEE Conf. Decision Control Eur. Control Conf. 2005, Seville, Spain. IEEE, Washington, DC, 1718-1722. (Pubitemid 46297216)
13. 3 fast-pivoting algorithm for the Gittins index and optimal stopping of a Markov chain. INFORMS J. Comput. 19 (4) 596-606.
14. Niño-Mora, J. 2008. Computing an index policy for multiarmed bandits with deadlines. ValueTools '08: Proc. Third Internat. Conf. Performance Evaluation Methodologies Tools, Athens, Greece. ICST, Brussels, Article 13.
15. Robbins, H. 1952. Some aspects of the sequential design of experiments. Bull. Amer. Math. Soc. 58 (5) 527-535.
16. Varaiya, P. P., J. C. Walrand, C. Buyukkoc. 1985. Extensions of the multiarmed bandit problem: The discounted case. IEEE Trans. Automat. Control 30 (5) 426-439. (Pubitemid 15478097)
17. Wang, Y.-G. 1997. Error bounds for calculation of the Gittins indices. Austral. J. Statist. 39 (2) 225-233.