The irrevocable multiarmed bandit problem

Operations Research

Volumn 59, Issue 2, 2011, Pages 383-399

  Farias, Vivek F ^a   Madan, Ritesh ^b  

^a MASSACHUSETTS INSTITUTE OF TECHNOLOGY   (United States)

^b New Jersey Research Center of QUALCOMM Inc   (United States)

Author keywords

Dynamic programming optimal control; Learning; Multiarmed bandit problem; Production scheduling; Sequencing; Stochastic

Indexed keywords

DYNAMIC PROGRAMMING/OPTIMAL CONTROL; LEARNING; MULTI-ARMED BANDIT PROBLEM; PRODUCTION/SCHEDULING; SEQUENCING; STOCHASTIC;

ALGORITHMS;

HEURISTIC METHODS;

EID: 79957458110     PISSN: 0030364X     EISSN: 15265463     Source Type: Journal    
DOI: 10.1287/opre.1100.0891     Document Type: Article

References (24)

1. Anantharam, V., P. Varaiya, J. Walrand. 1987a. Asymptotically efficient allocation rules for the multiarmed bandit problem with multiple plays-part I: i.i.d. rewards. IEEE Trans. Automatic Control 32(11) 968-976. (Pubitemid 18521625)
2. Anantharam, V., P. Varaiya, J. Walrand. 1987b. Asymptotically efficient allocation rules for the multiarmed bandit problem with multiple plays-part II: Markovian rewards. IEEE Trans. Automatic Control 32(11) 977-982. (Pubitemid 18521626)
3. Anscombe, F. J. 1963. Sequential medical trials. J. Amer. Statist. Assoc. 58(302) 365-383.
4. Arlotto, A., S. E. Chick, N. Gans. 2010. Optimal employee retention when inferring unknown learning curves. B. Johansson, S. Jain, J. Montoya-Torres, J. Hugan, E. Yücesan, eds. Proc. Winter Simulation Conf., 2010.
5. Armitage, P., G. Berry, J. N. S. Matthews. 2002. Statistical Methods in Medical Research, 4th ed. Blackwell, Malden, MA.
6. Bellman, R. E. 1956. A problem in the sequential design of experiments. Sankhya: Indian J. Statist. 16(3/4) 221-229.
7. Bergman, S. W., J. C. Gittins. 1985. Statistical Methods for Pharmaceutical Research Planning. Marcel Dekker, New York.
8. Berry, D. A, B. Fristedt. 1985. Monographs on statistics and applied probability. Bandit Problems: Sequential Allocation of Experiments. Chapman and Hall, London.
9. Bertsimas, D., A. J. Mersereau. 2007. A learning approach for interactive marketing to a customer segment. Oper. Res. 55(6) 1120-1135. (Pubitemid 351159521)
10. Bertsimas, D., J. Niño-Mora. 2000. Restless bandits, linear programming relaxations, and a primal-dual index heuristic. Oper. Res. 48(1) 80-90.
11. Boyd, S., L. Vandenberghe. 2004. Convex Optimization. Cambridge University Press, Cambridge, UK.
12. Caro, F., J. Gallien. 2007. Dynamic assortment with demand learning for seasonal consumer goods. Management Sci. 53(2) 276-292. (Pubitemid 46326180)
13. Dean, B. C., M. X. Goemans, J. Vondrak. 2008. Approximating the stochastic knapsack problem: The benefit of adaptivity. Math. Oper. Res. 33(4) 945-964.
14. Gittins, J. C. 1989. Multi-Armed Bandit Allocation Indices. Wiley, West Susex, UK.
15. Gittins, J. C., D. M. Jones. 1974. A dynamic allocation index for the sequential design of experiments. J. Gani, ed. Progress in Statistics. North-Holland, Amsterdam, 241-266.
16. Gittins, J., Y. G. Wang. 1992. The learning component of dynamic allocation indices. Ann. Statist. 20(3) 1625-1636.
17. Glazebrook, K. D., D. J. Wilkinson. 2000. Index-based policies for discounted multi-armed bandits on parallel machines. Ann. Appl. Probab. 10(3) 877-896.
18. Goel, A., S. Khanna, B. Null. 2009. The ratio index for budgeted learning, with applications. SODA'09, http://portal.acm.org/citation.cfm?id= 1496773.
19. Guha, S., K. Munagala. 2007. Approximation algorithms for budgeted learning problems. STOC '07: Proc. Thirty-Ninth Annual ACM Sympos. Theory Comput., ACM, New York, 104-113. (Pubitemid 47630726)
20. Lai, T., H. Robbins. 1985. Asymptotically efficient adaptive allocation rules. Adv. Appl. Math. 6(1) 4-22.
21. Pich, M. T., L. Van der Heyden. 2002. Marks and Spencer and Zara: Process competition in the textile apparel industry. INSEAD Business Case Study, INSEAD, Fontainebleau, France.
22. Shor, N. Z. 1985. Minimization Methods for Non-Differentiable Functions. Springer-Verlag, New York.
23. Weber, R. R., G. Weiss. 1990. On an index policy for restless bandits. J. Appl. Probab. 27(3) 637-648.
24. Whittle, P. 1988. Restless bandits: activity allocation in a changing world. J. Appl. Probab. 25 287-298.