A modern Bayesian look at the multi-armed bandit

Applied Stochastic Models in Business and Industry

Volumn 26, Issue 6, 2010, Pages 639-658

  Scott, Steven L ^a  

^a GOOGLE INC   (United States)

Author keywords

Bayesian adaptive design; exploration vs exploitation; probability matching; sequential design

Indexed keywords

ADAPTIVE DESIGNS; BAYESIAN; BAYESIAN COMPUTATION; BAYESIAN POSTERIOR PROBABILITIES; EXPERIMENTAL DESIGN; EXPLORATION VS EXPLOITATION; MULTI ARMED BANDIT; PROBABILITY MATCHING; SEQUENTIAL DESIGN; UNKNOWN PARAMETERS;

BAYESIAN NETWORKS; DESIGN; OPTIMIZATION;

PROBABILITY DISTRIBUTIONS;

EID: 78650505735     PISSN: 15241904     EISSN: 15264025     Source Type: Journal    
DOI: 10.1002/asmb.874     Document Type: Article

References (27)

1. Google. Available from:, 2010.
2. Whittle P,. Discussion of 'bandit processes and dynamic allocation indices'. Journal of the Royal Statistical Society, Series B: Methodological 1979; 41: 165.
3. Gittins JC,. Bandit processes and dynamic allocation indices. Journal of the Royal Statistical Society, Series B: Methodological 1979 41: 148-177.
4. Bellman RE,. A problem in the sequential design of experiments. Sankhya Series A 1956; 30: 221-252.
5. Brezzi M, Lai TL,. Incomplete leraning from endogenous data in dynamic allocation. Econometrica 2000; 68 (6): 1511-1516.
6. Thompson WR,. On the likelihood that one unknown probability exceeds another in view of the evidence of two samples. Biometrika 1933; 25: 285-294.
7. Thompson WR,. On the theory of apportionment. American Journal of Mathematics 1935; 57 (2): 450-456.
8. Cox DR, Reid N,. The Theory of the Design of Experiments. Chapman & Hall, CRC: London, Boca Raton, 2000.
9. Chaloner K, Verdinelli I,. Bayesian experimental design: a review. Statistical Science 1995; 10: 273-304.
10. Tierney L,. Markov chains for exploring posterior distributions (disc: P1728-1762). The Annals of Statistics 1994; 22: 1701-1728.
11. Doucet A, De Frietas N, Gordon N,. Sequential Monte Carlo in Practice. Springer: Berlin, 2001.
12. Powell WB,. Approximate Dynamic Programming: Solving the Curses of Dimensionality. Wiley: New York, 2007.
13. Sutton RS, Barto AG,. Reinforcement Learning: An Introduction. MIT Press: Cambridge, MA, 1998.
14. Brezzi M, Lai TL,. Optimal learning and experimentation in bandit problems. Journal of Economic Dynamics and Control 2002; 27: 87-108.
15. Gittins J, Wang Y-G,. The learning component of dynamic allocation indices. The Annals of Statistics 1992; 20: 1625-1636.
16. Berry DA, Fristedt B,. Bandit Problems: Sequential Allocation of Experiments. Chapman & Hall: London, 1985.
17. Lai T-L, Robbins H,. Asymptotically efficient adaptive allocation rules. Advances in Applied Mathematics 1985; 6: 4-22.
18. Lai T-L,. Adaptive treatment allocation and the multi-armed bandit problem. The Annals of Statistics 1987; 15 (3): 1091-1114.
19. Agrawal R,. Sample mean based index policies with o(logn) regret for the multi-armed bandit problem. Advances in Applied Probability 1995; 27: 1054-1078.
20. Auer P, Cesa-Bianchi N, Fischer P,. Finite-time analysis of the multiarmed bandit problem. Machine Learning 2002; 47: 235-256.
21. Robbins H,. A sequential decision procedure with a finite memory. Proceedings of the National Academy of Science 1956; 42: 920-923.
22. Luce D,. Individual Choice Behavior. Wiley: New York, 1959.
23. Yang Y, Zhu D,. Randomized allocation with nonparametric estimation for a multi-armed bandit problem with covariates. The Annals of Statistics 2002; 30: 100-121.
24. Albert JH, Chib S,. Bayesian analysis of binary and polychotomous response data. Journal of the American Statistical Association 1993; 88: 669-679.
25. Whittle P,. Restless bandits: activity allocation in a changing world. Journal of Applied Probability 1988; 25A: 287-298.
26. West M, Harrison J,. Bayesian Forecasting and Dynamic Models. Springer: Berlin, 1997.
27. Whittle P,. Arm-acquiring bandits. The Annals of Probability 1981; 9 (2): 284-292.