The value of information in multi-armed bandits with exponentially distributed rewards

Procedia Computer Science

Volumn 4, Issue , 2011, Pages 1363-1372

  Ryzhov, Ilya O ^a   Powell, Warren B ^a  

^a Princeton University   (United States)

Author keywords

Exponential rewards; Knowledge gradient; Multi armed bandit; Optimal learning

Indexed keywords

BAYESIAN MODEL; EXPONENTIAL DISTRIBUTIONS; EXPONENTIAL REWARDS; GAMMA PRIOR; GENERAL APPROACH; KNOWLEDGE GRADIENT; LEARNING MODELS; LEARNING PROBLEM; MAKING DECISION; MULTI ARMED BANDIT; MULTI-ARMED BANDIT PROBLEM; OPTIMAL LEARNING; VALUE OF INFORMATION;

BAYESIAN NETWORKS;

OPTIMIZATION;

EID: 79958262308     PISSN: None     EISSN: 18770509     Source Type: Conference Proceeding    
DOI: 10.1016/j.procs.2011.04.147     Document Type: Conference Paper

References (31)

1. J. C. Gittins, D. M. Jones, A dynamic allocation index for the discounted multiarmed bandit problem, Biometrika 66 (3) (1979) 561-565. (Pubitemid 10218405)
2. D. Berry, L. Pearson, Optimal designs for clinical trials with dichotomous responses, Statistics in Medicine 4 (4) (1985) 497-508. (Pubitemid 16176681)
3. J. C. Gittins, D. M. Jones, A dynamic allocation index for the sequential design of experiments, in: J. Gani (Ed.), Progress in Statistics, 1974, pp. 241-266.
4. J. Gittins, Multi-Armed Bandit Allocation Indices, John Wiley and Sons, New York, 1989.
5. M. Brezzi, T. Lai, Optimal learning and experimentation in bandit problems, Journal of Economic Dynamics and Control 27 (1) (2002) 87-108.
6. Y. Yao, Some results on the Gittins index for a normal reward process, in: H. Ho, C. Ing, T. Lai (Eds.), Time Series and Related Topics: In Memory of Ching-Zong Wei, Institute of Mathematical Statistics, Beachwood, OH, USA, 2006, pp. 284-294.
7. S. Chick, N. Gans, Economic analysis of simulation selection options, Management Science 55 (3) (2009) 421-437.
8. T. Lai, Adaptive treatment allocation and the multi-armed bandit problem, The Annals of Statistics 15 (3) (1987) 1091-1114.
9. P. Auer, N. Cesa-Bianchi, P. Fischer, Finite-time analysis of the multiarmed bandit problem, Machine Learning 47 (2-3) (2002) 235-256. (Pubitemid 34126111)
10. J. Gittins, Y. Wang, The learning component of dynamic allocation indices, The Annals of Statistics 20 (3) (1992) 1625-1636.
11. T. L. Lai, H. Robbins, Asymptotically efficient adaptive allocation rules, Advances in Applied Mathematics 6 (1985) 4-22.
12. R. Agrawal, Sample mean based index policies with O (log n) regret for the multi-armed bandit problem, Advances in Applied Probability 27 (4) (1995) 1054-1078.
13. K. Liu, Q. Zhao, Distributed Learning in Multi-Armed Bandit with Multiple Players, IEEE Transactions on Signal Processing 58 (11) (2010) 5667-5681.
14. R. Sutton, A. Barto, Reinforcement Learning, The MIT Press, Cambridge, Massachusetts, 1998.
15. W. B. Powell, Approximate Dynamic Programming: Solving the curses of dimensionality, John Wiley and Sons, New York, 2007.
16. M. A. Lariviere, E. Porteus, Stalking information: Bayesian inventory management with unobserved lost sales, Management Science 45 (3) (1999) 1346-363.
17. V. Farias, B. Van Roy, Dynamic pricing with a prior on market response, Operations Research 58 (1) (2010) 16-29.
18. S. Gupta, K. Miescke, Bayesian look ahead one stage sampling allocations for selecting the largest normal mean, Statistical Papers 35 (1994) 169-177.
19. S. Gupta, K. Miescke, Bayesian look ahead one-stage sampling allocations for selection of the best population, Journal of statistical planning and inference 54 (2) (1996) 229-244. (Pubitemid 126161097)
20. P. I. Frazier, W. B. Powell, S. Dayanik, A knowledge gradient policy for sequential information collection, SIAM Journal on Control and Optimization 47 (5) (2008) 2410-2439.
21. P. I. Frazier, W. B. Powell, S. Dayanik, The knowledge-gradient policy for correlated normal rewards, INFORMS J. on Computing 21 (4) (2009) 599-613.
22. S. Chick, J. Branke, C. Schmidt, Sequential Sampling to Myopically Maximize the Expected Value of Information, INFORMS J. on Computing 22 (1) (2010) 71-80.
23. P. Frazier, W. Powell, Paradoxes in learning and the marginal value of information, Decision Analysis 7 (4) (2011) 378-403.
24. I. O. Ryzhov, W. B. Powell, The knowledge gradient algorithm for online subset selection, in: Proceedings of the 2009 IEEE Symposium on Adaptive Dynamic Programming and Reinforcement Learning, Nashville, TN, 2009, pp. 137-144.
25. I. O. Ryzhov, W. B. Powell, A Monte Carlo Knowledge Gradient Method For Learning Abatement Potential Of Emissions Reduction Technologies, in: M. Rosetti, R. Hill, B. Johansson, A. Dunkin, R. Ingalls (Eds.), Proceedings of the 2009 Winter Simulation Conference, 2009, pp. 1492-1502.
26. I. O. Ryzhov, W. B. Powell, P. I. Frazier, The knowledge gradient algorithm for a general class of online learning problems, Submitted for publication.
27. I. O. Ryzhov, P. I. Frazier, W. B. Powell, On the robustness of a one-period look-ahead policy in multi-armed bandit problems, in: Proceedings of the 2010 International Conference on Computational Science, 2010, pp. 1629-1638.
28. M. H. DeGroot, Optimal Statistical Decisions, John Wiley and Sons, 1970.
29. P. Key, E. Godolphin, On the Bayesian steady forecasting model, Journal of the Royal Statistical Society B43 (1) (1981) 92-96.
30. S. E. Chick, P. I. Frazier, The Conjunction Of The Knowledge Gradient And The Economic Approach To Simulation Selection, in: M. Rosetti, R. Hill, B. Johansson, A. Dunkin, R. Ingalls (Eds.), Proceedings of the 2009 Winter Simulation Conference, 2009, pp. 528-539.
31. I. O. Ryzhov, W. B. Powell, Information collection on a graph, Operations Research (to appear).