Optimal adaptive policies for sequential allocation problems

Advances in Applied Mathematics

Volumn 17, Issue 2, 1996, Pages 122-142

  Burnetas, Apostolos N ^a   Katehakis, Michael N ^b  

^a CASE WESTERN RESERVE UNIVERSITY   (United States)

^b RUTGERS UNIVERSITY   (United States)

Author keywords

[No Author keywords available]

Indexed keywords

EID: 0030159874     PISSN: 01968858     EISSN: None     Source Type: Journal    
DOI: 10.1006/aama.1996.0007     Document Type: Article

References (33)

1. 1. R. Acost-Abreu and O. Hernandez-Lerma, Iterative adaptive control of denumerable state, average-cost Markov systems, Control Cybernet. 14 (1985), 313-322.
2. 2. R. Agrawal, D. Teneketzis, and V. Anantharam, Asymtotically efficient adaptive allocation schemes for controlled Markov chains: Finite parameter space, Trans. Automat. Control IEEE 34 (1989), 1249-1259.
3. 3. R. Bellman, "Dynamic Programming," Princeton Univ. Press, Princeton, NJ, 1957.
4. 4. D. A. Berry and B. Fristedt, "Bandit Problems: Sequential Allocation of Experiments," Chapman & Hall, London (1985).
5. 5. F. J. Beutler and D. Teneketzis, Routing in queueing networks under imperfect information: Stochastic dominance and thresholds, Stochastics Stochastics Rep. 26 (1989), 81-100.
6. 6. A. N. Burnetas and M. N. Katehakis, "On Optimal Sequential Allocation Policies for the Finite Horizon One-Armed Bandit Problem," Technical Report, Rutgers University, 1989.
7. 7. A. N. Burnetas and M. N. Katehakis, On sequencing two types of tasks on a single processor under incomplete information, Probab. Engrg. Inform. Sci. 7 (1993) 85-119.
8. 8. A. N. Burnetas and M. N. Katehakis, "Optimal Adaptive Policies for Dynamic Programming," Technical Report, Rutgers University, 1994.
9. 9. A. Dembo and O. Zeitouni, "Large Deviations Techniques and Applications," Jones & Bartlett, Boston, 1993.
10. 10. E. B. Dynkin and A. A. Yushkevich, "Controlled Markov Processes," Springer-Verlag, Berlin/New York, 1979.
11. 11. R. S. Ellis, "Entropy, Large Deviations and Statistical Mechanics," Springer-Verlag, Berlin/New York, 1985.
12. 12. A. Federgruen and P. Schweitzer, Nonstatoinary Markov decision problems with converging parameters, J. Optim. Theory Appl. 34 (1981), 207-241.
13. 13. W. Feller, "An Introduction to Probability Theory and Its Applications," Vol. 1, 3rd ed., Wiley, New York, 1967.
14. 14. B. L. Fox and J. F. Rolph, Adaptive policies for Markov renewal programs, Ann. Statist. 1 (1973), 334-341.
15. 15. J. C. Gittins, Bandit processes and dynamic allocation indices (with discussion), J. Roy. Statist. Soc. Ser. B 41 (1979), 335-340.
16. 16. J. C. Gittins and K. D. Glazebrook, On Bayesian models in stochastic scheduling, J. Appl. Probab. 14 (1977), 556-565.
17. 17. J. C. Gittins and D. M. Jones, A dynamic allocation index for the discounted multarmed bandit problem, Biometrika 66 (1979), 561-565.
18. 18. Z. Govindarajulu, "The Sequential Statistical Analysis of Hypothesis Testing, Point and Interval Estimation, and Decision Theory," 2nd ed., Am. Sci. Press, Syracuse, NY, 1987.
19. 19. M. N. Katehakis and C. Derman, Computing optimal sequential allocation rules in clinical trials, in "Adaptive Statistical Procedures and Related Topics," (J. van Ryzin, Ed. Vol. 8, pp. 29-39, IMS Lecture Notes - Monograph Series, Inst. Math .Statist., Hayward, CA, 1985.
20. 20. M. N. Katehakis and A. F. Vienott, Jr., The multi-armed bandit problem: Decomposition and computation, Math. Oper. Res. 12 (1987), 262-268.
21. 21. M.N. Katehakis and H. Robbins, "Sequential Allocation Involving Normal Populations," Technical Report, Rutgers University, 1994.
22. 22. P. R. Kumar, A survey of some results in stochastic adaptive control, SIAM J. Control Optim. 23 (1985) 329-380.
23. 23. T. L. Lai, Adaptive treatment allocation and the multi-armed bandit problem, Ann. Statist. 15 (1987), 1091-1114.
24. 24. T. L. Lai and H. Robbins, Asymptotically optimal allocation of treatments in sequential experiments, in "Design of Experients: Ranking and Selection: Essays in Honor of Robert E. Bechhofer" (T. J. Santner and A. C. Tamhane, Eds.), Vol. 56, pp. 127-142, Dekker, New York, 1984.
25. 25. T. L. Lai and H. Robbins, Asymptotically efficient adaptive allocation rules, Adv. in Appl. Math. 6 (1985), 4-22.
26. 26. Z. Li and C. Zhang, Asymptotically efficient allocation rules for two Bernoulli populations, J. Roy. Statist. Soc. Ser. B 54 (1992), 609-616.
27. 27. P. Mandl, Estimation and control in Markov chains, Adv. in Appl. Probab. 6 (1974), 40-60.
28. 28. K. M. Van Hee, Markov decision processes with unknown transition law: The average return case, in "Recent Developments in Markov Decision Processes" (D. J. White R. Hartley, and L. C. Thomas, Eds.), pp. 227-244. Academic Press, New York, 1980.
29. 29. R. A. Milito and J. B. Cruz, Jr., A weak contrast function approach to adaptive semi-Markov decision models, in "Stochastic Large Scale Engineering Systems" (S. Tzafestas and C. Watanabe, Eds.), pp. 253-278, Dekker, New York, 1992.
30. 30. U. Rieder and J. Weishaupt, Customer scheduling with incomplete information, Probab. Engrg. Inform. Sci. to appear.
31. 31. H. Robbins, Some aspects of the sequential design of experiments, Bull. Amer. Math. Monthly 58 (1952), 527-536.
32. 32. H. R. Varian, "Microeconomic Analysis," 2nd ed., Norton, New York, 1984.
33. 33. S. Yakowitz and W. Lowe, Nonparametric bandit methods, Ann. Oper. Res. 28 (1991), 297-312.