An adaptive sampling algorithm for solving Markov decision processes

Operations Research

Volumn 53, Issue 1, 2005, Pages 126-139

  Chang, Hyeong Soo ^a   Fu, Michael C ^b   Hu, Jiaqiao ^c   Marcus, Steven I ^c  

^a Sogang University   (South Korea)

^b UNIVERSITY OF MARYLAND   (United States)

^c UNIVERSITY OF MARYLAND   (United States)

Author keywords

Dynamic programming optimal control: Markov finite state

Indexed keywords

ADAPTIVE SAMPLING; DYNAMIC PROGRAMMING/OPTIMAL CONTROL; MARKOV DECISION PROCESSES; MARKOV FINITE STATE;

ADAPTIVE ALGORITHMS; ALGORITHMS; DYNAMIC PROGRAMMING; INVENTORY CONTROL; OPTIMAL CONTROL SYSTEMS; PROBLEM SOLVING; RANDOM PROCESSES;

MARKOV PROCESSES;

EID: 14644444172     PISSN: 0030364X     EISSN: None     Source Type: Journal    
DOI: 10.1287/opre.1040.0145     Document Type: Article

References (17)

1. Agrawal, R. 1995. Sample mean based index policies with O(log n) regret for the multiarmed bandit problem. Advances Appl. Probab. 27 1054-1078.
2. Agrawal, R., D. Teneketzis, V. Anantharam. 1989. Asymptotically efficient adaptive allocation schemes for controlled Markov chains: Finite parameter space. IEEE Trans. Automat. Control 34 1249-1259.
3. Auer, P., N. Cesa-Bianchi, P. Fisher. 2002. Finite-time analysis of the multiarmed bandit problem. Machine Learning 47 235-256.
4. Bertsekas, D. P. 1995. Dynamic Programming and Optimal Control, Vols. 1 and 2. Athena Scientific, Belmont, MA.
5. Bertsekas, D. P. 1997. Differential training of rollout policies. Proc. 35th Allerton Conf. Communication, Control, Comput., Allerton Park, IL, 913-922.
6. Blondel, V. D., J. Tsitsiklis. 2000. A survey of computational complexity results in systems and control. Automatica 36 1249-1274.
7. Broadie, M., P. Glasserman. 1997. Pricing American-style securities using simulation. J. Econom. Dynamics Control 21 1323-1352.
8. Cesa-Bianchi, N., P. Fisher. 1998. Finite-time regret bounds for the multiarmed bandit problem. Proc. 15th Int. Conf. Machine Learning. Morgan Kaufmann Publishers, San Francisco, CA, 101-108.
9. Cormen, T. H., C. E. Leiserson, R. L. Rivest. 1990. Introduction to Algorithms. MIT Press, Cambridge, MA.
10. Graves, T. L., T. L. Lai. 1997. Asymptotically efficient adaptive choice of control laws in controlled Markov chains. SIAM J. Control Optim. 35 715-743.
11. Grimmett, G., D. Stirzaker. 2001. Probability and Random Processes, 3rd ed. Oxford University Press, New York.
12. Hernández-Lerma, O., J. B. Lasserre. 1990. Error bounds for rolling horizon policies in discrete-time Markov control processes. IEEE Trans. Automat. Control 35 1118-1124.
13. Hoeffding, W. 1963. Probability inequalities for sums of bounded random variables. J. Amer. Statist. Assoc. 58 13-30.
14. Kearns, M., Y. Mansour, A. Y. Ng. 2001. A sparse sampling algorithm for near-optimal planning in large Markov decision processes. Machine Learning 49 193-208.
15. Lai, T., H. Robbins. 1985. Asymptotically efficient adaptive allocation rules. Advances Appl. Math. 6 4-22.
16. Law, A. M., W. D. Kelton. 2000. Simulation Modeling and Analysis, 3rd ed. McGraw-Hill, New York.
17. Ross, S. 1995. Stochastic Process, 2nd ed. John Wiley and Sons, New York.