Loss bounds for uncertain transition probabilities in Markov decision processes

Proceedings of the IEEE Conference on Decision and Control

Volumn , Issue , 2012, Pages 6708-6715

  Mastin, Andrew ^a   Jaillet, Patrick ^a  

^a Department of Electrical Engineering and Computer Science   (United States)

Author keywords

[No Author keywords available]

Indexed keywords

DYNAMIC PROGRAMMING; MARKOV PROCESSES;

FINITE HORIZONS; INFINITE HORIZONS; MARKOV DECISION PROCESSES; NON NEGATIVES; PRECOMPUTING; TOTAL VARIATION; TRANSITION PROBABILITIES; VALUE FUNCTIONS;

PROBABILITY DISTRIBUTIONS;

EID: 84874269643     PISSN: 07431546     EISSN: 25762370     Source Type: Conference Proceeding    
DOI: 10.1109/CDC.2012.6426504     Document Type: Conference Paper

References (22)

1. E. A. Silver, "Markovian decision processes with uncertain transition probabilities or rewards," Massachusetts Institute of Technology, Tech. Rep. AD0417150, August 1963.
2. J. K. Satia and R. E. Lave Jr., "Markovian decision processes with uncertain transition probabilities," Operations Research, vol. 21, no. 3, pp. 728-740, 1973.
3. R. S. Filho and F. W. Trevizan, "Multilinear and integer programming for Markov decision processes with imprecise probabilities," in 5th International Symposium on Imprecise Probability: Theories and Applications, 2007.
4. M. Kurano, M. Hosaka, Y. Huang, and J. Song, "Controlled Markov set-chains with discounting." J. Appl. Probab., vol. 35, no. 2, pp. 293-302, 1998.
5. A. Nilim and L. El Ghaoui, "Robust control of Markov decision processes with uncertain transition matrices," Oper. Res., vol. 53, no. 5, pp. 780-798, September-October 2005.
6. K. V. Delgado, S. Sanner, and L. N. de Barros, "Efficient solutions to factored MDPs with imprecise transition probabilities," Artificial Intelligence, vol. 175, no. 910, pp. 1498 - 1527, 2011.
7. C. C. White III and H. K. Eldeib, "Markov decision processes with imprecise transition probabilities," Operations Research, vol. 42, pp. 739-749, 1994.
8. R. Givan, S. Leach, and T. Dean, "Bounded-parameter Markov decision processes," Artificial Intelligence, vol. 122, pp. 71-109, 2000.
9. W. J. Hopp, "Sensitivity analysis in discrete dynamic programming," J. Optim. Theory Appl., vol. 56, pp. 257-269, February 1988.
10. A. Müller, "How does the value function of a Markov decision process depend on the transition probabilities?" Math. Oper. Res., vol. 22, pp. 872-885, November 1997.
11. C. H. Tan and J. C. Hartman, Sensitivity Analysis and Dynamic Programming. John Wiley & Sons, Inc., 2010.
12. - "Sensitivity analysis in Markov decision processes with uncertain reward parameters," Journal of Applied Probability, vol. 48, no. 4, pp. 954 - 967, 2011.
13. S. P. Singh and R. C. Yee, "An upper bound on the loss from approximate optimal-value functions," Machine Learning, vol. 16, no. 3, pp. 227-233, 1994.
14. M. Kearns, Y. Mansour, and A. Y. Ng, "A sparse sampling algorithm for near-optimal planning in large Markov decision processes," in Machine Learning, 1999, pp. 1324-1331.
15. L. Mercier and P. Van Hentenryck, "Performance analysis of online anticipatory algorithms for large multistage stochastic integer programs," in Proceedings of the 20th international joint conference on Artifical intelligence, ser. IJCAI'07. San Francisco, CA, USA: Morgan Kaufmann Publishers Inc., 2007, pp. 1979-1984.
16. P. V. Hentenryck and R. Bent, Online Stochastic Combinatorial Optimization. The MIT Press, 2009.
17. D. P. Bertsekas and J. N. Tsitsiklis, Neuro-Dynamic Programming. Athena Scientific, 1996.
18. R. Munos, "Error bounds for approximate policy iteration," in ICML, 2003, pp. 560-567.
19. A. M. Farahmand, R. Munos, and C. Szepesvári, "Error propagation for approximate policy and value iteration," in NIPS, 2010, pp. 568-576.
20. W. B. Powell, Approximate Dynamic Programming. John Wiley and Sons, Inc., 2007.
21. R. F. Drenick, "Multilinear programming: Duality theories," Journal of Optimization Theory and Applications, vol. 72, pp. 459-486, 1992.
22. D. P. Bertsekas, Dynamic Programming and Optimal Control, 3rd ed. Athena Scientific, 2007.