Percentile optimization for Markov decision processes with parameter uncertainty

Operations Research

Volumn 58, Issue 1, 2010, Pages 203-213

  Delage, Erick ^a   Mannor, Shie ^b  

^a HEC MONTRÉAL   (Canada)

^b MCGILL UNIVERSITY   (Canada)

Author keywords

Chance constrained optimization; Finite state; Markov decision processes; Parameter uncertainty; Stochastic model applications; Stochastic programming; Value at risk

Indexed keywords

CHANCE-CONSTRAINED; FINITE STATE; MARKOV DECISION PROCESSES; PARAMETER UNCERTAINTY; STOCHASTIC MODEL APPLICATIONS; VALUE AT RISK;

CONSTRAINED OPTIMIZATION; MARKOV PROCESSES; STOCHASTIC MODELS; STOCHASTIC PROGRAMMING; STOCHASTIC SYSTEMS; VALUE ENGINEERING;

DECISION MAKING;

EID: 77249117255     PISSN: 0030364X     EISSN: 15265463     Source Type: Journal    
DOI: 10.1287/opre.1080.0685     Document Type: Article

References (26)

1. Avrachenkov, K. E., J. A. Filar, M. Haviv. 2002. Singular perturbations of Markov chains and decision processes. E. Feinberg, A. Shwartz, eds. Handbook of Markov Decision Processes: Methods and Applications. Kluwer, Boston, 113-152.
2. Bagnell, J., A. Y. Ng, J. Schneider. 2001. Solving uncertain Markov decision problems. Technical report CMU-RI-TR-01-25, Robotics Institute, Carnegie Mellon University, Pittsburgh.
3. Ben-Tal, A., A. Nemirovski. 1998. Robust convex optimization. Math. Oper. Res. 23(4) 769-805.
4. Bertsekas, D. P., J. N. Tsitsiklis. 1996. Neuro-Dynamic Programming. Athena Scientific, Cambridge, MA.
5. Calafiore, G., L. El Ghaoui. 2006. On distributionally robust chanceconstrained linear programs. Optim. Theory Appl. 130(1) 1-22.
6. Charnes, A., W. W. Cooper. 1959. Chance constrained programming. Management Sci. 6 73-79.
7. Dearden, R., N. Friedman, D. Andre. 1999. Model based Bayesian exploration. Proc. 15th Conf. Uncertainty in Artificial Intelligence, Stockholm, 150-159.
8. Filar, J. A., D. Krass, K. W. Ross. 1995. Percentile performance criteria for limiting average Markov control problems. IEEE Trans. Automatic Control 40 2-10.
9. Gelman, A., J. B. Carlin, H. S. Stern, D. B. Rubin. 2003. Bayesian Data Analysis, 2nd ed. Chapman and Hall/CRC, Boca Raton, FL.
10. Givan, R., S. M. Leach, T. Dean. 2000. Bounded-parameter Markov decision processes. Artificial Intelligence 122(1-2) 71-109.
11. Grant, M., S. Boyd. 2009. CVX: Matlab software for disciplined convex programming (Web page and software). February. http://www.stanford.edu/boyd/cvx.
12. Howard, R., J. Matheson. 1972. Risk-sensitive Markov decision processes. Management Sci. 18(7) 356-369.
13. Iyengar, G. 2005. Robust dynamic programming. Math. Oper. Res. 30(2) 257-280.
14. Lobo, M. S., L. Vandenberghe, S. Boyd, H. Lebret. 1998. Applications of second order cone programming. Linear Algebra and Its Appl. 284 193-228.
15. MacQueen, J. 1966. A modified dynamic programming method for Markov decision problems. J. Math. Anal. Appl. 14 28-43.
16. Mannor, S., D. Simester, P. Sun, J. N. Tsitsiklis. 2007. Bias and variance approximation in value function estimates. Management Sci. 53(2) 308-322.
17. Martin, J. J. 1967. Bayesian Decision Problems and Markov Chains. John Wiley and Sons, New York.
18. Nemirovski, A., A. Shapiro. 2006. Convex approximations of chance constrained programs. SIAM J. Optim. 17(4) 969-996.
19. Nilim, A., L. El Ghaoui. 2005. Robust control of Markov decision processes with uncertain transition matrices. Oper. Res. 53(5) 780-798.
20. Prékopa, A. 1995. Stochastic Programming. Kluwer Academic Publishers, Dordrecht, The Netherlands.
21. Puterman, M. L. 1994. Markov Decision Processes: Discrete Stochastic Dynamic Programming. Wiley, New York.
22. Satia, J. K., R. L. Lave. 1973. Markov decision processes with imprecise transition probabilities. Oper. Res. 21(3) 755-763.
23. Shapiro, A., A. J. Kleywegt. 2002. Minimax analysis of stochastic problems. Optim. Methods Software 17 523-542.
24. Silver, E. A. 1963. Markovian decision processes with uncertain transition probabilities or rewards. Technical report 1, Operations Research Center, Massachusetts Institute of Technology, Cambridge, MA.
25. van der Schaft, A. J. 1996. L2-Gain and Passivity Techniques in Non-Linear Control. Springer-Verlag, New York, Inc., Secaucus, NJ.
26. Wilks, S. S. 1962. Mathematical Statistics. John Wiley and Sons, New York