Robust control of Markov decision processes with uncertain transition matrices

Operations Research

Volumn 53, Issue 5, 2005, Pages 780-798

  Nilim, Arnab ^a   Ghaoui, Laurent El ^a  

^a UNIVERSITY OF CALIFORNIA   (United States)

Author keywords

Dynamic programming: Markov; Finite state; Game theory; programming: convex; Robustness; statistics: estimation; Uncertainty

Indexed keywords

DYNAMIC PROGRAMMING: MARKOV; FINITE STATE; GAME THEORY; PROGRAMMING: CONVEX; ROBUSTNESS; STATISTICS: ESTIMATION; UNCERTAINTY;

DECISION THEORY; FINITE ELEMENT METHOD; GAME THEORY; PROBABILITY; PROBLEM SOLVING; UNCERTAIN SYSTEMS;

MARKOV PROCESSES;

EID: 14344250395     PISSN: 0030364X     EISSN: 15265463     Source Type: Journal    
DOI: 10.1287/opre.1050.0216     Document Type: Article

References (27)

1. Abbad, M., J. A. Filar. 1992. Perturbation and stability theory for Markov control problems. IEEE Trans. Automatic Control 37 1415-1420.
2. Abbad, M., J. Filar, T. Bielecki. 1992. Algorithms for singularly perturbed limiting average Markov control problems. IEEE Trans. Automatic Control 37 1421-1425.
3. Altman, E., A. Hordijk. 1994. Zero-sum Markov games and worst-case optimal control of queueing systems. QUESTA 21(Special Issue on Optimization of Queueing Systems) 415-447.
4. Altman, E., E. A. Feinberg, A. Shwartz. 2000. Weighted discounted stochastic games with perfect information. Ann. Internat. Soc. Dynamic Games 5 303-323.
5. Bagnell, J., A. Ng, J. Schneider. 2001. Solving uncertain Markov decision problems. Technical report CMU-RI-TR-01-25, Robotics Institute, Carnegie Mellon University, Pittsburgh, PA.
6. Bertsekas, D., J. Tsitsiklis. 1996. Neuro-Dynamic Programming. Athena Scientific, Nashua, NH.
7. Boyd, S., L. Vandenberghe. 2004. Convex Optimization. Cambridge University Press, Cambridge, UK.
8. Epstein, L. G., M. Schneider. 2002. Learning under ambiguity. http://www.econ.rochster.edu/Faculty/Epstein.html.
9. Feinberg, E., A. Shwartz. 2002. Handbook of Markov Decision Processes, Methods and Applications. Kluwer Academic Publishers, Boston, MA.
10. Ferguson, T. 1974. Prior distributions on space of probability measures. Ann. Statist. 2(4) 615-629.
11. Givan, R., S. Leach, T. Dean. 1997. Bounded parameter Markov decision processes. Fourth European Conf. Planning, 234-246.
12. Iyengar, G. 2003. Robust dynamic programming. Technical report TR-2002-07, Columbia University, New York.
13. Kalyanasundaram, S., E. Chong, N. Shroff. 2001. Markov decision processes with uncertain transition rates: Sensitivity and robust control. Technical report, Department of ECE, Purdue University, West Lafayette, IN.
14. Lehmann, E. 1986. Testing Statistical Hypothesis. Wiley, New York.
15. Lehmann, E., G. Casella. 1998. Theory of Point Estimation. Springer-Verlag, New York.
16. Mine, H., S. Osaki. 1970. Markov Decision Processes. American Elsevier Publishing, New York.
17. Nilim, A., L. El Ghaoui. 2002. Robust solution to the Markov decision processes with uncertain transition matrices. Technical report UCB/ERL M02/31, Department of Electrical Engineering and Computer Sciences, University of California, Berkeley, CA.
18. Nilim, A., L. El Ghaoui, M. Hansen, V. Duong. 2001. Trajectory-based air traffic management (TB-ATM) under weather uncertainty. Proc. 4th USA/EUROPE ATM R&D Seminar, Santa Fe, NM, 64-72.
19. Nowak, A. S. 1984. On zero sum stochastic games with general state space. I. Probab. Math. Statist. 4(1) 13-32.
20. Pitman, J. 1993. Probability. Springer-Verlag, New York.
21. Poor, H. 1988. An Introduction to Signal Detection and Estimation. Springer-Verlag, New York.
22. Putterman, M. 1994. Markov Decision Processes: Discrete Stochastic Dynamic Programming. Wiley-Interscience, New York.
23. Satia, J. K., R. L. Lave. 1973. Markov decision processes with uncertain transition probabilities. Oper. Res. 21(3) 728-740.
24. Shapiro, A., A. J. Kleywegt. 2002. Minimax analysis of stochastic problems. Optim. Methods Software. 17(1) 523-592.
25. Siouris, G. 1995. Optimal Control and Estimation Theory. Wiley-Interscience, New York.
26. White, C. C., H. K. Eldeib. 1994. Markov decision processes with imprecise transition probabilities. Oper. Res. 42(4) 739-749.
27. Wilks, S. 1962. Mathematical Statistics. Wiley-Interscience, New York.