Distributionally robust markov decision processes

Mathematics of Operations Research

Volumn 37, Issue 2, 2012, Pages 288-300

  Xu, Huan ^a   Mannor, Shie ^b  

^a NATIONAL UNIVERSITY OF SINGAPORE   (Singapore)

^b TECHNION ISRAEL INSTITUTE OF TECHNOLOGY   (Israel)

Author keywords

Distributional robustness; Markov decision process; Parameter uncertainty

Indexed keywords

CONFIDENCE LEVELS; CONFIDENCE REGION; DECISION CRITERIONS; DECISION MAKERS; MARKOV DECISION PROCESSES; NESTED SETS; OPTIMAL STRATEGIES; PARAMETER UNCERTAINTY; POLYNOMIAL-TIME; PRIORI INFORMATION; PROBABILISTIC GUARANTEES; ROBUST STRATEGY; TECHNICAL CONDITIONS; UNKNOWN PARAMETERS;

LEARNING ALGORITHMS; MARKOV PROCESSES; POLYNOMIAL APPROXIMATION; PROBABILITY DISTRIBUTIONS;

UNCERTAINTY ANALYSIS;

EID: 84861386911     PISSN: 0364765X     EISSN: 15265471     Source Type: Journal    
DOI: 10.1287/moor.1120.0540     Document Type: Article

References (33)

1. Abbad M, Filar J (1992) Perturbation and stability theory for Markov control problems. IEEE Trans. Automatic Control 37(9):1415-1420. (Pubitemid 23555846)
2. Abbad M, Filar JA, Bielecki TR (1992) Algorithms for singularly perturbed limiting average Markov control problems. IEEE Trans. Automatic Control 37(9):1421-1425. (Pubitemid 23555845)
3. Avrachenkov KE, Filar J, Haviv M (2002) Singular perturbations of Markov chains and decision processes. Feinberg EA, Shwartz A, eds. Handbook of Markov Decision Processes: Methods and Applications.
4. Bagnell A, Ng A, Schneider J (2001) Solving uncertain Markov decision problems. Technical Report CMU-RI-TR-01-25, Carnegie Mellon University, Pittsburgh.
5. Baron J (2000) Thinking and Deciding (Cambridge University Press, New York).
6. Ben-Tal A, Nemirovski A (1999) Robust solutions of uncertain linear programs. Oper. Res. Lett. 25(1):1-13.
7. Bertsekas DP, Tsitsiklis JN (1996) Neuro-Dynamic Programming (Athena Scientific, Nashua, NH).
8. Blackwell D, Girshick M (1954) Theory of Games and Statistical Decisions (John Wiley & Sons Inc., New York).
9. Boyd S, Vandenberghe L (2004) Convex Optimization (Cambridge University Press, New York).
10. Calafiore G, El Ghaoui L (2006) On distributionally robust chance-constrained linear programs. J. Optimization Theory and Appl. 130(1):1-22. (Pubitemid 44951197)
11. Delage E, Mannor S (2010) Percentile optimization for Markov decision processes with parameter uncertainty. Oper. Res. 58(1):203-213.
12. Delage E, Ye Y (2010) Distributionally robust optimization under moment uncertainty with applications to data-driven problems. Oper. Res. 58(3):596-612.
13. Delebecque F, Quadrat JP (1981) Optimal control of Markov chains admitting strong and weak interactions. Automatica 17(2):281-296. (Pubitemid 11504180)
14. Dupacová J (1987) The minimax approach to stochastic programming and an illustrative application. Stochastics 20:73-88.
15. Dvoretzky A, Wald A, Wolfowitz J (1951) Elimination of randomization in certain statistical decision procedures and zero-sum two-person games. Ann. Math. Statist. 22(1):1-21.
16. Epstein LG, Schneider M (2007) Learning under ambiguity. Rev. Econom. Stud. 74(4):1275-1303. (Pubitemid 47382376)
17. Gilboa I, Schmeidler D (1989) Maxmin expected utility with a non-unique prior. J. Math. Econom. 18(2):141-153.
18. Goh J, Sim M (2010) Distributionally robust optimization and its tractable approximations. Oper. Res. 58(4):902-917.
19. Grötschel M, Lovász L, Schrijver A (1988) Geometric Algorithms and Combinatorial Optimization (Springer, Heidelberg).
20. Iyengar GN (2005) Robust dynamic programming. Math. Oper. Res. 30(2):257-280. (Pubitemid 43126832)
21. Kall P (1988) Stochastic programming with recourse: Upper bounds and moment problems, a review. Advances in Mathematical Optimization (Academie-Verlag, Berlin), 86-103.
22. Karlin S (1953) The theory of infinite games. Ann. Math. 58(2):371-401.
23. Kelsey D (1994) Maxmin expected utility and weight of evidence. Oxford Econom. Papers 46(3):425-444.
24. Mannor S, Simester D, Sun P, Tsitsiklis JN (2007) Bias and variance approximation in value function estimates. Management Sci. 53(2):308-322. (Pubitemid 46326182)
25. Nilim A, El Ghaoui L (2005) Robust control of Markov decision processes with uncertain transition matrices. Oper. Res. 53(5):780-798. (Pubitemid 41525849)
26. Popescu I (2007) Robust mean-covariance solutions for stochastic optimization. Oper. Res. 55(1):98-112. (Pubitemid 46663476)
27. Puterman ML (1994) Markov Decision Processes (John Wiley & Sons, New York).
28. Rockafellar RT (1970) Convex Analysis (Princeton University Press, Princeton, NJ).
29. Scarf H (1958) A min-max solution of an inventory problem. Arrow KJ, Karlin S, Scarf H, eds. Studies in Mathematical Theory of Inventory and Production (Stanford University Press, Stanford, CA), 201-209.
30. Shapiro A (2006) Worst-case distribution analysis of stochastic programs. Math. Programming 107(1):91-96. (Pubitemid 43334015)
31. Shapley LS (1953) Stochastic games. Proc. National Acad. Sci. USA 39(10):1095-1100.
32. Sion M (1958) On general minimax theorems. Pacific J. Math. 8(1):171-176.
33. White III CC, El Deib HK (1992) Markov decision processes with imprecise transition probabilities. Oper. Res. 42(4):739-748.