Efficient solutions to factored MDPs with imprecise transition probabilities

Artificial Intelligence

Volumn 175, Issue 9-10, 2011, Pages 1498-1527

  Delgado, Karina Valdivia ^a   Sanner, Scott ^b   De Barros, Leliane Nunes ^a  

^a UNIVERSITY OF SÃO PAULO   (Brazil)

^b AUSTRALIAN NATIONAL UNIVERSITY   (Australia)

Author keywords

Markov Decision Process; Probabilistic planning; Robust planning

Indexed keywords

APPROXIMATION TECHNIQUES; COMPUTATIONAL BOTTLENECKS; COMPUTATIONAL OVERHEADS; DECISION-THEORETIC; DYNAMIC PROGRAMMING METHODS; MARKOV DECISION PROCESSES; NON-LINEAR SOLVER; NONLINEAR CONSTRAINED OPTIMIZATION PROBLEMS; NONSTATIONARY; OPTIMAL SOLUTIONS; OPTIMIZATION ROUTINE; ORDERS OF MAGNITUDE; PLANNING PROBLEM; PROBABILISTIC PLANNING; ROBUST PLANNING; SOLUTION ALGORITHMS; TRANSITION MODEL; TRANSITION PROBABILITIES; VALUE ITERATION;

APPROXIMATION ALGORITHMS; CONSTRAINED OPTIMIZATION; MARKOV PROCESSES; PROBABILITY;

DYNAMIC PROGRAMMING;

EID: 79953864311     PISSN: 00043702     EISSN: None     Source Type: Journal    
DOI: 10.1016/j.artint.2011.01.001     Document Type: Article

References (32)

1. M.L. Puterman Markov Decision Processes Wiley Series in Probability and Mathematical Statistics 1994 John Wiley and Sons New York
2. C. Boutilier, S. Hanks, and T. Dean Decision-theoretic planning: Structural assumptions and computational leverage JAIR 11 1999 1 94 (Pubitemid 129628760)
3. J. Hoey, R. St-Aubin, A. Hu, and C. Boutilier SPUDD: Stochastic planning using decision diagrams Proceedings UAI 1999 Morgan Kaufmann 279 288
4. R. St-Aubin, J. Hoey, and C. Boutilier APRICODD: Approximate policy construction using decision diagrams Proceedings NIPS 2000 MIT Press 1089 1095
5. C. Guestrin, D. Koller, R. Parr, and S. Venkataraman Efficient solution algorithms for factored MDPs JAIR 19 2003 399 468 (Pubitemid 41525926)
6. J.K. Satia, and R.E. Lave Jr. Markovian decision processes with uncertain transition probabilities Oper. Res. 21 1970 728 740
7. C.C. White III, and H.K. El-Deib Markov decision processes with imprecise transition probabilities Oper. Res. 42 4 1994 739 749
8. T. Dean, and K. Kanazawa A model for reasoning about persistence and causation Comput. Intell. 5 3 1990 142 150
9. K.V. Delgado, L.N. de Barros, F.G. Cozman, R. Shirota, Representing and solving factored Markov decision processes with imprecise probabilities, in: Proceedings ISIPTA, Durham, United Kingdom, 2009.
10. D.P. Bertsekas, and J.N. Tsitsiklis An analysis of stochastic shortest path problems Math. Oper. Res. 16 3 1991 580 595
11. F.G. Cozman Credal networks Artificial Intelligence 120 2000 199 233
12. P. Walley Statistical Reasoning with Imprecise Probabilities 1991 Chapman and Hall London
13. A. Nilim, and L. El Ghaoui Robust control of Markov decision processes with uncertain transition matrices Oper. Res. 53 5 2005 780 798 (Pubitemid 41525849)
14. R. Shirota, F.G. Cozman, F.W. Trevizan, C.P. de Campos, L.N. de Barros, Multilinear and integer programming for Markov decision processes with imprecise probabilities, in: Proceedings ISIPTA, Prague, Czech Republic, 2007, pp. 395-404.
15. F.G. Cozman Graphical models for imprecise probabilities Internat. J. Approx. Reason. 39 2-3 2005 167 184 (Pubitemid 40467339)
16. C. Boutilier, N. Friedman, M. Goldszmidt, D. Koller, Context-specific independence in Bayesian networks, in: Proceedings UAI, 1996, pp. 115-123.
17. R.I. Bahar, E.A. Frohm, C.M. Gaona, G.D. Hachtel, E. Macii, A. Pardo, and F. Somenzi Algebraic decision diagrams and their applications Proceedings ICCAD 1993 IEEE Computer Society Press Los Alamitos, CA, USA 188 191
18. R.E. Bryant Symbolic Boolean manipulation with ordered binary-decision diagrams ACM Comput. Surv. 24 3 1992 293 318
19. R.E. Bryant Graph-based algorithms for Boolean function manipulation IEEE Trans. Comput. 35 8 1986 677 691
20. N.L. Zhang, D. Poole, A simple approach to Bayesian network computations, in: Proceedings of the Tenth Canadian Conference on Artificial Intelligence, 1994, pp. 171-178.
21. C.F. Daganzo The cell transmission model: a dynamic representation of highway traffic consistent with the hydrodynamic theory Transport. Res. B 28 4 1994 269 287
22. R. Givan, S. Leach, and T. Dean Bounded-parameter Markov decision processes Artificial Intelligence 122 2000 71 109 (39)
23. O. Buffet, D. Aberdeen, Robust planning with LRTDP, in: Proceedings IJCAI, 2005, pp. 1214-1219.
24. S. Cui, J. Sun, M. Yin, S. Lu, Solving uncertain Markov decision problems: an interval-based method, in: Proceedings ICNC (2), 2006, pp. 948-957. (Pubitemid 44618109)
25. M. Yin, J. Wang, W. Gu, Solving planning under uncertainty: quantitative and qualitative approach, in: Proceedings IFSA (2), 2007, pp. 612-620.
26. F.W. Trevizan, F.G. Cozman, L.N. de Barros, Planning under risk and knightian uncertainty, in: Proceedings IJCAI, 2007, pp. 2023-2028.
27. J.A. Bagnell, A.Y. Ng, J.G. Schneider, Solving uncertain Markov decision processes, Tech. rep., Carnegie Mellon University, 2001.
28. M.L. Littman Markov games as a framework for multi-agent reinforcement learning Proceedings ICML 1994 Morgan Kaufmann 157 163
29. L.S. Shapley Stochastic games Proc. Natl. Acad. Sci. USA 39 1953 327 332
30. A. Cano, and S. Moral Using probability trees to compute marginals with imprecise probabilities Internat. J. Approx. Reason. 29 1 2002 1 46 (Pubitemid 33060596)
31. S. Sanner, D. McAllester, Affine algebraic decision diagrams (AADDs) and their application to structured probabilistic inference, in: Proceedings IJCAI, 2005, pp. 1384-1390.
32. M.O. Duff, Optimal learning: Computational procedures for Bayes-adaptive Markov decision processes, Ph.D. thesis, University of Massachusetts, Amherst, January 2002.