Symmetric approximate linear programming for factored MDPs with application to constrained problems

Annals of Mathematics and Artificial Intelligence

Volumn 47, Issue 3-4, 2006, Pages 273-293

  Dolgov, Dmitri A ^a   Durfee, Edmund H ^b  

^a TOYOTA TECHNICAL CENTER   (United States)

^b UNIVERSITY OF MICHIGAN   (United States)

Author keywords

Approximate linear programming; Constrained Markov problems; Dual LP; Markov decision processes; Primal LP formulation

Indexed keywords

EID: 33847310802     PISSN: 10122443     EISSN: None     Source Type: Journal    
DOI: 10.1007/s10472-006-9038-x     Document Type: Conference Paper

References (25)

1. Altman, E.: Constrained Markov decision processes with total cost criteria: occupation measures and primal LP. Methods Models Oper. Res. 43(1), 45-72 (1996)
2. Altman, E.: Constrained Markov decision processes with total cost criteria: Lagrange approach and dual LP. Methods Models Oper. Res. 48, 387-417 (1998)
3. Altman, E., Shwartz, A.: Adaptive control of constrained Markov chains: criteria and policies. Ann. Oper. Res., special issue on Markov Decision Processes 28, 101-134 (1991)
4. Altman, E.: Constrained Markov Decision Processes. Chapman & Hall, London, UK (1999)
5. Bellman, R.: Adaptive Control Processes: A Guided Tour. Princeton University Press, Princeton, NJ (1961)
6. Bertele, U., Brioschi, F.: Nonserial Dynamic Programming. Academic, New York (1972)
7. Bertsekas, D.P., Tsitsiklis, J. N.: Neuro-dynamic Programming. Athena Scientific, Belmont, MA (1996)
8. Bertsimas, D., Tsitsiklis, J.N.: Introduction to Linear Optimization. Athena Scientific. Belmont, MA (1997)
9. Boutilier, C., Dearden, R., Goldszmidt, M.: Exploiting structure in policy construction. In: Proceedings of the Fourteenth International Joint Conference on Artificial Intelligence (IJCAI-95), pp.1104-1111 (1995)
10. Boutilier, C., Dearden, R., Goldszmidt, M.: Stochastic dynamic programming with factored representations. Artif. Intell. 121(1,2), 49-107 (2000)
11. de Farias, D.P., Van Roy, B.: The linear programming approach to approximate dynamic programming. Oper. Res. 51(6), 850-856 (2003)
12. de Parias, D.P., Van Roy, B.: On constraint sampling in the linear programming approach to approximate dynamic programming. Math. Oper. Res. 29(3), 462-478 (2004)
13. Dean, T., Kanazawa, K.: A model for reasoning about persistence and causation. Comput. Intell. 5(3). 142-150 (1989)
14. Dolgov, D.A., Durfee, E.H.: Graphical models in local, asymmetric multi-agent Markov decision processes. In: Proceedings of the Third International Joint Conference on Autonomous Agents and Multiagent Systems (AAMAS-04), pp. 956-963 (2004a)
15. Dolgov, D.A., Durfee, E.H.: Optimal resource allocation and policy formulation in loosely-coupled Markov decision processes. In: Proceedings of the Fourteenth International Conference on Automated Planning and Scheduling (ICAPS-04), pp. 315-324 (2004b)
16. Guestrin, C., Koller, D., Parr, R., Venkataraman, S.: Efficient solution algorithms for factored MDPs. J. Artif. Intell. Res. 19, 399-468 (2003)
17. Guestrin, C.: Planning Under Uncertainty in Complex Structured Environments. Ph.D. thesis, Computer Science Department, Stanford University (2003)
18. Kallenberg, L.: Linear Programming and Finite Markovian Control Problems. Math. Centrum, Amsterdam, Holland (1983)
19. Koller, D., Parr, R.: Computing factored value functions for policies in structured MDPs. In: Proceedings of the Sixteenth International Joint Conference on Artificial Intelligence IJCAI-99, pp.1332-1339 (1999)
20. Patrascu, R., Poupart, P., Schuurmans, D., Boutilier, C., Guestrin, C.: Greedy linear value-approximation for factored Markov decision processes. In: Eighteenth National Conference on Artificial Intelligence, pp. 285-291. American Association for Artificial Intelligence, Menlo Park, CA (2002)
21. Poupart, P., Boutilier, C., Patrascu, R., Schuurmans, D.: Piecewise linear value function approximation for factored MDPs. In: Eighteenth national conference on Artificial Intelligence, pp. 292-299. American Association for Artificial Intelligence, Menlo Park, CA (2002)
22. Puterman, M. L.: Markov Decision Processes. Wiley, New York (1994)
23. Schuurmans, D., Patrascu, R.: Direct value-approximation for factored MDPs. In: Proceedings of the Fourteenths Neural Information Processing Systems (NIPS) (2001)
24. Schweitzer, P., Seidmann, A.: Generalized polynomial approximations in Markovian decision processes. J. Math. Anal. Appl. 110, 568-582 (1985)
25. Sutton, R.S., Barto, A.G.: Reinforcement Learning: An Introduction. MIT Press, Cambridge, MA (1998)