Solving factored MDPs with exponential-family transition models

ICAPS 2006 - Proceedings, Sixteenth International Conference on Automated Planning and Scheduling

Volumn 2006, Issue , 2006, Pages 114-120

  Kveton, Branislav ^a   Hauskrecht, Milos ^b  

^a UNIVERSITY OF PITTSBURGH   (United States)

^b University of Pittsburgh   (United States)

Author keywords

[No Author keywords available]

Indexed keywords

HYBRID APPROXIMATE LINEAR PROGRAMMING (HALF); MARKOV DECISION PROCESSES (MDP); OPTIMAL VALUE FUNCTIONS; TRANSITION MODELS;

APPROXIMATION THEORY; DISCRETE TIME CONTROL SYSTEMS; HYBRID COMPUTERS; LINEAR PROGRAMMING; MARKOV PROCESSES; MATHEMATICAL MODELS; OPTIMIZATION;

DECISION MAKING;

EID: 33746054938     PISSN: None     EISSN: None     Source Type: Conference Proceeding    
DOI: None     Document Type: Conference Paper

References (24)

1. Andricu, C.; de Frcilas, N.; Doucel, A.; and Jordan, M. 2003. An introduction to MCMC for machine learning. Machine Learning 50:5-43.
2. Bellman, R.; Kalaba, R.; and Kolkin, B. 1963. Polynomial approximation - a new computational technique in dynamic programming: Allocation processes. Mathematics of Computation 17(82): 155-161.
3. Bellman, R. 1957. Dynamic Programming. Princeton, NJ: Princeton University Press.
4. Bertsekas, D., and Tsitsiklis, J. 1996. Neuro-Dynamic Programming. Belmont, MA: Athena Scientific.
5. Boutilier, C.; Dearden, R.; and Goldszmidt, M. 1995. Exploiting structure in policy construction. In Proceedings of the 14th International Joint Conference on Artificial Intelligence, 1104-1 111.
6. Bresina, J.; Dearden, R.; Meuleau, N.; Ramakrishnan, S.; Smith, D.; and Washington, R. 2002. Planning under continuous time and resource uncertainty: A challenge for AI. In Proceedings of the 18th Conference on Uncertainty in Artificial Intelligence, 77-84.
7. Chow, C.-S., and Tsitsiklis, J. 1991. An optimal one-way multigrid algorithm for discrete-time stochastic control. IEEE Transactions on Automatic Control 36(8):898-914.
8. de Parias, D. P., and Van Roy, B. 2003. The linear programming approach to approximate dynamic programming. Operations Research 51(6):850-856.
9. de Parias, D. P., and Van Roy, B. 2004. On constraint sampling for the linear programming approach to approximate dynamic programming. Mathematics of Operations Research 29(3):462-478.
10. Dean, T., and Kanazawa, K. 1989. A model for reasoning about persistence and causation. Computational Intelligence 5:142-150.
11. Feng, Z.; Dearden, R.; Meuleau, N.; and Washington, R. 2004. Dynamic programming for structured continuous Markov decision problems. In Proceedings of the 20th Conference on Uncertainty in A rtificial Intelligence, 154-161.
12. Guestrin, C.; Hauskrecht, M.; and Kveton, B. 2004. Solving factored MDPs with continuous and discrete variables. In Proceedings of the 20th Conference on Uncertainty in Artificial Intelligence, 235-242.
13. Hauskrecht, M., and Kveton, B. 2004. Linear program approximations for factored continuous-state Markov decision processes. In Advances in Neural Information Processing Systems 16, 895-902.
14. Jeffreys, H., and Jeffreys, B. 1988. Methods of Mathematical Physics. Cambridge, United Kingdom: Cambridge University Press.
15. Koller, D., and Parr, R. 1999. Computing factored value functions for policies in structured MDPs. In Proceedings of the 16th International Joint Conference on Artificial Intelligence, 1332-1339.
16. Kveton, B., and Hauskrecht, M. 2005. An MCMC approach to solving hybrid factored MDPs. In Proceedings of the 19th International Joint Conference on Artificial Intelligence, 1346-1351.
17. Mahadevan, S., and Maggioni, M. 2006. Value function approximation with diffusion wavelets and Laplacian eigenfunctions. In Advances in Neural Information Processing Systems 18, 843-850.
18. Mahadevan, S. 2005. Samuel meets Amarel: Automating value function approximation using global state space analysis. In Proceedings of the 20th National Conference on Artificial Intelligence, 1000-1005.
19. Munos, R., and Moore, A. 2002. Variable resolution discretization in optimal control. Machine Learning 49:291-323.
20. Patrascu, R.; Poupart, P.; Schuurmans, D.; Boutilier, C.; and Guestrin, C. 2002. Greedy linear value-approximation for factored Markov decision processes. In Proceedings of the 18th National Conference on Artificial Intelligence, 285-291.
21. Puterman, M. 1994. Markov Decision Processes: Discrete Stochastic Dynamic Programming. New York, NY: John Wiley & Sons.
22. Rust, J. 1997. Using randomization to break the curse of dimensionality. Econometrica 65(3):487-516.
23. Schweitzer, P., and Seidmann, A. 1985. Generalized polynomial approximations in Markovian decision processes. Journal of Mathematical Analysis and Applications 110:568-582.
24. Van Roy, B. 1998. Planning Under Uncertainty in Complex Structured Environments. Ph.D. Dissertation, Massachusetts Institute of Technology.