Symbolic dynamic programming for continuous state and action MDPs

Proceedings of the National Conference on Artificial Intelligence

Volumn 3, Issue , 2012, Pages 1839-1845

  Zamani, Zahra ^a   Sanner, Scott ^b   Fang, Cheng ^c  

^a AUSTRALIAN NATIONAL UNIVERSITY   (Australia)

^b CSIRO   (Australia)

^c Massachusetts Institute of Technology   (United States)

Author keywords

[No Author keywords available]

Indexed keywords

CLOSED FORM; COMPACT DATA STRUCTURE; CONTINUOUS STATE; DECISION DIAGRAM; DECISION-THEORETIC; EXACT SOLUTION; PIECE-WISE; PIECEWISE LINEAR; PLANNING PROBLEM; TWO DOMAINS; UNKNOWN STATE; VALUE FUNCTIONS;

ARTIFICIAL INTELLIGENCE; CONSTRAINED OPTIMIZATION; DATA STRUCTURES; OPERATIONS RESEARCH; PIECEWISE LINEAR TECHNIQUES;

DYNAMIC PROGRAMMING;

EID: 84868280144     PISSN: None     EISSN: None     Source Type: Conference Proceeding    
DOI: None     Document Type: Conference Paper

References (23)

1. Arrow, K.; Karlin, S.; and Scarf, H. 1958. Studies in the mathematical theory of inventory and production. Stanford University Press.
2. Athans, M. 1971. The role and use of the stochastic linear-quadratic- gaussian problem in control system design. IEEE Transaction on Automatic Control 16(6):529-552.
3. Bahar, R. I.; Frohm, E.; Gaona, C.; Hachtel, G.; Macii, E.; Pardo, A.; and Somenzi, F. 1993. Algebraic Decision Diagrams and their applications. In IEEE /ACM International Conference on CAD.
4. Bellman, R. E. 1957. Dynamic Programming. Princeton, NJ: Princeton University Press.
5. Bitran, G. R., and Yanasse, H. 1982. Computational complexity of the capacitated lot size problem. Management Science 28(10):1271-81.
6. Boutilier, C.; Dean, T.; and Hanks, S. 1999. Decision-theoretic planning: Structural assumptions and computational leverage. JAIR 11:1-94.
7. Boutilier, C.; Reiter, R.; and Price, B. 2001. Symbolic dynamic programming for first-order MDPs. In IJCAI-01, 690-697.
8. Boyan, J., and Littman, M. 2001. Exact solutions to time-dependent MDPs. In Advances in Neural Information Processing Systems NIPS-00, 1026-1032.
9. Bresina, J. L.; Dearden, R.; Meuleau, N.; Ramkrishnan, S.; Smith, D. E.; and Washington, R. 2002. Planning under continuous time and resource uncertainty:a challenge for ai. In Uncertainty in Artificial Intelligence (UAI-02).
10. Dean, T., and Kanazawa, K. 1989. A model for reasoning about persistence and causation. Computational Intelligence 5(3):142-150.
11. Feng, Z.; Dearden, R.; Meuleau, N.; and Washington, R. 2004. Dynamic programming for structured continuous markov decision problems. In Uncertainty in Artificial Intelligence (UA1-04), 154-161.
12. Hoey, J.; St-Aubin, R.; Hu, A.; and Boutilier, C. 1999. SPUDD: Stochastic planning using decision diagrams. In UAI-99, 279-288.
13. Kveton, B.; Hauskrecht, M.; and Guestrin, C. 2006. Solving factored mdps with hybrid state and action variables. Journal Artificial Intelligence Research (JAIR) 27:153-201.
14. Lamond, B., and Boukhtouta, A. 2002. Water reservoir applications of markov decision processes. In International Series in Operations Research and Management Science, Springer.
15. Li, L., and Littman, M. L. 2005. Lazy approximation for solving continuous finite-horizon mdps. In National Conference on Artificial Intelligence AAAI-05, 1175-1180.
16. Mahootchi, M. 2009. Storage System Management Using Reinforcement Learning Techniques and Nonlinear Models. Ph.D. Dissertation, University of Waterloo,Canada.
17. Marecki, J.; Koenig, S.; and Tambe, M. 2007. A fast analytical algorithm for solving markov decision processes with real-valued resources. In International Conference on Uncertainty in Artificial Intelligence IJCAI, 2536-2541.
18. Meuleau, N.; Benazera, E.; Brafman, R. I.; Hansen, E. A.; and Mausam. 2009. A heuristic search approach to planning with continuous resources in stochastic domains. Journal Artificial Intelligence Research (JAIR) 34:27-59.
19. Remi Munos, A. M. 2002. Variable resolution discretization in optimal control. Machine Learning 49, 2-3:291-323.
20. Sanner, S.; Delgado, K. V.; and de Barros, L. N. 2011. Symbolic dynamic programming for discrete and continuous state mdps. In Proceedings of the 27th Conference on Uncertainty in AI (UAI-2011).
21. St-Aubin, R.; Hoey, J.; and Boutilier, C. 2000. APRICODD: Approximate policy construction using decision diagrams. In NIPS-2000, 1089-1095.
22. Wu, T.; Shi, L.; and Duffie, N. A. 2010. An hnp-mp approach for the capacitated multi-item lot sizing problem with setup times. IEEE T. Automation Science and Engineering 7(3):500-511.
23. Yeh, W. G. 1985. Reservoir management and operations models: A state-of-the-art review. Water Resources research 21,12:17971818.