Nearly deterministic abstractions of Markov decision processes

Proceedings of the National Conference on Artificial Intelligence

Volumn , Issue , 2002, Pages 260-266

  Lane, Terran ^a   Kaelbling, Leslie Pack ^a  

^a MASSACHUSETTS INSTITUTE OF TECHNOLOGY   (United States)

Author keywords

[No Author keywords available]

Indexed keywords

ALGORITHMS; COMPUTER SIMULATION; MARKOV PROCESSES; NAVIGATION; OPTIMIZATION; PROBLEM SOLVING;

MARKOV DECISION PROCESSES (MDP);

MOBILE ROBOTS;

EID: 0036931070     PISSN: None     EISSN: None     Source Type: Conference Proceeding    
DOI: None     Document Type: Conference Paper

References (23)

1. Arora, S., and Karakostas, G. 1999. Approximation schemes for minimum latency problems. In ACM Symposium on Theory of Computing, 688-693.
2. Baum, J., and Nicholson, A. E. 1998. Dynamic non-uniform abstractions for approximate planning in large structured stochastic domains. In Proceedings of the 5th Pacific Rim International Conference on Artificial Intelligence, 587-598.
3. Boutilier, C.; Dearden, R.; and Goldszmidt, M. 2000. Stochastic dynamic programming with factored representations. Artificial Intelligence 121(1-2):49-107.
4. Dean, T., and Givan, R. 1997. Model minimization in Markov decision processes. In Proceedings of the Fourteenth National Conference on Artificial Intelligence (AAAI-97), 106-111. Providence, RI: AAAI Press/MIT Press.
5. Dean, T.; Kaelbling, L. P.; Kirman, J.; and Nicholson, A. 1995. Planning under time constraints in stochastic domains. Artificial Intelligence 76.
6. Goemans, M., and Kleinberg, J. 1996. An improved approximation ratio for the minimum latency problem. In SODA: Proceedings of the Seventh ACM-SIAM Symposium on Discrete Algorithms.
7. Guestrin, C., and Ormoneit, D. 2001. Robust combination of local controllers. In Breese, J., and Koller, D., eds., Proceedings of the Seventeenth Conference on Uncertainty in Artificial Intelligence (UAI-01), 178-185. Seattle, WA: Morgan Kaufmann.
8. Hauskrecht, M.; Meuleau, N.; Boutilier, C.; Kaelbling, L. P.; and Dean, T. 1998. Hierarchical solution of Markov decision processes using macro-actions. In Cooper, G. F., and Moral, S., eds., Proceedings of the Fourteenth Conference on Uncertainty in Artificial Intelligence (UAI-98). Morgan Kaufmann.
9. Johnson, D. S., and McGeoch, L. A. 2001. The Traveling Salesman Problem and its Variations. Kluwer Academic Publishers, chapter Experimental Analysis of Heuristics for the STSP. To appear.
10. Kaelbling, L. P. 1993. Hierarchical learning in stochastic domains: Preliminary results. In Proceedings of the Tenth International Conference on Machine Learning, 167-173.
11. Kearns, M., and Koller, D. 1999. Efficient reinforcement learning in factored MDPs. In Dean, T., ed., Proceedings of the Sixteenth International Joint Conference on Artificial Intelligence (IJCAI-99), 740-747. Stockholm, Sweden: Morgan Kaufmann.
12. Kemeny, J. G., and Snell, J. L. 1976. Finite Markov Chains. Undergraduate Texts in Mathematics. New York: Springer-Verlag.
13. Koller, D., and Parr, R. 2000. Policy iteration for factored MDPs. In Uncertainty in Artificial Intelligence: Proceedings of the Sixteenth Conference (UAI 2000). Morgan Kaufmann.
14. Littman, M. 1997. Probabilisitic propositional planning: Representations and complexity. In Proceedings of the Fourteenth National Conference on Artificial Intelligence (AAAI-97), 748-754. Providence, RI: AAAI Press/MIT Press.
15. Lusena, C.; Mundhenk, M.; and Goldsmith, J. 2001. Non-approximability results for partially observable Markov decision processes. Journal of Artificial Intelligence Research 14:83-103.
16. McGovern, A.; Sutton, R. S.; and Fagg, A. H. 1997. Roles of macro-actions in accelerating reinforcement learning. In Proceedings of the 1997 Grace Hopper Celebration of Women in Computing, 13-18.
17. Moore, A. W.; Baird, L. C.; and Kaelbling, L. 1999. Multi-value-functions: Efficient automatic action hierarchies for multiple goal MDPs. In Dean, T., ed., Proceedings of the Sixteenth International Joint Conference on Artificial Intelligence (IJCAI-99). Stockholm, Sweden: Morgan Kaufmann.
18. Mundhenk, M.; Goldsmith, J.; Lusena, C.; and Allender, E. 2000. Complexity of finite-horizon markov decision process problems. Journal of the ACM 47(4):681-720.
19. Parr, R. 1998a. Flexible decomposition algorithms for weakly coupled Markov decision problems. In Cooper, G. F., and Moral, S., eds., Proceedings of the Fourteenth Conference on Uncertainty in Artificial Intelligence (UAI-98). Morgan Kaufmann.
20. Parr, R. E. 1998b. Hierarchical Control and Learning for Markov Decision Processes. Ph.D. Dissertation, University of California at Berkeley.
21. Precup, D. 2000. Temporal Abstraction in Reinforcement Learning. Ph.D. Dissertation, University of Massachusetts, Amherst, Department of Computer Science.
22. Puterman, M. L. 1994. Markov Decision Processes: Discrete Stochastic Dynamic Programming. New York: John Wiley & Sons.
23. Sutton, R. S.; Precup, D.; and Singh, S. 1999. Between MDPs and semi-MDPs: A framework for temporal abstraction in reinforcement learning. Artificial Intelligence 112:181-211.