A causal approach to hierarchical decomposition of factored MDPs

ICML 2005 - Proceedings of the 22nd International Conference on Machine Learning

Volumn , Issue , 2005, Pages 401-408

  Jonsson, Anders ^a   Barto, Andrew ^a  

^a The Manning College of Information and Computer Sciences   (United States)

Author keywords

[No Author keywords available]

Indexed keywords

DECISION THEORY; HIERARCHICAL SYSTEMS; LEARNING SYSTEMS;

CAUSAL RELATIONSHIPS; MARKOV DECISION PROCESSES; VARIABLE INFLUENCE STRUCTURE ANALYSIS;

LEARNING ALGORITHMS;

EID: 31844455449     PISSN: None     EISSN: None     Source Type: Conference Proceeding    
DOI: 10.1145/1102351.1102402     Document Type: Conference Paper

References (19)

1. Boutilier, C., Dearden, R., & Goldszmidt, M. (1995) Exploiting structure in policy construction. IJCAI, 14: 1104-1113.
2. Dean, T., & Kanazawa, K. (1989) A model for reasoning about persistence and causation. Computational Intelligence, 5(3): 142-150.
3. Dietterich, T. (2000). Hierarchical reinforcement learning with the MAXQ value function decomposition. Journal of Artificial Intelligence Research, 13: 227-303.
4. Digney, B. (1996) Emergent hierarchical control structures: Learning reactive/hierarchical relationships in reinforcement environments. From animals to animate, 4: 363-372.
5. Feng, Z., Hansen, E., & Zilberstein, Z. (2003) Symbolic generalization for on-line planning. UAI, 19: 209-216.
6. Ghavamzadeh, M., & Mahadevan, S. (2001) Continuous-time hierarchical reinforcement learning. ICML, 18: 186-193.
7. Guestrin, C., Koller, D., & Parr, R. (2001) Max-norm projections for factored MDPs. IJCAI, 17: 673-680.
8. Helmert, M. (2004) A planning heuristic based on causal graph analysis. ICAPS, 16: 161-170.
9. Hengst, B. (2002) Discovering hierarchy in reinforcement learning with HEXQ. ICML, 19: 243-250.
10. Hoey, J., St-Aubin, R., Hu, A., & Boutilier, C. (1999) SPUDD: Stochastic Planning using Decision Diagrams. UAI, 15: 279-288.
11. Kearns, M., & Koller, D: (1999) Efficient reinforcement learning in factored MDPs. IJCAI, 16: 740-747.
12. Mannor, S., Menache, I., Hoze, A., & Klein, U. (2004) Dynamic abstraction in reinforcement learning via clustering. ICML, 21: 560-567.
13. McGovern, A., & Barto, A. (2001) Automatic discovery of subgoals in reinforcement learning using diverse density. ICML, 18: 361-368.
14. Menache, I., Mannor, S., & Shimkin, N. (2002) Q-Cut - Dynamic discovery of sub-goals in reinforcement learning. ECML, 14: 295-306.
15. Parr, R., & Russell, S. (1998) Reinforcement learning with hierarchies of machines. NIPS, 10: 1043-1049.
16. Pickett, M., & Barto, A. (2002) PolicyBlocks: An algorithm for creating useful macro-actions in reinforcement learning. ICML, 19: 506-513.
17. Şimşek, Ö., & Barto, A. (2004) Using relative novelty to identify useful temporal abstractions in reinforcement learning. ICML, 21: 751-758.
18. Button, R., Precup, D., & Singh, S. (1999) Between MDPs and semi-MDPs: A framework for temporal abstraction in reinforcement learning. Artificial Intelligence, 112: 181-211.
19. Thrun, S., & Schwartz, A. (1995) Finding structure in reinforcement learning. NIPS, 8: 385-392.