Identifying useful subgoals in reinforcement learning by local graph partitioning

ICML 2005 - Proceedings of the 22nd International Conference on Machine Learning

Volumn , Issue , 2005, Pages 817-824

  Şimşek, Özgür ^a   Wolfe, Alicia P ^a   Barto, Andrew G ^a  

^a Biologically Inspired Neural and Dynamical Systems Laboratory   (United States)

Author keywords

[No Author keywords available]

Indexed keywords

ALGORITHMS; COMPUTATION THEORY; COSTS; GRAPH THEORY;

LOCAL GRAPH PARTITIONING; STATE TRANSITION GRAPHS; SUBGOALS;

LEARNING SYSTEMS;

EID: 31844447221     PISSN: None     EISSN: None     Source Type: Conference Proceeding    
DOI: None     Document Type: Conference Paper

References (18)

1. Burridge, R., Rizzi, A. A., & Koditschek, D. E. (1999). Sequential composition of dynamically dexterous robot behaviors. The International Journal of Robotics Research, 18, 534-555.
2. Dietterich, T. G. (2000). Hierarchical reinforcement learning with the MAXQ value function decomposition. Journal of Artificial Intelligence Research, 13, 227-303.
3. Digney, B. (1998). Learning hierarchical control structure for multiple tasks and changing environments. From Animals to Animats 5: The Fifth Conference on the Simulation of Adaptive Behaviour, MIT Press.
4. Duda, R. O., Hart, P. E., & Stork, D. G. (2001). Pattern classification. New York: Wiley.
5. Hagen, L., & Kahng, A. B. (1992). New spectral methods for ratio cut partitioning and clustering. IEEE Transactions on Computer-Aided Design of Integrated Circuits and Systems, 11, 1074-1085.
6. Hengst, B. (2002). Discovering hierarchy in reinforcement learning with HEXQ. Proceedings of the Nineteenth International Conference on Machine Learning (pp. 243-250). Morgan Kaufmann.
7. Lin, L. (1992). Self-Improving reactive agents based on reinforcement learning, planning and teaching. Machine Learning, 8, 293-321.
8. Mannor, S., Menache, I., Hoze, A., & Klein, U. (2004). Dynamic abstraction in reinforcement learning via clustering. Proceedings of the Twenty-First International Conference on Machine Learning (pp. 560-567). ACM Press.
9. McGovern, A., & Barto, A. G. (2001). Automatic discovery of subgoals in reinforcement learning using diverse density, Proceedings of the Eighteenth International Conference on Machine Learning (pp. 361-368). Morgan Kaufmann.
10. Menache, I., Mannor, S., & Shimkin, N. (2002). Q-Cut -Dynamic discovery of sub-goals in reinforcement learning. Proceedings of the Thirteenth European Conference on Machine Learning (pp. 295-306). Springer.
11. Parr, R. (1998). Hierarchical control and learning for markov decision processes. Doctoral dissertation, Computer Science Division, University of California, Berkeley.
12. Pickett, M., & Barto, A. G. (2002). PolicyBlocks: An algorithm for creating useful macro-actions in reinforcement learning. Proceedings of the Nineteenth International Conference on Machine Learning (pp. 506-513). Morgan Kaufmann.
13. Precup, D. (2000). Temporal abstraction in reinforcement learning. Doctoral dissertation, University of Massachusetts Amherst.
14. Shi, J., & Malik, J. (2000). Normalized cuts and image segmentation. IEEE Transactions on Pattern Analysis and Machine Intelligence, 22, 888-905.
15. Şimşek, Ö., & Barto, A. G. (2004). Using relative novelty to identify useful temporal abstractions in reinforcement learning. Proceedings of the Twenty-First International Conference on Machine Learning (pp. 751-758). ACM Press.
16. Sutton, R. S., Precup, D., & Singh, S. P. (1999). Between MDPs and Semi-MDPs: A framework for temporal abstraction in reinforcement learning. Artificial Intelligence, 112, 181-211.
17. Thrun, S., & Schwartz, A. (1995). Finding structure in reinforcement learning. Advances in Neural Information Processing Systems (pp. 385-392). MIT Press.
18. Wu, Z., & Leahy, R. (1993). An optimal graph theoretic approach to data clustering: theory and its application to image segmentation. IEEE Transactions on Pattern Analysis and Machine Intelligence, 15, 1101-1113.