Automatic abstraction in reinforcement learning using data mining techniques

Robotics and Autonomous Systems

Volumn 57, Issue 11, 2009, Pages 1119-1128

  Kheradmandian, Ghorban ^a   Rahmati, Mohammad ^a  

^a AMIRKABIR UNIVERSITY OF TECHNOLOGY   (Iran)

Author keywords

Abstraction; Clustering; Data mining; Reinforcement learning; Sequence mining

Indexed keywords

ABSTRACTION; ACTION SEQUENCES; CLUSTERING; DATA MINING ALGORITHM; DATA MINING TECHNIQUES; DATA SETS; LARGE DATASETS; LEARNING AGENTS; Q-LEARNING ALGORITHMS; SEQUENCE MINING; STATE CLUSTERING; STATE SPACE; STATE TRANSITIONS; TEMPORAL ABSTRACTION;

ABSTRACTING; DATA MINING; EDUCATION; LEARNING ALGORITHMS; MINING; REINFORCEMENT; REINFORCEMENT LEARNING; SET THEORY;

CLUSTERING ALGORITHMS;

EID: 70350053106     PISSN: 09218890     EISSN: None     Source Type: Journal    
DOI: 10.1016/j.robot.2009.07.002     Document Type: Article

References (23)

1. Sutton R., and Barto A. Introduction to Reinforcement Learning (1998), MIT Press, Cambridge
2. Kaelbling L.P., Littman M.L., and Moore A.W. Reinforcement learning: A survey. Journal of Artificial Intelligence Research 4 (1996) 237-285
3. Barto A., and Mahadevan S. Recent advances in hierarchical reinforcement learning. Discrete Event Systems Journal 13 (2003) 41-77
4. Dietterich T. Hierarchical reinforcement learning with the MAXQ value function decomposition. Journal of Artificial Intelligence Research 3 (2000) 227-303
5. Sutton R., Precup D., and Singh S. Between MDPs and Semi-MDPs: A framework for temporal abstraction in reinforcement learning. Artificial Intelligence 112 (1999) 181-211
6. Amy E. McGovern, Autonomous discovery of temporal abstraction from interaction with an environment, Ph.D. Thesis, University of Massachusetts, Amherst, MA 2002
7. Bruce L. Digney, Learning hierarchical control structures for multiple tasks and changing environments, in: Proceedings of the Fifth International Conference on Simulation of Adaptive Behaviour, 1998
8. Menache I., Mannor S., and Shimkin N. Q-Cut: Dynamic Discovery of Sub-Goals in Reinforcement Learning. Lecture Notes in Computer Science vol. 24 (2002), Springer
9. Simsek O., and Barto A. Using relative novelty to identify useful temporal abstractions in reinforcement learning. Proceedings of the International Conferenceon Machine Learning vol. 21 (2004) 751-758
10. Mannor S., Menache I., Hoze A., and Klein U. Dynamic abstraction in reinforcement learning via clustering. Proceedings of the International Conference on Machine Learning vol. 21 (2004) 560-567
11. O. Simsek, A. Wolfe, A. Barto, Identifying useful sub-goals in reinforcement learning by local graph partitioning, in: Proceedings of the International Conference on Machine Learning, 2005
12. Hengst Bernhard, Discovering hierarchy in reinforcement learning, Ph.D. Thesis, University of New South Wales, Australia, 2004
13. Jonsson Anders, A causal approach to hierarchical decomposition in reinforcement learning, Ph.D. Thesis, University of Massachusetts Amherst, 2006
14. Parr R., and Russell S. Reinforcement learning with hierarchies of machines. Advances in Neural Information Processing Systems 10 (1998) 1043-1049
15. Thrun S., and Schwartz A. Finding structure in reinforcement learning. Advances in Neural Information Processing Systems (1995), MIT Press 385-392
16. M. Pickett, A.G. Barto, PolicyBlocks: An algorithm for creating useful macro-actions in reinforcement learning, in: Proceedings of the Nineteenth International Conference on Machine Learning 2002, pp. 506-513
17. Menache I., Mannor S., and Shimkin N. Q-Cut-Dynamic discovery of sub-goals in reinforcement learning. Proceedings of the Thirteenth European Conference on Machine Learning (2002), Springer 295-306
18. Amy McGovern, Richard S. Sutton, Macro-actions in reinforcement learning: An empirical analysis, Technical Report, University of Massachusetts, Amherst, 1998
19. Dayan P., and Watkins C. Q-learning. Machine Learning 8 (1992) 279-292
20. Han J., and Kamber M. Data Mining: Concepts and Techniques (2000), Morgan Kaufmann Publishers
21. Bart Goethals, Survey on frequent pattern mining, HIIT Basic Research unit, University of Helsinki, Finland
22. Park K.H., Kim Y.J., and Kim J.H. Modular Q-learning based multi-agent cooperation for robot soccer. Robotics and Autonomous Systems 35 (2001) 109-122
23. Kamal M.A.S., and Murata J. Reinforcement learning for problems with symmetrical restricted states. Robotics and Autonomous Systems 56 9 (30 September 2008) 717-727