Using cases as heuristics in reinforcement learning: A transfer learning application

IJCAI International Joint Conference on Artificial Intelligence

Volumn , Issue , 2011, Pages 1211-1217

  Celiberto Jr , Luiz A ^a   Matsuura, Jackson P ^a   De Mantaras, Ramon Lopez ^b   Bianchi, Reinaldo A C ^c  

^a INSTITUTO TECNOLÓGICO DE AERONÁUTICA   (Brazil)

^b UNIVERSITAT AUTÒNOMA DE BARCELONA   (Spain)

^c CENTRO UNIVERSITÁRIO DA FEI   (Brazil)

Author keywords

[No Author keywords available]

Indexed keywords

AI TECHNIQUES; EMPIRICAL EVALUATIONS; HUMANOID ROBOT; LEARNING PERFORMANCE; OPTIMAL POLICIES; THIRD PHASE; TRANSFER LEARNING; TWO DOMAINS;

ANTHROPOMORPHIC ROBOTS; COMPUTER AIDED INSTRUCTION; LEARNING ALGORITHMS; NEURAL NETWORKS; THREE DIMENSIONAL;

REINFORCEMENT LEARNING;

EID: 84871606206     PISSN: 10450823     EISSN: None     Source Type: Conference Proceeding    
DOI: 10.5591/978-1-57735-516-8/IJCAI11-206     Document Type: Conference Paper

References (24)

1. David W. Aha, Matthew Molineaux, and Gita Sukthankar. Case-based reasoning in transfer learning. In Lorraine McGinty and David C. Wilson, editors, 8th International Conference on Case-Based Reasoning, volume 5650 of Lecture Notes in Computer Science, pages 29-44. Springer, 2009.
2. Bryan Auslander, Stephen Lee-Urban, Chad Hogg, and Héctor Muñoz-Avila. Recognizing the enemy: Combining reinforcement learning with strategy selection using case-based reasoning. In Klaus-Dieter Althoff, Ralph Bergmann, Mirjam Minor, and Alexandre Hanft, editors, 9th European Conference on Case-Based Reasoning, volume 5239 of Lecture Notes in Computer Science, pages 59-73. Springer, 2008.
3. Bikramjit Banerjee and Peter Stone. General game learning using knowledge transfer. In Manuela M. Veloso, editor, Proceedings of the 20th International Joint Conference on Artificial Intelligence, pages 672-677. AAAI Press, 2007.
4. Reinaldo A. C. Bianchi, Carlos H. C. Ribeiro, and Anna H. R. Costa. Heuristically Accelerated Q-learning: a new approach to speed up reinforcement learning. Lecture Notes in Artificial Intelligence, 3171:245-254, 2004.
5. Reinaldo A. C. Bianchi, Carlos H. C. Ribeiro, and Anna H. R. Costa. Accelerating autonomous learning by using heuristic selection of actions. Journal of Heuristics, 14(2):135-168, 2008.
6. Reinaldo A. C. Bianchi, Raquel Ros, and Ramon López de Mántaras. Improving reinforcement learning by using case based heuristics. In Lorraine McGinty and David C. Wilson, editors, 8th International Conference on Case-Based Reasoning, volume 5650 of Lecture Notes in Computer Science, pages 75-89. Springer, 2009.
7. Joschka Boedecker, Klaus Dorer, Markus Rollmann andYuan Xu, Feng Xue, Marian Buchta, and Hedayat Vatankhah. Spark 3D Simulation System. 2010.
8. Ramon López de Mántaras, David McSherry, Derek Bridge, David Leake, Barry Smyth, Susan Craw, Boi Faltings, Mary Lou Maher, Michael T. Cox, Kenneth Forbus, Mark Keane, Agnar Aamodt, and Ian Watson. Retrieval, reuse, revision and retention in case-based reasoning. Knowl. Eng. Rev., 20(3):215-240, 2005.
9. Chris Drummond. Accelerating reinforcement learning by composing solutions of automatically identified subtasks. Journal of Artificial Intelligence Research, 16:59-104, 2002.
10. Fernando Fernández and Manuela Veloso. Probabilistic policy reuse in a reinforcement learning agent. In Hideyuki Nakashima, Michael P. Wellman, Gerhard Weiss and Peter Stone, editors, Proceedings of the 5th International Joint Conference on Autonomous Agents and Multiagent Systems, pages 720-727. ACM, 2006.
11. Thomas Gabel and Martin A. Riedmiller. CBR for state value function approximation in reinforcement learning. In Héctor Muñoz-Avila and Francesco Ricci, editors, 6th International Conference on Case-Based Reasoning, volume 3620 of Lecture Notes in Computer Science, pages 206-221. Springer, 2005.
12. Paul Juell and Patrick Paulson. Using reinforcement learning for similarity assessment in case-based systems. IEEE Intelligent Systems, 18(4):60-67, 2003.
13. Alessandro Lazaric, Marcello Restelli, and Andrea Bonarini. Transfer of samples in batch reinforcement learning. In William W. Cohen, Andrew McCallum and Sam T. Roweis, editors, 25th International Conference on Machine Learning, pages 544-551. ACM, 2008.
14. Yang Li, Chen Zonghai, and Chen Feng. A case-based reinforcement learning for probe robot path planning. In 4th World Congress on Intelligent Control and Automation, pages 1161-1165. 2002.
15. Raquel Ros, Josep Lluis Arcos, Ramon López de Mántaras, and Manuela Veloso. A case-based approach for coordinated action selection in robot soccer. Artificial Intelligence, 173(9-10):1014-1039, 2009.
16. Manu Sharma, Michael Holmes, Juan Carlos Santamaría, Arya Irani, Charles Lee Isbell Jr., and Ashwin Ram. Transfer learning in real-time strategy games using hybrid CBR/RL. In Manuela M. Veloso, editor, Proceedings of the 20th International Joint Conference on Artificial Intelligence, pages 1041-1046. AAAI Press, 2007.
17. Vishal Soni and Satinder Singh. Using homomorphisms to transfer options across continuous reinforcement learning domains. In Proceedings of the 21st National Conference on Artificial Intelligence, volume 1, pages 494-499. AAAI Press, 2006.
18. R. S. Sutton and A. G. Barto. Reinforcement Learning: An Introduction. MIT Press, Cambridge, MA, 1998.
19. Matthew E. Taylor and Peter Stone. Transfer learning for reinforcement learning domains: A survey. Journal of Machine Learning Research, 10(1):1633-1685, 2009.
20. Matthew E. Taylor, Nicholas K. Jong, and Peter Stone. Transferring instances for model-based reinforcement learning. In Walter Daelemans, Bart Goethals and Katharina Morik, editors, 19th European Conference on Machine Learning, volume 5212 of Lecture Notes in Artificial Intelligence, pages 488-505. Springer, 2008.
21. E. L. Thorndike and R. S. Woodworth. The influence of improvement in one mental function upon the efficiency of other functions. Psychological Review, 8:247-261, 1901.
22. Lisa Torrey, Trevor Walker, Jude W. Shavlik, and Richard Maclin. Using advice to transfer knowledge acquired in one reinforcement learning task to another. In João Gama, Rui Camacho, Alípio Jorge, and Luís Torgo, editors, 16th European Conference on Machine Learning, volume 3720 of Lecture Notes in Computer Science, pages 412-424. Springer, 2005.
23. Andreas von Hessling and Ashok K. Goel. Abstracting reusable cases from reinforcement learning. In Stefanie Brüninghaus, editor, 6th International Conference on Case-Based Reasoning, Workshop Proceedings, pages 227-236, 2005.
24. Christopher J. C. H. Watkins. Learning from Delayed Rewards. PhD thesis, University of Cambridge, 1989.