Constructing skill trees for reinforcement learning agents from demonstration trajectories

Advances in Neural Information Processing Systems 23: 24th Annual Conference on Neural Information Processing Systems 2010, NIPS 2010

Volumn , Issue , 2010, Pages

  Konidaris, George ^a   Kuindersmay, Scott ^a   Barto, Andrew ^a   Grupen, Roderic ^a  

^a Biologically Inspired Neural and Dynamical Systems Laboratory   (United States)

Author keywords

[No Author keywords available]

Indexed keywords

ABSTRACTING; CHAINS; DEMONSTRATIONS; FORESTRY; MANIPULATORS; REINFORCEMENT LEARNING; TREES (MATHEMATICS);

CHANGE POINT DETECTION; CONTINUOUS DOMAIN; CONTINUOUS REINFORCEMENT; DETECTION METHODS; MOBILE MANIPULATOR; REINFORCEMENT LEARNING AGENT; REINFORCEMENT LEARNINGS;

TRAJECTORIES;

FORESTRY; MATHEMATICS; TREES;

EID: 85162033542     PISSN: None     EISSN: None     Source Type: Conference Proceeding    
DOI: None     Document Type: Conference Paper

References (26)

1. A.G. Barto and S. Mahadevan. Recent advances in hierarchical reinforcement learning. Discrete Event Dynamic Systems, 13:41-77, 2003. Special Issue on Reinforcement Learning.
2. R.S. Sutton and A.G. Barto. Reinforcement Learning: An Introduction. MIT Press, Cambridge, MA, 1998.
3. G.D. Konidaris and A.G. Barto. Skill discovery in continuous reinforcement learning domains using skill chaining. In Advances in Neural Information Processing Systems 22, pages 1015-1023, 2009.
4. G.D. Konidaris and A.G. Barto. Efficient skill learning using abstraction selection. In Proceedings of the Twenty First International Joint Conference on Artificial Intelligence, July 2009.
5. B. Argall, S. Chernova, M. Veloso, and B. Browning. A survey of robot learning from demonstration. Robotics and Autonomous Systems, 57:469-483, 2009.
6. M. Huber and R.A. Grupen. A feedback control structure for on-line learning tasks. Robotics and Autonomous Systems, 22(3-4):303-315, 1997.
7. M. Rosenstein and A.G. Barto. Supervised actor-critic reinforcement learning. In J. Si, A.G. Barto, A. Powell, and D. Wunsch, editors, Learning and Approximate Dynamic Programming: Scaling up the Real World, pages 359-380. John Wiley & Sons, Inc., New York, 2004.
8. P. Fearnhead and Z. Liu. On-line inference for multiple changepoint problems. Journal of the Royal Statistical Society B, 69:589-605, 2007.
9. R.S. Sutton, D. Precup, and S.P. Singh. Between MDPs and semi-MDPs: A framework for temporal abstraction in reinforcement learning. Artificial Intelligence, 112(1-2):181-211, 1999.
10. T. Lozano-Perez, M.T. Mason, and R.H. Taylor. Automatic synthesis of fine-motion strategies for robots. The International Journal of Robotics Research, 3(1):3-24, 1984.
11. R.R. Burridge, A.A. Rizzi, and D.E. Koditschek. Sequential composition of dynamically dextrous robot behaviors. International Journal of Robotics Research, 18(6):534-555, 1999.
12. S. Boccaletti, A. Farini, E.J. Kostelich, and F.T. Arecchi. Adaptive targeting of chaos. Physical Review E, 55(5):4845-4848, 1997.
13. G.D. Konidaris and S. Osentoski. Value function approximation in reinforcement learning using the Fourier basis. Technical Report UM-CS-2008-19, Department of Computer Science, University of Massachusetts Amherst, June 2008.
14. N. Mehta, S. Ray, P. Tadepalli, and T. Dietterich. Automatic discovery and transfer of MAXQ hierarchies. In Proceedings of the Twenty Fifth International Conference on Machine Learning, pages 648-655, 2008.
15. J. Mugan and B. Kuipers. Autonomously learning an action hierarchy using a learned qualitative state representation. In Proceedings of the 21st International Joint Conference on Artificial Intelligence, 2009.
16. G. Neumann, W. Maass, and J. Peters. Learning complex motions by sequencing simpler motion templates. In Proceedings of the 26th International Conference on Machine Learning, 2009.
17. R. Tedrake. LQR-Trees: Feedback motion planning on sparse randomized trees. In Proceedings of Robotics: Science and Systems, pages 18-24, 2009.
18. X. Xuan and K. Murphy. Modeling changing dependency structure in multivariate time series. In Proceedings of the Twenty-Fourth International Conference on Machine Learning, 2007.
19. E.B. Fox, E.B. Sudderth, M.I. Jordan, and A.S. Willsky. Nonparametric Bayesian learning of switching linear dynamical systems. In Advances in Neural Information Processing Systems 21, 2008.
20. O.C. Jenkins and M. Matarić. Performance-derived behavior vocabularies: data-driven acquisition of skills from motion. International Journal of Humanoid Robotics, 1(2):237-288, 2004.
21. D.H. Grollman and O.C. Jenkins. Incremental learning of subtasks from unsegmented demonstration. In International Conference on Intelligent Robots and Systems, 2010.
22. J. Butterfield, S. Osentoski, G. Jay, and O.C. Jenkins. Learning from demonstration using a multi-valued function regressor for time-series data. In Proceedings of the Tenth IEEE-RAS International Conference on Humanoid Robots, 2010.
23. J. Peters and S. Schaal. Natural actor-critic. Neurocomputing, 71(7-9):1180-1190, 2008.
24. P. Abbeel and A.Y. Ng. Apprenticeship learning via inverse reinforcement learning. In Proceedings of the 21st International Conference on Machine Learning, 2004.
25. S. Chernova and M. Veloso. Confidence-based policy learning from demonstration using Gaussian mixture models. In Proceedings of the 6th International Joint Conference on Autonomous Agents and Multiagent Systems, 2007.
26. G.D. Konidaris and A.G. Barto. Building portable options: Skill transfer in reinforcement learning. In Proceedings of the Twentieth International Joint Conference on Artificial Intelligence, 2007.