The option-critic architecture

31st AAAI Conference on Artificial Intelligence, AAAI 2017

Volumn , Issue , 2017, Pages 1726-1734

  Bacon, Pierre Luc ^a   Harb, Jean ^a   Precup, Doina ^a  

^a MCGILL UNIVERSITY   (Canada)

Author keywords

[No Author keywords available]

Indexed keywords

ABSTRACTING; REINFORCEMENT LEARNING;

NEW OPTIONS; POLICY GRADIENT; SCALING-UP; SUBGOALS; TEMPORAL ABSTRACTION; TERMINATION CONDITION;

ARTIFICIAL INTELLIGENCE;

EID: 85030457046     PISSN: None     EISSN: None     Source Type: Conference Proceeding    
DOI: None     Document Type: Conference Paper

References (30)

1. Baird, L. C. 1993. Advantage updating. Technical Report WL-TR-93-1146, Wright Laboratory.
2. Bellemare, M. G.; Naddaf, Y.; Veness, J.; and Bowling, M. 2013. The arcade learning environment: An evaluation platform for general agents. Journal of Artificial Intelligence Research 47: 253-279.
3. Comanici, G., and Precup, D. 2010. Optimal policy switching algorithms for reinforcement learning. In AAMAS, 709-714.
4. ŞimŞek, O., and Barto, A. G. 2009. Skill characterization based on betweenness. In NIPS 21, 1497-1504.
5. Daniel, C.; van Hoof, H.; Peters, J.; and Neumann, G. 2016. Probabilistic inference for determining options in reinforcement learning. Machine Learning, Special Issue 104 (2): 337-357.
6. Harb, J. 2016. Learning options in deep reinforcement learning. Master's thesis, McGill University.
7. Konda, V. R., and Tsitsiklis, J. N. 2000. Actor-critic algorithms. In NIPS 12, 1008-1014.
8. Konidaris, G., and Barto, A. 2009. Skill discovery in continuous reinforcement learning domains using skill chaining. In NIPS 22, 1015-1023.
9. Konidaris, G.; Kuindersma, S.; Grupen, R. A.; and Barto, A. G. 2011. Autonomous skill acquisition on a mobile manipulator. In AAAI.
10. Krishnamurthy, R.; Lakshminarayanan, A. S.; Kumar, P.; and Ravindran, B. 2016. Hierarchical reinforcement learning using spatio-temporal abstractions and deep neural networks. CoRR abs/1605. 05359.
11. Kulkarni, T.; Narasimhan, K.; Saeedi, A.; and Tenenbaum, J. 2016. Hierarchical deep reinforcement learning: Integrating temporal abstraction and intrinsic motivation. In NIPS 29.
12. Levy, K. Y., and Shimkin, N. 2011. Unified inter and intra options learning using policy gradient methods. In EWRL, 153-164.
13. Mankowitz, D. J.; Mann, T. A.; and Mannor, S. 2016. Adaptive skills, adaptive partitions (ASAP). In NIPS 29.
14. Mann, T. A.; Mankowitz, D. J.; and Mannor, S. 2014. Timeregularized interrupting options (TRIO). In ICML, 1350-1358.
15. Mann, T. A.; Mannor, S.; and Precup, D. 2015. Approximate value iteration with temporally extended actions. Journal of Artificial Intelligence Research 53: 375-438.
16. McGovern, A., and Barto, A. G. 2001. Automatic discovery of subgoals in reinforcement learning using diverse density. In ICML, 361-368.
17. Menache, I.; Mannor, S.; and Shimkin, N. 2002. Q-cut-dynamic discovery of sub-goals in reinforcement learning. In ECML, 295-306.
18. Mnih, V.; Kavukcuoglu, K.; Silver, D.; Graves, A.; Antonoglou, I.; Wierstra, D.; and Riedmiller, M. A. 2013. Playing atari with deep reinforcement learning. CoRR abs/1312. 5602.
19. Mnih, V.; Badia, A. P.; Mirza, M.; Graves, A.; Lillicrap, T. P.; Harley, T.; Silver, D.; and Kavukcuoglu, K. 2016. Asynchronous methods for deep reinforcement learning. In ICML.
20. Niekum, S. 2013. Semantically Grounded Learning from Unstructured Demonstrations. Ph. D. Dissertation, University of Massachusetts, Amherst.
21. Precup, D. 2000. Temporal abstraction in reinforcement learning. Ph. D. Dissertation, University of Massachusetts, Amherst.
22. Puterman, M. L. 1994. Markov Decision Processes: Discrete Stochastic Dynamic Programming. John Wiley & Sons, Inc.
23. Silver, D., and Ciosek, K. 2012. Compositional planning using optimal option models. In ICML.
24. Sorg, J., and Singh, S. P. 2010. Linear options. In AAMAS, 31-38.
25. Stolle, M., and Precup, D. 2002. Learning options in reinforcement learning. In Abstraction, Reformulation and Approximation, 5th International Symposium, SARA Proceedings, 212-223.
26. Sutton, R. S.; McAllester, D. A.; Singh, S. P.; and Mansour, Y. 2000. Policy gradient methods for reinforcement learning with function approximation. In NIPS 12. 1057-1063.
27. Sutton, R. S.; Precup, D.; and Singh, S. P. 1999. Between mdps and semi-mdps: A framework for temporal abstraction in reinforcement learning. Artificial Intelligence 112 (1-2): 181-211.
28. Sutton, R. S. 1984. Temporal Credit Assignment in Reinforcement Learning. Ph. D. Dissertation.
29. Thomas, P. 2014. Bias in natural actor-critic algorithms. In ICML, 441-448.
30. Vezhnevets, A. S.; Mnih, V.; Agapiou, J.; Osindero, S.; Graves, A.; Vinyals, O.; and Kavukcuoglu, K. 2016. Strategic attentive writer for learning macro-actions. In NIPS 29.