Increasing the action gap: New operators for reinforcement learning

30th AAAI Conference on Artificial Intelligence, AAAI 2016

Volumn , Issue , 2016, Pages 1476-1483

  Bellemare, Marc G ^a   Ostrovski, Georg ^a   Guez, Arthur ^a   Thomas, Philip ^a,b   Munos, Rémi ^a  

^a DEEPMIND   (United Kingdom)

^b CARNEGIE MELLON UNIVERSITY   (United States)

Author keywords

[No Author keywords available]

Indexed keywords

ARTIFICIAL INTELLIGENCE; REINFORCEMENT LEARNING;

CONTINUOUS SPACES; EMPIRICAL STUDIES; ESTIMATION ERRORS; INCREASING OPERATORS; LOCAL CONSISTENCY; LOCAL POLICIES; TABULAR REPRESENTATIONS; UNDESIRABLE EFFECTS;

LEARNING ALGORITHMS;

EID: 85007236718     PISSN: None     EISSN: None     Source Type: Conference Proceeding    
DOI: None     Document Type: Conference Paper

References (32)

1. Atkeson, C. G. 1991. Using locally weighted regression for robot learning. In Proceedings of 1991 IEEE International Conference on Robotics and Automation, 958-963.
2. Azar, M. G.; Munos, R.; Gavamzadeh, M.; and Kappen, H. J. 2011. Speedy Q-learning. In Advances in Neural Information Processing Systems 24.
3. Baird, L. C. 1999. Reinforcement learning through gradient descent. Ph.D. Dissertation, Carnegie Mellon University.
4. Bellemare, M. G.; Naddaf, Y.; Veness, J.; and Bowling, M. 2013. The Arcade Learning Environment: An evaluation platform for general agents. Journal of Artificial Intelligence Research 47:253-279.
5. Bellman, R. E. 1957. Dynamic programming. Princeton, NJ: Princeton University Press.
6. Bertsekas, D. P., and Tsitsiklis, J. N. 1996. Neuro-Dynamic Programming. Athena Scientific.
7. Bertsekas, D. P., and Yu, H. 2012. Q-learning and enhanced policy iteration in discounted dynamic programming. Mathematics of Operations Research 37(1):66-94.
8. Bertsekas, D. P. 2011. Approximate policy iteration: A survey and some new methods. Journal of Control Theory and Applications 9(3):310-335.
9. Deisenroth, M. P., and Rasmussen, C. E. 2011. PILCO: A model-based and data-efficient approach to policy search. In Proceedings of the International Conference on Machine Learning.
10. Ernst, D.; Geurts, P.; and Wehenkel, L. 2005. Tree-based batch mode reinforcement learning. Journal of Machine Learning Research 6:503-556.
11. Gordon, G. 1995. Stable function approximation in dynamic programming. In Proceedings of the Twelfth International Conference on Machine Learning.
12. Hoburg, W., and Tedrake, R. 2009. System identification of post stall aerodynamics for UAV perching. In Proceedings of the AIAA Infotech Aerospace Conference.
13. Jiang, D. R., and Powell, W. B. 2015. Optimal hour ahead bidding in the real time electricity market with battery storage using approximate dynamic programming. INFORMS Journal on Computing 27(3):525 - 543.
14. Kushner, H., and Dupuis, P. G. 2001. Numerical methods for stochastic control problems in continuous time. Springer.
15. Lee, D.; Defourny, B.; and Powell, W. B. 2013. Biascorrected Q-learning to control max-operator bias in Qlearning. In Symposium on Adaptive Dynamic Programming And Reinforcement Learning.
16. Mnih, V.; Kavukcuoglu, K.; Silver, D.; Rusu, A. A.; Veness, J.; Bellemare, M. G.; Graves, A.; Riedmiller, M.; Fidjeland, A. K.; Ostrovski, G.; et al. 2015. Humanlevel control through deep reinforcement learning. Nature 518(7540):529-533.
17. Munos, R., and Moore, A. 1998. Barycentric interpolators for continuous space & time reinforcement learning. In Advances in Neural Information Processing Systems 11.
18. Munos, R., and Moore, A. 2002. Variable resolution discretization in optimal control. Machine learning 49(2- 3):291-323.
19. Ormoneit, D., and Sen, S. 2002. Kernel-based reinforcement learning. Machine learning 49(2-3):161-178.
20. Randlov, J., and Alstrom, P. 1998. Learning to drive a bicycle using reinforcement learning and shaping. In Proceedings of the Fifteenth International Conference on Machine Learning.
21. Riedmiller, M.; Gabel, T.; Hafner, R.; and Lange, S. 2009. Reinforcement learning for robot soccer. Autonomous Robots 27(1):55-73.
22. Rummery, G. A., and Niranjan, M. 1994. On-line Qlearning using connectionist systems. Technical report, Cambridge University Engineering Department.
23. Sutton, R. S., and Barto, A. G. 1998. Reinforcement learning: An introduction. MIT Press.
24. Sutton, R.; Modayil, J.; Delp, M.; Degris, T.; Pilarski, P.; White, A.; and Precup, D. 2011. Horde: A scalable real-Time architecture for learning knowledge from unsupervised sensorimotor interaction. In Proceedings of the Tenth International Conference on Autonomous Agents and Multiagents Systems.
25. Sutton, R. S. 1988. Learning to predict by the methods of temporal differences. Machine Learning 3(1):9-44.
26. Sutton, R. S. 1996. Generalization in reinforcement learning: Successful examples using sparse coarse coding. In Advances in Neural Information Processing Systems 8, 1038- 1044.
27. Tesauro, G. 1995. Temporal difference learning and TDGammon. Communications of the ACM 38(3).
28. Togelius, J.; Karakovskiy, S.; Koutńik, J.; and Schmidhuber, J. 2009. Super Mario evolution. In Symposium on Computational Intelligence and Games.
29. van Hasselt, H.; Guez, A.; and Silver, D. 2016. Deep reinforcement learning with double Q-learning. In Proceedings of the AAAI Conference on Artificial Intelligence (to appear).
30. van Hasselt, H. 2010. Double Q-learning. In Advances in Neural Information Processing Systems 23.
31. Veness, J.; Bellemare, M. G.; Hutter, M.; Chua, A.; and Desjardins, G. 2015. Compress and control. In Proceedings of the AAAI Conference on Artificial Intelligence.
32. Watkins, C. 1989. Learning From Delayed Rewards. Ph.D. Dissertation, Cambridge University, Cambridge, England.