Deep reinforcement learning with double Q-Learning

30th AAAI Conference on Artificial Intelligence, AAAI 2016

Volumn , Issue , 2016, Pages 2094-2100

  Van Hasselt, Hado ^a   Guez, Arthur ^a   Silver, David ^a  

^a DEEPMIND   (United Kingdom)

Author keywords

[No Author keywords available]

Indexed keywords

APPROXIMATION ALGORITHMS; ARTIFICIAL INTELLIGENCE; REINFORCEMENT LEARNING;

DEEP NEURAL NETWORKS; Q-LEARNING; Q-LEARNING ALGORITHMS; SCALE FUNCTIONS; SPECIFIC ADAPTATIONS;

LEARNING ALGORITHMS;

EID: 85007210890     PISSN: None     EISSN: None     Source Type: Conference Proceeding    
DOI: None     Document Type: Conference Paper

References (26)

1. R. Agrawal. Sample mean based index policies with O(log n) regret for the multi-Armed bandit problem. Advances in Applied Probability, pages 1054-1078, 1995.
2. P. Auer, N. Cesa-Bianchi, and P. Fischer. Finite-Time analysis of the multiarmed bandit problem. Machine learning, 47(2-3):235-256, 2002.
3. L. Baird. Residual algorithms: Reinforcement learning with function approximation. In Machine Learning: Proceedings of the Twelfth International Conference, pages 30-37, 1995.
4. M. G. Bellemare, Y. Naddaf, J. Veness, and M. Bowling. The arcade learning environment: An evaluation platform for general agents. J. Artif. Intell. Res. (JAIR), 47:253-279, 2013.
5. R. I. Brafman and M. Tennenholtz. R-max-A general polynomial time algorithm for near-optimal reinforcement learning. The Journal of Machine Learning Research, 3:213-231, 2003.
6. K. Fukushima. Neocognitron: A hierarchical neural network capable of visual pattern recognition. Neural networks, 1(2):119-130, 1988.
7. L. P. Kaelbling, M. L. Littman, and A. W. Moore. Reinforcement learning: A survey. Journal of Artificial Intelligence Research, 4: 237-285, 1996.
8. Y. LeCun, L. Bottou, Y. Bengio, and P. Haffner. Gradient-based learning applied to document recognition. Proceedings of the IEEE, 86(11):2278-2324, 1998.
9. L. Lin. Self-improving reactive agents based on reinforcement learning, planning and teaching. Machine learning, 8(3):293-321, 1992.
10. H. R. Maei. Gradient temporal-difference learning algorithms. PhD thesis, University of Alberta, 2011.
11. V. Mnih, K. Kavukcuoglu, D. Silver, A. A. Rusu, J. Veness, M. G. Bellemare, A. Graves, M. Riedmiller, A. K. Fidjeland, G. Ostrovski, S. Petersen, C. Beattie, A. Sadik, I. Antonoglou, H. King, D. Kumaran, D. Wierstra, S. Legg, and D. Hassabis. Human-level control through deep reinforcement learning. Nature, 518(7540): 529-533, 2015.
12. A. Nair, P. Srinivasan, S. Blackwell, C. Alcicek, R. Fearon, A. D. Maria, V. Panneershelvam, M. Suleyman, C. Beattie, S. Petersen, S. Legg, V. Mnih, K. Kavukcuoglu, and D. Silver. Massively parallel methods for deep reinforcement learning. In Deep Learning Workshop, ICML, 2015.
13. M. Riedmiller. Neural fitted Q iteration - first experiences with a data efficient neural reinforcement learning method. In Proceedings of the 16th European Conference on Machine Learning, pages 317-328. Springer, 2005.
14. B. Sallans and G. E. Hinton. Reinforcement learning with factored states and actions. The Journal of Machine Learning Research, 5: 1063-1088, 2004.
15. A. L. Strehl, L. Li, and M. L. Littman. Reinforcement learning in finite MDPs: PAC analysis. The Journal of Machine Learning Research, 10:2413-2444, 2009.
16. R. S. Sutton. Learning to predict by the methods of temporal differences. Machine learning, 3(1):9-44, 1988.
17. R. S. Sutton. Integrated architectures for learning, planning, and reacting based on approximating dynamic programming. In Proceedings of the seventh international conference on machine learning, pages 216-224, 1990.
18. R. S. Sutton and A. G. Barto. Introduction to reinforcement learning. MIT Press, 1998.
19. R. S. Sutton, A. R. Mahmood, and M. White. An emphatic approach to the problem of off-policy temporal-difference learning. arXiv preprint arXiv:1503.04269, 2015.
20. I. Szita and A. Lorincz. The many faces of optimism: A unifying approach. In Proceedings of the 25th international conference on Machine learning, pages 1048-1055. ACM, 2008.
21. G. Tesauro. Temporal difference learning and td-gammon. Communications of the ACM, 38(3):58-68, 1995.
22. S. Thrun and A. Schwartz. Issues in using function approximation for reinforcement learning. In M. Mozer, P. Smolensky, D. Touretzky, J. Elman, and A. Weigend, editors, Proceedings of the 1993 Connectionist Models Summer School, Hillsdale, NJ, 1993. Lawrence Erlbaum.
23. J. N. Tsitsiklis and B. Van Roy. An analysis of temporal-difference learning with function approximation. IEEE Transactions on Automatic Control, 42(5):674-690, 1997.
24. H. van Hasselt. Double Q-learning. Advances in Neural Information Processing Systems, 23:2613-2621, 2010.
25. H. van Hasselt. Insights in Reinforcement Learning. PhD thesis, Utrecht University, 2011.
26. C. J. C. H. Watkins. Learning from delayed rewards. PhD thesis, University of Cambridge England, 1989.