Deep exploration via bootstrapped DQN

Advances in Neural Information Processing Systems

Volumn , Issue , 2016, Pages 4033-4041

  Osband, Ian ^a,b   Blundell, Charles ^b   Pritzel, Alexander ^b   Van Roy, Benjamin ^a  

^a STANFORD UNIVERSITY   (United States)

^b DEEPMIND   (United Kingdom)

Author keywords

[No Author keywords available]

Indexed keywords

COMPUTER AIDED INSTRUCTION; DEEP LEARNING; REINFORCEMENT LEARNING;

COMPLEX ENVIRONMENTS; DATA REQUIREMENTS; LEARNING ENVIRONMENTS; LEARNING SPEED; PARAMETERIZED; VALUE FUNCTIONS;

DEEP NEURAL NETWORKS;

EID: 85019259487     PISSN: 10495258     EISSN: None     Source Type: Conference Proceeding    
DOI: None     Document Type: Conference Paper

References (27)

1. Marc G Bellemare, Yavar Naddaf, Joel Veness, and Michael Bowling. The arcade learning environment: An evaluation platform for general agents. arXiv preprint arXiv: 1207.4708, 2012.
2. Peter J Bickel and David A Freedman. Some asymptotic theory for the bootstrap. The Annals of Statistics, pages 1196-1217, 1981.
3. Charles Blundell, Julien Cornebise, Koray Kavukcuoglu, and Daan Wierstra. Weight uncertainty in neural networks. ICML, 2015.
4. Christoph Dann and Emma Brunskill. Sample complexity of episodic fixed-horizon reinforcement learning. In Advances in Neural Information Processing Systems, pages 2800-2808, 2015.
5. Bradley Efron. The jackknife, the bootstrap and other resampling plans, Volume 38. SIAM, 1982.
6. Bradley Efron and Robert J Tibshirani. An introduction to the bootstrap. CRC press, 1994.
7. Yarin Gal and Zoubin Ghahramani. Dropout as a bayesian approximation: Representing model uncertainty in deep learning. arXiv preprint arXiv: 1506.02142, 2015.
8. Arthur Guez, David Silver, and Peter Dayan. Efficient bayes-adaptive reinforcement learning using sample-based search. In Advances in Neural Information Processing Systems, pages 1025-1033, 2012.
9. Thomas Jaksch, Ronald Ortner, and Peter Auer. Near-optimal regret bounds for reinforcement learning. Journal of Machine Learning Research, 11: 1563-1600, 2010.
10. Sham Kakade. On the Sample Complexity of Reinforcement Learning. PhD thesis, University College London, 2003.
11. Tze Leung Lai and Herbert Robbins. Asymptotically efficient adaptive allocation rules. Advances in applied mathematics, 6(1): 4-22, 1985.
12. Volodymyr et al. Mnih. Human-level control through deep reinforcement learning. Nature, 518(7540): 529-533, 2015.
13. Ian Osband, Daniel Russo, and Benjamin Van Roy. (More) efficient reinforcement learning via posterior sampling. In NIPS, pages 3003-3011. Curran Associates, Inc., 2013.
14. Ian Osband and Benjamin Van Roy. Model-based reinforcement learning and the eluder dimension. In Advances in Neural Information Processing Systems, pages 1466-1474, 2014.
15. Ian Osband and Benjamin Van Roy. Bootstrapped thompson sampling and deep exploration. arXiv preprint arXiv: 1507.00300, 2015.
16. Ian Osband, Benjamin Van Roy, and Zheng Wen. Generalization and exploration via randomized value functions. arXiv preprint arXiv: 1402.0635, 2014.
17. Art B Owen, Dean Eckles, et al. Bootstrapping data arrays of arbitrary order. The Annals of Applied Statistics, 6(3): 895-927, 2012.
18. Tom Schaul, John Quan, Ioannis Antonoglou, and David Silver. Prioritized experience replay. arXiv preprint arXiv: 1511.05952, 2015.
19. Nitish Srivastava, Geoffrey Hinton, Alex Krizhevsky, Ilya Sutskever, and Ruslan Salakhutdinov. Dropout: A simple way to prevent neural networks from overfitting. The Journal of Machine Learning Research, 15(1): 1929-1958, 2014.
20. Bradly C Stadie, Sergey Levine, and Pieter Abbeel. Incentivizing exploration in reinforcement learning with deep predictive models. arXiv preprint arXiv: 1507.00814, 2015.
21. Malcolm J. A. Strens. A bayesian framework for reinforcement learning. In ICML, pages 943-950, 2000.
22. Richard Sutton and Andrew Barto. Reinforcement Learning: An Introduction. MIT Press, March 1998.
23. Gerald Tesauro. Temporal difference learning and td-gammon. Communications of the ACM, 38(3): 58-68, 1995.
24. W.R. Thompson. On the likelihood that one unknown probability exceeds another in view of the evidence of two samples. Biometrika, 25(3/4): 285-294, 1933.
25. Hado Van Hasselt, Arthur Guez, and David Silver. Deep reinforcement learning with double q-learning. arXiv preprint arXiv: 1509.06461, 2015.
26. Ziyu Wang, Nando de Freitas, and Marc Lanctot. Dueling network architectures for deep reinforcement learning. arXiv preprint arXiv: 1511.06581, 2015.
27. Zheng Wen and Benjamin Van Roy. Efficient exploration and value function generalization in deterministic systems. In NIPS, pages 3021-3029, 2013.