Deep learning for real-time Atari game play using offline Monte-Carlo tree search planning

Advances in Neural Information Processing Systems

Volumn 4, Issue January, 2014, Pages 3338-3346

  Guo, Xiaoxiao ^a   Singh, Satinder ^a   Lee, Honglak ^a   Lewis, Richard ^a   Wang, Xiaoshi ^a  

^a UNIVERSITY OF MICHIGAN   (United States)

Author keywords

[No Author keywords available]

Indexed keywords

ARTIFICIAL INTELLIGENCE; BENCHMARKING; COMPUTER AIDED INSTRUCTION; INFORMATION SCIENCE;

COMBINING MODEL; DEEP LEARNING; HUMAN PLAYERS; LEARNING ENVIRONMENTS; MONTE-CARLO TREE SEARCHES; ORDERS OF MAGNITUDE; POLICY SELECTION; TRAINING DATA;

REINFORCEMENT LEARNING;

EID: 84937779024     PISSN: 10495258     EISSN: None     Source Type: Conference Proceeding    
DOI: None     Document Type: Conference Paper

References (23)

1. M. G. Bellemare, Y. Naddaf, J. Veness, and M. Bowling. The arcade learning environment: an evaluation platform for general agents. Journal of Artificial Intelligence Research, 47(1): 253-279, 2013.
2. M. G. Bellemare, J. Veness, and M. Bowling. Investigating contingency awareness using Atari 2600 games. In the 26th AAAI Conference on Artificial Intelligence, 2012.
3. M. G. Bellemare, J. Veness, and M. Bowling. Sketch-based linear value function approximation. In Advances in Neural Information Processing Systems, pages 2222-2230, 2012.
4. M. G. Bellemare, J. Veness, and E. Talvitie. Skip context tree switching. In Proceedings of the International Conference on Machine Learning, 2014.
5. Y. Bengio. Learning deep architectures for AI. Foundations and trends in Machine Learning, 2(1): 1-127, 2009.
6. D. Ciresan, U. Meier, and J. Schmidhuber. Multi-column deep neural networks for image classification. In IEEE Conference on Computer Vision and Pattern Recognition, 2012.
7. D. Erhan, Y. Bengio, A. Courville, and P. Vincent. Visualizing higher-layer features of a deep network. Technical report, University of Montreal, 2009.
8. A. Graves, A. Mohamed, and G. Hinton. Speech recognition with deep recurrent neural networks. In IEEE International Conference on Acoustics, Speech and Signal Processing, pages 6645-6649, 2013.
9. M. Hausknecht, P. Khandelwal, R. Miikkulainen, and P. Stone. HyperNEAT-GGP: A hyperNEAT-based Atari general game player. In Proceedings of the Fourteenth International Conference on Genetic and Evolutionary Computation Conference, pages 217-224, 2012.
10. A. Karpathy, G. Toderici, S. Shetty, T. Leung, R. Sukthankar, and L. Fei-Fei. Large-scale video classification with convolutional neural networks. In IEEE Conference on Computer Vision and Pattern Recognition, 2014.
11. M. Kearns, Y. Mansour, and A. Y. Ng. A sparse sampling algorithm for near-optimal planning in large Markov decision processes. Machine Learning, 49(2-3): 193-208, 2002.
12. L. Kocsis and C. Szepesvári. Bandit based Monte-Carlo planning. In European Conference on Machine Learning, pages 282-293. 2006.
13. A. Krizhevsky, I. Sutskever, and G. E. Hinton. ImageNet classification with deep convolutional neural networks. In Advances in Neural Information Processing Systems, 2012.
14. Q. V. Le, M. Ranzato, R. Monga, M. Devin, K. Chen, G. S. Corrado, J. Dean, and A. Y. Ng. Building high-level features using large scale unsupervised learning. In Proceedings of the 29th International Conference on Machine Learning, 2012.
15. Q. V. Le, W. Y. Zou, S. Y. Yeung, and A. Y. Ng. Learning hierarchical invariant spatio-temporal features for action recognition with independent subspace analysis. In IEEE Conference on Computer Vision and Pattern Recognition, pages 3361-3368, 2011.
16. Y. LeCun, L. Bottou, Y. Bengio, and P. Haffner. Gradient-based learning applied to document recognition. Proceedings of the IEEE, 86(11): 2278-2324, 1998.
17. H. Lee, R. Grosse, R. Ranganath, and A. Y. Ng. Convolutional deep belief networks for scalable unsupervised learning of hierarchical representations. In Proceedings of the 26th International Conference on Machine Learning, pages 609-616, 2009.
18. H. Lee, P. Pham, Y. Largman, and A. Y. Ng. Unsupervised feature learning for audio classification using convolutional deep belief networks. In Advances in Neural Information Processing Systems, pages 1096-1104, 2009.
19. V. Mnih, K. Kavukcuoglu, D. Silver, A. Graves, I. Antonoglou, D. Wierstra, and M. Riedmiller. Playing Atari with deep reinforcement learning. In Deep Learning, Neural Information Processing Systems Workshop, 2013.
20. A. Mohamed, G. E. Dahl, and G. Hinton. Acoustic modeling using deep belief networks. IEEE Transactions on Audio, Speech, and Language Processing, 20(1): 14-22, 2012.
21. S. Ross, G. J. Gordon, and J. A. Bagnell. A reduction of imitation learning and structured prediction to no-regret online learning. In Proceedings of the 14th International Conference on Artificial Intelligence and Statistics, 2011.
22. J. Schmidhuber. Deep learning in neural networks: An overview. arXiv preprint arXiv: 1404.7828, 2014.
23. G. Tesauro. Temporal difference learning and TD-gammon. Communications of the ACM, 38(3): 58-68, 1995.