Curiosity-driven exploration by self-supervised prediction

34th International Conference on Machine Learning, ICML 2017

Volumn 6, Issue , 2017, Pages 4261-4270

  Pathak, Deepak ^a   Agrawal, Pulkit ^a   Efros, Alexei A ^a   Darrell, Trevor ^a  

^a UNIVERSITY OF CALIFORNIA   (United States)

Author keywords

[No Author keywords available]

Indexed keywords

ARTIFICIAL INTELLIGENCE; FORECASTING; LEARNING ALGORITHMS;

CONTINUOUS STATE SPACE; EXTRINSIC REWARDS; HIGH-DIMENSIONAL; INTRINSIC REWARDS; INVERSE DYNAMICS MODELS; REAL-WORLD SCENARIO; VISUAL FEATURE;

LEARNING SYSTEMS;

EID: 85040005905     PISSN: None     EISSN: None     Source Type: Conference Proceeding    
DOI: None     Document Type: Conference Paper

References (45)

1. Agrawal, Pulkit, Carreira, Joao, and Malik, Jitendra. Learning to see by moving. In ICCV, 2015.
2. Agrawal, Pulkit, Nair, Ashvin, Abbeel, Pieter, Malik, Jitendra, and Levine, Sergey. Learning to poke by poking: Experiential learning of intuitive physics. NIPS, 2016.
3. Bellemare, Marc, Srinivasan, Sriram, Ostrovski, Georg, Schaul, Tom, Saxton, David, and Munos, Remi. Unifying count-based exploration and intrinsic motivation. In NIPS, 2016.
4. Brafman, Ronen I and Tennenholtz, Moshe. R-max-a general polynomial time algorithm for near-optimal reinforcement learning. JMLR, 2002.
5. Brockman, Greg, Cheung, Vicki, Pettersson, Ludwig, Schneider, Jonas, Schulman, John, Tang, Jie, and Zaremba, Wojciech. Openai gym. arXiv:1606.01540, 2016.
6. Doersch, Carl, Gupta, Abhinav, and Efros, Alexei A. Unsupervised visual representation learning by context prediction. In ICCV, 2015.
7. Dosovitskiy, Alexey and Koltun, Vladlen. Learning to act by predicting the future. ICLR, 2016.
8. Fu, Justin, Co-Reyes, John D, and Levine, Sergey. Ex2: Exploration with exemplar models for deep reinforcement learning. arXiv:1703.01260, 2017.
9. Goroshin, Ross, Bruna, Joan, Tompson, Jonathan, Eigcn, David, and Le Cun, Yann. Unsupervised feature learning from temporal data. arXiv:1504.02518, 2015.
10. Gregor, Karol, Rezende, Danilo Jimenez, and Wierstra, Daan. Variational intrinsic control. ICLR Workshop, 2017.
11. Houthooft, Rein, Chen, Xi, Duan, Yan, Schulman, John, De Turck, Filip, and Abbeel, Pieter. Vime: Variational information maximizing exploration. In NIPS, 2016.
12. Jaderberg, Max, Mnih, Volodymyr, Czamecki, Wojciech Marian, Schaul, Tom, Leibo, Joel Z, Silver, David, and Kavukcuoglu, Koray. Reinforcement learning with unsupervised auxiliary tasks. ICLR, 2017.
13. Jayaraman, Dinesh and Grauman, Kristen. Learning image representations tied to ego-motion. In ICCV, 2015.
14. Jordan, Michael I and Rumelhart, David E. Forward models: Supervised learning with a distal teacher. Cognitive science, 1992.
15. Kearns, Michael and Roller, Daphne. Efficient reinforcement learning in factored mdps. In IJCAI, 1999.
16. Kcmpka, Michal, Wydmuch, Marck, Rune, Grzegorz, Toczck, Jakub, and Jaskowski, Wojciech. Vizdoom: A doom-based ai research platform for visual reinforcement learning. arXiv:1605.02097 2016.
17. Klyubin, Alexander S, Polani, Daniel, and Nehaniv, Chrystopher L. Empowerment: A universal agent- centric measure of control. In Evolutionary Computation, 2005.
18. Lillicrap, Timothy P, Hunt, Jonathan J, Pritzel, Alexander, Heess, Nicolas, Erez, Tom, Tassa, Yuval, Silver, David, and Wierstra, Daan. Continuous control with deep reinforcement learning. ICLR, 2016.
19. Little, Daniel Y and Sommer, Friedrich T. Learning and exploration in action-perception loops. Closing the Loop Around Neural Systems, 2014.
20. Lopes, Manuel, Lang, Tobias, Toussaint, Marc, and Oudeyer, Pierre-Yves. Exploration in model-based reinforcement learning by empirically estimating learning progress. In NIPS, 2012.
21. Mirowski, Piotr, Pascanu, Razvan, Viola, Fabio, Soyer, Hubert, Ballard, Andy, Banino, Andrea, Denil, Misha, Goroshin, Ross, Sifre, Laurent, Kavukcuoglu, Koray, et al. Learning to navigate in complex environments. ICLR, 2017.
22. Mnih, Volodymyr, Kavukcuoglu, Koray, Silver, David, Rusu, Andrei A, Veness, Joel, Bellemare, Marc G, Graves, Alex, Riedmiller, Martin, Fidjeland, Andreas K, Ostrovski, Georg, et al. Human-level control through deep reinforcement learning. Nature, 2015.
23. Mnih, Volodymyr, Badia, Adria Puigdomenech, Mirza, Mehdi, Graves, Alex, Lillicrap, Timothy P, Harley, Tim, Silver, David, and Kavukcuoglu, Koray. Asynchronous methods for deep reinforcement learning. In ICML, 2016.
24. Mohamed, Shakir and Rezende, Danilo Jimenez. Variational information maximisation for intrinsically motivated reinforcement learning. In NIPS, 2015.
25. Oh, Junhyuk, Guo, Xiaoxiao, Lee, Honglak, Lewis, Richard L, and Singh, Satinder. Action-conditional video prediction using deep networks in atari games. In NIPS, 2015.
26. Osband, Ian, Blundell, Charles, Pritzel, Alexander, and Van Roy, Benjamin. Deep exploration via bootstrapped dqn. In NIPS, 2016.
27. Oudeycr, Pierre-Yves and Kaplan, Frederic. What is intrinsic motivation? a typology of computational approaches. Frontiers in neurorobotics, 2009.
28. Oudeyer, Pierre-Yves, Kaplan, Frdric, and Hafner, Verena V. Intrinsic motivation systems for autonomous mental development. Evolutionary Computation, 2007.
29. Paquette, Philip. Super mario bros. In openai gym. github:ppaquette/gym-super-mario, 2016.
30. Pathak, Deepak, Krahenbuhl, Philipp, Donahue, Jeff, Darrell, Trevor, and Efros, Alexei A. Context encoders: Feature learning by inpainting. In CVPR, 2016.
31. Poupart, Pascal, Vlassis, Nikos, Hoey, Jesse, and Regan, Kevin. An analytic solution to discrete bayesian reinforcement learning. In ICML, 2006.
32. Ryan, Richard; Deci, Edward L. Intrinsic and extrinsic motivations: Classic definitions and new directions. Contemporary Educational Psychology, 2000.
33. Schmidhuber, Jurgen. A possibility for implementing curiosity and boredom in model-building neural controllers. In From animals to animats: Proceedings of the first international conference on simulation of adaptive behavior, 1991.
34. Schmidhuber, Jurgen. Formal theory of creativity, fun, and intrinsic motivation (1990-2010). IEEE Transactions on Autonomous Mental Development, 2010.
35. Shelhamer, Evan, Mahmoudieh, Parsa, Argus, Max, and Darrell, Trevor. Loss is its own reward: Self-supervision for reinforcement learning. arXiv:1612.07307, 2017.
36. Silvia, Paul J. Curiosity and motivation. In The Oxford Handbook of Human Motivation, 2012.
37. Singh, Satinder P, Barto, Andrew G, and Chentanez, Nuttapong. Intrinsically motivated reinforcement learning. In NIPS, 2005.
38. Stadie, Bradly C, Levine, Sergey, and Abbeel, Pieter. In- centivizing exploration in reinforcement learning with deep predictive models. NIPS Workshop, 2015.
39. Still, Susanne and Precup, Doina. An information-theoretic approach to curiosity-driven reinforcement learning. Theory in Biosciences, 2012.
40. Storck, Jan, Hochreiter, Sepp, and Schmidhuber, Jurgen. Reinforcement driven information acquisition in non- deterministic environments. In ICANN, 1995.
41. Sukhbaatar, Sainbayar, Kostrikov, Ilya, Szlam, Arthur, and Fergus, Rob. Intrinsic motivation and automatic curricula via asymmetric self-play. arXiv:1703.05407, 2017.
42. Sun, Yi, Gomez, Faustino, and Schmidhuber, Jurgen. Planning to be surprised: Optimal bayesian exploration in dynamic environments. In AGI, 2011.
43. Tang, Haoran, Houthooft, Rein, Foote, Davis, Stooke, Adam, Chen, Xi, Duan, Yan, Schulman, John, De Turck, Filip, and Abbeel, Pieter. # exploration: A study of count-based exploration for deep reinforcement learning. arXiv:1611.04717, 2016.
44. Wang, Xiaolong and Gupta, Abhinav. Unsupervised learning of visual representations using videos. In ICCV, 2015.
45. Wolpert, Daniel M, Ghahramani, Zoubin, and Jordan, Michael I. An internal model for sensorimotor integration. Science-AAAS-Weekly Paper Edition, 1995.