Epopt: Learning robust neural network policies using model ensembles

5th International Conference on Learning Representations, ICLR 2017 - Conference Track Proceedings

Volumn , Issue , 2017, Pages

  Rajeswaran, Aravind ^a   Ghotra, Sarvjeet ^b   Ravindran, Balaraman ^c   Levine, Sergey ^d  

^a UNIVERSITY OF WASHINGTON   (United States)

^b NATIONAL INSTITUTE OF TECHNOLOGY KARNATAKA   (India)

^c INDIAN INSTITUTE OF TECHNOLOGY MADRAS   (India)

^d UNIVERSITY OF CALIFORNIA   (United States)

Author keywords

[No Author keywords available]

Indexed keywords

BAYESIAN NETWORKS; DEEP NEURAL NETWORKS; PROBABILITY DISTRIBUTIONS;

APPROXIMATE BAYESIAN; DOMAIN ADAPTATION; FUNCTION APPROXIMATORS; MODEL ENSEMBLES; MODEL-BASED METHOD; REAL-WORLD TASK; SAMPLE COMPLEXITY; SIMULATED TRAININGS;

REINFORCEMENT LEARNING;

EID: 85064811489     PISSN: None     EISSN: None     Source Type: Conference Proceeding    
DOI: None     Document Type: Conference Paper

References (39)

1. Pieter Abbeel, Morgan Quigley, and Andrew Y. Ng. Using inaccurate models in reinforcement learning. In ICML, 2006.
2. Brenna D. Argall, Sonia Chernova, Manuela Veloso, and Brett Browning. A survey of robot learning from demonstration. Robotics and Autonomous Systems, 57(5):469 - 483, 2009.
3. Greg Brockman, Vicki Cheung, Ludwig Pettersson, Jonas Schneider, John Schulman, Jie Tang, and Wojciech Zaremba. OpenAI Gym, 2016.
4. Bruno Castro da Silva, George Konidaris, and Andrew G. Barto. Learning parameterized skills. In ICML, 2012.
5. Marc Peter Deisenroth, Gerhard Neumann, and Jan Peters. A survey on policy search for robotics. Foundations and Trends in Robotics, 2(12):1-142, 2013.
6. Erick Delage and Shie Mannor. Percentile optimization for markov decision processes with parameter uncertainty. Operations Research, 58(1):203-213, 2010.
7. Yan Duan, Xi Chen, Rein Houthooft, John Schulman, and Pieter Abbeel. Benchmarking deep reinforcement learning for continuous control. In ICML, 2016.
8. Michael O. Duff. Design for an optimal probe. In ICML, 2003.
9. Tom Erez, Yuval Tassa, and Emanuel Todorov. Infinite-horizon model predictive control for periodic tasks with contacts. In Proceedings of Robotics: Science and Systems, 2011.
10. Javier García and Fernando Fernández. A comprehensive survey on safe reinforcement learning. Journal of Machine Learning Research, 2015.
11. Mohammad Ghavamzadeh, Shie Mannor, Joelle Pineau, and Aviv Tamar. Bayesian reinforcement learning: A survey. Foundations and Trends in Machine Learning, 8(5-6):359-483, 2015.
12. Sham Kakade. A natural policy gradient. In NIPS, 2001.
13. Sham Kakade. On the Sample Complexity of Reinforcement Learning. PhD thesis, University College London, 2003.
14. Sham Kakade and John Langford. Approximately optimal approximate reinforcement learning. In ICML, 2002.
15. Sergey Levine and Vladlen Koltun. Guided policy search. In ICML, 2013.
16. T. P. Lillicrap, J. J. Hunt, A. Pritzel, N. Heess, T. Erez, Y. Tassa, D. Silver, and D. Wierstra. Continuous control with deep reinforcement learning. ArXiv e-prints, September 2015.
17. Shiau Hong Lim, Huan Xu, and Shie Mannor. Reinforcement learning in robust markov decision processes. In NIPS. 2013.
18. Lennart Ljung. System Identification, pp. 163-173. Birkhäuser Boston, Boston, MA, 1998.
19. Volodymyr Mnih et al. Human-level control through deep reinforcement learning. Nature, 518(7540): 529-533, Feb 2015.
20. I. Mordatch, K. Lowrey, and E. Todorov. Ensemble-CIO: Full-body dynamic motion planning that transfers to physical humanoids. In IROS, 2015a.
21. Igor Mordatch, Kendall Lowrey, Galen Andrew, Zoran Popovic, and Emanuel V. Todorov. Interactive control of diverse complex characters with neural networks. In NIPS. 2015b.
22. Arnab Nilim and Laurent El Ghaoui. Robust control of markov decision processes with uncertain transition matrices. Operations Research, 53(5):780-798, 2005.
23. Xue Bin Peng, Glen Berseth, and Michiel van de Panne. Terrain-adaptive locomotion skills using deep reinforcement learning. ACM Transactions on Graphics (Proc. SIGGRAPH 2016), 2016.
24. Josep M. Porta, Nikos A. Vlassis, Matthijs T. J. Spaan, and Pascal Poupart. Point-based value iteration for continuous pomdps. Journal of Machine Learning Research, 7:2329-2367, 2006.
25. Pascal Poupart, Nikos A. Vlassis, Jesse Hoey, and Kevin Regan. An analytic solution to discrete bayesian reinforcement learning. In ICML, 2006.
26. S. Ross, B. Chaib-draa, and J. Pineau. Bayesian reinforcement learning in continuous pomdps with application to robot navigation. In ICRA, 2008.
27. Stephane Ross and Drew Bagnell. Agnostic system identification for model-based reinforcement learning. In ICML, 2012.
28. John Schulman, Sergey Levine, Philipp Moritz, Michael Jordan, and Pieter Abbeel. Trust region policy optimization. In ICML, 2015.
29. David Silver et al. Mastering the game of go with deep neural networks and tree search. Nature, 529 (7587):484-489, Jan 2016.
30. Aviv Tamar, Dotan Di Castro, and Ron Meir. Integrating a partial model into model free reinforcement learning. Journal of Machine Learning Research, 2012.
31. Aviv Tamar, Yonatan Glassner, and Shie Mannor. Optimizing the cvar via sampling. In AAAI Conference on Artificial Intelligence, 2015.
32. Matthew E. Taylor and Peter Stone. Transfer learning for reinforcement learning domains: A survey. Journal of Machine Learning Research, 10:1633-1685, December 2009.
33. Philip Thomas, Georgios Theocharous, and Mohammad Ghavamzadeh. High-confidence off-policy evaluation. In AAAI Conference on Artificial Intelligence. 2015.
34. E. Todorov, T. Erez, and Y. Tassa. Mujoco: A physics engine for model-based control. In 2012 IEEE/RSJ International Conference on Intelligent Robots and Systems, pp. 5026-5033, Oct 2012.
35. Nikos Vlassis, Mohammad Ghavamzadeh, Shie Mannor, and Pascal Poupart. Bayesian Reinforcement Learning, pp. 359-386. Springer Berlin Heidelberg, Berlin, Heidelberg, 2012.
36. Jack M. Wang, David J. Fleet, and Aaron Hertzmann. Optimizing walking controllers for uncertain inputs and environments. ACM Trans. Graph., 2010.
37. Pawel Wawrzynski. Real-time reinforcement learning by sequential actor-critics and experience replay. Neural Networks, 22:1484-1497, 2009.
38. Ronald J. Williams. Simple statistical gradient-following algorithms for connectionist reinforcement learning. Machine Learning, 8(3):229-256, 1992.
39. Kemin Zhou, John C. Doyle, and Keith Glover. Robust and Optimal Control. Prentice-Hall, Inc., Upper Saddle River, NJ, USA, 1996. ISBN 0-13-456567-3.