Learning deep neural network policies with continuous memory states

Proceedings - IEEE International Conference on Robotics and Automation

Volumn 2016-June, Issue , 2016, Pages 520-527

  Zhang, Marvin ^a   McCarthy, Zoe ^a   Finn, Chelsea ^a   Levine, Sergey ^a   Abbeel, Pieter ^a  

^a UNIVERSITY OF CALIFORNIA   (United States)

Author keywords

[No Author keywords available]

Indexed keywords

AERODYNAMICS; COMPLEX NETWORKS; MANIPULATORS; ROBOTS; SUPERVISED LEARNING;

CONTINUOUS CONTROL; CONTINUOUS SYSTEM; DEEP NEURAL NETWORKS; HIGH-DIMENSIONAL; LEARNING POLICY; POLICY LEARNING; ROBOTIC MANIPULATORS; TRAJECTORY OPTIMIZATION;

ROBOTICS;

EID: 84977555800     PISSN: 10504729     EISSN: None     Source Type: Conference Proceeding    
DOI: 10.1109/ICRA.2016.7487174     Document Type: Conference Paper

References (24)

1. G. Shani, J. Pineau, and R. Kaplow, "A survey of point-based pomdp solvers, " Autonomous Agents and Multi-Agent Systems, vol. 27, no. 1, pp. 1-51, 2013.
2. N. Meuleau, L. Peshkin, K.-E. Kim, and L. P. Kaelbling, "Learning finite-state controllers for partially observable environments, " in Proceedings of the Fifteenth conference on Uncertainty in Artificial Intelligence (UAI), 1999, pp. 427-436.
3. L. Peshkin, N. Meuleau, and L. Kaelbling, "Learning policies with external memory, " in Proceedings of the Sixteenth International Conference on Machine Learning (ICML), 2001, pp. 307-314.
4. M. Toussaint, L. Charlin, and P. Poupart, "Hierarchical pomdp controller optimization by likelihood maximization. " in Proceedings of the 24th conference on Uncertainty in Artificial Intelligence (UAI), 2008, pp. 562-570.
5. M. Deisenroth and C. E. Rasmussen, "PILCO: A model-based and data-efficient approach to policy search, " in Proceedings of the 28th International Conference on Machine Learning (ICML), 2011, pp. 465-472.
6. M. Watter, J. T. Springenberg, J. Boedecker, and M. Riedmiller, "Embed to control: A locally linear latent dynamics model for control from raw images, " in Advances in Neural Information Processing Systems (NIPS), 2015.
7. M. Deisenroth, G. Neumann, and J. Peters, "A survey on policy search for robotics, " Foundations and Trends in Robotics, vol. 2, no. 1-2, pp. 1-142, 2013.
8. D. Wierstra, A. Foerster, J. Peters, and J. Schmidhuber, "Solving deep memory pomdps with recurrent policy gradients, " in Artificial Neural Networks-ICANN 2007. Springer, 2007, pp. 697-706.
9. S. Levine and P. Abbeel, "Learning neural network policies with guided policy search under unknown dynamics, " 2014.
10. S. Levine, C. Finn, T. Darrell, and P. Abbeel, "End-to-end training of deep visuomotor policies, " arXiv preprint arXiv:1504. 00702, 2015.
11. S. Ross, J. Pineau, S. Paquet, and B. Chaib-Draa, "Online planning algorithms for pomdps, " Journal of Artificial Intelligence Research, pp. 663-704, 2008.
12. E. Brunskill, L. P. Kaelbling, T. Lozano-Perez, and N. Roy, "Continuous-state pomdps with hybrid dynamics. " in ISAIM, 2008.
13. R. Pascanu, T. Mikolov, and Y. Bengio, "On the difficulty of training recurrent neural networks, " arXiv preprint arXiv:1211. 5063, 2012.
14. S. Hochreiter and J. Schmidhuber, "Long short-term memory, " Neural computation, vol. 9, no. 8, pp. 1735-1780, 1997.
15. K. Cho, B. van Merrienboer, C. Gulcehre, F. Bougares, H. Schwenk, and Y. Bengio, "Learning phrase representations using RNN encoder-decoder for statistical machine translation, " arXiv preprint arXiv:1406. 1078, 2014.
16. S. Levine, N. Wagener, and P. Abbeel, "Learning contact-rich manipulation skills with guided policy search, " in International Conference on Robotics and Automation (ICRA), 2015.
17. S. Ross, G. Gordon, and A. Bagnell, "A reduction of imitation learning and structured prediction to no-regret online learning, " Journal of Machine Learning Research, vol. 15, pp. 627-635, 2011.
18. S. Levine and V. Koltun, "Learning complex neural network policies with trajectory optimization, " in International Conference on Machine Learning (ICML), 2014.
19. J. A. Bagnell and J. Schneider, "Covariant policy search, " in International Joint Conference on Artificial Intelligence (IJCAI), 2003.
20. J. Peters and S. Schaal, "Reinforcement learning of motor skills with policy gradients, " Neural Networks, vol. 21, no. 4, pp. 682-697, 2008.
21. J. Peters, K. Mülling, and Y. Altün, "Relative entropy policy search, " in AAAI Conference on Artificial Intelligence, 2010.
22. I. Sutskever, O. Vinyals, and Q. V. Le, "Sequence to sequence learning with neural networks, " in Advances in Neural Information Processing Systems (NIPS), 2014, pp. 3104-3112.
23. J. Peters and S. Schaal, "Applying the episodic natural actor-critic architecture to motor primitive learning, " in European Symposium on Artificial Neural Networks (ESANN), 2007.
24. J. Kober and J. Peters, "Learning motor primitives for robotics, " in International Conference on Robotics and Automation (ICRA), 2009.