Deep spatial autoencoders for visuomotor learning

Proceedings - IEEE International Conference on Robotics and Automation

Volumn 2016-June, Issue , 2016, Pages 512-519

  Finn, Chelsea ^a   Tan, Xin Yu ^a   Duan, Yan ^a   Darrell, Trevor ^a   Levine, Sergey ^a   Abbeel, Pieter ^a  

^a UNIVERSITY OF CALIFORNIA   (United States)

Author keywords

[No Author keywords available]

Indexed keywords

AUTOMATION; LEARNING SYSTEMS; ROBOTICS; ROBOTS;

AUTO ENCODERS; CAMERA IMAGES; CLOSED-LOOP CONTROL; FLEXIBLE FRAMEWORK; LOCAL LINEAR MODELS; REINFORCEMENT LEARNING METHOD; STATE REPRESENTATION; VISUO-MOTOR LEARNING;

REINFORCEMENT LEARNING;

EID: 84977501896     PISSN: 10504729     EISSN: None     Source Type: Conference Proceeding    
DOI: 10.1109/ICRA.2016.7487173     Document Type: Conference Paper

References (38)

1. S. Lange, A. Voigtlaender, and M. Riedmiller, "Autonomous reinforcement learning on raw visual input data in a real world application, " in International Joint Conference on Neural Networks (IJCNN), 2012.
2. R. Jonschkowski and O. Brock, "State representation learning in robotics: Using prior knowledge about physical interaction, " in Robotics: Science and Systems (RSS), 2014.
3. M. Watter, J. T. Springenberg, J. Boedecker, and M. Riedmiller, "Embed to control: A locally linear latent dynamics model for control from raw images, " in Advances in Neural Information Processing Systems (NIPS), 2015.
4. T. Lampe and M. Riedmiller, "Acquiring visual servoing reaching and grasping skills using neural reinforcement learning, " in International Joint Conference on Neural Networks (IJCNN), 2013.
5. Y. LeCun, Y. Bengio, and G. Hinton, "Deep learning, " Nature, vol. 521, pp. 436-444, 2015.
6. H. Lee, C. Ekanadham, and A. Y. Ng, "Sparse deep belief net model for visual area V2, " in Neural Information Processing Systems (NIPS), 2008.
7. S. Levine and P. Abbeel, "Learning neural network policies with guided policy search under unknown dynamics, " in Advances in Neural Information Processing Systems (NIPS), 2014.
8. S. Levine, N. Wagener, and P. Abbeel, "Learning contact-rich manipulation skills with guided policy search, " in International Conference on Robotics and Automation (ICRA), 2015.
9. J. Kober, E. Oztop, and J. Peters, "Reinforcement learning to adjust robot movements to new situations, " in Robotics: Science and Systems (RSS), 2010.
10. M. Deisenroth, C. Rasmussen, and D. Fox, "Learning to control a low-cost manipulator using data-efficient reinforcement learning, " in Robotics: Science and Systems (RSS), 2011.
11. P. Pastor, H. Hoffmann, T. Asfour, and S. Schaal, "Learning and generalization of motor skills by learning from demonstration, " in International Conference on Robotics and Automation (ICRA), 2009.
12. G. Endo, J. Morimoto, T. Matsubara, J. Nakanishi, and G. Cheng, "Learning CPG-based biped locomotion with a policy gradient method: Application to a humanoid robot, " International Journal of Robotics Research (IJRR), vol. 27, no. 2, pp. 213-228, 2008.
13. T. Geng, B. Porr, and F. Wörgötter, "Fast biped walking with a reflexive controller and realtime policy searching, " in Advances in Neural Information Processing Systems (NIPS), 2006.
14. N. Kohl and P. Stone, "Policy gradient reinforcement learning for fast quadrupedal locomotion, " in International Conference on Robotics and Automation (IROS), 2004.
15. R. Tedrake, T. Zhang, and H. Seung, "Stochastic policy gradient reinforcement learning on a simple 3d biped, " in International Conference on Intelligent Robots and Systems (IROS), 2004.
16. B. Sallans and G. E. Hinton, "Reinforcement learning with factored states and actions, " The Journal of Machine Learning Research (JMLR), vol. 5, pp. 1063-1088, 2004.
17. L. Baird et al., "Residual algorithms: Reinforcement learning with function approximation, " in Proceedings of the twelfth international conference on machine learning, 1995, pp. 30-37.
18. S. Singh, M. L. Littman, N. K. Jong, D. Pardoe, and P. Stone, "Learning predictive state representations, " in International Conference on Machine Learning (ICML), 2003.
19. B. Boots, S. M. Siddiqi, and G. J. Gordon, "Closing the learningplanning loop with predictive state representations, " The International Journal of Robotics Research (IJRR), vol. 30, no. 7, 2011.
20. W. Böhmer, J. T. Springenberg, J. Boedecker, M. Riedmiller, and K. Obermayer, "Autonomous learning of state representations for control, " KI-Knstliche Intelligenz, 2015.
21. R. Goroshin, M. Mathieu, and Y. LeCun, "Learning to linearize under uncertainty, " arXiv preprint arXiv:1506. 03011, 2015.
22. V. Mnih, K. Kavukcuoglu, D. Silver, A. Graves, I. Antonoglou, D. Wierstra, and M. Riedmiller, "Playing Atari with deep reinforcement learning, " NIPS '13 Workshop on Deep Learning, 2013.
23. S. Levine, C. Finn, T. Darrell, and P. Abbeel, "End-to-end training of deep visuomotor policies, " arXiv preprint arXiv:1504. 00702, 2015.
24. B. Espiau, F. Chaumette, and P. Rives, "A new approach to visual servoing in robotics, " IEEE Transactions on Robotics and Automation, vol. 8, no. 3, 1992.
25. K. Mohta, V. Kumar, and K. Daniilidis, "Vision based control of a quadrotor for perching on planes and lines, " in International Conference on Robotics and Automation (ICRA), 2014.
26. W. J. Wilson, C. W. W. Hulls, and G. S. Bell, "Relative endeffector control using cartesian position based visual servoing, " IEEE Transactions on Robotics and Automation, vol. 12, no. 5, 1996.
27. M. Jägersand, O. Fuentes, and R. C. Nelson, "Experimental evaluation of uncalibrated visual servoing for precision manipulation, " in International Conference on Robotics and Automation (ICRA), 1997.
28. B. H. Yoshimi and P. K. Allen, "Active, uncalibrated visual servoing, " in International Conference on Robotics and Automation (ICRA), 1994.
29. C. Szegedy, W. Liu, Y. Jia, P. Sermanet, S. Reed, D. Anguelov, D. Erhan, V. Vanhoucke, and A. Rabinovich, "Going deeper with convolutions, " arXiv preprint arXiv:1409. 4842, 2014.
30. J. Zhao, M. Mathieu, R. Goroshin, and Y. LeCun, "Stacked whatwhere auto-encoders, " arXiv preprint arXiv:1506. 02351, 2015.
31. M. Deisenroth, G. Neumann, and J. Peters, "A survey on policy search for robotics, " Foundations and Trends in Robotics, 2013.
32. W. Li and E. Todorov, "Iterative linear quadratic regulator design for nonlinear biological movement systems, " in ICINCO (1), 2004.
33. J. A. Bagnell and J. Schneider, "Covariant policy search, " in International Joint Conference on Artificial Intelligence (IJCAI), 2003.
34. J. Peters and S. Schaal, "Reinforcement learning of motor skills with policy gradients, " Neural Networks, vol. 21, no. 4, pp. 682-697, 2008.
35. J. Peters, K. Mülling, and Y. Altün, "Relative entropy policy search, " in AAAI Conference on Artificial Intelligence, 2010.
36. S. Ioffe and C. Szegedy, "Batch normalization: Accelerating deep network training by reducing internal covariate shift, " in International Conference on Machine Learning (ICML), 2015.
37. J. Deng, W. Dong, R. Socher, L. Li, K. Li, and L. Fei-Fei, "ImageNet: A large-scale hierarchical image database, " in Computer Vision and Pattern Recognition (CVPR), 2009.
38. S. Ross and A. Bagnell, "Agnostic system identification for modelbased reinforcement learning, " in International Conference on Machine Learning (ICML), 2012.