Stable reinforcement learning with autoencoders for tactile and visual data

IEEE International Conference on Intelligent Robots and Systems

Volumn 2016-November, Issue , 2016, Pages 3928-3934

  Van Hoof, Herke ^a   Chen, Nutan ^b   Karl, Maximilian ^b   Van Der Smagt, Patrick ^b,c   Peters, Jan ^a,d  

^a DARMSTADT UNIVERSITY OF TECHNOLOGY   (Germany)

^b TECHNICAL UNIVERSITY OF MUNICH   (Germany)

^c Fortiss   (Germany)

^d MAX PLANCK INSTITUTE FOR INTELLIGENT SYSTEMS   (Germany)

Author keywords

[No Author keywords available]

Indexed keywords

FEEDBACK; INTELLIGENT ROBOTS; LEARNING SYSTEMS; REINFORCEMENT LEARNING; ROBOTS; VISUAL COMMUNICATION; VISUAL SERVOING;

AUTO ENCODERS; COMPACT REPRESENTATION; CONTINUOUS STATE SPACE; HIGH-DIMENSIONAL; NON-TRIVIAL; TRIAL AND ERROR; VISUAL DATA; VISUAL FEEDBACK;

LEARNING ALGORITHMS;

EID: 85006416116     PISSN: 21530858     EISSN: 21530866     Source Type: Conference Proceeding    
DOI: 10.1109/IROS.2016.7759578     Document Type: Conference Paper

References (36)

1. J. Kober, J. A. Bagnell, and J. Peters, "Reinforcement learning in robotics: A survey", Int. J. Robotics Research, vol. 11, no. 32, pp. 1238-1274, 2013.
2. L. P. Kaelbling, M. L. Littman, and A. W. Moore, "Reinforcement learning: A survey", J. Artificial Intell. Research, vol. 4, 1996.
3. T. Lampe and M. Riedmiller, "Acquiring visual servoing reaching and grasping skills using neural reinforcement learning", in IJCNN, 2013.
4. O. Kroemer, C. Daniel, G. Neumann, H. van Hoof, and J. Peters, "Towards learning hierarchical skills for multi-phase manipulation tasks", in ICRA, 2015.
5. M. Kalakrishnan, L. Righetti, P. Pastor, and S. Schaal, "Learning force control policies for compliant manipulation", in IROS, 2011.
6. H. Zhang and N. Chen, "Control of contact via tactile sensing", Trans. Robotics and Automation, vol. 16, no. 5, pp. 482-495, 2000.
7. L. Jentoft, Q. Wan, and R. Howe, "Limits to compliance and the role of tactile sensing in grasping", in ICRA, 2014, pp. 6394-6399.
8. K. Hsiao, S. Chitta, M. Ciocarlie, and E. Jones, "Contact-reactive grasping of objects with partial shape information", in IROS, 2010.
9. P. Pastor, M. Kalakrishnan, L. Righetti, and S. Schaal, "Towards associative skill memories", in Humanoids, 2012.
10. P. L. Bartlett, "An introduction to reinforcement learning theory: Value function methods", in Advanced lectures on Machine Learning. Springer Berlin Heidelberg, 2003, pp. 184-202.
11. Y. Chebotar, O. Kroemer, and J. Peters, "Learning robot tactile sensing for object manipulation", in IROS, 2014, pp. 3368-3375.
12. O. Kroemer, R. Detry, J. Piater, and J. Peters, "Combining active learning and reactive control for robot grasping", Robotics and Autonomous Syst., vol. 58, no. 9, pp. 1105-1116, 2010.
13. P. Pastor, M. Kalakrishnan, S. Chitta, E. Theodorou, and S. Schaal, "Skill learning and task outcome prediction for manipulation", in ICRA, 2011.
14. J. Kober, B. Mohler, and J. Peters, "Learning perceptual coupling for motor primitives", in IROS, 2008, pp. 834-839.
15. H. van Hoof, J. Peters, and G. Neumann, "Learning of non-parametric control policies with high-dimensional state features", in AISTATS, 2015.
16. H. van Hoof, T. Hermans, G. Neumann, and J. Peters, "Learning robot in-hand manipulation with tactile features", in Humanoids, 2015.
17. P. Vincent, H. Larochelle, Y. Bengio, and P.-A. Manzagol, "Extracting and composing robust features with denoising autoencoders", in ICML, 2008, pp. 1096-1103.
18. Y. Bengio, "Learning deep architectures for AI", Found. Trends Mach. Learn., vol. 2, no. 1, pp. 1-127, 2009.
19. D. P. Kingma and M. Welling, "Auto-encoding variational Bayes", in ICLR, 2014.
20. N. Chen, J. Bayer, S. Urban, and P. van der Smagt, "Efficient movement representation by embedding dynamic movement primitives in deep autoencoders", in Humanoids, 2015.
21. J. Mattner, S. Lange, and M. Riedmiller, "Learn to swing up and balance a real pole based on raw visual input data", in ICONIP, 2012.
22. C. Finn, X. Y. Tan, Y. Duan, T. Darrell, S. Levine, and P. Abbeel, "Deep spatial autoencoders for visuomotor learning", in ICRA, 2016.
23. J.-A. M. Assael, N. Wahlström, T. B. Schön, and M. P. Deisenroth, "Data-efficient learning of feedback policies from image pixels using deep dynamical models", ArXiv, Tech. Rep., 2015.
24. N. Wahlström, T. B. Schön, and M. P. Deisenroth, "From pixels to torques: Policy learning with deep dynamical models", ArXiv, Tech. Rep. 1502.02251, 2015.
25. S. Lange, M. Riedmiller, and A. Voigtländer, "Autonomous reinforcement learning on raw visual input data in a real world application", in IJCNN, 2012.
26. M. Riedmiller, "Neural fitted Q iteration-first experiences with a data efficient neural reinforcement learning method", in ECML, 2005.
27. V. Mnih, K. Kavukcuoglu, D. Silver, A. A. Rusu, J. Veness, M. G. Bellemare, A. Graves, M. Riedmiller, A. K. Fidjeland, G. Ostrovski, S. Petersen, C. Beattie, A. Sadik, I. Antonoglou, H. King, D. Kumaran, D. Wierstra, S. Legg, and D. Hassabis, "Human-level control through deep reinforcement learning", Nature, vol. 518, no. 7540, 2015.
28. J. Koutník, G. Cuccu, J. Schmidhuber, and F. Gomez, "Evolving largescale neural networks for vision-based reinforcement learning", in Annu. Conf. Genetic and Evolutionary Computation, 2013.
29. T. P. Lillicrap, J. J. Hunt, A. Pritzel, N. Heess, T. Erez, Y. Tassa, D. Silver, and D. Wierstra, "Continuous control with deep reinforcement learning", arXiv, Tech. Rep. 1509.02971, 2015.
30. J. Schulman, S. Levine, P. Moritz, M. Jordan, and P. Abbeel, "Trust region policy optimization", in ICML, 2015.
31. S. Levine, C. Finn, T. Darrell, and P. Abbeel, "End-to-end training of deep visuomotor policies", Journal of Machine Learning Research, vol. 17, no. 39, pp. 1-40, 2016.
32. M. Watter, J. T. Springenberg, J. Boedecker, and M. Riedmiller, "Embed to control: A locally linear latent dynamics model for control from raw images", in NIPS, 2015, pp. 2728-2736.
33. R. G. Krishnan, U. Shalit, and D. Sontag, "Deep Kalman filters", in NIPS Advances in Variational Inference Workshop, 2015.
34. J. Peters, K. Mülling, and Y. Altün, "Relative entropy policy search", in AAAI, 2010, pp. 1607-1612.
35. J. Bagnell and J. Schneider, "Covariant policy search", in International Joint Conference on Artificial Intelligence, 2003.
36. A. Rahimi and B. Recht, "Random features for large-scale kernel machines", in NIPS, 2007, pp. 1177-1184.