Acquiring visual servoing reaching and grasping skills using neural reinforcement learning

Proceedings of the International Joint Conference on Neural Networks

Volumn , Issue , 2013, Pages

  Lampe, Thomas ^a   Riedmiller, Martin ^a  

^a UNIVERSITY OF FREIBURG   (Germany)

Author keywords

[No Author keywords available]

Indexed keywords

INTEROPERATIONS; PRIOR KNOWLEDGE; ROBOTIC PLATFORMS; VISUAL FEEDBACK CONTROL;

REINFORCEMENT LEARNING; ROBUST CONTROL; VISUAL COMMUNICATION; VISUAL SERVOING;

NEURAL NETWORKS;

EID: 84893561637     PISSN: None     EISSN: None     Source Type: Conference Proceeding    
DOI: 10.1109/IJCNN.2013.6707053     Document Type: Conference Paper

References (37)

1. O. Kromer, R. Detry, J. Piater, and J. Peters, "Combining active learning and reactive control for robot grasping, " Robotics and Autonomous Systems, vol. 58, no. 9, pp. 1105-1116, 2010.
2. M. Kazemi, J.S. Valois, J. A. D. Bagnell, and N. Pollard, "Robust object grasping using force compliant motion primitives, " in Robotics: Science and Systems, July 2012.
3. P. Pastor, L. Righetti, M. Kalakrishnan, and S. Schaal, "Online movement adaptation based on previous sensor experiences, " in IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), 2011.
4. J. Steffen, R. Haschke, and H. Ritter, "Experiencebased and tactiledriven dynamic grasp control, " in IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), November 2007, pp. 2938- 2943.
5. K. Hsiao, P. Nangeroni, M. Huber, A. Saxena, and A. Y. Ng, "Reactive grasping using optical proximity sensors, " in IEEE International Conference on Robotics and Automation (ICRA), May 2009, pp. 2098- 2105.
6. N. Ratliff, M. Zucker, J. A. D. Bagnell, and S. Srinivasa, "CHOMP: Gradient optimization techniques for efficient motion planning, " in IEEE International Conference on Robotics and Automation (ICRA), May 2009.
7. P. Pastor, M. Kalakrishnan, S. Chitta, E. Theodorou, and S. Schaal, "Skill learning and task outcome prediction for manipulation, " in IEEE International Conference on Robotics and Automation (ICRA), 2011.
8. K. Hashimoto, Ed., Visual Servoing: RealTime Control of Robot Manipulators Based on Visual Sensory Feedback. World Scientific, October 1993.
9. J. Piater, S. Jodogne, R. Detry, D. Kraft, N. Kruger, O. Kromer, and J. Peters, "Learning visual representations for perceptionaction systems, " Int. J. Rob. Res., vol. 30, no. 3, pp. 294-307, March 2011.
10. P. K. Allen, B. Yoshimi, and A. Timcenko, "Realtime visual servoing, " in IEEE International Conference on Robotics and Automation (ICRA), 1991, pp. 851-856.
11. A. Namiki and M. Ishikawa, "Optimal grasping using visual and tactile feedback, " in IEEE/SICE/RSJ Inernational Conference on Multisensor Fusion and Integration for Intelligent Systems, Whashington, DC, 1996, pp. 589-596.
12. W. Hong and J.J. E. Slotine, "Experiments in handeye coordination using active vision, " in 4th International Symposium on Experimental Robotics. London, UK: SpringerVerlag, 1997, pp. 130-139.
13. F. Chaumette and S. Hutchinson, "Visual servo control, part I: Basic approaches, " IEEE Robotics Automation Magazine, vol. 13, no. 4, pp. 82-90, December 2006.
14. S. Schaal, "Is imitation learning the route to humanoid robots?" Trends in Cognitive Sciences, vol. 3, no. 6, pp. 233-242, 1999.
15. M. Riedmiller, "Neural fitted Q iteration-first experiences with a data efficient neural reinforcement learning method, " in 16th European Conference on Machine Learning. Springer, 2005, pp. 317-328.
16. R. Hafner and M. Riedmiller, "Neural reinforcement learning controllers for a real robot application, " in IEEE International Conference on Robotics and Automation (ICRA), 2007, pp. 2098-2103.
17. T. C. Kietzmann and M. Riedmiller, "The neuro slot car racer: Reinforcement learning in a real world setting, " in IEEE International Conference on Machine Learning and Applications (ICMLA). Washington, DC, USA: IEEE Computer Society, 2009, pp. 311-316.
18. B. E. Inria and B. Espiau, "Effect of camera calibration errors on visual servoing in robotics, " in 3rd International Symposium on Experimental Robotics, 1993, pp. 187-193.
19. J. A. Piepmeier, G. V. McMurray, and H. Lipkin, "Uncalibrated dynamic visual servoing, " IEEE Transactions on Robotics, vol. 20, no. 1, pp. 143-147, 2004.
20. A. Shademan, A.m. Farahmand, and M. Jaagersand, "Robust Jacobian estimation for uncalibrated visual servoing, " in IEEE International Conference on Robotics and Automation (ICRA), May 2010, pp. 5564- 5569.
21. F. Stulp, E. Theodorou, J. Buchli, and S. Schaal, "Learning to grasp under uncertainty, " in International Conference on Robotics and Automation (ICRA), 2011.
22. N. Ratliff, J. A. D. Bagnell, and S. Srinivasa, "Imitation learning for locomotion and manipulation, " in IEEERAS International Conference on Humanoid Robots, November 2007.
23. F. Stulp, E. Theodorou, M. Kalakrishnan, P. Pastor, L. Righetti, and S. Schaal, "Learning motion primitive goals for robust manipulation, " in IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), September 2011, pp. 325-331.
24. D.J. Kim, R. Lovelett, and A. Behal, "Eyeinhand stereo visual servoing of an assistive robot arm in unstructured environments, " in IEEE International Conference on Robotics and Automation (ICRA), May 2009, pp. 2326-2331.
25. M. Riedmiller, "High quality thermostat control by reinforcement learning-a case study, " in Conald Workshop, CarnegieMellon University, 1998.
26. R. Detry, D. Kraft, O. Kromer, L. Bodenhagen, J. Peters, N. Kruger, and J. Piater, "Learning grasp affordance densities, " Paladyn Journal of Behavioral Robotics, no. 2(1), pp. 117, 2011, intelligent Autonomous Systems.
27. A. Morales, P. J. Sanz, A. P. del Pobil, and A. H. Fagg, "Visionbased threefinger grasp synthesis constrained by hand geometry, " Robotics and Autonomous Systems, vol. 54, no. 6, pp. 496-512, March 2006.
28. D. Aarno, J. Sommerfeld, D. Kragic, S. Kalkan, F. Worgotter, N. Pugeault, D. Kraft, and N. Kruger, "Early reactive grasping with second order 3D feature relations, " in IEEE International Conference on Advanced Robotics, 2007.
29. P. Kulchenko and E. Todorov, "Firstexit model predictive control of fast discontinuous dynamics: Application to ball bouncing, " in IEEE International Conference on Robotics and Automation (ICRA), 2011.
30. M. Morari and J. H. Lee, "Model predictive control: Past, present and future, " Computers and Chemical Engineering, vol. 23, pp. 667-682, 1997.
31. P. Kulchenko and E. Todorov, "Policy gradient methods with model predictive control applied to ball bouncing, " in IEEE Adaptive Dy namic Programming and Reinforcement Learning, 2011.
32. G. Gordon, "Stable function approximation in dynamic programming, " in Proceedings of the 12th International Conference on Machine Learning (ICML), A. Prieditis and S. Russell, Eds., San Francisco, CA, 1995.
33. D. E. Rumelhart and J. McClelland, Parallel Distributed Processing, 1986.
34. M. Riedmiller and H. Braun, "A direct adaptive method for faster backpropagation learning: The RPROP algorithm, " in IEEE Interna tional Conference on Neural Networks, H. Ruspini, Ed., vol. 1, San Francisco, CA, 1993, pp. 586-591.
35. M. Riedmiller, "10 steps and some tricks to set up neural reinforcement controllers, " in Neural Networks: Tricks of the Trade, ser. Lecture Notes in Computer Science, G. Montavon, G. Orr, and K.R. Muller, Eds., 2012, vol. 7700.
36. The ICRA 2012 mobile manipulation challenge. [Online]. Available: http://mobilemanipulationchallenge.org/[Accessed: 09/11/2012].
37. R. A. Brooks, "Intelligence without reason, " in 12th International Joint Conference on Artificial Intelligence, R. Myopoulos, John; Reiter, Ed. Sydney, Australia: Morgan Kaufmann, August 1991, pp. 569-595.