Neural network Reinforcement Learning for visual control of robot manipulators

Expert Systems with Applications

Volumn 40, Issue 5, 2013, Pages 1721-1736

  Miljković, Zoran ^a   Mitić, Marko ^a   Lazarević, Mihailo ^a   Babić, Bojan ^a  

^a UNIVERSITY OF BELGRADE   (Serbia)

Author keywords

Image Based Visual Servo control; Intelligent hybrid control; Neural network; Reinforcement Learning; Robot manipulator

Indexed keywords

CALIBRATION ERROR; CENTERING BEHAVIOR; IMAGE BASED VISUAL SERVO CONTROL; IMAGE BASED VISUAL SERVOING; IMAGE FEATURES; IMAGE NOISE; INTELLIGENT HYBRID CONTROL; LEARNING CONTROLLERS; LEARNING SAMPLES; LOW COSTS; MACHINE LEARNING TECHNIQUES; MODELING ERRORS; NETWORK REINFORCEMENTS; PERFORMANCE ANALYSIS; PERFORMANCE EVALUATION; Q-LEARNING; REAL WORLD EXPERIMENT; REAL-TIME LEARNING; ROBOT BEHAVIOR; ROBOT MANIPULATOR; ROBOT MOTION; SWITCHING CONTROL; TEMPORAL-DIFFERENCE ALGORITHM; VISION BASED CONTROL; VISION SYSTEMS; VISUAL CONTROL; VISUAL SERVO; VISUAL SERVO CONTROL;

COMPUTER VISION; CONTROLLERS; FLEXIBLE MANIPULATORS; HUMAN COMPUTER INTERACTION; INDUSTRIAL ROBOTS; LEARNING ALGORITHMS; MEASUREMENT ERRORS; MODULAR ROBOTS; NEURAL NETWORKS; PNEUMATIC CONTROL EQUIPMENT; ROBOT APPLICATIONS; VISUAL SERVOING;

REINFORCEMENT LEARNING;

EID: 84872011415     PISSN: 09574174     EISSN: None     Source Type: Journal    
DOI: 10.1016/j.eswa.2012.09.010     Document Type: Article

References (44)

1. E. Alpaydin Introduction to machine learning 2004 MIT Press
2. M. Asada, S. Noda, S. Tawaratsumida, and K. Hosoda Purposive behavior acquisition on a real robot by vision-based reinforcement learning Machine Learning 23 1996 279 303 (Pubitemid 126724375)
3. Baird, L. (1995). Residual algorithms: Reinforcement learning with function approximation. In Machine learning: 12th international conference, San Francisco.
4. R. Basri, E. Rivlin, and I. Shimshoni Visual homing: Surfing on the epipoles International Journal of Computer Vision 33 2 1999 117 137 (Pubitemid 30532483)
5. H. Bay, A. Ess, T. Tuytelaars, and L.V. Gool SURF: Speeded up robust features Computer Vision & Image Understanding 110 3 2008 346 359
6. Busquets, D., de Mantaras, R. L., Sierra, C., & Ditterich, T. G. (2002). Reinforcement learning for landmark-based robot navigation. In Proceedings of the international joint conference on autonomous agents and multiagent systems (AAMAS 2002) (pp. 841-843).
7. Z. Ceren, and E. Altuǧ Image based and hybrid visual servo control of an unmanned aerial vehicle Journal of Intelligent & Robotic Systems 65 1-4 2012 325 344
8. F. Chaumette, and S. Hutchinson Visual servo control. Part I: Basic approaches IEEE Robotics and Automation Magazine 13 4 2006 82 90 (Pubitemid 46454544)
9. G. Chesi, K. Hashimoto, D. Prattichizzo, and A. Vicino Keeping features in the field of view in eye-in-hand visual servoing: A switching approach IEEE Transactions on Robotics 20 5 2004 908 913
10. G. Chesi, and Y.S. Hung Global path-planning for constrained and optimal visual servoing IEEE Transactions on Robotics 23 5 2007 1050 1060 (Pubitemid 47577980)
11. P.I. Corke, and S.A. Hutchinson A new partitioned approach to Image Based Visual servo control IEEE Transactions on Robotics and Automation 17 4 2001 507 515 (Pubitemid 33048337)
12. Cowan, N. J., & Koditschek, D. E. (1999). Planar image based visual servoing as a navigation problem. In Proceedings of the 1999 IEEE international conference on robotics and automation (ICRA 1999) (pp. 611-617).
13. C. Distante, A. Anglani, and F. Taurisano Target reaching by using visual information and Q-learning controllers Autonomous Robots 9 2000 41 50
14. B. Espiau, F. Chaumette, and P. Rives A new approach to visual servoing in robotics IEEE Transactions on Robotics and Automation 8 3 1992 313 326
15. R.T. Fomena, T. Omar, and F. Chaumette Distance-based and orientation-based visual servoing from three points IEEE Transactions on Robotics 27 2 2011 266 273
16. N. Garcia-Aracil, C. Perez-Vidal, J.M. Sabater, R. Morales, and F.J. Badesa Robust and cooperative Image Based Visual servoing system using a redundant architecture Sensors 11 12 2011 11885 11900
17. Gaskett, C. (2002). Q-Learning for robot control. PhD thesis, The Australian National University.
18. Gaskett, C., Fletcher, L., & Zelinsky, A. (2000). Reinforcement learning for a vision based mobile robot. In Proceedings of the 2000 IEEE/RSJ international conference on intelligent robots and systems (IROS 2000) (pp. 403-409). (Pubitemid 32380242)
19. Hafner, R., & Riedmiller, M. (2003). Reinforcement learning on an omnidirectional mobile robot. In Proceedings of the 2003 IEEE/RSJ international conference on intelligent robots and systems (IROS 2003) (pp. 418-423).
20. T. Hamel, and R. Mahony Visual servoing of an under-Actuated dynamic rigid-body system: An Image Based approach IEEE Transactions on Robotics and Automation 18 2 2002 187 198 (Pubitemid 34659022)
21. S. Hutchinson, G. Hager, and P. Corke A tutorial on visual servo control IEEE Transactions on Robotics and Automation 12 1996 651 670 (Pubitemid 126786243)
22. C. Jiang, and Z. Sheng Case-based reinforcement learning for dynamic inventory control in a multi-Agent supply-chain system Expert Systems with Applications 36 2009 6520 6526
23. S.G. Khan, G. Herrmann, F.L. Lewis, T. Pipe, and C. Melhuish Reinforcement learning and optimal adaptive control: An overview and implementation examples Annual Reviews in Control 36 2012 42 59
24. Kragic, D., & Christensen, H. I. (2002). Survey on visual servoing for manipulation. Technical report ISRN KTH/NA/P-02/01-SE, Royal Institute of Technology, Stockholm.
25. L. Lin, H. Xie, D. Zhang, and L. Shen Supervised neural Q-learning based motion control for bionic underwater robots Journal of Bionic Engineering 7 Suppl 2010 S177 S184
26. Y. Ma, J. Kosecka, and S.S. Sastry Vision guided navigation for a nonholonomic mobile robot IEEE Transactions on Robotics and Automation 15 3 1999 521 536
27. Malis, E. (2004). Improving vision-based control using efficient second-order minimization techniques. In Proceedings of the 2004 IEEE international conference on robotics and automation (ICRA 2004) (pp. 1843-1848).
28. G.L. Mariottini, G. Oriolo, and D. Prattichizzo Image Based Visual servoing for nonholonomic mobile robots using epipolar geometry IEEE Transactions on Robotics 23 1 2007 87 100 (Pubitemid 46443344)
29. Martínez-Marín, T., & Duckett, T. (2005). Fast reinforcement learning for vision-guided mobile robots. In Proceedings of the 2005 IEEE international conference on robotics and automation (ICRA 2005) (pp. 4170-4175). (Pubitemid 46084435)
30. Y. Mezouar, and F. Chaumette Path planning for robust Image Based control IEEE Transactions on Robotics and Automation 18 4 2002 534 549 (Pubitemid 35372517)
31. Miljković, Z., & Aleksendrić, D. (2009). Artificial neural networks-Solved examples with theoretical background (in Serbian). University of Belgrade, Faculty of Mechanical Engineering, Belgrade.
32. Miljković, Z., Vuković, N., Mitić, M., & Babić, B. (in press). New Hybrid vision-based control approach for automated guided vehicles. The International Journal of Advanced Manufacturing Technology. http://dx.doi.org/10.1007/s00170-012-4321-y.
33. M. Mitić, Z. Miljković, and B. Babić Empirical control system development for intelligent mobile robot based on the elements of the reinforcement machine learning and axiomatic design theory FME Transactions 39 1 2011 1 8
34. Perez, M. C. (2003). A proposal of a behavior based control architecture with reinforcement learning for an autonomous underwater robot. PhD thesis, University of Girona.
35. A. Remazeilles, and F. Chaumette Image Based robot navigation from an image memory Robotics and Autonomous Systems 55 4 2007 345 356 (Pubitemid 46413710)
36. F. Sahba, H.R. Tizhoosh, and M.M.M.A. Salama A reinforcement agent for object segmentation in ultrasound images Expert Systems with Applications 35 2008 772 780
37. F. Schramm, and G. Morel Ensuring visibility in calibration-free path planning for Image Based Visual servoing IEEE Transactions on Robotics 22 4 2006 848 854 (Pubitemid 44263870)
38. M. Shin, K. Ryu, and M. Jung Reinforcement learning approach to goal-regulation in a self-evolutionary manufacturing system Expert Systems with Applications 39 2012 8736 8743
39. K.-T. Song, and T.-S. Chu Reinforcement learning and its application to force control of an industrial robot Control Engineering Practice 6 1 1998 37 44 (Pubitemid 28230151)
40. R.S. Sutton, and A.G. Barto Reinforcement learning: An introduction 1998 MIT Press Cambridge
41. Takahashi, Y., Takeda, M., & Asada, M. (1999). Continuous valued Q-learning for vision-guided behaviour acquisition. In Proceedings of the 1999 IEEE international conference on multisensor fusion and integration for intelligent systems (pp. 255-260). (Pubitemid 30525256)
42. Y. Wang, H.X. Lang, and C.W. de Silva A hybrid visual servo controller for robust grasping by wheeled mobile robots IEEE/ASME Transactions on Mechatronics 15 5 2010 757 769
43. Watkins, C. (1989). Learning from delayed rewards. PhD thesis, King's College, Cambridge.
44. C. Watkins, and P. Dayan Technical note: Q-learning Machine Learning 8 3-4 1992 279 292