A novel Q-learning based adaptive optimal controller implementation for a humanoid robotic arm

IFAC Proceedings Volumes (IFAC-PapersOnline)

Volumn 44, Issue 1 PART 1, 2011, Pages 13528-13533

  Khan, Said G ^a   Herrmann, Guido ^b   Lewis, Frank L ^c   Pipe, Tony ^a   Melhuish, Chris ^a,b  

^a UNIVERSITY OF THE WEST OF ENGLAND   (United Kingdom)

^b UNIVERSITY OF BRISTOL   (United Kingdom)

^c UNIVERSITY OF TEXAS AT ARLINGTON   (United States)

Author keywords

[No Author keywords available]

Indexed keywords

ANTHROPOMORPHIC ROBOTS; CONTROLLERS; COST FUNCTIONS; OPTIMAL CONTROL SYSTEMS; REINFORCEMENT LEARNING; ROBOTIC ARMS; ROBOTICS;

ADAPTIVE DYNAMIC PROGRAMMING; FULL INFORMATIONS; HUMANOID ROBOTICS; JOINT CONSTRAINT; OPTIMAL CONTROL DESIGN; OPTIMAL CONTROLLER; OPTIMAL POLICIES; TRACKING CONTROLLER;

DYNAMIC PROGRAMMING;

EID: 84866760675     PISSN: 14746670     EISSN: None     Source Type: Conference Proceeding    
DOI: 10.3182/20110828-6-IT-1002.02232     Document Type: Conference Paper

References (18)

1. ∞ Constrained Feedback Control: A Practical Approach Using Neural Networks. Springer Verlag, UK, 2006.
2. Murad Abu-Khalaf and Frank L. Lewis. Nearly optimal control laws for nonlinear systems with saturating actuators using a neural network hjb approach. Automatica, 41(5):779-791, 2005. ISSN 0005-1098.
3. Asma Al-Tamimi and Frank Lewis. Model-free q-learning designs for linear discrete-time zero-sum games with application to h1 control. Automatica, 43(3):47-481, 2007.
4. Asma Al-Tamimi, Frank Lewis, and Murad Abu-Khalaf. Discrete-time nonlinear hjb solution using approximate dynamic programming: Convergence proof. IEEE Transaction on System, Man and Cybernetics-Part B: Cybernetics, 38(4):943-949, August 2008.
5. C. G. Atkeson and S. Schaal. learning tasks from a single demonstration. In ieee international conference on robotics and automation (icra97), pages 1706-1712, 1997.
6. Ihsan Omur Bucak and Mohamed Ali Zohdy. Reinforcement learning control of nonlinear multi-link system. Engineering Applications of Artificial Intelligence, 14 (5):563-575, 2001.
7. H. Hoffmann, E. Theodorou, and S. Schaal. behavioral experiments on reinforcement learning in human motor control. In abstracts of the eighteenth annual meeting of neural control of movement, 2008.
8. F.L. Lewis, D.M. Dawson, and C.T. Abdallah. Robot Manipulator Control: Theory and Practice. Marcel Dekker Inc., New York, USA, 2003.
9. Frank L. Lewis and Draguna Vrabie. Reinforcement learning and adaptive dynamic programming for feedback control. Cir. and Sys. Mag., 09(3), 2009.
10. J. Peters and S. Schaal. learning to control in operational space. international journal of robotics research, pages 197-212, 2008a.
11. Jan Peters and Stefan Schaal. Natural actor-critic. Neurocomputing, 71(7-9):1180-1190, 2008b. ISSN 0925-2312. Progress in Modeling, Theory, and Application of Computational Intelligenc - 15th European Symposium on Artificial Neural Networks 2007, 15th European Symposium on Artificial Neural Networks 2007.
12. S. Peters, J. Vijayakumar and S. Schaal. Reinfrocement learning for humanoid robotics. In Third IEEE-RAS International Conference on Humanoid Robots, Karlsruhe, Germany, Sep. 2003.
13. J. Sarangapani. Neural Network Control of Nonlinear Discrete-Time Systems. Taylor & Francis, USA, 2006.
14. P.E Stingu and F.L. Lewis. Adaptive dynamic programming applied to a 6dof quadrotor. In Boris Igelnik, editor, Computational Modeling and Simulation of Intellect: Current State and Future Perspectives. IGI Global, 2010.
15. R. S. Sutton and A. G. Barto. Reinforcement Learning: An Introduction. The MIT Press, Cambridge, Massachusetts London, England, 1998.
16. E Theodorou, J Peters, and S. Schaal. reinforcement learning for optimal control of arm movements. In abstracts of the 37st meeting of the society of neuroscience, 2007.
17. D. Vrabie, O. Pastravanu,M. Abu-Khalaf, and F.L. Lewis. Adaptive optimal control for continuous-time linear systems based on policy iteration. Automatica, 45(2): 477-484, 2009. ISSN 0005-1098.
18. Draguna Vrabie and Frank Lewis. Neural network approach to continuous-time direct adaptive optimal control for partially unknown nonlinear systems. Neural Networks, 22(3):237-246, 2009. ISSN 0893-6080.