Optimal tracking control of nonlinear partially-unknown constrained-input systems using integral reinforcement learning

Automatica

Volumn 50, Issue 7, 2014, Pages 1780-1792

  Modares, Hamidreza ^a   Lewis, Frank L ^a  

^a UNIVERSITY OF TEXAS AT ARLINGTON   (United States)

Author keywords

Input constrainers; Integral reinforcement learning; Neural networks; Optimal tracking control

Indexed keywords

CONTROL; NAVIGATION; NEURAL NETWORKS; OPTIMIZATION;

CONTINUOUS TIME NONLINEAR SYSTEMS; INPUT CONSTRAINERS; NEAR-OPTIMAL CONTROL; OPTIMAL CONTROL SOLUTION; OPTIMAL TRACKING CONTROL; OPTIMIZATION PROBLEMS; PERFORMANCE FUNCTIONS; PERSISTENCE OF EXCITATION;

REINFORCEMENT LEARNING;

EID: 84904739156     PISSN: 00051098     EISSN: None     Source Type: Journal    
DOI: 10.1016/j.automatica.2014.05.011     Document Type: Article

References (33)

1. M. Abou-Khalaf, and F.L. Lewis Nearly optimal control laws for nonlinear systems with saturating actuators using a neural network HJB approach Automatica 41 2005 779 791
2. D.P. Bertsekas, and J.N. Tsitsiklis Neuro-dynamic programming 1996 Athena Scientific MA
3. S. Bhasin, R. Kamalapurkar, M. Johnson, K.G Vamvoudakis, F.L Lewis, and W.E. Dixon A novel actor-critic-identifier architecture for approximate optimal control of uncertain nonlinear systems Automatica 49 2012 82 92
4. Dierks, T.; Jagannathan, S. (2009). Optimal tracking control of affine nonlinear discrete-time systems with unknown internal dynamics. In Joint 48th IEEE conference on decision and control and 28th Chinese control conference Shanghai, PR China (pp. 6750-6755).
5. Dierks, T.; Jagannathan, S. (2010). Optimal control of affine nonlinear continuous-time systems. In Proc. Am. control conf. (pp. 1568-1573).
6. B.A. Finlayson The method of weighted residuals and variational principles 1990 Academic Press New York
7. K. Hornik, M. Stinchcombe, and H. White Universal approximation of an unknown mapping and its derivatives using multilayer feedforward networks Neural Networks 3 1990 551 560
8. R.A. Howard Dynamic programming and markov processes 1960 MIT Press Cambridge, MA
9. Y. Jiang, and Z.P. Jiang Computational adaptive optimal control for continuous-time linear systems with completely unknown dynamics Automatica 48 2012 2699 2704
10. B. Kiumarsi, F.L. Lewis, H. Modares, A. Karimpour, and M.B. Naghibi-Sistani Reinforcement Q-learning for optimal tracking control of linear discrete-time systems with unknown dynamics Automatica 50 2014 1167 1175
11. J.Y. Lee, J.B. Park, and Y.H. Choi Integral Q-learning and explorized policy iteration for adaptive optimal control of continuous-time linear systems Automatica 48 2012 2850 2859
12. F.L. Lewis, and D. Liu Reinforcement learning and approximate dynamic programming for feedback control 2012 Wiley New York
13. F.L. Lewis, D. Vrabie, and V. Syrmos Optimal control 3rd ed. 2012 Wiley
14. D. Liu, and Q. Wei Finite-approximation-error-based optimal control approach for discretetime nonlinear systems IEEE Transactions on Cybernetics 43 2013 779 789
15. D. Liu, X. Yang, and H. Li Adaptive optimal control for a class of continuous-time affine nonlinear systems with unknown internal dynamics Neural Computing and Applications 2013 10.1007/s00521-012-1249-y
16. Lyshevski, S.E. (1998). Optimal control of nonlinear continuous-time systems: design of bounded controllers via generalized nonquadratic functionals. In Proceedings of American control conference (pp. 205-209).
17. H. Modares, M.B. Naghibi-Sistani, and F.L. Lewis A policy iteration approach to online optimal control of continuous- time constrained-input systems ISA Transactions 52 2013 611 621
18. H. Modares, M.B. Naghibi-Sistani, and F.L. Lewis Integral reinforcement learning and experience replay for adaptive optimal control of partially-unknown constrained-input continuous-time systems Automatica 50 2014 193 202
19. W.B. Powell Approximate dynamic programming: solving the curses of dimensionality 2007 Wiley-Interscience
20. S. Sastry Nonlinear systems: analysis, stability, and control 1991 Springer-Verlag
21. J.E. Slotine, and W. Li Applied nonlinear control 1991 Prentice Hall Englewood Cliffs, NJ
22. R.S. Sutton, and A.G. Barto Reinforcement learning - an introduction 1998 MIT Press Cambridge, MA
23. K. Vamvoudakis, and F.L. Lewis Online actor-critic algorithm to solve the continuous infinite-time horizon optimal control problem Automatica 46 2010 878 888
24. K. Vamvoudakis, D. Vrabie, and F.L. Lewis Online adaptive algorithm for optimal control with integral reinforcement learning International Journal of Robust and Nonlinear Control 2013 10.1002/rnc in press
25. D. Vrabie, and F.L. Lewis Neural network approach to continuous-time direct adaptive optimal control for partially unknown nonlinear systems Neural Networks 22 2009 237 246
26. D. Vrabie, O. Pastravanu, M. Abou-Khalaf, and F.L. Lewis Adaptive optimal control for continuous-time linear systems based on policy iteration Automatica 45 2009 477 484
27. D. Vrabie, K.G. Vamvoudakis, and F.L. Lewis Optimal adaptive control and differential games by reinforcement learning principles 2013 Institution of Engineering and Technology London
28. D. Wang, D. Liu, and Q. Wei Finite-horizon neuro-optimal tracking control for a class of discrete-time nonlinear systems using adaptive dynamic programming approach Neurocomputing 78 2012 14 22
29. Werbos, P.J. (1989). Neural networks for control and system identification. In Proc. IEEE conf. of decision control, Tampa, FL (pp. 260-265).
30. P.J. Werbos Approximate dynamic programming for real time control and neural modeling D.A. White, D.A. Sofge, Handbook of intelligent control 1992 Multiscience Press
31. H. Zhang, L. Cui, X. Zhang, and X. Luo Data-driven robust approximate optimal tracking control for unknown general nonlinear systems using adaptive dynamic programming method IEEE Transactions on Neural Networks 22 2011 2226 2236
32. H. Zhang, D. Liu, Y. Luo, and D. Wang Adaptive dynamic programming for control. algorithms and stability 2012 Springer-Verlag London
33. H. Zhang, Q. Wei, and Y. Luo A novel infinite-time optimal tracking control scheme for a class of discrete-time nonlinear systems via the greedy HDP iteration algorithm IEEE Transactions on Systems, Man and Cybernetics, Part B: Cybernetics 38 2008 937 942