Data-based approximate policy iteration for affine nonlinear continuous-time optimal control design

Automatica

Volumn 50, Issue 12, 2014, Pages 3281-3290

  Luo, Biao ^a   Wu, Huai Ning ^b   Huang, Tingwen ^c   Liu, Derong ^a  

^a INSTITUTE OF AUTOMATION   (China)

^b BEIHANG UNIVERSITY   (China)

^c TEXAS A AND M UNIVERSITY AT QATAR   (Qatar)

Author keywords

Data based approximate policy iteration; Hamilton Jacobi Bellman equation; Neural network; Nonlinear optimal control; Off policy; Reinforcement learning

Indexed keywords

CONTINUOUS TIME SYSTEMS; COST FUNCTIONS; DESIGN; DYNAMIC PROGRAMMING; LEAST SQUARES APPROXIMATIONS; NEURAL NETWORKS; NONLINEAR EQUATIONS; OPTIMAL CONTROL SYSTEMS; PARTIAL DIFFERENTIAL EQUATIONS; REINFORCEMENT LEARNING;

APPROXIMATE POLICY ITERATION (API); HAMILTON JACOBI BELLMAN EQUATION; HAMILTON-JACOBI-BELLMAN EQUATIONS; NON-LINEAR OPTIMAL CONTROL; NONLINEAR OPTIMAL CONTROL PROBLEMS; NONLINEAR PARTIAL DIFFERENTIAL EQUATIONS; POLICY ITERATION; REINFORCEMENT LEARNING TECHNIQUES;

ITERATIVE METHODS;

EID: 84919448289     PISSN: 00051098     EISSN: None     Source Type: Journal    
DOI: 10.1016/j.automatica.2014.10.056     Document Type: Article

References (32)

1. Murad Abu-Khalaf, and Frank L. Lewis Nearly optimal control laws for nonlinear systems with saturating actuators using a neural network HJB approach Automatica 41 5 2005 779 791
2. Asma Al-Tamimi, Frank L. Lewis, and Murad Abu-Khalaf Discrete-time nonlinear HJB solution using approximate dynamic programming: convergence proof IEEE Transactions on Systems, Man, and Cybernetics, Part B: Cybernetics 38 4 2008 943 949
3. Randal W. Beard, George N. Saridis, and John T. Wen Galerkin approximations of the generalized Hamilton-Jacobi-Bellman equation Automatica 33 12 1997 2159 2177
4. Dimitri P. Bertsekas Dynamic programming and optimal control, Vol. 1 2005 Athena Scientific Nashua
5. Richard Courant, and David Hilbert Methods of mathematical physics, Vol. 1 2004 Wiley
6. Bruce A. Finlayson The method of weighted residuals and variational principles: with applications in fluid mechanics, heat and mass transfer, Vol. 87 1972 Academic Press, Inc. New York
7. Ali Heydari, and Sivasubramanya N. Balakrishnan Finite-horizon control-constrained nonlinear optimal control using single network adaptive critics IEEE Transactions on Neural Networks and Learning Systems 24 1 2013 147 157
8. David G. Hull Optimal control theory for applications 2003 Springer Troy, NY
9. Yu Jiang, and Z.-P. Jiang Robust adaptive dynamic programming and feedback stabilization of nonlinear systems IEEE Transactions on Neural Networks and Learning Systems 25 5 2014 882 893
10. Yu Jiang, and Zhong-Ping Jiang Computational adaptive optimal control for continuous-time linear systems with completely unknown dynamics Automatica 48 10 2012 2699 2704
11. Zhong-Ping Jiang, and Yu Jiang Robust adaptive dynamic programming for linear and nonlinear systems: an overview European Journal of Control 19 5 2013 417 425
12. David L. Kleinman On an iterative technique for Riccati equation computations IEEE Transactions on Automatic Control 13 1 1968 114 115
13. Jae Young Lee, Jin Bae Park, and Yoon Ho Choi Integral Q-learning and explorized policy iteration for adaptive optimal control of continuous-time linear systems Automatica 48 11 2012 2850 2859
14. Frank L. Lewis, and Derong Liu Reinforcement learning and approximate dynamic programming for feedback control, Vol. 17 2013 John Wiley & Sons, Inc. Hoboken, New Jersey
15. Frank L. Lewis, Draguna Vrabie, and Vassilis L. Syrmos Optimal control 2013 John Wiley & Sons, Inc. Hoboken, New Jersey
16. Frank L. Lewis, Draguna Vrabie, and Kyriakos G. Vamvoudakis Reinforcement learning and feedback control: using natural decision methods to design optimal adaptive controllers IEEE Control Systems 32 6 2012 76 105
17. Qun Lin, Ryan Loxton, and Kok Lay Teo The control parameterization method for nonlinear optimal control: a survey Journal of Industrial and Management Optimization 10 1 2014 275 309
18. Derong Liu, Ding Wang, and Hongliang Li Decentralized stabilization for a class of continuous-time nonlinear interconnected systems using online learning optimal control approach IEEE Transactions on Neural Networks and Learning Systems 25 2 2014 418 428
19. ∞ control design IEEE Transactions on Cybernetics 2014 10.1109/TCYB. 2014.2319577 in press
20. Biao Luo, H.-N. Wu, and H.-X. Li Adaptive optimal control of highly dissipative nonlinear spatially distributed processes with neuro-dynamic programming IEEE Transactions on Neural Networks and Learning Systems 2014 10.1109/TNNLS.2014.2320744 in press
21. Biao Luo, Huai-Ning Wu, and Han-Xiong Li Data-based suboptimal neuro-control design with reinforcement learning for dissipative spatially distributed processes Industrial & Engineering Chemistry Research 53 29 2014 8106 8119
22. Vesna Nevistić, and James A. Primbs Optimality of nonlinear design techniques: a converse HJB approach. Technical report TR96-022 1996 California Institute of Technology
23. G. Peter Lepage A new algorithm for adaptive multidimensional integration Journal of Computational Physics 27 2 1978 192 203
24. Precup, Doina, Sutton, Richard S., & Dasgupta, Sanjoy (2001). Off-policy temporal-difference learning with function approximation. In Proceedings of the 18th international conference on machine learning (pp. 417-424).
25. George N. Saridis, and Chun-Sing G. Lee An approximation theory of optimal control for trainable manipulators IEEE Transactions on Systems, Man and Cybernetics 9 3 1979 152 159
26. Jennie Si, and Yu-Tsung Wang Online learning control by association and reinforcement IEEE Transactions on Neural Networks 12 2 2001 264 276
27. Richard S. Sutton, and Andrew G. Barto Reinforcement learning: an introduction 1998 Cambridge Univ Press Massachusetts London, England
28. Kyriakos G. Vamvoudakis, and Frank L. Lewis Online actor-critic algorithm to solve the continuous-time infinite horizon optimal control problem Automatica 46 5 2010 878 888
29. Draguna Vrabie, and Frank L. Lewis Neural network approach to continuous-time direct adaptive optimal control for partially unknown nonlinear systems Neural Networks 22 3 2009 237 246
30. Ling Yun Wang, Wei Hua Gui, Kok Lay Teo, Ryan C. Loxton, and Chun Hua Yang Time delayed optimal control problems with multiple characteristic time points: computation and industrial applications Journal of Industrial and Management Optimization 5 4 2009 705 718
31. Ding Wang, Derong Liu, Qinglai Wei, Dongbin Zhao, and Ning Jin Optimal control of unknown nonaffine nonlinear discrete-time systems based on adaptive dynamic programming Automatica 48 8 2012 1825 1832
32. Huaguang Zhang, Lili Cui, Xin Zhang, and Yanhong Luo Data-driven robust approximate optimal tracking control for unknown general nonlinear systems using adaptive dynamic programming method IEEE Transactions on Neural Networks 22 12 2011 2226 2236