Data-Based Adaptive Critic Designs for Nonlinear Robust Optimal Control with Uncertain Dynamics

IEEE Transactions on Systems, Man, and Cybernetics: Systems

Volumn 46, Issue 11, 2016, Pages 1544-1555

  Wang, Ding ^a   Liu, Derong ^b   Zhang, Qichao ^a   Zhao, Dongbin ^a  

^a INSTITUTE OF AUTOMATION   (China)

^b UNIVERSITY OF SCIENCE AND TECHNOLOGY BEIJING   (China)

Author keywords

Adaptive critic designs; adaptive dynamic programming; intelligent control; neural networks; policy iteration; robust optimal control; system identification; uncertain nonlinear systems

Indexed keywords

CLOSED LOOP SYSTEMS; CONTINUOUS TIME SYSTEMS; CONTROLLERS; COST FUNCTIONS; DYNAMIC PROGRAMMING; IDENTIFICATION (CONTROL SYSTEMS); INTELLIGENT CONTROL; ITERATIVE METHODS; NEURAL NETWORKS; NONLINEAR SYSTEMS; OPTIMAL CONTROL SYSTEMS;

ADAPTIVE CRITIC DESIGNS; ADAPTIVE DYNAMIC PROGRAMMING; POLICY ITERATION; ROBUST OPTIMAL CONTROL; UNCERTAIN NONLINEAR SYSTEMS;

ADAPTIVE CONTROL SYSTEMS;

EID: 84992727041     PISSN: 21682216     EISSN: 21682232     Source Type: Journal    
DOI: 10.1109/TSMC.2015.2492941     Document Type: Article

References (50)

1. S. Hussain, S. Q. Xie, and P. K. Jamwal, "Robust nonlinear control of an intrinsically compliant robotic gait tranning orthosis, " IEEE Trans. Syst., Man, Cybern. Syst., vol. 43, no. 3, pp. 655-665, May 2013.
2. H. Gao, X. Meng, and T. Chen, "A new design of robust H2 filters for uncertain systems, " Syst. Control Lett., vol. 57, no. 7, pp. 585-593, Jul. 2008.
3. Z. Wang and F. T. S. Chan, "A robust replenishment and production control policy for a single-stage production/inventory system with inventory inaccuracy, " IEEE Trans. Syst., Man, Cybern. Syst., vol. 45, no. 2, pp. 326-337, Feb. 2015.
4. F. Lin, R. D. Brandt, and J. Sun, "Robust control of nonlinear systems: Compensating for uncertainty, " Int. J. Control, vol. 56, no. 6, pp. 1453-1459, 1992.
5. F. Lin and R. D. Brandt, "An optimal control approach to robust control of robot manipulators, " IEEE Trans. Robot. Autom., vol. 14, no. 1, pp. 69-77, Feb. 1998.
6. D. Wang, D. Liu, and H. Li, "Policy iteration algorithm for online design of robust control for a class of continuous-time nonlinear systems, " IEEE Trans. Autom. Sci. Eng., vol. 11, no. 2, pp. 627-632, Apr. 2014.
7. P. J. Werbos, "Approximate dynamic programming for real-time control and neural modeling, " in Handbook of Intelligent Control: Neural, Fuzzy, and Adaptive Approaches, D. A. White and D. A. Sofge, Eds. New York, NY, USA: Van Nostrand Reinhold, 1992, ch. 13.
8. F. L. Lewis and D. Liu, Reinforcement Learning and Approximate Dynamic Programming for Feedback Control. Hoboken, NJ, USA: Wiley, 2013.
9. H. Zhang, D. Liu, Y. Luo, and D. Wang, Adaptive Dynamic Programming for Control: Algorithms and Stability. London, U. K. : Springer, 2013.
10. F. L. Lewis and D. Vrabie, "Reinforcement learning and adaptive dynamic programming for feedback control, " IEEE Circuits Syst. Mag., vol. 9, no. 3, pp. 32-50, Sep. 2009.
11. A. Al-Tamimi, F. L. Lewis, and M. Abu-Khalaf, "Discrete-time nonlinear HJB solution using approximate dynamic programming: Convergence proof, " IEEE Trans. Syst., Man, Cybern. B, Cybern., vol. 38, no. 4, pp. 943-949, Aug. 2008.
12. M. Abu-Khalaf and F. L. Lewis, "Nearly optimal control laws for nonlinear systems with saturating actuators using a neural network HJB approach, " Automatica, vol. 41, no. 5, pp. 779-791, May 2005.
13. K. G. Vamvoudakis and F. L. Lewis, "Online actor-critic algorithm to solve the continuous-time infinite horizon optimal control problem, " Automatica, vol. 46, no. 5, pp. 878-888, May 2010.
14. S. Bhasin et al., "A novel actor-critic-identifier architecture for approximate optimal control of uncertain nonlinear systems, " Automatica, vol. 49, no. 1, pp. 82-92, Jan. 2013.
15. H. Modares, M.-B. Naghibi-Sistani, and F. L. Lewis, "A policy iteration approach to online optimal control of continuous-time constrained-input systems, " ISA Trans., vol. 52, no. 5, pp. 611-621, Sep. 2013.
16. H. Modares, F. L. Lewis, and M.-B. Naghibi-Sistani, "Adaptive optimal control of unknown constrained-input systems using policy iteration and neural networks, " IEEE Trans. Neural Netw. Learn. Syst., vol. 24, no. 10, pp. 1513-1525, Oct. 2013.
17. D. P. Bertsekas, M. L. Homer, D. A. Logan, S. D. Patek, and N. R. Sandell, "Missile defense and interceptor allocation by neurodynamic programming, " IEEE Trans. Syst., Man, Cybern. A, Syst., Humans, vol. 30, no. 1, pp. 42-51, Jan. 2000.
18. D. V. Prokhorov and D. C. Wunsch, "Adaptive critic designs, " IEEE Trans. Neural Netw., vol. 8, no. 5, pp. 997-1007, Sep. 1997.
19. J. Fu, H. He, and X. Zhou, "Adaptive learning and control for MIMO system based on adaptive dynamic programming, " IEEE Trans. Neural Netw., vol. 22, no. 7, pp. 1133-1148, Jul. 2011.
20. Z. Ni, H. He, and J. Wen, "Adaptive learning in tracking control based on the dual critic network design, " IEEE Trans. Neural Netw. Learn. Syst., vol. 24, no. 6, pp. 913-928, Jun. 2013.
21. X. Xu, Z. Hou, C. Lian, and H. He, "Online learning control using adaptive critic designs with sparse kernel machines, " IEEE Trans. Neural Netw. Learn. Syst., vol. 24, no. 5, pp. 762-775, May 2013.
22. Y. Luo, Q. Sun, H. Zhang, and L. Cui, "Adaptive critic design-based robust neural network control for nonlinear distributed parameter systems with unknown dynamics, " Neurocomputing, vol. 148, pp. 200-208, Jan. 2015.
23. D. Wang and D. Liu, "Neuro-optimal control for a class of unknown nonlinear dynamic systems using SN-DHP technique, " Neurocomputing, vol. 121, pp. 218-225, Dec. 2013.
24. M. Palanisamy, H. Modares, F. L. Lewis, and M. Aurangzeb, "Continuous-time Q-learning for infinite-horizon discounted cost linear quadratic regulator problems, " IEEE Trans. Cybern., vol. 45, no. 2, pp. 165-176, Feb. 2015.
25. D. Wang, D. Liu, Q. Wei, D. Zhao, and N. Jin, "Optimal control of unknown nonaffine nonlinear discrete-time systems based on adaptive dynamic programming, " Automatica, vol. 48, no. 8, pp. 1825-1832, Aug. 2012.
26. D. Liu, D. Wang, and H. Li, "Decentralized stabilization for a class of continuous-time nonlinear interconnected systems using online learning optimal control approach, " IEEE Trans. Neural Netw. Learn. Syst., vol. 25, no. 2, pp. 418-428, Feb. 2014.
27. D. Liu, H. Li, and D. Wang, "Online synchronous approximate optimal learning algorithm for multi-player non-zero-sum games with unknown dynamics, " IEEE Trans. Syst., Man, Cybern. Syst., vol. 44, no. 8, pp. 1015-1027, Aug. 2014.
28. T. Dierks and S. Jagannathan, "Online optimal control of affine nonlinear discrete-time systems with unknown internal dynamics by using timebased policy update, " IEEE Trans. Neural Netw. Learn. Syst., vol. 23, no. 7, pp. 1118-1129, Jul. 2012.
29. D. Nodland, H. Zargarzadeh, and S. Jagannathan, "Neural networkbased optimal adaptive output feedback control of a helicopter UAV, " IEEE Trans. Neural Netw. Learn. Syst., vol. 24, no. 7, pp. 1061-1073, Jul. 2013.
30. A. Heydari and S. N. Balakrishnan, "Finite-horizon control-constrained nonlinear optimal control using single network adaptive critics, " IEEE Trans. Neural Netw. Learn. Syst., vol. 24, no. 1, pp. 145-157, Jan. 2013.
31. J. Liang, G. K. Venayagamoorthy, and R. G. Harley, "Wide-area measurement based dynamic stochastic optimal power flow control for smart grids with high variability and uncertainty, " IEEE Trans. Smart Grid, vol. 3, no. 1, pp. 59-69, Mar. 2012.
32. B. Luo, H. N. Wu, T. Huang, and D. Liu, "Data-based approximate policy iteration for affine nonlinear continuous-time optimal control design, " Automatica, vol. 50, no. 12, pp. 3281-3290, Dec. 2014.
33. Z.-P. Jiang and Y. Jiang, "Robust adaptive dynamic programming for linear and nonlinear systems: An overview, " Eur. J. Control, vol. 19, no. 5, pp. 417-425, Sep. 2013.
34. Y. Jiang and Z.-P. Jiang, "Computational adaptive optimal control for continuous-time linear systems with completely unknown dynamics, " Automatica, vol. 48, no. 10, pp. 2699-2704, Oct. 2012.
35. Y. Jiang and Z.-P. Jiang, "Robust adaptive dynamic programming and feedback stabilization of nonlinear systems, " IEEE Trans. Neural Netw. Learn. Syst., vol. 25, no. 5, pp. 882-893, May 2014.
36. D. Liu, D. Wang, F.-Y. Wang, H. Li, and X. Yang, "Neural-networkbased online HJB solution for optimal robust guaranteed cost control of continuous-time uncertain nonlinear systems, " IEEE Trans. Cybern., vol. 44, no. 12, pp. 2834-2847, Dec. 2014.
37. D. Wang, D. Liu, H. Li, and H. Ma, "Neural-network-based robust optimal control design for a class of uncertain nonlinear systems via adaptive dynamic programming, " Inf. Sci., vol. 282, pp. 167-179, Oct. 2014.
38. J. Na and G. Herrmann, "Online adaptive approximate optimal tracking control with simplified dual approximation structure for continuoustime unknown nonlinear systems, " IEEE/CAA J. Autom. Sinica, vol. 1, no. 4, pp. 412-422, Oct. 2014.
39. D. Liu, X. Yang, D. Wang, and Q. Wei, "Reinforcement-learning-based robust controller design for continuous-time uncertain nonlinear systems subject to input constraints, " IEEE Trans. Cybern., vol. 45, no. 7, pp. 1372-1385, Jul. 2015.
40. D. Liu, H. Li, and D. Wang, "Error bounds of adaptive dynamic programming algorithms for solving undiscounted optimal control problems, " IEEE Trans. Neural Netw. Learn. Syst., vol. 26, no. 6, pp. 1323-1334, Jun. 2015.
41. X. Yang, D. Liu, and Y. Huang, "Neural-network-based online optimal control for uncertain nonlinear continuous-time systems with control constraints, " IET Control Theory Appl., vol. 7, no. 17, pp. 2037-2047, Nov. 2013.
42. B. Luo, H.-N. Wu, and H.-X. Li, "Adaptive optimal control of highly dissipative nonlinear spatially distributed processes with neuro-dynamic programming, " IEEE Trans. Neural Netw. Learn. Syst., vol. 26, no. 4, pp. 684-696, Apr. 2015.
43. D. Wang, D. Liu, D. Zhao, Y. Huang, and D. Zhang, "A neuralnetwork-based iterative GDHP approach for solving a class of nonlinear optimal control problems with control constraints, " Neural Comput. Appl., vol. 22, no. 2, pp. 219-227, Feb. 2013.
44. R. Song, W. Xiao, and C. Sun, "A new self-learning optimal control laws for a class of discrete-time nonlinear systems based on ESN architecture, " Sci. China Inf. Sci., vol. 57, no. 6, pp. 1-10, Jun. 2014.
45. X. Zhong, H. He, H. Zhang, and Z. Wang, "Optimal control for unknown discrete-time nonlinear Markov jump systems using adaptive dynamic programming, " IEEE Trans. Neural Netw. Learn. Syst., vol. 25, no. 12, pp. 2141-2155, Dec. 2014.
46. D. Wang, D. Liu, H. Li, H. Ma, and C. Li, "A neural-networkbased online optimal control approach for nonlinear robust decentralized stabilization, " Soft Comput., vol. 20, no. 2, pp. 707-716, Feb. 2016.
47. Z. Ni, H. He, X. Zhong, and D. V. Prokhorov, "Model-free dual heuristic dynamic programming, " IEEE Trans. Neural Netw. Learn. Syst., vol. 26, no. 8, pp. 1834-1839, Aug. 2015.
48. D. Liu and Q. Wei, "Finite-approximation-error-based optimal control approach for discrete-time nonlinear systems, " IEEE Trans. Cybern., vol. 43, no. 2, pp. 779-789, Apr. 2013.
49. X. Yang, D. Liu, and D. Wang, "Reinforcement learning for adaptive optimal control of unknown continuous-time nonlinear systems with input constraints, " Int. J. Control, vol. 87, no. 3, pp. 553-566, 2014.
50. H. Zhang, L. Cui, X. Zhang, and Y. Luo, "Data-driven robust approximate optimal tracking control for unknown general nonlinear systems using adaptive dynamic programming method, " IEEE Trans. Neural Netw., vol. 22, no. 12, pp. 2226-2236, Dec. 2011.