Neural-network-based optimal tracking control scheme for a class of unknown discrete-time nonlinear systems using iterative ADP algorithm

Neurocomputing

Volumn 125, Issue , 2014, Pages 46-56

  Huang, Yuzhu ^a   Liu, Derong ^a  

^a INSTITUTE OF AUTOMATION   (China)

Author keywords

Adaptive dynamic programming; Convergence analysis; Heuristic dynamic programming; Neural networks; Optimal tracking control; Reinforcement learning

Indexed keywords

ADAPTIVE DYNAMIC PROGRAMMING; CONVERGENCE ANALYSIS; DISCRETE-TIME NONLINEAR SYSTEMS; HEURISTIC DYNAMIC PROGRAMMING; ITERATIVE ALGORITHM; OPTIMAL TRACKING CONTROL; PARAMETRIC STRUCTURE; REGULATION PROBLEMS;

ALGORITHMS; DYNAMIC PROGRAMMING; FEEDFORWARD NEURAL NETWORKS; NAVIGATION; NEURAL NETWORKS; NONLINEAR SYSTEMS; OPTIMIZATION; REINFORCEMENT LEARNING;

ITERATIVE METHODS;

ALGORITHM; ANALYTICAL ERROR; ARTICLE; ARTIFICIAL NEURAL NETWORK; CONTROL; CONTROL SYSTEM; CONTROLLED STUDY; CONVERGENCE ANALYSIS; FEEDBACK SYSTEM; HEURISTIC DYNAMIC PROGRAMMING; INFORMATION PROCESSING; ITERATIVE ADAPTIVE DYNAMIC PROGRAMMING; NONLINEAR SYSTEM; OPTIMAL TRACKING CONTROL; PRIORITY JOURNAL; PROCESS MODEL; SYSTEM ANALYSIS; SYSTEM TRANSFORMATION;

EID: 84888030007     PISSN: 09252312     EISSN: 18728286     Source Type: Journal    
DOI: 10.1016/j.neucom.2012.07.047     Document Type: Article

References (39)

1. Ha I.J., Gilbert E.G. Robust tracking in nonlinear systems. IEEE Trans. Autom. Control 1987, 32:763-771.
2. Devasiad S., Chen D., Paden B. Nonlinear inversion-based output tracking. IEEE Trans. Autom. Control 1996, 41:930-942.
3. T.W. Mclain, C.A. Bailry, R.W. Beard, Synthesis and experimental testing of a nonlinear optimal tracking controller, in: Proceedings of American Control Conference, San Diego, CA, June 1999, pp. 2847-2851.
4. Cui L., Zhang H., Chen B., Zhang Q. Asymptotic tracking control scheme for mechanical systems with external disturbances and friction. Neurocomputing 2010, 73:1293-1302.
5. Song R., Zhang H., Luo Y., Wei Q. Optimal control laws for time-delay systems with saturating actuators based on heuristic dynamic programming. Neurocomputing 2010, 73:3020-3027.
6. Yadav V., Padhi R., Balakrishnan S.M. Robust/optimal temperature profile control of a high-speed aerospace vehicle using neural networks. IEEE Trans. Neural Networks 2007, 18:1115-1128.
7. Bellman R.E. Dynamic Programming 1957, Princeton University Press, Princeton, NJ.
8. Lewis F.L., Syrmos V.L. Optimal Control 1995, Wiley, New York.
9. Jagannathan S. Neural Network Control of Nonlinear Discrete-Time Systems 2006, CRC Press, Boca Raton, FL.
10. Werbos P.J. Advanced forecasting methods for global crisis warning and models of intelligence. Gen. Syst. Yearb. 1977, 22:25-38.
11. Werbos P.J. Approximate dynamic programming for real-time control and neural modeling. Handbook of Intelligent Control 1992, Van Nostrand Reinhold, New York, (Chapter 13). D.A. White, D.A. Sofge (Eds.).
12. P.J. Werbos, Using ADP to understand and replicate brain intelligence: the next level design, in: Proceedings of the IEEE Symposium on Approximate Dynamic Programming and Reinforcement Learning, Honolulu, HI, April 2007, pp. 209-216.
13. Werbos P.J. ADP. the key direction for future research in intelligent control and understanding brain intelligence. IEEE Trans. Syst. Man Cybern. Part B Cybern. 2008, 38:898-900.
14. Wang F.Y., Zhang H., Liu D. Adaptive dynamic programming. an introduction. IEEE Comput. Intell. Mag. 2009, 4:39-47.
15. Lewis F.L., Vrabie D. Reinforcement learning and adaptive dynamic programming for feedback control. IEEE Circuits Syst. Mag. 2009, 9:32-50.
16. Bertsekas D.P., Tsitsiklis J.N. Neuro-Dynamic Programming 1996, Athena Scientific, Belmont, MA.
17. Prokhorov D.V., Santiago R.A., Wunsch D.C. Adaptive critic designs. a case study for neuro-control. Neural Networks 1995, 8:1367-1372.
18. Prokhorov D.V., Wunsch D.C. Adaptive critic designs. IEEE Trans. Neural Networks 1997, 8:997-1007.
19. Al-Tamimi A., Lewis F.L., Abu-Khalaf M. Discrete-time nonlinear HJB solution using approximate dynamic programming. convergence proof. IEEE Trans. Syst. Man Cybern. Part B Cybern. 2008, 38:943-949.
20. Vrabie D., Lewis F.L. Neural network approach to continuous-time direct adaptive optimal control for partially unknown nonlinear systems. Neural Networks 2009, 22:237-246.
21. Wang F.Y., Jin N., Liu D., Wei Q. Adaptive dynamic programming for finite-horizon optimal control of discrete-time nonlinear systems with ε-error bound. IEEE Trans. Neural Networks 2011, 22:24-36.
22. Dierks T., Thumati B.T., Jagannathan S. Optimal control of unknown affine nonlinear discrete-time systems using offline-trained neural networks with proof of convergence. Neural Networks 2009, 22:851-860.
23. Park Y.M., Choi M.S., Lee K.Y. An optimal tracking neuro-controller for nonlinear dynamic systems. IEEE Trans. Neural Networks 1996, 7:1099-1110.
24. T. Dierks, S. Jagannathan, Optimal tracking control of affine nonlinear discrete-time systems with unknown internal dynamics, in: Proceedings of Joint 48th IEEE Conference on Decision and Control and 28th Chinese Control Conference, Shanghai, PR China, December 2009, pp. 6750-6755.
25. Zhang H., Wei Q., Luo Y. A novel infinite-time optimal tracking control scheme for a class of discrete-time nonlinear system via the greedy HDP iteration algorithm. IEEE Trans. Syst. Man Cybern. Part B Cybern. 2008, 38:937-942.
26. Wang D., Liu D., Wei Q. Finite-horizon neuro-optimal tracking control for a class of discrete-time nonlinear systems using adaptive dynamic programming approach. Neurocomputing 2012, 78:14-22.
27. Murray J.J., Cox C.J., Lendaris G.G., Saeks R. Adaptive dynamic programming. IEEE Trans. Syst. Man Cybern. Part C Appl. Rev. 2002, 32:140-153.
28. Handbook of Learning and Approximate Dynamic Programming 2004, IEEE Press, Wiley, New York. J. Si, A.G. Barto, W.B. Powell, D.C. Wunsch (Eds.).
29. Si J., Wang Y.T. On-line learning control by association and reinforcement. IEEE Trans. Neural Networks 2001, 12:264-276.
30. Zhang H., Luo Y., Liu D. Neural-network-based near-optimal control for a class of discrete-time affine nonlinear systems with control constraints. IEEE Trans. Neural Networks 2009, 9:1490-1503.
31. D. Liu, N. Jin, ε-Adaptive dynamic programming for discrete-time systems, in: Proceedings of International Joint Conference on Neural Networks, Hong Kong, June 2008, pp. 1417-1424.
32. Liu D., Zhang Y., Zhang H. A self-learning call admission control scheme for CDMA cellular networks. IEEE Trans. Neural Networks 2005, 16:1219-1228.
33. D. Liu, X. Xiong, Y. Zhang, Action-dependent adaptive critic designs, in: Proceedings of International Joint Conference on Neural Networks, Washington, DC, July 2001, pp. 990-995.
34. Zhang X., Zhang H., Sun Q., Luo Y. Adaptive dynamic programming-based optimal control of unknown nonaffine discrete-time systems with proof of convergence. Neurocomputing 2012, 91:48-55.
35. Abu-Khalaf M., Lewis F.L. Nearly optimal control laws for nonlinear systems with saturating actuators using a neural network HJB approach. Automatica 2005, 41:779-791.
36. Balakrishnan S.N., Ding J., Lewis F.L. Issues on stability of ADP feedback controllers for dynamic systems. IEEE Trans. Syst. Man Cybern. Part B Cybern. 2008, 38:913-917.
37. He P., Jagannathan S. Reinforcement learning neural-network-based controller for nonlinear discrete-time systems with input constraints. IEEE Trans. Syst. Man Cybern. Part B Cybern. 2007, 37:425-436.
38. Venayagamoorthy G.K., Harley R.G., Wunsch D.C. Implementation of adaptive critic-based neurocontrollers for turbogenerators in a multimachine power system. IEEE Trans. Neural Networks 2003, 14:1047-1064.
39. Venayagamoorthy G.K., Harley R.G., Wunsch D.C. Comparison of heuristic dynamic programming and dual heuristic programming adaptive critics for neurocontrol of a turbogenerator. IEEE Trans. Neural Networks 2002, 13:764-773.