Stable iterative adaptive dynamic programming algorithm with approximation errors for discrete-time nonlinear systems

Neural Computing and Applications

Volumn 24, Issue 6, 2014, Pages 1355-1367

  Wei, Qinglai ^a   Liu, Derong ^a  

^a INSTITUTE OF AUTOMATION   (China)

Author keywords

Adaptive critic designs; Adaptive dynamic programming; Approximate dynamic programming; Neural networks; Nonlinear systems; Optimal control

Indexed keywords

APPROXIMATION ALGORITHMS; CONTROL THEORY; DYNAMIC PROGRAMMING; NEURAL NETWORKS; NONLINEAR SYSTEMS; OPTIMAL CONTROL SYSTEMS; STABILITY;

ADAPTIVE CRITIC DESIGNS; ADAPTIVE DYNAMIC PROGRAMMING; APPROXIMATE DYNAMIC PROGRAMMING; APPROXIMATION ERRORS; DISCRETE-TIME NONLINEAR SYSTEMS; OPTIMAL CONTROL PROBLEM; OPTIMAL CONTROLS; STABILITY PROPERTIES;

ITERATIVE METHODS;

EID: 84898013913     PISSN: 09410643     EISSN: None     Source Type: Journal    
DOI: 10.1007/s00521-013-1361-7     Document Type: Article

References (41)

1. Abu-Khalaf M, Lewis FL (2005) Nearly optimal control laws for nonlinear systems with saturating actuators using a neural network HJB approach. Automatica 41(5): 779-791.
2. Adhyaru DM, Kar IN, Gopal M (2011) Bounded robust control of nonlinear systems using neural network-based HJB solution. Neural Comput Appl 20(1): 91-103.
3. ∞ control. IEEE Trans Syst Cybern B Cybern 37(1): 240-247.
4. Al-Tamimi A, Lewis FL (2007) Discrete-time nonlinear HJB solution using approximate dynamic programming: convergence proof. In: Proceedings of the IEEE symposium on approximate dynamic programming and reinforcement learning, pp 38-43.
5. Al-Tamimi A, Lewis FL, Abu-Khalaf M (2008) Discrete-time nonlinear HJB solution using approximate dynamic programming: convergence proof. IEEE Trans Syst Man Cybern B Cybern 38(4): 943-949.
6. Beard R (1995) Improving the closed-loop performance of nonlinear systems. Ph. D Thesis, Rensselaer Polytechnic Institute, Troy, NY.
7. Bertsekas DP, Tsitsiklis JN (1996) Neuro-dynamic programming. Athena Scientific, Belmont.
8. Chen Z, Jagannathan S (2008) Generalized Hamilton-Jacobi-Bellman formulation-based neural network control of affine nonlinear discretetime systems. IEEE Trans Neural Netw 19(1): 90-106.
9. Enns R, Si J (2003) Helicopter trimming and tracking control using direct neural dynamic programming. IEEE Trans Neural Netw 14(8): 929-939.
10. Chen D, Yang J, Mohler RR (2008) On near optimal neural control of multiple-input nonlinear systems. Neural Comput Appl 17(4): 327-337.
11. Hagen S, Krose B (2003) Neural Q-learning. Neural Comput Appl 12(2): 81-88.
12. Huang T, Liu D (2013) A self-learning scheme for residential energy system control and management. Neural Comput Appl 22(2): 259-269.
13. Jin N, Liu D, Huang T, Pang Z (2007) Discrete-time adaptive dynamic programming using wavelet basis function neural networks. In: Proceedings of the IEEE symposium on approximate dynamic programming and reinforcement learning, pp 135-142.
14. Lewis FL, Vrabie D (2009) Reinforcement learning and adaptive dynamic programming for feedback control. IEEE Circuits Syst Mag 9(3): 32-50.
15. Liao X, Wang L, Yu P (2007) Stability of dynamical systems. Elsevier Press, Amsterdam.
16. Liu D, Javaherian H, Kovalenko O, Huang T (2008) Adaptive critic learning techniques for engine torque and air-fuel ratio control. IEEE Trans Syst Man Cybern B Cybern 38(4): 988-993.
17. Liu D, Zhang Y, Zhang H (2005) A self-learning call admission control scheme for CDMA cellular networks. IEEE Trans Neural Netw 16(5): 1219-1228.
18. Liu Z, Zhang H, Zhang Q (2010) Novel stability analysis for recurrent neural networks with multiple delays via line integral-type L-K functional. IEEE Trans Neural Netw 21(11): 1710-1718.
19. Luo Y, Zhang H (2008) Approximate optimal control for a class of nonlinear discrete-time systems with saturating actuators. Prog Nat Sci 18(8): 1023-1029.
20. Murray JJ, Cox CJ, Lendaris GG, Saeks R (2002) Adaptive dynamic programming. IEEE Trans Syst Man Cybern C Appl Rev 32(2): 140-153.
21. Prokhorov DV, Wunsch DC (1997) Adaptive critic designs. IEEE Trans Neural Netw 8(5): 997-1007.
22. Si J, Wang YT (2001) On-line learning control by association and reinforcement. IEEE Trans Neural Netw 12(2): 264-276.
23. Sutton RS, Barto AG (1998) Reinforcement learning: an introduction. The MIT Press, Cambridge.
24. Song R, Zhang H (2013) The finite-horizon optimal control for a class of time-delay affine nonlinear system. Neural Comput Appl 22(2): 229-235.
25. Wang D, Liu D, Zhao D, Huang Y, Zhang D (2013) A neural-network-based iterative GDHP approach for solving a class of nonlinear optimal control problems with control constraints. Neural Comput Appl 22(2): 219-227.
26. Wang F, Jin N, Liu D, Wei Q (2011) Adaptive dynamic programming for finite-horizon optimal control of discrete-time nonlinear systems with ε-error bound. IEEE Trans Neural Netw 22(1): 24-36.
27. Wang F, Zhang H, Liu D (2009) Adaptive dynamic programming: an introduction. IEEE Comput Intell Mag 4(2): 39-47.
28. Watkins C (1989) Learning from delayed rewards. Ph. D Thesis, Cambridge University, Cambridge, England.
29. Wei Q, Liu D (2012) Adaptive dynamic programming with stable value iteration algorithm for discrete-time nonlinear systems. In Proceedings of international joint conference on neural networks, Brisbane, Australia, 1-6.
30. Wei Q, Liu D (2012) Finite-approximation-error based optimal control approach for discrete-time nonlinear systems. IEEE Trans Syst Man Cybern B Cybern. Available on-line: http://ieeexplore. ieee. org/stamp/stamp. jsp?tp=&arnumber=6328288.
31. Wei Q, Liu D (2012) An iterative ε-optimal control scheme for a class of discrete-time nonlinear systems with unfixed initial state. Neural Netw 32: 236-244.
32. Wei Q, Zhang H, Dai J (2009) Model-free multiobjective approximate dynamic programming for discrete-time nonlinear systems with general performance index functions. Neurocomputing 72(7-9): 839-1848.
33. Werbos PJ (1977) Advanced forecasting methods for global crisis warning and models of intelligence. Gen Syst Yearbook 22: 25-38.
34. Werbos PJ (1991) A menu of designs for reinforcement learning over time. In: Miller WT, Sutton RS, Werbos PJ (eds) Neural networks for control. The MIT Press, Cambridge, pp 67-95.
35. Werbos PJ (1992) Approximate dynamic programming for real-time control and neural modeling. In: White DA, Sofge DA, (eds) Handbook of intelligent control: neural, fuzzy, and adaptive approaches. van Nostrand Reinhold, New York, ch. 13.
36. Zhang H, Liu Z, Huang G, Wang Z (2010) Novel weighting-delay-based stability criteria for recurrent neural networks with time-varying delay. IEEE Trans Neural Netw 21(1): 91-106.
37. Zhang H, Luo Y, Liu D (2009) The RBF neural network-based near-optimal control for a class of discrete-time affine nonlinear systems with control constraint. IEEE Trans Neural Netw 20(9): 1490-1503.
38. Zhang H, Quan Y (2001) Modeling identification and control of a class of nonlinear system. IEEE Trans Fuzzy Syst 9(2): 349-354.
39. Zhang H, Song R, Wei Q, Zhang T (2011) Optimal tracking control for a class of nonlinear discrete-time systems with time delays based on heuristic dynamic programming. IEEE Trans Neural Netw 22(12): 1851-1862.
40. Zhang H, Wei Q, Liu D (2011) An iterative adaptive dynamic programming method for solving a class of nonlinear zero-sum differential games. Automatica 47(1): 207-214.
41. Zhang H, Wei Q, Luo Y (2008) A novel infinite-time optimal tracking control scheme for a class of discrete-time nonlinear systems via the greedy HDP iteration algorithm. IEEE Trans Syst Man Cybern B Cybern 38(4): 937-942.