Dual iterative adaptive dynamic programming for a class of discrete-time nonlinear systems with time-delays

Neural Computing and Applications

Volumn 23, Issue 7-8, 2013, Pages 1851-1863

  Wei, Qinglai ^a   Wang, Ding ^a   Zhang, Dehua ^a  

^a INSTITUTE OF AUTOMATION   (China)

Author keywords

Adaptive critic designs; Adaptive dynamic programming; Approximate dynamic programming; Nonlinear systems; Optimal control; Time delay

Indexed keywords

ADAPTIVE CRITIC DESIGNS; ADAPTIVE DYNAMIC PROGRAMMING; APPROXIMATE DYNAMIC PROGRAMMING; DISCRETE-TIME NONLINEAR SYSTEMS; OPTIMAL CONTROL POLICY; OPTIMAL CONTROL PROBLEM; OPTIMAL CONTROL SCHEME; OPTIMAL CONTROLS;

ALGORITHMS; CONTROL; DELAY CONTROL SYSTEMS; DYNAMIC PROGRAMMING; NONLINEAR SYSTEMS; OPTIMAL CONTROL SYSTEMS; TIME DELAY;

ITERATIVE METHODS;

EID: 84887490966     PISSN: 09410643     EISSN: None     Source Type: Journal    
DOI: 10.1007/s00521-012-1188-7     Document Type: Article

References (48)

1. Abu-Khalaf M, Lewis FL (2005) Nearly optimal control laws for nonlinear systems with saturating actuators using a neural network HJB approach. Automatica 41(5): 779-791.
2. Al-Tamimi A, Abu-Khalaf M, Lewis FL (2007) Adaptive critic designs for discrete-time zero-sum games with application to control. IEEE Trans Syst Cybern Part B Cybern 37(1): 240-247.
3. Al-Tamimi A, Lewis FL, Abu-Khalaf M (2008) Discrete-time nonlinear HJB solution using approximate dynamic programming: convergence proof. IEEE Trans Syst Man Cybern Part B Cybern 38(4): 943-949.
4. Basin M, Rodriguez-Gonzalez J (2006) Optimal control for linear systems with multiple time delays in control input. IEEE Trans Autom Control 51(1): 91-97.
5. Basin M, Rodriguez-Gonzaleza J, Fridman L (2007) Optimal and robust control for linear state-delay systems. J Franklin Inst 344(7): 830-845.
6. Bellman RE (1957) Dynamic programming. Princeton University Press, Princeton, NJ.
7. Busoniu L, Ernst D, Schutter BD, Babuska R (2010) Approximate dynamic programming with a fuzzy parameterization. Automatica 46(5): 804-814.
8. ∞ control for vehicle active suspension systems. IEEE Trans Control Syst Technol 18(1): 238-245.
9. Chen Z, Jagannathan S (2008) Generalized Hamilton-Jacobi-Bellman formulation-based neural network control of affine nonlinear discretetime systems. IEEE Trans Neural Netw 19(1): 90-106.
10. Chiasson J (2007) Applications of time delay systems. Springer, Berlin.
11. Halpin SM, Harley KA, Jones RA, Taylor LY (2008) Slope-permissive under-voltage load shed relay for delayed voltage recovery mitigation. IEEE Trans Power Syst 23(3): 1211-1216.
12. Han M, Han B, Xi J, Hirasawa K (2006) Universal learning network and its application for nonlinear system with long time delay. Comput Chem Eng 31(1): 13-20.
13. Hanselmann T, Noakes L, Zaknich A (2007) Continuous-time adaptive critics. IEEE Trans Neural Netw 18(3): 631-647.
14. Ho DWC, Li J, Niu Y (2005) Adaptive neural control for a class of nonlinearly parametric time-delay systems. IEEE Trans Neural Netw 16(3): 625-635.
15. Huang X, Ma M (2008) Optimal scheduling for minimum delay in passive star coupled WDM optical networks. IEEE Trans Commun 56(8): 1324-1330.
16. Lewis FL, Vrabie D (2009) Reinforcement learning and adaptive dynamic programming for feedback control. IEEE Circuits Syst Mag 9(3): 32-50.
17. Li T, Tong SC, Feng G (2010) A novel robust adaptive-fuzzy-tracking control for a class of nonlinear multi-input/multi-output systems. IEEE Trans Fuzzy Syst 18(1): 150-160.
18. Li T, Wang D, Feng G, Tong SC (2010) A DSC approach to robust adaptive NN tracking control for strict-feedback nonlinear systems. IEEE Trans Syst Man Cybern Part B Cybern 40(3): 915-927.
19. Li T, Feng, Wang D, Tong S (2010) Neural-network-based simple adaptive control of uncertain multi-input multi-output non-linear systems. IET Control Theory Appl 4(9): 1543-1557.
20. Liu D, Zhang Y, Zhang H (2005) A self-learning call admission control scheme for CDMA cellular networks. IEEE Trans Neural Netw 16(5): 1219-1228.
21. Malek-Zavarei M, Jashmidi M (1987) Time-delay systems: analysis, optimization and applications. North-Holland, Amsterdam.
22. Pindyck RS (1992) The distrete-time tracking problem with a time delay in the control. IEEE Trans Autom Control 17(6): 397-398.
23. Murray JJ, Cox CJ, Lendaris GG, Saeks R (2002) Adaptive dynamic programming. IEEE Trans Syst Man Cybern Part C Appl Rev 32(2): 140-153.
24. Prokhorov DV, Wunsch DC (1997) Adaptive critic designs. IEEE Trans Neural Netw 8(5): 997-1007.
25. Richard JP (2003) Time-delay systems: an overview of some recent advances and open problems. Automatica 39(10): 1667-1694.
26. Schenato L (2008) Optimal estimation in networked control systems subject to random delay and packet drop. IEEE Trans Autom Control 53(5): 1311-1317.
27. Si J, Wang YT (2001) On-line learning control by association and reinforcement. IEEE Trans Neural Netw 12(2): 264-276.
28. Silva GJ (2005) PID Controllers for time-delay systems. Birkhuser, Boston, MA.
29. Song R, Zhang H, Luo Y, Wei Q (2010) Optimal control laws for time-delay systems with saturating actuators based on heuristic dynamic programming. Neurocomputing 73(16-18): 3020-3027.
30. Sun Q, Li Z, Yang J, Luo Y (2010) Load distribution model and voltage static profile of Smart Grid. J Central S Univ Technol 17(4): 824-829.
31. Vamvoudakis KG, Lewis FL (2010) Online actor-critic algorithm to solve the continuous-time infinite horizon optimal control problem. Automatica 46(5): 878-888.
32. Wang D, Liu D, Wei Q (2012) Finite-horizon neuro-optimal tracking control for a class of discrete-time nonlinear systems using adaptive dynamic programming approach. Neurocomputing 78(1): 14-22.
33. Wang FY, Jin N, Liu D, Wei Q (2011) Adaptive dynamic programming for finite-horizon optimal control of discrete-time nonlinear systems with ε-error bound. IEEE Trans Neural Netw 22(1): 24-36.
34. Wang FY, Zhang H, Liu D (2009) Adaptive dynamic programming: an introduction. IEEE Comput Intell Mag 4(2): 39-47.
35. Watkins C (1989) Learning from delayed rewards. Ph. D. Thesis. Cambridge University, Cambridge.
36. Wei Q, Zhang H, Dai J (2009) Model-free multiobjective approximate dynamic programming for discrete-time nonlinear systems with general performance index functions. Neurocomputing 72(7-9): 1839-1848.
37. Werbos PJ (1991) A menu of designs for reinforcement learning over time. In: Miller WT, Sutton RS, Werbos PJ (eds) Neural networks for control. MIT Press, Cambridge, pp 67-95.
38. Werbos PJ (1992) Approximate dynamic programming for real-time control and neural modeling. In: White DA, Sofge DA (eds) Handbook of intelligent control: neural, fuzzy, and adaptive approaches ch. 13. van Nostrand Reinhold, New York.
39. Widrow B, Gupta N, Maitra S (1973) Punish/reward: learning with a critic in adaptive threshold systems. IEEE Trans Syst Man Cybern 3: 455-465.
40. Yadav V, Padhi R, Balakrishnan SN (2007) Robust/optimal temperature profile control of a high-speed aerospace vehicle using neural networks. IEEE Trans Neural Netw 18(4): 1115-1128.
41. Yang Y, Feng G, Ren J (2004) A combined backstepping and small-gain approach to robust adaptive fuzzy control for strict-feedback nonlinear systems. IEEE Trans Syst Man Cybern Part A Syst Humans 34(3): 406-420.
42. Zhang H, Basin MV, Skliar M (2007) Ito-Volterra optimal state estimation with continuous, multirate, randomly sampled, and delayed measurements. IEEE Trans Autom Control 52(3): 401-416.
43. Zhang H, Quan Y (2001) Modeling, identification and control of a class of nonlinear system. IEEE Trans Fuzzy Syst 9(2): 349-354.
44. Zhang H, Wang Y, Liu D (2008) Delay-dependent guaranteed cost control for uncertain stochastic fuzzy systems with multiple time delays. IEEE Trans Syst Man Cybern Part B Cybern 38(1): 125-140.
45. Zhang H, Wei Q, Luo Y (2008) A novel infinite-time optimal tracking control scheme for a class of discrete-time nonlinear systems via the greedy HDP iteration algorithm. IEEE Trans Syst Man Cybern Part B Cybern 38(4): 937-942.
46. Zhang H, Wei Q, Liu D (2011) An iterative adaptive dynamic programming method for solving a class of nonlinear zero-sum differential games. Automatica 47(1): 207-214.
47. Zhang H, Song R, Wei Q, Zhang T (2011) Optimal tracking control for a class of nonlinear discrete-time systems with time delays based on heuristic dynamic programming. IEEE Trans Neural Netw 22(12): 1851-1862.
48. Zhang H, Yang D, Chai T (2007) Guaranteed cost networked control for T-S fuzzy systems with time delay. IEEE Trans Syst Man Cybern Part C Appl Rev 37(2): 160-172.