Online approximate optimal control for affine non-linear systems with unknown internal dynamics using adaptive dynamic programming

IET Control Theory and Applications

Volumn 8, Issue 16, 2014, Pages 1676-1688

  Yang, Xiong ^a   Liu, Derong ^a   Wei, Qinglai ^a  

^a INSTITUTE OF AUTOMATION   (China)

Author keywords

[No Author keywords available]

Indexed keywords

ADAPTIVE CONTROL SYSTEMS; CLOSED LOOP SYSTEMS; CONTINUOUS TIME SYSTEMS; LINEAR SYSTEMS; MEMORY ARCHITECTURE; NETWORK ARCHITECTURE; ONLINE SYSTEMS; OPTIMAL CONTROL SYSTEMS;

ADAPTIVE DYNAMIC PROGRAMMING; HAMILTON JACOBI BELLMAN EQUATION; INTERNAL DYNAMICS; LYAPUNOV DIRECT METHOD; NEURAL NETWORKS (NNS); OPTIMAL CONTROL PROBLEM; PERSISTENCE OF EXCITATION; UNIFORM ULTIMATE BOUNDEDNESS;

DYNAMIC PROGRAMMING;

EID: 84908454874     PISSN: 17518644     EISSN: 17518652     Source Type: Journal    
DOI: 10.1049/iet-cta.2014.0186     Document Type: Article

References (46)

1. Bryson, A.E., Ho, Y.C.: 'Applied optimal control: optimization', Estimation and Control (Taylor & Francis, 1975)
2. Lewis, F.L., Vrabie, D., Syrmos, V.L.: 'Optimal control' (John Wiley & Sons, 2012)
3. Li, H., Liu, D.: 'Optimal control for discrete-time affine nonlinear systems using general value iteration', IET Control Theory Appl., 2012, 6, (18), pp. 2725-2736
4. Yang, X., Liu, D., Huang, Y: 'Neural-network-based online optimal control for uncertain non-linear continuous-time systems with control constraints', IET Control Theory Appl., 2013, 7, (17), pp. 2037-2047
5. Yang, X., Liu, D., Wang, D.: 'Reinforcement learning for adaptive optimal control of unknown continuous-time nonlinear systems with input constraints', Int. J. Control, 2014, 87, (3), pp. 553-566
6. Bellman, R.E.: 'Dynamic programming' (Princeton University Press, 1957)
7. Werbos, P.J.: 'Beyond regression: new tools for prediction and analysis in the behavioral sciences'. PhD thesis, Harvard University, 1974
8. Werbos, P.J.: 'Approximate dynamic programming for real-time control and neural modeling', in White, D.A., Sofge, D.A. (Eds.): 'Handbook of intelligent control: neural, fuzzy, and adaptive approaches' (Van Nostrand Reinhold, 1992)
9. Murray, J.J., Cox, C.J., Lendaris, G.G., Saeks, R.: 'Adaptive dynamic programming', IEEE Trans. Syst., Man Cybern. C, Appl. Rev., 2002, 32, (2), pp. 140-153
10. Wang, F.Y., Zhang, H., Liu, D.: 'Adaptive dynamic programming: an introduction', IEEE Comput. Intell. Mag., 2009, 4, (2), pp. 39-47
11. Liu, D., Wang, D., Yang, X.: 'An iterative adaptive dynamic programming algorithm for optimal control of unknown discrete-time nonlinear systems with constrained inputs', Inf. Sci., 2013, 220, pp. 331-342
12. Liu, D., Wei, Q.: 'Finite-approximation-error-based optimal control approach for discrete-time nonlinear systems', IEEE Trans. Cybern., 2013, 43, (2), pp. 779-789
13. Wei, Q., Liu, D.: 'Numerical adaptive learning control scheme for discrete-time non-linear systems', IET Control Theory Appl., 2013, 7, (11), pp. 1472-1486
14. Powell, W.B.: 'Approximate dynamic programming: solving the curses of dimensionality' (Wiley, 2011, 2nd edn.)
15. Liu, D., Zhang, Y., Zhang, H.: 'A self-learning call admission control scheme for CDMA cellular networks', IEEE Trans. Neural Netw., 2005, 16, (5), pp. 1219-1228
16. Liu, D., Javaherian, H., Kovalenko, O., Huang, T.: 'Adaptive critic learning techniques for engine torque and air-fuel ratio control', IEEE Trans. Syst. Man Cybern. B, Cybern., 2008, 38, (4), pp. 988-993
17. Prokhorov, D.V., Wunsch, D.C.: 'Adaptive critic designs', IEEE Trans. Neural Netw., 1997, 8, (5), pp. 997-1007
18. Si, J., Wang, Y.T.: 'On-line learning control by association and reinforcement', IEEE Trans. Neural Netw., 2001, 12, (2), pp. 264-276
19. Sutton, R.S., Barto, A.G.: 'Reinforcement learning-an introduction' (MIT Press, 1998)
20. Lewis, F.L., Vrabie, D., Vamvoudakis, K.G.: 'Reinforcement learning and feedback control: using natural decision methods to design optimal adaptive controllers', IEEE Control Syst. Mag., 2012, 32, (6), pp. 76-105
21. Liu, D., Yang, X., Li, H.: 'Adaptive optimal control for a class of continuous-time affine nonlinear systems with unknown internal dynamics', Neural Comput. Appl., 2013, 23, (7-8), pp. 1843-1850
22. ∞ Control', IEEE Trans. Neural Netw. Learn. Syst., 2012, 23, (12), pp. 1884-1895
23. Ni, Z., He, H., Wu, J.: 'Adaptive learning in tracking control based on the dual critic network design', IEEE Trans. Neural Netw. Learn. Syst., 2013, 24, (6), pp. 913-928
24. Zhang, H., Cui, L., Zhang, X., Luo, Y.: 'Data-driven robust approximate optimal tracking control for unknown general nonlinear systems using adaptive dynamic programming method', IEEE Trans. Neural Netw., 2011, 22, (12), pp. 2226-2236
25. Abu-Khalaf, M., Lewis, F.L.: 'Nearly optimal control laws for nonlinear systems with saturating actuators using a neural network HJB approach', Automatica, 2005, 41, (5), pp. 779-791
26. Vrabie, D., Lewis, F.L.: 'Neural network approach to continuous-time direct adaptive optimal control for partially unknown nonlinear systems', Neural Netw., 2009, 22, (3), pp. 237-246
27. Vamvoudakis, K.G., Lewis, F.L.: 'Online actor-critic algorithm to solve the continuous-time infinite horizon optimal control problem', Automatica, 2010, 46, (5), pp. 878-888
28. Bhasin, S., Kamalapurkar, R., Johnson, M., Vamvoudakis, K.G., Lewis, F.L., Dixon, W.E.: 'A novel actor-critic-identifier architecture for approximate optimal control of uncertain nonlinear systems', Automatica, 2013, 49, (1), pp. 82-92
29. Dierks, T., Jagannathan, S.: 'Optimal control of affine nonlinear continuous-time systems'. Am. Control Conf., Baltimore, MD, USA, June-July 2010, pp. 1568-1573
30. Zhang, H., Cui, L., Luo, Y.: 'Near-optimal control for nonzero-sum differential games of continuous-time nonlinear systems using single-network ADP', IEEE Trans. Cybern., 2013, 43, (1), pp. 206-216
31. Nodland, D., Zargarzadeh, H., Jagannathan, S.: 'Neural network-based optimal adaptive output feedback control of a helicopter UAV', IEEE Trans. Neural Netw. Learn. Syst., 2013, 24, (7), pp. 1061-1073
32. Haykin, S.: 'Neural networks and learning machines' (Prentice-Hall, 2008, 3rd edn.)
33. Khalil, H.K.: 'Nonlinear systems' (Prentice-Hall, 2001, 3rd edn.)
34. Abdollahi, F., Talebi, H.A., Patel, R.V.: 'A stable neural network-based observer with application to flexible-joint manipulators', IEEE Trans. Neural Netw., 2006, 17, (1), pp. 118-129
35. Hornik, K., Stinchcombe, M., White, H.: 'Universal approximation of an unknown mapping and its derivatives using multilayer feedforward networks', Neural Netw., 1990, 3, (5), pp. 551-560
36. Lewis, F.L., Yesildirek, A., Liu, K.: 'Multilayer neural-net robot controller with guaranteed tracking performance', IEEE Trans. Neural Netw., 1996, 7, (2), pp. 388-399
37. Yu, W.: 'Recent advances in intelligent control systems' (Springer-Verlag, 2009)
38. Yang, X., Liu, D., Wang, D., Wei, Q.: 'Discrete-time online learning control for a class of unknown nonaffine nonlinear systems using reinforcement learning', Neural Netw., 2014, 55, pp. 30-41
39. Rudin, W.: 'Principles of mathematical analysis' (McGraw-Hill', Inc., 1976, 3rd edn.)
40. Lewis, F.L., Jagannathan, S., Yesildirek, A.: 'Neural network control of robot manipulators and nonlinear systems' (Taylor & Francis, 1999)
41. Gampbell, S.L., Meger, C.D.: 'Generalized inverses of linear transformations' (Dover Publications, 1991)
42. Horn, R.A., Johnson, C.R.: 'Matrix analysis' (Cambridge University Press, 2012, 2nd edn.)
43. Beard, R., Saridis, G., Wen, J.: 'Galerkin approximations of the generalized Hamilton-Jacobi-Bellman equation', Automatica, 1997, 33, (12), pp. 2159-2177
44. Chowdhary, G.V.: 'Concurrent learning for convergence in adaptive control without persistency of excitation'. PhD thesis, Georgia Institute of Technology, 2010
45. Padhi, R., Unnikrishnan, N., Wang, X., Balakrishnan, S.N.: 'A single network adaptive critic (SNAC) architecture for optimal control synthesis for a class of nonlinear systems', Neural Netw., 2006, 19, (10), pp. 1648-1660
46. Chowdhary, G.V.: 'A singular value maximizing data recording algorithm for concurrent learning'. American Control Conf., San Francisco, CA, USA, 2011, pp. 3547-3552