Complete stability analysis of a heuristic approximate dynamic programming control design

Automatica

Volumn 59, Issue , 2015, Pages 9-18

  Sokolov, Yury ^a   Kozma, Robert ^a   Werbos, Ludmilla D ^b   Werbos, Paul J ^b  

^a University of Memphis   (United States)

^b UNIVERSITY OF MEMPHIS   (United States)

Author keywords

Abstract Adaptive dynamic programming; Action Dependent Heuristic Dynamic Programming; Adaptive control; Adaptive critic; Gradient descent; Lyapunov function; Neural network

Indexed keywords

ADAPTIVE CONTROL SYSTEMS; DEEP LEARNING; GRADIENT METHODS; HEURISTIC PROGRAMMING; LYAPUNOV FUNCTIONS; MULTILAYER NEURAL NETWORKS; NETWORK LAYERS; NEURAL NETWORKS;

ACTION DEPENDENT HEURISTIC DYNAMIC PROGRAMMING; ADAPTIVE CONTROL; ADAPTIVE CRITIC; ADAPTIVE DYNAMIC PROGRAMMING; GRADIENT DESCENT;

DYNAMIC PROGRAMMING;

EID: 84937840990     PISSN: 00051098     EISSN: None     Source Type: Journal    
DOI: 10.1016/j.automatica.2015.06.001     Document Type: Article

References (29)

1. M. Abu-Khalaf, and F.L. Lewis Nearly optimal control laws for nonlinear systems with saturating actuators using a neural network HJB approach Automatica 41 2005 779 791
2. A. Al-Tamimi, F.L. Lewis, and M. Abu-Khalaf Model-free Q-learning designs for linear discrete-time zero-sum games with application to H-infinity control Automatica 43 2007 473 481
3. A. Barron Universal approximation bounds for superpositions of a sigmoidal function IEEE Transactions on Information Theory 39 1993 930 944
4. A. Barron Approximation and estimation bounds for artificial neural networks Machine Learning 14 1994 113 143
5. A.G. Barto, R.S. Sutton, and C.W. Anderson Neuronlike elements that can solve difficult learning control problems IEEE Transactions on Systems Man and Cybernetics, SMC 13 1983 835 846
6. D.P. Bertsekas, and J.N. Tsitsiklis Neuro-dynamic programming 1996 Athena Scientific MA
7. A.E. Bryson, and Y.C. Ho Applied optimal control 1975 Hemisphere Washington, DC
8. H. He Self-adaptive systems for machine intelligence 2011 Wiley
9. R. Kozma, and Y. Sokolov Improved stability criteria of ADP control for efficient context-aware decision support systems Int. conf. on awareness science and technology (iCAST2013) 2013 IEEE Press Aizu-Wakamatsu, Japan 41 46
10. G.G. Lendaris Adaptive dynamic programming approach to experience-based systems identification and control Neural Networks 22 5 2009 822
11. F. Lewis, and D. Liu Reinforcement learning and approximate dynamic programming for feedback control 2012 Wiley-IEEE Press
12. F. Liu, J. Sun, J. Si, W. Guo, and S. Mei A boundness result for the direct heuristic dynamic programming Neural Networks 32 2012 229 235
13. A. Michel, L. Hou, and D. Liu Stability of dynamical system 2008 Birkhäuser
14. W. Powell Approximate dynamic programming 2011 Wiley
15. D. Prokhorov, and D. Wunsch Adaptive critic designs IEEE Transactions on Neural Networks 8 5 1997 997 1007
16. J. Sarangapani Neural network control of nonlinear discrete-time systems 2006 Taylor and Francis
17. J. Si, A.G. Barto, W.B. Powell, and D.C. Wunsch Handbook of learning and approximate dynamic programming 2004 IEEE Press Piscataway, NJ
18. M.J. Valenti Approximate dynamic programming with applications in multi-agent systems (Ph.D. dissertation) 2007 MIT
19. G.K. Venayagamoorthy, R.G. Harley, and D.C. Wunsch Dual heuristic programming excitation neurocontrol for generatos in a multimachine power system IEEE Transactions on Industry Applications 39 2 2003 382 394
20. Vrabie, D., & Lewis, F. L. (2009). Generalized policy iteration for continuous-time systems. In Proceeding of international joint conference on neural networks(pp. 3224-3231).
21. D. Wang, D. Liu, Q. Wei, D. Zhao, and N. Jin Optimal control of unknown nonaffine nonlinear discrete-time systems based on adaptive dynamic programming Automatica 48 2012 1825 1832
22. P.J. Werbos Beyond regression: new tools for prediction and analysis in the behavioral science (Ph.D. thesis) 1974 Committee on Applied Mathematics, Harvard Univ. Cambridge, MA
23. P.J. Werbos Consistency of HDP applied to a simple reinforcement learning problem Neural Networks 3 1990 179 189
24. P.J. Werbos Approximate dynamic programming for real-time control and neural modeling D.A. White, D.A. Sofge, Handbook of intelligent control 1992 Van Nostrand Reinhold New York
25. Werbos, P.J. (2012). Stable adaptive control using new critic designs. http://arxiv.org/abs/adap-org/9810001.
26. D.A. White, and D.A. Sofge Handbook of intelligent control: neural, fuzzy, and adaptive approaches 1992 (Chapter 3, 10 and 13)
27. H. Zhang, Q. Wei, and D. Liu An iterative adaptive dynamic programming method for solving a class of nonlinear zero-sum differential games Automatica 47 1 2011 207 214
28. H. Zhang, D. Liu, Y. Luo, and D. Wang Adaptive dynamic programming for control: algorithms and stability 2013 Springer-Verlag London
29. J. Zhang, H. Zhang, Y. Luo, and H. Liang Nearly optimal control scheme using adaptive dynamic programming based on generalized fuzzy hyperbolic model Acta Automatica Sinica 39 2 2013 142 148