Online concurrent reinforcement learning algorithm to solve two-player zero-sum games for partially unknown nonlinear continuous-time systems

International Journal of Adaptive Control and Signal Processing

Volumn 29, Issue 4, 2015, Pages 473-493

  Yasini, Sholeh ^a   Karimpour, Ali ^a   Naghibi Sistani, Mohammad Bagher ^a   Modares, Hamidreza ^a  

^a FERDOWSI UNIVERSITY OF MASHHAD   (Iran)

Author keywords

H control; neural networks; online concurrent reinforcement learning algorithm; two player zero sum games

Indexed keywords

CLOSED LOOP SYSTEMS; CONCURRENCY CONTROL; CONTINUOUS TIME SYSTEMS; E-LEARNING; GAME THEORY; LEARNING SYSTEMS; NEURAL NETWORKS; NONLINEAR CONTROL SYSTEMS; ONLINE SYSTEMS; OPTIMAL CONTROL SYSTEMS; REINFORCEMENT LEARNING;

ADAPTIVE OPTIMAL CONTROL; CONTROL PROBLEMS; HAMILTON-JACOBI-ISAACS EQUATIONS; LYAPUNOV TECHNIQUES; NEURAL NETWORKS (NNS); NONLINEAR CONTINUOUS-TIME SYSTEMS; PERSISTENCE OF EXCITATION; ZERO-SUM GAME;

LEARNING ALGORITHMS;

EID: 84927697693     PISSN: 08906327     EISSN: 10991115     Source Type: Journal    
DOI: 10.1002/acs.2485     Document Type: Article

References (40)

1. Zames G,. Feedback and optimal sensitivity: model reference transformations, multiplicative seminorms, and approximate inverses. IEEE Transactions on Automatic Control 1981; 26 (2): 301-320.
2. ∞ control via measurement feedback for general nonlinear systems. IEEE Transactions on Automatic Control 1995; 40 (3): 446-472.
3. Zhou K, Doyle JC, Glover K,. Robust and Optimal Control. Prentice-Hall: Upper Saddle River, NJ, 1996.
4. 2-Gain and Passivity Techniques in Nonlinear Control. Springer-Verlag: Berlin, Germany, 1996.
5. ∞ Optimal Control and Related Minimax Design Problems. Birkhäuser: Boston, MA, 1995.
6. nd edn). SIAM, Vol. 23, SIAM's Classic in Applied Mathematics: Philadelphia, PA, 1999.
7. ∞ control. Automatica 2009; 45 (4): 881-888.
8. Lanzon A, Feng Y, Anderson BDO, Rothowitz M,. Computing the positive stabilizing solution to algebraic Riccati equations. IEEE Transactions on Automatic Control 2008; 53 (8): 2280-2291.
9. Beard R, Saridis GN, Wen J,. Galerkin approximations of the generalized Hamilton-Jacobi-Bellman equation. Automatica 1997; 33 (10): 2159-2177.
10. Beard R, McLain T,. Successive Galerkin approximation algorithms for nonlinear optimal and robust control. International Journal of Control 1998; 71 (5): 717-743.
11. ∞ control. IEEE Transactions on Automatic Control 1992; 37 (6): 770-784.
12. ∞ state feedback control with input saturation. IEEE Transactions on Automatic Control 2006; 51 (10): 1989-1995.
13. Abu-Khalaf M, Lewis FL, Huang J,. Neurodynamic programming and zero-sum games for constrained control systems. IEEE Transactions on Neural Networks 2008; 19 (5): 1243-1252.
14. Sutton RS, Barto AG,. Reinforcement Learning-An Introduction. MIT Press: Cambridge, MA, 1998.
15. Lewis FL, Vrabie D,. Reinforcement learning and adaptive dynamic programming for feedback control. IEEE Circuits and Systems Magazine 2009; 9 (3): 32-50.
16. Lewis FL, Vrabie D, Vamvoudakis KG,. Reinforcement learning and feedback control. IEEE Control Systems Magazine 2012; 32 (6): 76-105.
17. Wang D, Liu D, Wei Q, Zhao D, Jin N,. Optimal control of nonlinear discrete-time systems based on adaptive dynamic programming approach. Automatica 2012; 48 (6): 1825-1832.
18. Al-Tamimi A, Lewis FL, Abu-Khalaf M,. Discrete-time nonlinear HJB solution using approximate dynamic programming. IEEE Transactions on Systems, Man, and Cybernetics, Part B: Cybernetics 2008; 38 (4): 943-949.
19. Lian C, Xu X, Zuo L, Huang Z,. Adaptive critic design with graph Laplacian for online learning control of nonlinear systems. International Journal of Adaptive Control and Signal Processing 2012. DOI: 10.1002/acs.2344.
20. Liu D, Li H, Wang D,. Neural-network-based zero-sum game for discrete-time nonlinear systems via iterative adaptive dynamic programming algorithm. Neurocomputing 2013; 110: 92-100.
21. Morimoto J, Doya K,. Robust reinforcement learning. Neural Computation 2005; 17 (2): 335-359.
22. Vrabie D, Lewis FL,. Adaptive dynamic programming for online solution of a zero-sum differential game. Journal of Control Theory and Applications 2011; 9 (3): 353-360.
23. Vamvoudakis KG, Lewis FL,. Online actor-critic algorithm to solve the continuous infinite time horizon optimal control problem. Automatica 2010; 46: 878-888.
24. Vamvoudakis KG, Lewis FL,. Online solution of nonlinear two-player zero-sum games using synchronous policy iteration. International Journal of Robust and Nonlinear Control 2012; 22: 1460-1483.
25. Modares H, Naghibi Sistani MB, Lewis FL,. A policy iteration approach to online optimal control of continuous-time constrained-input systems. ISA Transactions 2013; 52: 611-621.
26. ∞ control of constrained input systems. International Journal of Adaptive Control and Signal Processing 2012. DOI: 10.1002/acs.2348.
27. ∞ state feedback control with Galerkin's method. International Journal of Robust and Nonlinear Control 2013; 23: 991-1012.
28. Vrabie D, Lewis FL,. Neural network approach to continuous time direct adaptive optimal control for partially unknown nonlinear systems. Neural Networks 2009; 22: 237-246.
29. Vamvoudakis KG, Vrabie D, Lewis FL,. Online learning algorithm for optimal control with integral reinforcement learning algorithm. International Journal of Robust and Nonlinear Control 2013. DOI: 10.1002/rnc.3018.
30. Ioannou P, Fidan B,. Adaptive Control Tutorial. SIAM, Advances in Design and Control: PA, 2006.
31. Chowdhary GV,. Concurrent learning for convergence in adaptive control without persistency of excitation, PhD Thesis, Georgia Institute of Technology, 2010.
32. Chowdhary GV, Yucelen T, Mhlegg M, Johnson EN,. Concurrent learning adaptive control of linear systems with exponentially convergent bounds. International Journal of adaptive Control and Signal Processing 2012; 27 (4): 280-301.
33. Modares H, Lewis FL, Naghibi Sistani MB, Chowdhary GV, Yucelen T,. Adaptive optimal control for the partially-unknown constrained-input using policy iteration with experience replay. Proceedings of AIAA Guidance, Navigation, and Control Conference, Boston, Massachusetts, August 2013; pp. 1-11.
34. Modares H, Lewis FL, Naghibi Sistani MB,. Integral reinforcement learning and experience replay for adaptive optimal control of partially-unknown constrained-input continuous-time systems. Automatica 2014; 50 (1): 193-202.
35. Lewis FL, Vrabie D, Syrmos V,. Optimal Control (Third edn). Wiley: New York, 2012.
36. Werbos PJ,. Approximate dynamic programming for real-time control and neural modeling. In Handbook of Intelligent Control, White DA, Sofge DA, (eds). Van Nostrand Reinhold: New York, 1992.
37. Bertsekas DP, Tsitsiklis JN,. Neuro-Dynamic Programming. Athena Scientific: MA, 1996.
38. Finlayson BA,. The Method of Weighted Residuals and Variational Principles. Academic Press: New York, 1990.
39. Hornik K, Stinchcombe M, White H,. Universal approximation of an unknown mapping and its derivatives using multilayer feedforward networks. Neural Networks 1990; 3: 551-560.
40. Khalil HK,. Nonlinear Systems. Prentice-Hall: New Jersey, 1996.