Online solution of nonquadratic two-player zero-sum games arising in the H∞ control of constrained input systems

International Journal of Adaptive Control and Signal Processing

Volumn 28, Issue 3-5, 2014, Pages 232-254

  Modares, Hamidreza ^a   Lewis, Frank L ^b   Sistani, Mohammad Bagher Naghibi ^a  

^a FERDOWSI UNIVERSITY OF MASHHAD   (Iran)

^b UNIVERSITY OF TEXAS AT ARLINGTON   (United States)

Author keywords

Control; H ; Input constraints; Neural networks; Policy iteration; Two player zero sum games

Indexed keywords

ALGORITHMS; CONTROL; GAME THEORY; ITERATIVE METHODS; NEURAL NETWORKS; NONLINEAR CONTROL SYSTEMS;

CONSTRAINED INPUT SYSTEMS; INPUT CONSTRAINTS; NON-LINEAR BENCHMARK PROBLEMS; ONLINE LEARNING ALGORITHMS; PERSISTENCE OF EXCITATION; POLICY ITERATION; POLICY ITERATION ALGORITHMS; ZERO-SUM GAME;

ONLINE SYSTEMS;

EID: 84899093084     PISSN: 08906327     EISSN: 10991115     Source Type: Journal    
DOI: 10.1002/acs.2348     Document Type: Article

References (42)

1. Zames G. Feedback and optimal sensitivity: model reference transformations, multiplicative seminorms, and approximate inverses. IEEE Transactions on Automatic Control 1981; 26 (2):301-320.
2. Doyle JH, Glover K, Khargonekar P, Francis B. State-space solutions to standard H2 and H∞ control problems. IEEE Transactions on Automatic Control 1989; 34 (8):831-847.
3. Van der Schaft AJ. L2-gain analysis of nonlinear systems and nonlinear state feedback H∞ control. IEEE Transactions on Automatic Control 1992; 37 (6):770-784.
4. Ball J, Helton W. Viscosity solutions of Hamilton-Jacobi equations arising in nonlinear H∞ control. Journal of Mathematical Systems, Estimation, and Control 1996; 6 (1):1-22.
5. Bardi M, Capuzzo-Dolcetta I. Optimal Control and Viscosity Solutions of Hamilton-Jacobi-Bellman Equations. Birkhauser: Boston, MA, 1997.
6. Isidori A, Lin W. Global L2-gain design for a class of nonlinear systems. Systems and Control Letters 1998; 34 (5):245-252.
7. Isidori A. H∞ control via measurement feedback for affine nonlinear systems. International Journal of Robust and Nonlinear Control 1994; 4:553-574.
8. Basar T, Bernard P. H∞ Optimal Control and Related Minimax Design Problems. Birkhäuser: Boston, MA, 1995.
9. Lanzon A, Feng Y, Anderson BDO, Rotkowitz M. Computing the positive stabilizing solution to algebraic Riccati equations with an indefinite quadratic term via a recursive method. IEEE Transactions on Automatic Control 2008; 53 (10):2280-2291.
10. Cherfi L, Abou-Kandil H, Bourles H. Iterative method for general algebraic Riccati equation. In Proceedings of International Conference on Automatic Control and System Engineering, Cairo, Egypt, 2005; 85-88.
11. Damm T. Rational Matrix Equations in Stochastic Control. Springer-Verlag: Berlin, 2004.
12. Huang J, Lin CF. Numerical approach to computing nonlinear H∞ control laws. Journal of Guidance, Control, and Dynamics 1995; 18 (5):989-994.
13. Crandall MG, Ishii H, Lions PL. User's guide to viscosity solutions of second order partial differential equations. Bulletin of the American Mathematical Society 1992; 27:1-67.
14. Bertsekas DP, Tsitsiklis JN. Neuro-Dynamic Programming. Athena Scientific: MA, 1996.
15. Howard RA. Dynamic Programming and Markov Processes. MIT Press: Cambridge, Massachusetts, 1960.
16. Si J, Barto A, Powel W, Wunch D. Handbook of Learning and Approximate Dynamic Programming. John Wiley: New Jersey, 2004.
17. Sutton RS, Barto AG. Reinforcement Learning-An Introduction. MIT Press: Cambridge, Massachusetts, 1998.
18. Werbos PJ. Beyond regression: new tools for prediction and analysis in the behavior sciences. PhD Thesis, 1974.
19. Werbos PJ. Approximate dynamic programming for real-time control and neural modeling. In Handbook of Intelligent Control, White DA, Sofge DA (eds). Multiscience Press: New York, 1992.
20. Prokhorov DV, Wunsch DC. Adaptive critic designs. IEEE Transactions on Neural Networks 1997; 8 (5):997-1007.
21. Vrabie D, Vamvoudakis K, Lewis FL. Adaptive optimal controllers based on generalized policy iteration in a continuous-time framework. In Proceedings of the IEEE Mediterranean Conference on Control and Automation, Makedonia Palace, Thessaloniki, Greece, 2009; 1402-1409.
22. Beard R, McLain T. Successive Galerkin approximation algorithms for nonlinear optimal and robust control. International Journal of Control 1998; 71 (5):717-743.
23. Abu-Khalaf M, Lewis FL, Huang J. Neurodynamic programming and zero-sum games for constrained control systems. IEEE Transactions on Neural Networks 2008; 19 (7):1243-1252.
24. Zhang H, Wei Q, Liu D. An iterative adaptive dynamic programming method for solving a class of nonlinear zero-sum differential games. Automatica 2011; 47 (1):207-214.
25. Morimoto J, Doya K. Robust reinforcement learning. Neural Computation 2005; 17 (2):335-359.
26. Vrabie D, Lewis FL. Adaptive dynamic programming for online solution of a zero-sum differential game. Journal of Control Theory and Applications 2011; 9 (3):353-360.
27. Vamvoudakis K, Lewis FL. Online Gaming: Real Time Solution of Nonlinear Two-Player Zero-Sum Games Using Synchronous Policy Iteration, Advances in Reinforcement Learning, Abdelhamid Mellouk (Ed.) INTECH: India, 2011; 331-360.
28. Vamvoudakis K, Lewis FL. Online solution of nonlinear two-player zero-sum games using synchronous policy iteration. In Proceedings of 49th IEEE Conference on Decision and Control, Atlanta, 2010; 3040-3047.
29. Vamvoudakis K, Lewis FL. Multi-player non-zero-sum games: online adaptive learning solution of coupled Hamilton-Jacobi equations. Automatica 2011; 47 (8):1556-1569.
30. Lyshevski SE. Constrained optimization and control of nonlinear systems: new results in optimal control. In Proceedings of IEEE Conference on Decision and Control, Kobe, Japan, 1996; 541-546.
31. Abu-Khalaf M, Lewis FL. Nearly optimal control laws for nonlinear systems with saturating actuators using a neural network HJB approach. Automatica 2005; 41:779-791.
32. Adhyaru D, Kar IN, Gopal M. Bounded robust control of nonlinear systems using neural network-based HJB solution. Neural Computing and Applications 2011; 20 (1):91-103.
33. Abu-Khalaf M, Lewis FL, Huang J. Policy iterations on the Hamilton-Jacobi-Isaacs equations for H∞ state feedback control with input saturation. IEEE Transactions on Automatic Control 2006; 51 (12):1989-1995.
34. Lewis FL, Jagannathan S, Yesildirek A. Neural Network Control of Robot Manipulators and Nonlinear Systems. Taylor & Francis: London, 1999.
35. Funahashi K. On the approximate realization of continuous mappings by neural networks. Neural Networks 1989; 2:183-192.
36. Zhang H, Luo Y, Liu D. Neural-network-based near-optimal control for a class of discrete-time affine nonlinear systems with control constraints. IEEE Transactions on Neural Networks 2009; 20 (9):1490-1503.
37. Feng Y, Anderson BD, Rotkowitz M. A game theoretic algorithm to compute local stabilizing solutions to HJBI equations in nonlinear H∞ control. Automatica 2009; 45 (4):881-888.
38. Vamvoudakis K, Lewis FL. Online actor-critic algorithm to solve the continuous infinite time horizon optimal control problem. Automatica 2010; 46:878-888.
39. Bupp R, Bernstein D, Coppola V. A benchmark problem for nonlinear control design. International Journal of Robust and Nonlinear Control 1998; 8:307-310.
40. Tsiotras P, Corless M, Rotea M. An L2 disturbance attenuations solution to the nonlinear benchmark problem. International Journal of Robust Nonlinear Control 1998; 8:311-330.
41. Deng F, Huang J. Computer-aided design of nonlinear H∞ control law: the benchmark problem. In Proceedings of Chinese Control Conference, Dalin, China, 2001; 840-845.
42. Mracek CP, Cloutier CJ. A preliminary control design for the nonlinear benchmark problem. In Proceedings of IEEE International Conference on Control Applications, Dearborn, Mi, 1996; 265-272.