Nonlinear two-player zero-sum game approximate solution using a Policy Iteration algorithm

Proceedings of the IEEE Conference on Decision and Control

Volumn , Issue , 2011, Pages 142-147

  Johnson, M ^a   Bhasin, S ^a   Dixon, W E ^a  

^a UNIVERSITY OF FLORIDA   (United States)

Author keywords

[No Author keywords available]

Indexed keywords

CONTINUOUS TIME SYSTEMS; GAME THEORY; GRADIENT METHODS; ROBUST CONTROL;

APPROXIMATE SOLUTION; DYNAMIC NEURAL NETWORKS; GRADIENT DESCENT METHOD; OPTIMAL VALUE FUNCTIONS; POLICY ITERATION ALGORITHMS; TEMPORAL DIFFERENCE ERRORS; UNCERTAIN DYNAMICS; UNIFORMLY ULTIMATELY BOUNDED;

CLOSED LOOP SYSTEMS;

EID: 84860670757     PISSN: 07431546     EISSN: 25762370     Source Type: Conference Proceeding    
DOI: 10.1109/CDC.2011.6160778     Document Type: Conference Paper

References (40)

1. R. Isaacs, Differential games: a mathematical theory with applications to warfare and pursuit, control and optimization. Dover Pubns, 1999.
2. S. Tijs, Introduction to Game Theory. Hindustand Book Agency, 2003.
3. T. Basar and G. Olsder, Dynamic Noncooperative Game Theory. SIAM, PA, 1999.
4. M. Abu-Khalaf and F. Lewis, "Nearly optimal control laws for nonlinear systems with saturating actuators using a neural network HJB approach," Automatica, vol. 41, no. 5, pp. 779-791, 2005.
5. T. Basar and P. Bernhard, H-infinity Optimal Control and Related Minimax Design Problems. Boston: Birkhäuser, 2008.
6. A. Barto, R. Sutton, and C. Anderson, "Neuron-like adaptive elements that can solve difficult learning control problems," IEEE Trans. Syst. Man Cybern., vol. 13, no. 5, pp. 834-846, 1983.
7. R. S. Sutton and A. G. Barto, Reinforcement Learning: An Introduction. MIT Press, 1998.
8. R. Sutton, A. Barto, and R. Williams, "Reinforcement learning is direct adaptive optimal control," IEEE Contr. Syst. Mag., vol. 12, no. 2, pp. 19-22, 1992.
9. J. Campos and F. Lewis, "Adaptive critic neural network for feedforward compensation," in Proc. Am. Control Conf., vol. 4, 1999.
10. A. Al-Tamimi, F. L. Lewis, and M. Abu-Khalaf, "Adaptive critic designs for discrete-time zero-sum games with application to h-[infinity] control," IEEE Trans. Syst. Man Cybern. Part B Cybern., vol. 37, pp. 240-247, 2007.
11. P. Werbos, "Approximate dynamic programming for real-time control and neural modeling," in Handbook of Intelligent Control: Neural, Fuzzy, and Adaptive Approaches, D. A. White and D. A. Sofge, Eds. New York: Van Nostrand Reinhold, 1992.
12. D. Bertsekas and J. Tsitsiklis, Neuro-Dynamic Programming. Athena Scientific, 1996.
13. D. V. Prokhorov and I. Wunsch, D. C., "Adaptive critic designs," IEEE Trans. Neural Networks, vol. 8, pp. 997-1007, 1997.
14. A. Al-Tamimi, F. L. Lewis, and M. Abu-Khalaf, "Discrete-time nonlinear HJB solution using approximate dynamic programming: Convergence proof," IEEE Trans. Syst. Man Cybern. Part B Cybern., vol. 38, pp. 943-949, 2008.
15. -, "Model-free q-learning designs for linear discrete-time zero-sum games with application to h-[infinity] control," Automatica, vol. 43, pp. 473-481, 2007.
16. S. Balakrishnan, "Adaptive-critic-based neural networks for aircraft optimal control," J. Guid. Contr. Dynam., vol. 19, no. 4, pp. 893-898, 1996.
17. G. Lendaris, L. Schultz, and T. Shannon, "Adaptive critic design for intelligent steering and speed control of a 2-axle vehicle," in Int. Joint Conf. Neural Netw., 2000, pp. 73-78.
18. S. Ferrari and R. Stengel, "An adaptive critic global controller," in Proc. Am. Control Conf., vol. 4, 2002.
19. D. Han and S. Balakrishnan, "State-constrained agile missile control with adaptive-critic-based neural networks," IEEE Trans. Control Syst. Technol., vol. 10, no. 4, pp. 481-489, 2002.
20. P. He and S. Jagannathan, "Reinforcement learning neural-network-based controller for nonlinear discrete-time systems with input constraints," IEEE Trans. Syst. Man Cybern. Part B Cybern., vol. 37, no. 2, pp. 425-436, 2007.
21. L. Baird, "Advantage updating," Wright Lab, Wright-Patterson Air Force Base, OH, Tech. Rep., 1993.
22. K. Doya, "Reinforcement learning in continuous time and space," Neural Comput., vol. 12, no. 1, pp. 219-245, 2000.
23. J. Murray, C. Cox, G. Lendaris, and R. Saeks, "Adaptive dynamic programming," IEEE Trans. Syst. Man Cybern. Part C Appl. Rev., vol. 32, no. 2, pp. 140-153, 2002.
24. R. Beard, G. Saridis, and J. Wen, "Galerkin approximations of the generalized Hamilton-Jacobi-Bellman equation," Automatica, vol. 33, pp. 2159-2178, 1997.
25. D. Vrabie and F. Lewis, "Neural network approach to continuous-time direct adaptive optimal control for partially unknown nonlinear systems," Neural Networks, vol. 22, no. 3, pp. 237 - 246, 2009.
26. K. Vamvoudakis and F. Lewis, "Online synchronous policy iteration method for optimal control," in Recent Advances in Intelligent Control Systems. Springer, 2009, pp. 357-374.
27. -, "Online neural network solution of nonlinear two-player zero-sum games using synchronous policy iteration," in Proc. IEEE Conf. Decis. Control, 2010.
28. S. Bhasin, M. Johnson, and W. E. Dixon, "A model-free robust policy iteration algorithm for optimal control of nonlinear systems," in Proc. IEEE Conf. Decis. Control, 2010, pp. 3060-3065.
29. R. Sutton, "Learning to predict by the methods of temporal differences," Mach. Learn., vol. 3, no. 1, pp. 9-44, 1988.
30. F. L. Lewis, Optimal Control. John Wiley & Sons, 1986.
31. A. Van der Schaft, "L2-gain analysis of nonlinear systems and nonlinear H-[infinity] control," IEEE Trans. Autom. Control, vol. 37, no. 6, pp. 770-784, 1992.
32. D. Kirk, Optimal Control Theory: An Introduction. Dover Pubns, 2004.
33. M. Polycarpou and P. Ioannou, "Identification and control of nonlinear systems using neural network models: Design and stability analysis," Systems Report 91-09-01, University of Southern California, 1991.
34. G. Cybenko, "Approximation by superpositions of a sigmoidal function," Math. Control Signals Syst., vol. 2, pp. 303-314, 1989.
35. F. L. Lewis, R. Selmic, and J. Campos, Neuro-Fuzzy Control of Industrial Systems with Actuator Nonlinearities. Philadelphia, PA, USA: Society for Industrial and Applied Mathematics, 2002.
36. F. L. Lewis, "Nonlinear network structures for feedback control," Asian J. Control, vol. 1, no. 4, pp. 205-228, 1999.
37. J. Hopfield, "Neurons with graded response have collective computational properties like those of two-state neurons," Proc. Nat. Acad. Sci. U.S.A., vol. 81, no. 10, p. 3088, 1984.
38. A. Poznyak, E. Sanchez, and W. Yu, Differential neural networks for robust nonlinear control: identification, state estimation and trajectory tracking. World Scientific Pub Co Inc, 2001.
39. J. Slotine and W. Li, Applied Nonlinear Control. Prentice Hall, 1991.
40. H. K. Khalil, Nonlinear Systems, 3rd ed. Prentice Hall, 2002.