Near-optimal control for nonzero-sum differential games of continuous-time nonlinear systems using single-network ADP

IEEE Transactions on Cybernetics

Volumn 43, Issue 1, 2013, Pages 206-216

  Zhang, Huaguang ^a   Cui, Lili ^b   Luo, Yanhong ^c  

^a STATE KEY LABORATORY OF SYNTHETICAL AUTOMATION FOR PROCESS INDUSTRIES   (China)

^b SHENYANG NORMAL UNIVERSITY   (China)

^c NORTHEASTERN UNIVERSITY   (China)

Author keywords

Adaptive dynamic programming (ADP); Continuous time nonlinear systems; Neural networks (NNs); Nonzero sum differential games; Optimal control

Indexed keywords

ADAPTIVE DYNAMIC PROGRAMMING; CONTINUOUS TIME NONLINEAR SYSTEMS; NEURAL NETWORKS (NNS); NONZERO-SUM DIFFERENTIAL GAME; OPTIMAL CONTROLS;

CONTINUOUS TIME SYSTEMS; CONTROL; GAME THEORY; NEURAL NETWORKS; NONLINEAR SYSTEMS;

DYNAMIC PROGRAMMING;

EID: 84885835001     PISSN: 21682267     EISSN: None     Source Type: Journal    
DOI: 10.1109/TSMCB.2012.2203336     Document Type: Article

References (30)

1. H. Mukaidani, "Newton's method for solving cross-coupled sign-indefinite algebraic Riccati equations for weakly coupled large-scale systems," Appl. Math. Comput., vol. 188, no. 1, pp. 103-115, May 2007.
2. V. Shah, "Power Control for Wireless Data Services Based on Utility and Pricing," M.S. thesis, Rutgers Univ., Piscataway, NJ, 1998.
3. A. W. Starr and Y. C. Ho, "Nonzero-sum differential games," J. Optim. Theory Appl., vol. 3, no. 3, pp. 184-206, 1969.
4. T. Baser and G. J. Olsder, Dynamic Noncooperative Game Theory. Philadelphia, PA: SIAM, 1998.
5. R. E. Bellman, Dynamic Programming. Princeton, NJ: Princeton Univ. Press, 1957.
6. M. Jungers, E. De Pieri, and H. Abou-Kandil, "Solving coupled riccati equations for closed-loop Nash strategy, by lack of trust approach," Int. J. Tomogr. Stat., vol. 7, no. F07, pp. 49-54, 2007.
7. F.-Y. Wang, H. Zhang, and D. Liu, "Adaptive dynamic programming: An introduction," IEEE Comput. Intell. Mag., vol. 4, no. 2, pp. 39-47, May 2009.
8. D. P. Bertsekas and J. N. Tsitsiklis, Neuro-Dynamic Programming. Belmont, MA: Athena Scientific, 1996.
9. J. Si, A. G. Barto, W. B. Powell, and D. W. II, Handbook of Learning and Approximate Dynamic Programming. New York: Wiley, 2004.
10. M. Abu-Khalaf and F. L. Lewis, "Nearly optimal control laws for nonlinear systems with saturating actuators using a neural network HJB approach," Automatica, vol. 41, no. 5, pp. 779-791, May 2005.
11. J. J. Murray, C. J. Cox, G. G. Lendaris, and R. Saeks, "Adaptive dynamic programming," IEEE Trans. Syst., Man, Cybern. C, Appl. Rev., vol. 32, no. 2, pp. 140-153, May 2002.
12. D. Vrabie, O. Pastravanu, M. Abu-Khalaf, and F. L. Lewis, "Adaptive optimal control for continuous-time linear systems based on policy iteration," Automatica, vol. 45, no. 2, pp. 477-484, Feb. 2009.
13. K. G. Vamvoudakis and F. L. Lewis, "Online actor-critic algorithm to solve the continuous-time infinite horizon optimal control problem," Automatica, vol. 46, no. 5, pp. 878-888, May 2010.
14. T. Hanselmann, L. Noakes, and A. Zaknich, "Continuous-time adaptive critics," IEEE Trans. Neural Netw., vol. 18, no. 3, pp. 631-647, May 2007.
15. T. Dierks and S. Jagannathan, "Optimal control of affine nonlinear continuous-time systems," in Proc. IEEE Amer. Control Conf., Baltimore, MD, Jun. 2010, pp. 1568-1573.
16. A. Al-Tamimi, F. L. Lewis, and M. Abu-Khalaf, "Discrete-time nonlinear HJB solution using approximate dynamic programming: Convergence proof," IEEE Trans. Syst., Man, Cybern. B, Cybern., vol. 38, no. 4, pp. 943-949, Aug. 2008.
17. F. L. Lewis and K. G. Vamvoudakis, "Reinforcement learning for partially observable dynamic processes: Adaptive dynamic programming using measured output data," IEEE Trans. Syst., Man, Cybern. B, Cybern., vol. 41, no. 1, pp. 14-25, Feb. 2011.
18. H. Zhang, Y. Luo, and D. Liu, "Neural-network-based near-optimal control for a class of discrete-time affine nonlinear systems with control constraints," IEEE Trans. Neural Netw., vol. 20, no. 9, pp. 1490-1503, Sep. 2009.
19. H. Zhang, Q. Wei, and Y. Luo, "A novel infinite-time optimal tracking control scheme for a class of discrete-time nonlinear system based on greedy HDP iteration algorithm," IEEE Trans. Syst., Man, Cybern. B, Cybern., vol. 38, no. 4, pp. 937-942, Aug. 2008.
20. T. Dierks and S. Jagannathan, "Optimal tracking control of affine nonlinear discrete-time systems with unknown internal dynamics," in Proc. Joint 48th IEEE Conf. Decision Control/28th Chin. Control Conf., Shanghai, China, Dec. 2009, pp. 6750-6755.
21. T. Dierks, T. Balaje, and S. Jagannathan, "Optimal control of unknown affine nonlinear discrete-time systems using offline-trained neural networks with proof of convergence," Neural Netw., vol. 22, no. 5/6, pp. 851-860, Aug. 2009.
22. R. Padhi, N. Unnikrishnan, X. Wang, and S. N. Balakrishnan, "A single network adaptive critic (SNAC) architecture for optimal control synthesis for a class of nonlinear systems," Neural Netw., vol. 19, no. 10, pp. 1648-1660, Dec. 2006.
23. M. Abu-Khalaf, F. L. Lewis, and J. Huang, "Neurodynamic programming and zero-sum games for constrained control systems," IEEE Trans. Neural Netw., vol. 19, no. 7, pp. 1243-1252, Jul. 2008.
24. Q. Wei, H. Zhang, and L. Cui, "Data-based optimal control for discretetime zero-sum games of 2-D systems using adaptive critic designs," ACTA Autom. Sin., vol. 35, no. 6, pp. 682-692, Jun. 2009.
25. A. Al-Tamimi, M. Abu-Khalaf, and F. L. Lewis, "Adaptive critic designs for discrete-time zero-sum games with application to H? control," IEEE Trans. Syst., Man, Cybern. B, Cybern., vol. 37, no. 1, pp. 240-247, Feb. 2007.
26. H. Zhang, Q. Wei, and D. Liu, "An iterative adaptive dynamic programming method for solving a class of nonlinear zero-sum differential games," Automatica, vol. 47, no. 1, pp. 207-214, Jan. 2011.
27. T. Dierks and S. Jagannathan, "Optimal control of affine nonlinear continuous-time systems using an online Hamilton-Jacobi-Isaacs formulation," in Proc. 49th IEEE Conf. Decision Control, Atlanta, GA, Dec. 2010, pp. 3048-3053.
28. D. Vrabie and F. L. Lewis, "Integral reinforcement learning for online computation of feedback Nash strategies of nonzero-sum differential games," in Proc. 49th IEEE Conf. Decision Control, Atlanta, GA, Dec. 2010, pp. 3066-3071.
29. K. G. Vamvoudakisand and F. L. Lewis, "Multi-player non-zero-sum games: Online adaptive learning solution of coupled Hamilton-Hacobi equations," Automatica, vol. 47, no. 8, pp. 1556-1569, Aug. 2011. doi:DOI:10.1016/j.automatica.2011.03.005.
30. H. K. Khalil, Nonlinear System. Englewood Cliffs, NJ: Prentice-Hall, 1996.