Policy iteration algorithm for distributed networks and graphical games

Proceedings of the IEEE Conference on Decision and Control

Volumn , Issue , 2011, Pages 128-135

  Vamvoudakis, Kyriakos G ^a   Lewis, F L ^a  

^a UNIVERSITY OF TEXAS AT ARLINGTON   (United States)

Author keywords

best response; cooperative Hamilton Jacobi equations; graphical games; Nash equilibrium; Policy Iteration

Indexed keywords

COMPUTATION THEORY; DYNAMICAL SYSTEMS; GRAPH ALGORITHMS; ITERATIVE METHODS; MULTI AGENT SYSTEMS; REINFORCEMENT LEARNING; TOPOLOGY;

BEST RESPONSE; COOPERATIVE HAMILTON-JACOBI EQUATIONS; GRAPHICAL GAMES; NASH EQUILIBRIA; POLICY ITERATION;

GAME THEORY;

EID: 84860652054     PISSN: 07431546     EISSN: 25762370     Source Type: Conference Proceeding    
DOI: 10.1109/CDC.2011.6160491     Document Type: Conference Paper

References (38)

1. H. Abou-Kandil, G. Freiling, V. Ionescu, and G. Jank, Matrix Riccati Equations in Control and Systems Theory, Birkhäuser, 2003.
2. nd ed. Philadelphia, PA: SIAM, 1999.
3. D. P. Bertsekas and J. N. Tsitsiklis, Neuro-Dynamic Programming, Athena Scientific, MA, 1996.
4. J.W. Brewer, "Kronecker products and matrix calculus in system theory," IEEE Transactions Circuits and Systems, vol. 25, 1978, pp. 772-781.
5. L. Busoniu, R. Babuska, B. De Schutter, "A Comprehensive Survey of Multi-Agent Reinforcement Learning," IEEE Transactions on Systems, Man, and Cybernetics - Part C: Applications and Reviews, vol. 38, no. 2, pp. 156-172, 2008.
6. T. Dierks and S. Jagannathan, Optimal Control of Affine Nonlinear Continuous-time Systems Using an Online Hamilton-Jacobi-Isaacs Formulation1, Proc. IEEE Conf Decision and Control, Atlanta, pp. 3048-3053, 2010.
7. J. Fax and R. Murray, "Information flow and cooperative control of vehicle formations," IEEE Trans. Autom. Control, vol. 49, no. 9, pp. 1465-1476, Sep. 2004.
8. G. Freiling, G. Jank, H. Abou-Kandil, "On global existence of Solutions to Coupled Matrix Riccati equations in closed loop Nash Games," IEEE Transactions on Automatic Control, vol. 41, no. 2, pp. 264-269, 2002.
9. Y. Hong, J. Hu, and L. Gao, "Tracking control for multi-agent consensus with an active leader and variable topology," Automatica, vol. 42, no. 7, pp. 1177-1182, 2006.
10. Z. Gajic and T-Y. Li, "Simulation results for two new algorithms for solving coupled algebraic Riccati equations," Third Int. Symp. On Differential Games, Sophia, Antipolis, France, June 1988.
11. A. Jadbabaie, J. Lin, and A. Morse, "Coordination of groups of mobile autonomous agents using nearest neighbor rules," IEEE Trans. Autom. Control, vol. 48, no. 6, pp. 988-1001, Jun. 2003.
12. M. Johnson, T. Hiramatsu, N. Fitz-Coy, and W. E. Dixon, "Asymptotic Stackelberg Optimal Control Design for an Uncertain Euler Lagrange System," IEEE Conference on Decision and Control, pp. 6686-6691, 2010
13. S. Kakade, M. Kearns, J. Langford, and L. Ortiz, "Correlated equilibria in graphical games," Proc. 4th ACM Conference on Electronic Commerce, pp. 42-47, 2003.
14. th Annual Conference on Uncertainty in Artificial Intelligence, pp. 253-260, 2001.
15. S. Khoo, L. Xie, and Z. Man, "Robust Finite-Time Consensus Tracking Algorithm for Multirobot Systems," IEEE Transactions on Mechatronics, vol. 14, pp. 219-228.
16. R. J. Leake, Ruey-Wen Liu, "Construction of Suboptimal Control Sequences," J. SIAM Control, vol. 5, no, 1, pp. 54-63, 1967.
17. F. L. Lewis, V. L. Syrmos, Optimal Control, John Wiley, 1995.
18. F. Lewis, Applied Optimal Control and Estimation: Digital Design and Implementation, New Jersey: Prentice-Hall, 1992.
19. X. Li, X. Wang, and G. Chen, "Pinning a complex dynamical network to its equilibrium," IEEE Trans. Circuits Syst. I, Reg. Papers, vol. 51, no. 10, pp. 2074-2087, Oct. 2004.
20. M.L. Littman, "Value-function reinforcement learning in Markov games," Journal of Cognitive Systems Research 1, 2001.
21. R. Olfati-Saber, J. Fax, and R. Murray, "Consensus and cooperation in networked multi-agent systems," Proc. IEEE, vol. 95, no. 1, pp. 215-233, Jan. 2007.
22. R. Olfati-Saber and R.M. Murray, "Consensus Problems in Networks of Agents with Switching Topology and Time-Delays," IEEE Transaction of Automatic Control, vol. 49, pp. 1520-1533, 2004.
23. Z. Qu, Cooperative Control of Dynamical Systems: Applications to Autonomous Vehicles, New York: Springer-Verlag, 2009.
24. W. Ren, R. Beard, and E. Atkins, "A survey of consensus problems in multi-agent coordination," in Proc. Amer. Control Conf, Portland, OR, pp. 1859-1864, 2005.
25. W. Ren and R. Beard, "Consensus seeking in multiagent systems under dynamically changing interaction topologies," IEEE Trans. Autom. Control, vol. 50, no. 5, pp. 655-661, May 2005.
26. W. Ren and R.W. Beard, Distributed Consensus in Multi-vehicle Cooperative Control, Springer, Berlin, 2008.
27. W. Ren, K. Moore, and Y. Chen, "High-order and model reference consensus algorithms in cooperative control of multivehicle systems," J. Dynam. Syst., Meas., Control, vol. 129, no. 5, pp. 678-688, 2007.
28. Y. Shoham, K. Leyton-Brown, Multiagent Systems: Algorithmic, Game-Theoretic, and Logical Foundations, Cambridge University Press, 2009.
29. R. S. Sutton, A. G. Barto, Reinforcement Learning - An Introduction, MIT Press, Cambridge, Massachusetts, 1998.
30. S. Tijs, Introduction to Game Theory, Hindustan Book Agency, India, 2003.
31. J. Tsitsiklis, "Problems in Decentralized Decision Making and Computation," Ph.D. dissertation, Dept. Elect. Eng. and Comput. Sci., MIT, Cambridge, MA 1984.
32. K.G Vamvoudakis, and F. L. Lewis, "Online Actor-Critic Algorithm to Solve the Continuous-Time Infinite Horizon Optimal Control Problem," Automatica, vol. 46, no. 5, pp. 878-888, 2010.
33. K.G. Vamvoudakis, and F. L. Lewis, "Multi-Player Non-Zero Sum Games: Online Adaptive Learning Solution of Coupled Hamilton- Jacobi Equations," Automatica, vol. 47, no. 8, pp. 1556-1569, 2011.
34. D. Vrabie, O. Pastravanu, F. L. Lewis, & M. Abu-Khalaf, "Adaptive Optimal Control for continuous-time linear systems based on policy iteration," Automatica, 45(2), 477-484, 2009
35. P. Vrancx, K. Verbeeck, and A. Nowe, "Decentralized learning in markov games," IEEE Transactions on Systems, Man and Cybernetics, vol. 38, no. 4, pp. 976-981, August 2008.
36. X. Wang and G. Chen, "Pinning control of scale-free dynamical networks," Physica A, vol. 310, no. 3-4, pp. 521-531, 2002.
37. P.J. Werbos, Beyond Regression: New Tools for Prediction and Analysis in the Behavior Sciences, Ph.D. Thesis, 1974.
38. P. J. Werbos, "Approximate dynamic programming for real-time control and neural modeling," Handbook of Intelligent Control, ed. D.A White and D.A. Sofge, New York: Van Nostrand Reinhold, 1992.