Adaptive adversarial multi-armed bandit approach to two-person zero-sum markov games

IEEE Transactions on Automatic Control

Volumn 55, Issue 2, 2010, Pages 463-468

  Chang, Hyeong Soo ^a   Hu, Jiaqiao ^b   Fu, Michael C ^c   Marcus, Steven I ^d  

^a Sogang University   (South Korea)

^b Stony Brook University   (United States)

^c UNIVERSITY OF MARYLAND   (United States)

^d UNIVERSITY OF MARYLAND   (United States)

Author keywords

Multi armed bandit; Sample average approximation; Sampling; Two person zero sum Markov game (MG)

Indexed keywords

ASYMPTOTIC CONVERGENCE; EQUILIBRIUM VALUE; FINITE HORIZONS; ITERATION BOUND; MARKOV GAMES; MULTI ARMED BANDIT; MULTI-ARMED BANDIT PROBLEM; SAMPLE AVERAGE APPROXIMATION; SAMPLING-BASED ALGORITHMS; STATE SPACE; TECHNICAL NOTES; TIME AND SPACE;

EID: 76949084015     PISSN: 00189286     EISSN: None     Source Type: Journal    
DOI: 10.1109/TAC.2009.2036333     Document Type: Article

References (13)

1. E. Altman, "Zero-sum Markov games and worst-cast optimal control of queueing systems," Queueing Syst., Theory Appl., vol.21, pp. 415-447, 1995.
2. P. Auer, N. Cesa-Bianchi, Y. Freund, and R. E. Schapire, "The nonstochastic multiarmed bandit problem," SIAM J. Comput., vol.32, no.1, pp. 48-77, 2002.
3. N. Cesa-Bianchi and G. Lugosi, Prediction, Learning, and Games. Cambridge, U.K.: Cambridge Univ. Press, 2006.
4. H. S. Chang and S. I. Marcus, "Two-person zero-sum Markov games: Receding horizon approach," IEEE Trans. Autom. Control, vol.48, no.11, pp. 1951-1961, Nov. 2003.
5. J. Filar and K. Vrieze, Competitive Markov Decision Processes. New York: Springer-Verlag, 1996.
6. M. Kearns, Y. Mansour, and S. Singh, "Fast planning in stochastic games," in Proc. 16th Conf. Uncertainty Artif. Intell., 2000, pp. 309-316.
7. A. J. Kleywegt, A. Shapiro, and T. Homem-De-Mello, "The sample average approximation method for stochastic discrete optimization," SIAM J. Optim., vol.12, no.2, pp. 479-502, 2001.
8. S. Lakshmivarahan and K. S. Narendra, "Learning algorithms for twoperson zero-sum stochastic games with incomplete information," Math. Oper. Res., vol.6, pp. 379-386, 1981.
9. S. Lakshmivarahan and K. S. Narendra, "Learning algorithms for twoperson zero-sum stochastic games with incomplete information: A unified approach," SIAM J. Control Optim., vol.20, pp. 541-552, 1982.
10. K. Rajaraman and P. S. Sastry, "Finite time analysis of the pursuit algorithm for learning automata," IEEE Trans. Syst., Man, Cybern. B, vol.26, no.4, pp. 590-598, Aug. 1996.
11. H. Robbins, "Some aspects of the sequential design of experiments," Bull. Amer. Math. Soc., vol.55, pp. 527-535, 1952.
12. P. S. Sastry, V. V. Phansalkar, and M. A. L. Thathachar, "Decentralized learning of Nash equilibria in multi-person stochastic games with incomplete information," IEEE Trans. Syst., Man, Cybern., vol.24, no.5, pp. 769-777, May 1994.
13. J. Van Der Wal, "Stochastic Dynamic Programming: Successive Approximations and Nearly Optimal Strategies for Markov Decision Processes and Markov Games," Ph.D. dissertation, Department of Mathematics, Technische Hogeschool Eindhoven, Eindhoven, The Netherlands, 1980.