Convergent multiple-timescales reinforcement learning algorithms in normal form games

Annals of Applied Probability

Volumn 13, Issue 4, 2003, Pages 1231-1251

  Leslie, David S ^a   Collins, E J ^a  

^a UNIVERSITY OF BRISTOL   (United Kingdom)

Author keywords

Best response dynamics; Reinforcement learning; Repeated normal form games; Stochastic approximation

Indexed keywords

EID: 0346913265     PISSN: 10505164     EISSN: None     Source Type: Journal    
DOI: 10.1214/aoap/1069786497     Document Type: Article

References (20)

1. BENAÏM, M. (1999). Dynamics of stochastic approximation algorithms. Le Séminaire de Probabilités XXXIII. Lecture Notes in Math. 1709 1-68. Springer, Berlin.
2. BENAÏM, M. and HIRSCH, M. W. (1999). Mixed equilibria and dynamical systems arising from fictitious play in perturbed games, Games Econom. Behav. 29 36-72.
3. BERTSEKAS, D. P. and TSITSIKLIS, J. N. (1996). Neuro-Dynamic Programming. Athena Scientific, Belmont, MA.
4. BORKAR, V. S. (1997). Stochastic approximation with two timescales. Systems Control Lett. 29 291-294.
5. BORKAR, V. S. (2002). Reinforcement learning in Markovian evolutionary games. Available at www.tcs.tifr.res.in/̃borkar/games.ps.
6. COWAN, S. (1992), Dynamical systems arising from game theory. Ph.D. dissertation, Univ. California, Berkeley.
7. FUDENBERG, D. and KREPS, D. M. (1993). Learning mixed equilibria. Games Econom. Behav. 5 320-367.
8. FUDENBERG, D. and LEVINE, D. K. (1998). The Theory of Learning in Games. MIT Press, Cambridge, MA.
9. HARSANYI, J. (1973). Games with randomly disturbed payoffs: A new rationale for mixed-strategy equilibrium points. Internat. J. Game Theory 2 1-23.
10. HOFBAUER, J. and HOPKINS, E. (2002). Learning in perturbed asymmetric games. Available at www.econ.ed.ac.uk/pdf/perturb.pdf.
11. HOPKINS, E. (1999). A note on best response dynamics. Games Econom. Behav. 29 138-150.
12. JONES, C. K. R. T. (1995). Geometric singular perturbation theory. Dynamical Systems. Lecture Notes in Math. 1609 44-118. Springer, Berlin.
13. JORDAN, J. S. (1993). Three problems in learning mixed strategy equilibria. Games Econom. Behav. 5 368-386.
14. KONDA, V. R. and BORKAR, V. S. (2000). Actor-critic-type learning algorithms for Markov decision process. SIAM J. Control Opt. 38 94-123.
15. KUSHNER, H. J. and CLARK, D. S. (1978). Stochastic Approximation Methods for Constrained and Unconstrained Systems. Springer, New York.
16. LITTMAN, M. and STONE, P. (2001). Implicit negotiation in repeated games. Intelligent agents VIII: Agent Theories, Architectures and Languages. Lecture Notes in Comput. Sci. 2333 393-404. Springer, Berlin.
17. NASH, J. (1951). Non-cooperative games. Ann. Math. 54 286-295.
18. PEMANTLE, R. (1990). Nonconvergence to unstable points in urn models and stochastic approximations. Ann. Probab. 18 698-712.
19. SHAPLEY, L. S. (1964). Some topics in two person games. In Advances in Game Theory (M. Dresher, L. S. Shapley and A. W. Tucker, eds.) 1-28. Princeton Univ. Press.
20. SUTTON, R. S. and BARTO, A. G. (1998). Reinforcement Learning: An Introduction. MIT Press.