Transient and asymptotic dynamics of reinforcement learning in games

Games and Economic Behavior

Volumn 61, Issue 2, 2007, Pages 259-276

  Izquierdo, Luis R ^a   Izquierdo, Segismundo S ^b   Gotts, Nicholas M ^c   Polhill, J Gary ^c  

^a UNIVERSITY OF BURGOS   (Spain)

^b UNIVERSITY OF VALLADOLID   (Spain)

^c SCOTTISH CROP RESEARCH INSTITUTE   (United Kingdom)

Author keywords

Bush and Mosteller; Distance diminishing; Learning in games; Reinforcement learning; Slow learning; Stochastic approximation

Indexed keywords

EID: 35048853187     PISSN: 08998256     EISSN: 10902473     Source Type: Journal    
DOI: 10.1016/j.geb.2007.01.005     Document Type: Article

References (46)

1. Arthur W.B. Designing economic agents that act like human agents: A behavioral approach to bounded rationality. Amer. Econ. Rev. 81 2 (1991) 353-359
2. Beggs A. Stochastic evolution with slow learning. Econ. Theory 19 (2002) 379-405
3. Beggs A.W. On the convergence of reinforcement learning. J. Econ. Theory 122 (2005) 1-36
4. Bendor J., Mookherjee D., and Ray D. Aspiration-based reinforcement learning in repeated interaction games: An overview. Int. Game Theory Rev. 3 2-3 (2001) 159-174
5. Bendor J., Mookherjee D., and Ray D. Reinforcement learning in repeated interaction games. Adv. Theor. Econ. 1 1 (2001) Article 3
6. Benveniste A., Métivier M., and Priouret P. Adaptive Algorithms and Stochastic Approximations (1990), Springer-Verlag, Berlin
7. Binmore K., and Samuelson L. An economist's perspective on the evolution of norms. J. Institutional Theoret. Econ. 150 (1993) 45-63
8. Binmore K., Samuelson L., and Vaughan R. Musical chairs: Modeling noisy evolution. Games Econ. Behav. 11 1 (1995) 1-35
9. Börgers T., and Sarin R. Learning through reinforcement and replicator dynamics. J. Econ. Theory 77 (1997) 1-14
10. Börgers T., and Sarin R. Naive reinforcement learning with endogenous aspirations. Int. Econ. Rev. 41 (2000) 921-950
11. Boylan R.T. Laws of large numbers for dynamical systems with randomly matched individuals. J. Econ. Theory 57 (1992) 473-504
12. Boylan R.T. Continuous approximation of dynamical systems with randomly matched individuals. J. Econ. Theory 66 (1995) 615-625
13. Bush R., and Mosteller F. Stochastic Models of Learning (1955), Wiley & Sons, New York
14. Camerer C.F. Behavioral Game Theory: Experiments in Strategic Interaction (2003), Russell Sage Foundation, New York
15. Chen Y., and Tang F. Learning and incentive-compatible mechanisms for public goods provision: An experimental study. J. Polit. Economy 106 (1998) 633-662
16. Cross J.G. A stochastic learning model of economic behavior. Quart. J. Econ. 87 (1973) 239-266
17. Duffy J. Agent-based models and human subject experiments. In: Tesfatsion L., and Judd K.L. (Eds). Handbook of Computational Economics II: Agent-Based Computational Economics (2006), Elsevier, North-Holland 949-1011 (Chapter 19)
18. Erev I., and Roth A.E. Predicting how people play games: Reinforcement learning in experimental games with unique, mixed strategy equilibria. Amer. Econ. Rev. 88 4 (1998) 848-881
19. Erev I., and Roth A.E. Simple reinforcement learning models and reciprocation in the Prisoner's Dilemma game. In: Gigerenzer G., and Selten R. (Eds). Bounded Rationality: The Adaptive Toolbox (2001), MIT Press, Cambridge, MA 216-231 (Chapter 12)
20. Erev I., Bereby-Meyer Y., and Roth A.E. The effect of adding a constant to all payoffs: Experimental investigation, and implications for reinforcement learning models. J. Econ. Behav. Organ. 39 1 (1999) 111-128
21. Flache A., and Macy M.W. Stochastic collusion and the power law of learning. J. Conflict Resolution 46 5 (2002) 629-653
22. Hopkins E. Two competing models of how people learn in games. Econometrica 70 (2002) 2141-2166
23. Hopkins E., and Posch M. Attainability of boundary points under reinforcement learning. Games Econ. Behav. 53 1 (2005) 110-125
24. Hyslop J.M. Infinite Series. fifth ed (1965), Oliver & Boyd, Edinburgh
25. Ianni, A., 2001. Reinforcement learning and the power law of practice. Mimeo. University of Southampton
26. Karandikar R., Mookherjee D., Ray D., and Vega-Redondo F. Evolving aspirations and cooperation. J. Econ. Theory 80 (1998) 292-331
27. Kushner H.J., and Yin G.G. Stochastic Approximation Algorithms and Applications (1997), Springer-Verlag, New York
28. Laslier J., and Walliser B. A reinforcement learning process in extensive form games. Int. J. Game Theory 33 (2005) 219-227
29. Laslier J., Topol R., and Walliser B. A behavioral learning process in games. Games Econ. Behav. 37 (2001) 340-366
30. Macy M.W., and Flache A. Learning dynamics in social dilemmas. Proc. Natl. Acad. Sci. USA 99 3 (2002) 7229-7236
31. Maier N.R.F., and Schneirla T.C. Principles of Animal Psychology (1964), Dover Publications, New York
32. McAllister P.H. Adaptive approaches to stochastic programming. Ann. Oper. Res. 30 (1991) 45-62
33. Mookherjee D., and Sopher B. Learning behavior in an experimental matching pennies game. Games Econ. Behav. 7 (1994) 62-91
34. Mookherjee D., and Sopher B. Learning and decision costs in experimental constant sum games. Games Econ. Behav. 19 (1997) 97-132
35. Norman M.F. Some convergence theorems for stochastic learning models with distance diminishing operators. J. Math. Psychol. 5 (1968) 61-101
36. Norman M.F. Markov Processes and Learning Models (1972), Academic Press, New York
37. Palomino F., and Vega-Redondo F. Convergence of aspirations and (partial) cooperation in the Prisoner's Dilemma. Int. J. Game Theory 28 4 (1999) 465-488
38. Phansalkar V.V., Sastry P.S., and Thathachar M.A.L. Absolutely expedient algorithms for learning Nash equilibria. Proc. Indian Acad. Sci. Math. Sci. 104 1 (1994) 279-294
39. Posch M. Cycling in a stochastic learning algorithm for normal form games. J. Evolutionary Econ. 7 (1997) 193-207
40. Roth A.E., and Erev I. Learning in extensive-form games: Experimental data and simple dynamic models in the intermediate term. Games Econ. Behav. 8 (1995) 164-212
41. Rustichini A. Optimal properties of stimulus-response learning models. Games Econ. Behav. 29 (1999) 244-273
42. Sastry P.S., Phansalkar V.V., and Thathachar M.A.L. Decentralized learning of Nash equilibria in multi-person stochastic games with incomplete information. IEEE Trans. Syst. Man Cybernet. 24 5 (1994) 769-777
43. Selten R. Re-examination of the perfectness concept for equilibrium points in extensive games. Int. J. Game Theory 4 (1975) 25-55
44. Thorndike E.L. Animal Intelligence: An Experimental Study of the Associative Processes in Animals. Psychological Review, Monograph Supplements vol. 8 (1898), MacMillan, New York
45. Weibull J.W. Evolutionary Game Theory (1995), MIT Press, Cambridge, MA
46. Wustmann G., Rein K., Wolf R., and Heisenberg M. A new paradigm for operant conditioning of drosophila melanogaster. J. Comp. Physiol. (A) 179 (1996) 429-436