Numerical analysis of a reinforcement learning model with the dynamic aspiration level in the iterated Prisoner's dilemma

Journal of Theoretical Biology

Volumn 278, Issue 1, 2011, Pages 55-62

  Masuda, Naoki ^a,b   Nakamura, Mitsuhiro ^a  

^a UNIVERSITY OF TOKYO   (Japan)

^b JAPAN SCIENCE AND TECHNOLOGY AGENCY   (Japan)

Author keywords

Cooperation; Direct reciprocity; Prisoner's dilemma; Reinforcement learning

Indexed keywords

ENVIRONMENTAL CUE; LEARNING; NUMERICAL METHOD; SOCIAL BEHAVIOR;

ARTICLE; CONTROLLED STUDY; COOPERATION; ENVIRONMENTAL FACTOR; MATHEMATICAL ANALYSIS; MATHEMATICAL MODEL; MOTIVATION; PRIORITY JOURNAL; PRISONER; PROBABILITY; REINFORCEMENT; SIMULATION; SOCIAL BEHAVIOR;

ANIMALS; ASPIRATIONS (PSYCHOLOGY); BIOLOGICAL EVOLUTION; COMPUTER SIMULATION; GAME THEORY; HUMANS; LEARNING; MODELS, BIOLOGICAL; NUMERICAL ANALYSIS, COMPUTER-ASSISTED; REINFORCEMENT (PSYCHOLOGY); TIME FACTORS;

ANIMALIA;

EID: 79952856548     PISSN: 00225193     EISSN: 10958541     Source Type: Journal    
DOI: 10.1016/j.jtbi.2011.03.005     Document Type: Article

References (47)

1. Axelrod R. Evolution of Cooperation 1984, Basic Books, NY.
2. Bendor J., Diermeier D., Ting M. A behavioral model of turnout. Am. Political Sci. Rev. 2003, 97(2):261-280.
3. Börgers T., Sarin R. Naive reinforcement learning with endogenous aspirations. Int. Econ. Rev. 2000, 41(4):921-950.
4. Boyd R., Richerson P.J. Culture and the Evolutionary Process 1985, The University of Chicago Press, Chicago.
5. Camerer C.F. Behavioral Game Theory 2003, Princeton University Press, NJ.
6. Cho I.K., Matsui A. Learning aspiration in repeated games. J. Econ. Theory 2005, 124(2):171-201.
7. Cross J.G. A Theory of Adaptive Economic Behavior 1983, Cambridge University Press, Cambridge.
8. Daw N.D., Doya K. The computational neurobiology of learning and reward. Curr. Opin. Neurobiol. 2006, 16:199-204.
9. Dixon H.D. Keeping up with the joneses: competition and the evolution of collusion. J. Econ. Behav. Organ. 2000, 43(2):223-238.
10. Erev, I., 1998. Predicting how people play games: reinforcement learning in experimental games with unique, mixed strategy equilibria. Am. Econ. Rev. 88, 848-881.
11. Erev I., Roth A.E. Simple reinforcement learning models and reciprocation in the Prisoner's dilemma game. The Adaptive Toolbox 2001, 215-231. G. Gigerenzer, R. Selten (Eds.).
12. Fudenberg D., Levine D.K. The Theory of Learning in Games 1998, MIT Press, MA.
13. Glimcher P.W., Camerer C.F., Fehr E., Poldrack R.A. Neuroeconomics-Decision Making and the Brain 2009, Academic Press, London.
14. Gutnisky D.A., Zanutto B.S. Cooperation in the iterated Prisoner's dilemma is learned by operant conditioning mechanisms. Artif. Life 2004, 10(4):433-461.
15. Hauert C., Stenull O. Simple adaptive strategy wins the Prisoner's dilemma. J. Theor. Biol. 2002, 218(3):261-272.
16. Izquierdo L.R., Izquierdo S.S., Gotts N.M., Polhill J.G. Transient and asymptotic dynamics of reinforcement learning in games. Games Econ. Behav. 2007, 61(2):259-276.
17. Izquierdo S.S., Izquierdo L.R., Gotts N.M. Reinforcement learning dynamics in social dilemmas. J. Artif. Soc. Soc. Simul. 2008, 11(2):1.
18. Karandikar R., Mookherjee D., Ray D., Vega-Redondo F. Evolving aspirations and cooperation. J. Econ. Theory 1998, 80(2):292-331.
19. Kim Y. Satisficing and optimality in 2×2 common interest games. Econ. Theory 1999, 13(2):365-375.
20. Kraines D., Kraines V. Pavlov and the Prisoner's dilemma. Theory Decis. 1989, 26(1):47-79.
21. Lindgren K. Evolutionary phenomena in simple dynamics. Proceedings of Artificial Life II 1991, 295-312.
22. Macy M. Natural selection and social learning in Prisoner's dilemma. Sociol. Meth. Res. 1996, 25(1):103-137.
23. Macy M.W. Learning to cooperate: stochastic and tacit collusion in social exchange. Am. J. Sociol. 1991, 97(3):808-843.
24. Macy M.W., Flache A. Learning dynamics in social dilemmas. Proc. Natl. Acad. Sci. USA 2002, 99:7229-7236.
25. Masuda N., Ohtsuki H. A theoretical analysis of temporal difference learning in the iterated Prisoner's dilemma game. Bull. Math. Biol. 2009, 71:1818-1850.
26. Montague P.R., Berns G.S. Neural economics and the biological substrates of valuation. Neuron 2002, 36:265-284.
27. Napel S. Aspiration adaptation in the ultimatum minigame. Games Econ. Behav. 2003, 43(1):86-106.
28. Nowak M. Stochastic strategies in the Prisoner's dilemma. Theor. Popul. Biol. 1990, 38:93-112.
29. Nowak M., Sigmund K. Game-dynamical aspects of the Prisoner's dilemma. Appl. Math. Comput. 1989, 30:191-213.
30. Nowak M., Sigmund K. The evolution of stochastic strategies in the Prisoner's dilemma. Acta Applicandae Math. 1990, 20:247-265.
31. Nowak M., Sigmund K. A strategy of win-stay, lose-shift that outperforms tit-for-tat in the Prisoner's dilemma game. Nature 1993, 364:56-58.
32. Nowak M.A. Evolutionary Dynamics 2006, The Belknap Press of Harvard University Press, MA.
33. Nowak M.A., Sigmund K. Tit for tat in heterogeneous populations. Nature 1992, 355:250-253.
34. Nowak M.A., Sigmund K., El-Sedy E. Automata, repeated games and noise. J. Math. Biol. 1995, 33(7):703-722.
35. Oechssler J. Cooperation as a result of learning with aspiration levels. J. Econ. Behav. Organ. 2002, 49(3):405-409.
36. Palomino F., Vega-Redondo F. Convergence of aspirations and (partial) cooperation in the Prisoner's dilemma. Int. J. Game Theory 1999, 28(4):465-488.
37. Pazgal A. Satisficing leads to cooperation in mutual interests games. Int. J. Game Theory 1997, 26(4):439-453.
38. Posch M., Pichler A., Sigmund K. The efficiency of adapting aspiration levels. Proc. R. Soc. London B 1999, 266(1427):1427-1435.
39. Rapoport A., Chammah A.M. Prisoner's Dilemma: A Study in Conflict and Cooperation 1965, Michigan University Press, Ann Arbor, MI.
40. Sandholm T.W., Crites R.H. Multiagent reinforcement learning in the iterated Prisoner's dilemma. Biosystems 1996, 37(1-2):147-166.
41. Schultz W., Dayan P., Montague P.R. A neural substrate of prediction and reward. Science 1997, 275:1593-1599.
42. Sigmund K. The Calculus of Selfishness 2010, Princeton University Press, Princeton, NJ.
43. Simon H.A. Theories of decision-making in economics and behavioral science. Am. Econ. Rev. 1959, 49(3):253-283.
44. Szabó G., Tke C. Evolutionary Prisoner's dilemma game on a square lattice. Phys. Rev. E 1998, 58(1):69-73.
45. Taiji M., Ikegami T. Dynamics of internal models in game players. Physica D 1999, 134(2):253-266.
46. Traulsen A., Nowak M.A., Pacheco J.M. Stochastic dynamics of invasion and fixation. Phys. Rev. E 2006, 74(1):011909.
47. Trivers R.L. The evolution of reciprocal altruism. Q. Rev. Biol. 1971, 46:35-57.