Predicting How People Play Games: Reinforcement Learning in Experimental Games with Unique, Mixed Strategy Equilibria

American Economic Review

Volumn 88, Issue 4, 1998, Pages 848-881

  Erev, Ido ^a,b   Roth, Alvin E ^c,d  

^a TECHNION ISRAEL INSTITUTE OF TECHNOLOGY   (Israel)

^b UNIVERSITY OF PITTSBURGH   (United States)

^c HARVARD UNIVERSITY   (United States)

^d HARVARD BUSINESS SCHOOL   (United States)

Author keywords

[No Author keywords available]

Indexed keywords

EID: 0038829878     PISSN: 00028282     EISSN: None     Source Type: Journal    
DOI: None     Document Type: Article

References (105)

1. Anderson, John R. "Acquisition of Cognitive Skill." Psychological Review, July 1982, 89(4), pp. 369-403.
2. _. The architecture of cognition. Cambridge, MA: Harvard University Press, 1993.
3. Arthur, W. Brian. "Designing Economic Agents That Act Like Human Agents: A Behavioral Approach to Bounded Rationality." American Economic Review, May 1991 (Papers and Proceedings), 81(2), pp. 353-59.
4. _. "On Designing Economic Agents That Behave Like Human Agents." Journal of Evolutionary Economics, February 1993, 3(1), pp. 1-22.
5. Atkinson, Richard C. and Suppes, Patrick. "An Analysis of Two-Person Game Situations in Terms of Statistical Learning Theory." Journal of Experimental Psychology, April 1958, 55(4), pp. 369-78.
6. Bereby-Meyer, Yoella and Erev, Ido. "On Learning to Become a Successful Loser: A Comparison of Alternative Abstractions of Learning in the Loss Domain." Journal of Mathematical Psychology, 1998 (forthcoming).
7. Binmore, Kenneth G.; Gale, John and Samuelson, Larry. "Learning to Be Imperfect: The Ultimatum Game." Games and Economic Behavior, January 1995, 8(1), pp. 56-90.
8. Blackburn, J. M. "Acquisition of Skill: An Analysis of Learning Curves." IHRB Report No. 73, 1936.
9. Bolton, Gary. "A Comparative Model of Bargaining: Theory and Evidence." American Economic Review, December 1991, 81(5), pp. 1096-136.
10. Bolton, Gary and Ockenfels, Axel. "ERC: A Theory of Equity, Reciprocity and Competition." Mimeo, Penn State University, 1997.
11. Borgers, Tilman and Sarin, Rajiv. "Learning Through Reinforcement and Replicator Dynamics." Mimeo, University College London, 1994.
12. _. "Naive Reinforcement Learning with Endogenous Aspirations." Mimeo, University College London, 1995.
13. Bornstein, Gary; Erev, Ido and Goren, Harel. "The Effect of Repeated Play in the IPG and IPD Team Games." Journal of Conflict Resolution, December 1994, 38(4), pp. 690-707.
14. Bornstein, Gary; Winter, Eyal and Goren, Harel. "Experimental Study of Repeated Team Games." European Journal of Political Economy, December 1996, 12(4), pp. 629-39.
15. Brandts, Jordi and Holt, Charles A. "An Experimental Test of Equilibrium Dominance in Signaling Games." American Economic Review, December 1992, 82(5), pp. 1350-65.
16. _. "Adjustment Patterns and Equilibrium Selection in Experimental Signaling Games." International Journal of Game Theory, 1993, 22(3), pp. 279-302.
17. Brown, George W. "Iterative Solutions of Games by Fictitious Play," in Tjalling C. Koopmans, ed., Activity analysis of production and allocation. New York: Wiley, 1951, pp. 374-76.
18. Brown, J. S.; Clark, F. R. and Stein, L. "A New Technique for Studying Spatial Generalization with Voluntary Responses." Journal of Experimental Psychology, April 1958, 55(4), pp. 359-62.
19. Brown, James N. and Rosenthal, Robert W. "Testing the Minimax Hypothesis: A Reexamination of O'Neill's Game Experiment." Econometrica, September 1990, 58(5), pp. 1065-81.
20. Busemeyer, Jerome R. and Myung, In Jae. "An Adaptive Approach to Human Decision Making: Learning Theory, Decision Theory, and Human Performance." Journal of Experimental Psychology: General, June 1992, 121(2), pp. 177-94.
21. Bush, Robert and Mosteller, Frederick. Stochastic models for learning. New York: Wiley, 1955.
22. Camerer, Colin. "Behavioral Game Theory," in Robin Hogarth, ed., Insights in decision making: A tribute to Hillel J. Einhorn. Chicago: University of Chicago Press, 1990, pp. 311-36.
23. Camerer, Colin and Ho, Teck-Hua. "Experience Weighted Attraction Learning in Normal-Form Games." Econometrica, 1998a (forthcoming).
24. _. "Experience-Weighted Attraction Learning in Games: Estimates from Weak-Link Games," in David V. Budescu, Ido Erev, and Rami Zwick, eds., Games and human behavior: Essays in honor of Amnon Rapoport's 60th birthday. Hillsdale, NJ: Erlbaum, 1998b (forthcoming).
25. Cheung, Yin-Wong and Friedman, Daniel. "Learning in Evolutionary Games: Some Laboratory Results." Mimeo, University of California, Santa Cruz, 1994.
26. _. "Individual Learning in Normal Form Games: Some Laboratory Results." Mimeo, University of California, Santa Cruz, December 1995.
27. _. "A Comparison of Learning and Replicator Dynamics Using Experimental Data." Mimeo, University of California, Santa Cruz, October 1996.
28. Cooper, David and Feltovich, Nick. "Reinforcement-Based Learning vs. Bayesian Learning: A Comparison." Mimeo, University of Pittsburgh, 1996.
29. Cooper, David; Garvin, Susan and Kagel, John. "Signalling and Adaptive Learning in An Entry Limit Pricing Game." Mimeo, University of Pittsburgh, 1994.
30. Cox, James; Shacht, Jason and Walker, Mark. "An Experiment to Evaluate Bayesian Learning of Nash Equilibrium." Mimeo, University of Arizona, 1995.
31. Crawford, Vincent P. "An 'Evolutionary' Interpretation of Van Huyck, Battalio, and Beil's Experimental Results on Coordination." Games and Economic Behavior, February 1991, 3(1), pp. 25-59.
32. _. "Adaptive Dynamics in Coordination Games." Mimeo, University of California, San Diego, 1992.
33. Cross, John G. A theory of adaptive economic behavior. Cambridge: Cambridge University Press, 1983.
34. Crossman, E. R. F. W. "A Theory of Acquisition of Speed-Skill." Ergonomics, November 1958, 2(1), pp. 153-66.
35. Daniel, Terry E.; Seale, Darryl A. and Rapoport, Amnon. "Strategic Play and Adaptive Learning in the Sealed Bid Bargaining Mechanism." Mimeo, University of Arizona, April 1996.
36. Dickhaut, John; Mukherji, Arijit; Rajan, Vijay and Sevcik, Galen. "An Experimental Study of Learning, Equilibrium Refinements and Local Interaction in Games." Mimeo, University of Minnesota, 1995.
37. Duffy, John and Feltovich, Nick. "The Effect of Information on Learning in Strategic Environments: An Experimental Study." International Journal of Game Theory, 1998 (forthcoming).
38. Erev, Ido. "Signal Detection by Human Observers: A Cutoff Reinforcement Learning Model of Categorization Decisions Under Uncertainty." Psychological Review, 1998, 105(2), pp. 280-98.
39. Erev, Ido; Gopher, Daniel; Itkin, Revital and Greenshpan, Yaacov. "Toward a Generalization of Signal Detection Theory to N-Person Games: The Example of Two-Person Safety Problem." Journal of Mathematical Psychology, December 1995, 39(4), pp. 360-75.
40. Erev, Ido; Maital, Shlomo and Or-Hof, Ori. "Melioration, Adaptive Learning and the Effect of Constant Re-evaluations of Strategies," in G. Antoniedes, F. van Raaij, and S. Maital, eds., Advances in economic psychology. Sussex, U.K.: Wiley, 1997, pp. 237-53.
41. Erev, Ido and Rapoport, Amnon. "Magic, Reinforcement Learning, and Coordination in a Market Entry Game." Games and Economic Behavior, 1998, 23, pp. 146-75.
42. Erev, Ido and Roth, Alvin E. "On the Need for Low Rationality, Cognitive Game Theory: Reinforcement Learning in Experimental Games with Unique Mixed Strategy Equilibria." Mimeo, University of Pittsburgh, 1996.
43. Estes, William K. "Toward a Statistical Theory of Learning." Psychological Review, March 1950, 57(2), pp. 94-107.
44. Fehr, Ernst and Schmidt, Klaus. "How to Account for Fair and Unfair Outcomes - A Model of Biased Inequality Aversion." Paper presented at Gerzensee [Switzerland] Symposium on Economic Theory, July 1997.
45. Feltovich, Nick. "Learning and Equilibrium in an Asymmetric Information Game: An Experimental Study." Ph.D. dissertation, University of Pittsburgh, 1997.
46. Fudenberg, Drew and Levine, David K. "Measuring Players' Losses in Experimental Games." Quarterly Journal of Economics, May 1997a, 112(2), pp. 507-36.
47. _. Theory of learning in games. Draft, http://levine.sscnet.ucla.edu/papers/contents.htm (accessed March 10, 1997), 1997b.
48. Gilat, Sharon; Meyer, Joachim; Erev, Ido and Gopher, Daniel. "Beyond Bayes' Theorem: The Effect of Base Rate Information in Consensus Games." Journal of Experimental Psychology: Applied, June 1997, 3(2), pp. 83-104.
49. Guth, Werner. "On Ultimatum Bargaining Experiments - A Personal Review." Journal of Economic Behavior and Organization, August 1995, 27(3), pp. 329-44.
50. Guthrie, Edwin R. The psychology of learning. New York: Harper, 1952.
51. Guttman, N. and Kalish, H. "Discriminability and Stimulus Generalization." Journal of Experimental Psychology, January 1956, 51(1), pp. 79-88.
52. Harley, Calvin B. "Learning the Evolutionarily Stable Strategy." Journal of Theoretical Biology, April 1981, 89(4), pp. 611-33.
53. Herrnstein, Richard J. "On the Law of Effect." Journal of the Experimental Analysis of Behavior, March 1970, 13(2), pp. 243-66.
54. Kagel, John H. and Roth, Alvin, E. Handbook of experimental economics. Princeton, NJ: Princeton University Press, 1995.
55. Kahneman, Daniel and Tversky, Amos. "Prospect Theory: An Analysis of Decision Under Risk." Econometrica, March 1979, 47(2), pp. 263-91.
56. Lee, Wayne. Decision theory and human behavior. New York: Wiley, 1971.
57. Luce, Duncan R. Individual choice behavior. New York: Wiley, 1959.
58. Macy, Michael W. "Learning to Cooperate: Stochastic and Tacit Collusion in Social Exchange." American Journal of Sociology, November 1991, 97(3), pp. 808-43.
59. Malcolm, David and Lieberman, Bernhardt. "The Behavior of Responsive Individuals Playing a Two-Person, Zero-Sum Game Requiring the Use of Mixed Strategies." Psychonomic Science, June 1965, (12), pp. 373-74.
60. Maynard-Smith, John. Evolution and the theory of games. Cambridge: Cambridge University Press, 1982.
61. McKelvey, Richard D. and Palfrey, Thomas R. "Quantal Response Equilibria for Normal Form Games." Games and Economic Behavior, July 1995, 10(1), pp. 6-38.
62. Messick, David M. and Liebrand, Wim B. G. "Individual Heuristics and the Dynamics of Cooperation in Large Groups." Psychological Review, January 1995, 102(1), pp. 131-45.
63. Mookherjee, Dilip and Sopher, Barry. "Learning Behavior in an Experimental Matching Pennies Game." Games and Economic Behavior, July 1994, 7(1), pp. 62-91.
64. _. "Learning and Decision Costs in Experimental Constant Sum Games." Games and Economic Behavior, April 1997, 19 (1), pp. 97-132.
65. Nagel, Rosemarie. "Unraveling in Guessing Games: An Experimental Study." American Economic Review, December 1995, 85(5), pp. 1313-26.
66. Ochs, Jack. "Simple Games with Unique Mixed Strategy Equilibrium: An Experimental Study." Games and Economic Behavior, July 1995, 10(1), pp. 202-17.
67. Ochs, Jack and Roth, Alvin E. "An Experimental Study of Sequential Bargaining." American Economic Review, June 1989, 79(3), pp. 355-84.
68. O'Neill, Barry. "Nonmetric Test of the Minimax Theory of Two-Person Zerosum Games." Proceedings of the National Academy of Sciences, USA, April 1987, 84(7), pp. 2106-9.
69. Payne, John W.; Bettman, James R. and Johnson, Eric J. The adaptive decision maker. Cambridge: Cambridge University Press, 1993.
70. Rabin, Matthew. "Incorporating Fairness into Game Theory and Econometrics." American Economic Review, December 1993, 83(5), pp. 1281-303.
71. Rapoport, Amnon and Boebel, Richard B. "Mixed Strategies in Strictly Competitive Games: A Further Test of the Minmax Hypothesis." Games and Economic Behavior, April 1992, 4(2), pp. 261-83.
72. Rapoport, Amnon and Budescu, David V. "Generation of Random Series in Two-Person Strictly Competitive Games." Journal of Experimental Psychology: General, September 1992, 121(3), pp. 352-63.
73. Rapoport, Amnon; Erev, Ido; Abraham, Elizabeth V. and Olson, David E. "Randomization and Adaptive Learning in a Simplified Poker Game." Organizational Behavior and Human Decision Processes, January 1997, 69(1), pp. 31-49.
74. Rapoport, Amnon; Seale, Darryl A.; Erev, Ido and Sundali, James A. "Coordination Success in Market Entry Games: Tests of Equilibrium and Adaptive Learning Models." Management Science, 1998, 44, pp. 119-41.
75. Rapoport, Anatol; Guyer, Melvin J. and Gordon, David G. The 2 × 2 game. Ann Arbor, MI: University of Michigan Press, 1976.
76. Robinson, Julia. "An Iterative Method of Solving a Game." Annals of Mathematics, September 1951, 54(2), pp. 296-301.
77. Roth, Alvin E. "The Evolution of the Labor Market for Medical Interns and Residents: A Case Study in Game Theory." Journal of Political Economy, December 1984, 92(6), pp. 991-1016.
78. _. "New Physicians: A Natural Experiment in Market Organization." Science, December 14, 1990, 250(4987), pp. 1524-28.
79. _. "A Natural Experiment in the Organization of Entry-Level Labor Markets: Regional Markets for New Physicians and Surgeons in the United Kingdom." American Economic Review, June 1991, 81(3), pp. 415-40.
80. _. "Introduction to Experimental Economics," in John Kagel and Alvin E. Roth, eds., Handbook of experimental economics. Princeton, NJ: Princeton University Press, 1995, pp. 3-109.
81. _. "Individual Rationality as a Useful Approximation: Comments on Tversky's 'Rational Theory and Constructive Choice,'" in K. Arrow, E. Colombatto, M. Perlman, and C. Schmidt, eds., The rational foundations of economic behavior. London: Macmillan, 1996a, pp. 198-202; http:// www.economics.harvard.edu/faculty/roth/ rational.html.
82. _. "Report on the Design and Testing of an Applicant Proposing Matching Algorithm, and Comparison with the Existing NRMP Algorithm." http://www.pitt.edu/ economics.harvard.edu/faculty/roth/ phase1.html, 1996b.
83. Roth, Alvin E. and Erev, Ido. "Learning in Extensive-Form Games: Experimental Data and Simple Dynamic Models in the Intermediate Term." Games and Economic Behavior, Special Issue: Nobel Symposium, January 1995, 8(1), pp. 164-212.
84. Roth, Alvin E. and Peranson, Elliott. "The Effects of a Change in the NRMP Matching Algorithm." Journal of the American Medical Association, September 3, 1997, 278(9), pp. 729-32.
85. Roth, Alvin E.; Prasnikar, Vesna; Okuno-Fujiwara, Masahiro and Zamir, Shmuel. "Bargaining and Market Behavior in Jerusalem, Ljubljana, Pittsburgh, and Tokyo: An Experimental Study." American Economic Review, December 1991, 81(5), pp. 1068-95.
86. Roth, Alvin E. and Xing, Xiaolin. "Jumping the Gun: Imperfections and Institutions Related to the Timing of Market Transactions." American Economic Review, September 1994, 84(4), pp. 992-1044.
87. Selten, Reinhard. "Die Strategiemethode zur Erforschung des Eingeschrankt Rationalen Verhaltens im Rahmen eines Oligopolexperiments," in Heinz Sauermann, ed., Beitrage zur experimentellen Wirtschaftsforschung. Tubingen, Germany: Mohr, 1967, pp. 136-68.
88. _. "Axiomatic Characterization of the Quadratic Scoring Rule." Experimental Economics, 1998, 1(1), pp. 43-62.
89. Selten, Reinhard and Buchta, Joachim. "Experimental Sealed Bid First Price Auction with Directly Observed Bid Functions." Discussion Paper No. B270, University of Bonn, Germany, 1994.
90. Selten, Reinhard; Mitzkewitz, Michael and Uhlich, Gerald R. "Duopoly Strategies Programmed by Experienced Players." Mimeo, University of Bonn, Germany, 1988.
91. Skinner, B. F. Science and human behavior. New York: Macmillan, 1953.
92. Slonim, Robert and Roth, Alvin E. "Learning in High-Stakes Ultimatum Games: An Experiment in the Slovak Republic." Econometrica, May 1998, 66(3), pp. 569-96.
93. Stahl, Dale O. "Boundedly Rational Rule Learning in a Guessing Game." Games and Economic Behavior, October 1996a, 16(2), pp. 303-30.
94. _. "Evidence Based Rule Learning in Symmetric Normal-Form Games." Mimeo, University of Texas, April 1996b.
95. Suppes, Patrick and Atkinson, Richard C. Markov learning models for multiperson interactions. Stanford, CA: Stanford University Press, 1960.
96. Tang, Fang-Fang. "Anticipatory Learning in Two-Person Games: An Experimental Study, Part I, Equilibrium and Stability." Discussion Paper No. B-362, University of Bonn, Germany, March 1996a.
97. _. "Anticipatory Learning in Two-Person Games: An Experimental Study, Part II, Learning." Discussion Paper No. B-363, University of Bonn, Germany, March 1996b.
98. _. "Anticipatory Learning in Two-Person Games: An Experimental Study, Part III, Individual Analysis." Discussion Paper No. B-364, University of Bonn, Germany, March 1996c.
99. Thorndike, Edward L. "Animal Intelligence: An Experimental Study of the Associative Processes in Animals." Psychological Monographs, 1898, 2(8).
100. Tinkelepaugh, L. H. "An Experimental Study of Representative Factors in Monkeys." Journal of Comparative Psychology, June 1928, 8(3), pp. 197-236.
101. Tversky, Amos and Kahneman, Daniel. "Judgment Under Uncertainty: Huristics and Biases." Science, September 27, 1974, 185(4157), pp. 1124-31.
102. Van Huyck, John B.; Cook, Joseph P. and Battalio, Raymond C. "Selection Dynamics, Asymptotic Stability, and Adaptive Behavior." Journal of Political Economy, October 1994, 102(5), pp. 975-1005.
103. Watson, John B. Behaviorism, 2nd. Ed. Chicago: University of Chicago Press, 1930.
104. Weibull, Jorgen W. Evolution and the theory of games. Cambridge, MA: MIT Press, 1995.
105. Weisbuch, Gerard; Kirman, Alan and Herreiner, Dorothea. "Market Organization." http:// www.lps.ens.fr/∼weisbuch/mark.html (accessed May 21, 1996), 1996.