On adaptation, maximization, and reinforcement learning among cognitive strategies

Psychological Review

Volumn 112, Issue 4, 2005, Pages 912-931

  Erev, Ido ^a   Barron, Greg ^b  

^a TECHNION ISRAEL INSTITUTE OF TECHNOLOGY   (Israel)

^b HARVARD BUSINESS SCHOOL   (United States)

Author keywords

Case based reasoning; Decisions from experience; Learning in games; Probability learning; Stickiness effect

Indexed keywords

ADAPTIVE BEHAVIOR; ARTICLE; COGNITION; HUMAN; LEARNING; PSYCHOLOGIC TEST; REINFORCEMENT;

ADAPTATION, PSYCHOLOGICAL; COGNITION; HUMANS; LEARNING; PSYCHOLOGICAL TESTS; REINFORCEMENT (PSYCHOLOGY);

EID: 26844491600     PISSN: 0033295X     EISSN: None     Source Type: Journal    
DOI: 10.1037/0033-295X.112.4.912     Document Type: Article

References (84)

1. Allais, M. (1979). The foundations of a positive theory of choice involving risk and a criticism of the postulates and axioms of the American school. In M. Allais & O. Hagen (Eds.), Expected utility hypotheses and the Allais paradox: Contemporary discussions of decision under uncertainty with Allais' rejoinder (pp. 27-145). Dordrecht, the Netherlands: D. Reidel.
2. Barron, G. (2000). The effect of feedback on decision making under uncertainty and risk: The predictive value of descriptive models. Unpublished master's thesis, Technion-Israel Institute of Technology, Haifa, Israel.
3. Barron, G., & Erev, I. (2003). Small feedback-based decisions and their limited correspondence to description-based decisions. Journal of Behavioral Decision Making, 16, 215-233.
4. Bereby-Meyer, Y. (1997). On the robustness of the framing effect: A re-interpretation of the "payoff effect" in probability learning experiments. Unpublished doctoral dissertation, Technion-Israel Institute of Technology, Haifa, Israel.
5. Bereby-Meyer, Y., & Erev, I. (1998). On learning to become a successful loser: A comparison of alternative abstraction of learning processes in the loss domain. Journal of Mathematical Psychology, 42, 266-286.
6. Berry, D., & Fristedt, B. (1985). Bandit problems; Sequential allocation of experiments. London: Chapman & Hall.
7. Blavatskyy, P. R. (2004). Essays on cumulative prospect theory, individuals' preference for most probable winner, and the design of Olympic prizes. Unpublished doctoral dissertation, Charles University, Prague, Czech Republic.
8. Busemeyer, J. R. (1985). Decision making under uncertainty: A comparison of simple scalability, fixed sample, and sequential sampling models. Journal of Experimental Psychology: Learning, Memory, and Cognition, 11, 538-564.
9. Busemeyer, J. R., & Myung, I. J. (1987). Resource allocation decision making in an uncertain environment. Acta Psychologica 66, 1-19.
10. Busemeyer, J. R., & Myung, I. J. (1992). An adaptive approach to human decision making: Learning theory, decision theory, and human performance. Journal of Experimental Psychology: General, 121, 177-194.
11. Busemeyer, J. R., & Townsend, J. T. (1993). Decision field theory: A dynamic-cognitive approach to decision making in an uncertain environment. Psychological Review, 100, 432-459.
12. Busemeyer, J. R., & Wang, Y.-M. (2000). Model comparisons and model selections based on generalization criterion methodology. Journal of Mathematical Psychology, 44, 171-189.
13. Bush, R., & Mosteller, F. (1955). Stochastic models for learning. New York: Wiley.
14. Camerer, C. F., & Ho, T. (1999). Experience-weighted attraction in games. Econometrica, 64, 827-874.
15. Cheung, Y. W., & Friedman, D. (1998). A comparison of learning and replicator dynamics using experimental data. Journal of Economic Behavior and Organizations, 35, 263-280.
16. Dember, W. N., & Fowler, F. (1958). Spontaneous alternation behavior. Psychological Bulletin, 55, 412-428.
17. Denrell, J., & March, J. G. (2001). Adaptation as information restriction: The hot stove effect. Organization Science, 12, 523-538.
18. Diederich, A., & Busemeyer, J. R.. (1999). Conflict and the stochastic dominance principle of decision making. Psychological Science, 10, 353-359.
19. Edwards, W. (1961). Probability learning in 1000 trials. Journal of Experimental Psychology, 62, 385-394.
20. Erev, I. (1994). Convergence in the orange grove: Learning processes in a social dilemma setting. In U. Schul, W. Albers, & U. Mueller (Eds.), Social dilemmas and cooperation (pp. 187-206). New York: Springer-Verlag.
21. Erev, I. (1998). Signal detection by human observers: A cutoff reinforcement learning model of categorization decisions under uncertainty. Psychological Review, 105, 280-298.
22. Erev, I., Barron, G., & Remington, R. (2004). Right of way in the sky: Two problems in aircraft self-separation and the auction-based solution. Human Factors, 46, 277-287.
23. Erev, I., Bereby-Meyer, Y., & Roth, A. E. (1999). The effect of adding a constant to all payoffs: Experimental investigation, and implications for reinforcement learning models. Journal of Economic Behavior & Organization, 39, 111-128.
24. Erev, I., & Roth, A. E. (1998). Predicting how people play games: Reinforcement learning in games with unique strategy equilibrium. American Economic Review, 88, 848-881.
25. Erev, I., & Roth, A. E. (1999). On the role of reinforcement learning in experimental games: The cognitive game-theoretic approach. In D. Budescu, I. Erev, & R. Zwick (Eds.), Games and human behavior: Essays in honor of Amnon Rapoport (pp. 53-78). Mahwah, NJ: Erlbaum.
26. Erev, I., Roth, A. E., Slonim, S. L., & Barron, G. (2002). Predictive value and the usefulness of game theoretic models. International Journal of Forecasting, 18, 359-368.
27. Estes, W. K. (1950). Toward a statistical theory of learning. Psychological Review, 57, 94-107.
28. Estes, W. K. (1964). Probability learning. In A. W. Melton (Ed.), Categories of human learning (pp. 89-128). New York: Academic Press.
29. Estes, W. K. (1976). The cognitive side of probability learning. Psychological Review, 83, 37-64.
30. Feltovich, N. (2000). Reinforcement-based vs. belief-based learning models in experimental asymmetric-information games. Econometrica, 68, 605-641.
31. Fiedler, K. (2000). Beware of samples! A cognitive-ecological sampling approach to judgment biases. Psychological Review, 107, 659-676.
32. Friedman, J. W., & Mezzetti, C. (2001). Learning in games by random sampling. Journal of Economic Theory, 98, 55-84.
33. Fudenberg, D., & Levine, D. (1998). Theory of learning in games. Cambridge, MA: MIT Press.
34. Gigerenzer, G., Hoffrage, U., & Kleinulting, H. (1991). Probabilistic mental models: A Brunswikian theory of confidence. Psychological Review, 98, 506-528.
35. Gilboa, I., & Schmeidler, D. (1995). Case-based decision theory. The Quarterly Journal of Economics, 110, 605-639.
36. Gneezy, U., & Potters, J. (1997). An experiment on risk taking and evaluation periods. The Quarterly Journal of Economics, 112, 631-645.
37. Gonzalez, R., & Wu, G. (1999). On the shape of the probability weighting function. Cognitive Psychology, 38, 129-166.
38. Grant, D., Hake, H. W., & Hornseth, J. P. (1951). Acquisition and extinction of a verbal conditioned response with differing percentages of reinforcement. Journal of Experimental Psychology, 42, 1-5.
39. Green, L., Price, P. C., & Hamburger, M. E. (1995). Prisoner's dilemma and the pigeon: Control by immediate consequences. Journal of the Experimental Analysis of Behavior, 64, 1-17.
40. Grosskopf, B., Erev, I., & Yechiam, E. (2005). Forgone with the wind. Manuscript submitted for publication.
41. Haruvy, E., & Erev, I. (2001). On the application and interpretation of learning models. In R. Zwick & A. Rapoport (Eds.), Advances in experimental business research (pp. 285-300). Norwell, MA: Kluwer Academic.
42. Haruvy, E., & Erev, I. (2002). Variable pricing: A customer learning perspective. Unpublished manuscript, Technion-Israel Institute of Technology, Haifa, Israel.
43. Haruvy, E., Erev, I., & Sonsino, D. (2001). The medium prizes paradox: Evidence from a simulated casino. Journal of Risk and Uncertainty, 22, 251-261.
44. Herrnstein, R. J., Loewenstein, G. F., Prelec, D., & Vaughan, W., Jr. (1993). Utility maximization and melioration: Internalities in individual choice. Journal of Behavioral Decision Making, 6, 149-185.
45. Hertwig, R., & Ortman, A. (2001). Experimental practices in economics: A methodological challenge for psychologists? Behavioral and Brain Sciences, 24, 383-451.
46. Humphreys, L. G. (1939). The effect of random alternation of reinforcement on the acquisition and extinction of conditioned eyelid reactions. Journal of Experimental Psychology, 25, 141-158.
47. Kahneman, D., & Tversky, A. (1979). Prospect theory: An analysis of decision under risk. Econometrica, 47, 263-291.
48. Kareev, Y. (2000). Seven (indeed, plus or minus two) and the detection of correlations. Psychological Review, 107, 397-402.
49. Katz, L. (1964). Effects of differential monetary gain and loss on sequential two-choice behavior. Journal of Experimental Psychology, 68, 245-249.
50. Lee, W. (1971). Decision theory and human behavior. New York: Wiley.
51. Logan, G. (1988). Toward an instance theory of automatization. Psychological Review, 95, 492-527.
52. Luce, R. D. (1959). Individual choice behavior. New York: Wiley.
53. Luce, R. D., & Suppes, P. (1965). Preference, utility, and subjective probability. In R. D. Luce, R. R. Bush, & E. Galanter (Eds.), Handbook of mathematical psychology (Vol. 3, pp. 249-410). New York: Wiley.
54. March, J. G. (1996). Learning to be risk averse. Psychological Review, 103, 309-319.
55. Mazur, J. E. (1994). Learning and behavior. Englewood Cliffs, NJ: Prentice Hall.
56. Myers, J. L., Fort, J. G., Katz, L., & Suydam, M. M. (1963). Differential monetary gains and losses and event probability in a two-choice situation. Journal of Experimental Psychology, 66, 521-522.
57. Myers, J. L., Reilly, E., & Taub, H. A. (1961). Differential cost, gain, and relative frequency of reward in a sequential choice situation. Journal of Experimental Psychology, 62, 357-360.
58. Myers, J. L., & Sadler, E. (1960). Effects of range of payoffs as a variable in risk taking. Journal of Experimental Psychology, 60, 306-309.
59. Myers, J. L., Suydam, M. M., & Gambino, B. (1965). Contingent gains and losses in risk-taking situations. Journal of Mathematical Psychology, 2, 363-370.
60. Payne, J. W., Bettman, J. R., & Johnson, E. J. (1993). The adaptive decision maker. Cambridge, United Kingdom: Cambridge University Press.
61. Perry, O., Haruvy, E., & Erev, I. (2002). Frequent delayed probabilistic punishment in law enforcement. Economics of Governance, 3, 71-85.
62. Rapoport, A., & Budescu, D. V. (1992). Generation of random series in two-person strictly competitive games. Journal of Experimental Psychology: General, 121, 352-363.
63. Rapoport, A., Daniel, T. E., & Seale, D. A. (1998). Reinforcement-based adaptive learning in asymmetric two-person bargaining with incomplete information. Experimental Economics, 1, 221-253.
64. Rapoport, A., Erev, I., Abraham, E. V., & Olson, D. E. (1997). Randomization and adapative learning in a simplified poker game. Organizational Behavior and Human Decision Processes, 69, 31-49.
65. Riesbeck, C. K., & Schank, R. (1989). Inside case-based reasoning. Northvale, NJ: Erlbaum.
66. Rieskamp, J., Busemeyer, J., & Laine, T. (2003). How do people learn to allocate resources? Comparing two learning theories. Journal of Experimental Psychology: Learning, Memory, and Cognition, 29, 1066-1081.
67. Roe, R. M., Busemeyer, J. R., & Townsend, J. T. (2001). Multialternative decision field theory: A dynamic connectionist model of decision making. Psychological Review, 108, 370-392.
68. Roth, A. E., & Erev, I. (1995). Learning in extensive-form games: Experimental data and simple dynamic models in the intermediate term. Games and Economic Behavior, 8, 164-212.
69. Sarin, R., & Vahid, F. (2001). Predicting how people play games: A simple dynamic model of choice. Games and Economic Behavior, 34, 104-122.
70. Shanks, D. R., Tunney, R. J., & McCarthy, J. D. (2002). A re-examination of probability matching and rational choice. Journal of Behavioral Decision Making, 15, 233-250.
71. Shor, M. (2002). Learning to respond: The use of heuristics in dynamic games. Unpublished manuscript, Vanderbilt University, Nashville, TN.
72. Siegel, S., & Goldstein, D. A. (1959). Decision-making behavior in a two-choice uncertain outcome situation. Journal of Experimental Psychology, 57, 37-42.
73. Skinner, B. F. (1953). Science and human behavior. New York: Macmillan.
74. Stahl, D. (1996). Boundedly rational rule learning in a guessing game. Games and Economic Behavior, 16, 303-330.
75. Thaler, R., Tversky, A., Kahneman, D., & Schwartz, A. (1997). The effect of myopia and loss aversion on risk taking: An experimental test. Quarterly Journal of Economics, 112, 647-661.
76. Thorndike, E. L. (1898). Animal intelligence: An experimental study of the associative processes in animals. Psychological Monographs, 2(4).
77. Tolman, E. C. (1925). Purpose and cognition: The determiners of animal learning. Psychological Review, 32, 285-297.
78. Tversky, A., & Kahneman, D. (1992). Advances in prospect theory: Cumulative representation of uncertainty. Journal of Risk and Uncertainty, 9, 195-230.
79. von Neumann, J., & Morgenstern, O. (1947). The theory of games and economic behavior. Princeton, NJ: Princeton University Press.
80. Vulkan, N. (2000). An economist's perspective on probability matching. Journal of Economic Surveys, 14, 101-118.
81. Weber, E. U., Shafir, S., & Biais, A.-R. (2004). Predicting risk sensitivity in humans and lower animals: Risk as variance or coefficient of variation. Psychological Review, 111, 430-445.
82. Williams, R. W., & Herrup, K. (1988). The control of neuron number. Annual Review of Neuroscience, 11, 423-453.
83. Yechiam, E., & Busemeyer, J. R. (2005). Comparison of basic assumptions embedded in learning models for experience based decision-making. Psychonomic Bulletin & Review, 12, 387-402.
84. Yechiam, E., Erev, I., & Gopher, D. (2001). On the potential value and limitations of emphasis change and other exploration-enhancing training methods. Journal of Experimental Psychology: Applied, 7, 277-285.