Learning, risk attitude and hot stoves in restless bandit problems

Journal of Mathematical Psychology

Volumn 53, Issue 3, 2009, Pages 155-167

  Biele, Guido ^a   Erev, Ido ^b   Ert, Eyal ^b  

^a MAX PLANCK INSTITUTE FOR HUMAN DEVELOPMENT   (Germany)

^b TECHNION ISRAEL INSTITUTE OF TECHNOLOGY   (Israel)

Author keywords

Case based reasoning; Dynamic decision making; Probability matching; The recency hot stove paradox; Underweighting of rare events

Indexed keywords

EID: 67349181341     PISSN: 00222496     EISSN: 10960880     Source Type: Journal    
DOI: 10.1016/j.jmp.2008.05.006     Document Type: Article

References (41)

1. Barron G., and Erev I. Small feedback-based decisions and their limited correspondence to description based decisions. Journal of Behavioral Decision Making 16 (2003) 215-233
2. Bush R.B., and Mosteller F. Stochastic models for learning (1955), Wiley, New York
3. Busemeyer J.R., and Townsend J.T. Decision field theory: A dynamic-cognitive approach to decision making in an uncertain environment. Psychological Review 100 (1993) 432-459
4. Busemeyer J.R., and Wang Y.M. Model comparisons and model selections based on the generalization criterion methodology. Journal of Mathematical Psychology 44 (2000) 171-189
5. Camerer C.F., and Ho T.H. Experience-weighted attraction learning in normal form games. Econometrica 67 (1999) 827-874
6. Daw N.D., O'Doherty J.P., Dayan P., Seymour B., and Dolan R.J. Cortical substrates for exploratory decisions in humans. Nature 441 (2006) 876-879
7. Denrell J. Why most people disapprove of me: Experience sampling in impression formation. Psychological Review 112 (2005) 951-978
8. Denrell J. Adaptive learning and risk taking. Psychological Review 114 (2007) 177-187
9. Denrell J., and March J.G. Adaptation as information restriction: The hot stove effect. Organization Science 12 (2001) 523-538
10. Erev I., and Roth A.E. Predicting how people play games: Reinforcement learning in experimental games with unique, mixed strategy equilibria. American Economic Review 88 (1998) 848-881
11. Erev I., and Barron G. On adaptation, maximization, and reinforcement learning among cognitive strategies. Psychological Review 112 (2005) 912-931
12. Erev, I., Ert, E., & Yechiam, E. (in press). Loss aversion, diminishing sensitivity, and the effect of experience on repeated decision. Journal of Behavioral Decision Making
13. Erev I., and Haruvy E. Generality, repetition, and the role of descriptive learning models. Journal of Mathematical Psychology 49 (2005) 357-371
14. Erev I., Roth A.E., Slonim R.L., and Barron G. Learning and equilibrium as useful approximations: Accuracy of prediction on randomly selected constant sum games. Economic Theory 33 (2007) 29-51
15. Estes W.K., and Maddox W. Risks of drawing inferences about cognitive processes from model fits to individual versus average performance. Psychonomic Bulletin & Review 12 (2005) 403-408
16. Fiedler K. Beware of samples! A cognitive-ecological sampling approach to judgment biases. Psychological Review 107 (2000) 659-676
17. Garcia J., Ervin F.R., and Koelling R.A. Learning with prolonged delay of reinforcement. Psychonomic Science 5 (1966) 121-122
18. Gilboa I., and Schmeidler D. Case-based decision theory. The Quarterly Journal of Economics 30 (1995) 605-639
19. Gittins J.C. Bandit processes and dynamic allocation indexes. Journal of the Royal Statistical Society Series B-Methodological 41 (1979) 148-177
20. Glazebrook K.D., Ruiz-Hernandez D., and Kirkbride C. Some indexable families of restless bandit problems. Advances in Applied Probability 38 (2006) 643-672
21. Gonzalez C., Lerch J.F., and Lebiere C. Instance-based learning in dynamic decision making. Cognitive Science 27 (2003) 591-635
22. Grosskopf B., Erev I., and Yechiam E. Forgone with the wind: Indirect payoff information and its implications for choice. International Journal of Game Theory 34 (2006) 285-302
23. Hertwig R., Barron G., Weber E.U., and Erev I. Decisions from experience and the effect of rare events in risky choices. Psychological Science 15 (2004) 534-539
24. Hochman, G., & Erev, I. (2007). The partial reinforcement extinction effect and the role of contingent sampling. Working paper, Technion
25. Kareev Y. Seven (indeed, plus or minus two) and the detection of correlations. Psychological Review 107 (2000) 397-402
26. Lebiere, C., Gonzalez, C., & Martin, M. (2007). Instance-based decision making model of repeated binary choice. In Proceedings of the 8th international conference on cognitive modeling
27. Logan G.D. Toward an instance theory of automatization. Psychological Review 95 (1988) 492-527
28. March J.G. Exploration and exploitation in organizational learning. Organization Science 2 (1991) 71-87
29. March J.G. Learning to become risk averse. Psychological Review 103 (1996) 309-319
30. Monahan G.E. State of the art-A survey of partially observable Markov decision-processes: Theory, models, and algorithms. Management Science 28 (1982) 1-16
31. Munichor N., Erev I., and Lotem A. Risk attitude in small time saving decisions. Journal of Experimental Psychology: Applied 12 (2006) 129-141
32. Myers J.L., and Sadler E. Effects of range of payoffs as a variable in risk taking. Journal of Experimental Psychology 60 (1960) 306-309
33. Nowak M., and Sigmund K. A strategy of win-stay, lose-shift that outperforms tit-for-tat in the prisoner's dilemma game. Nature 364 (1993) 56-58
34. Osborne M.J., and Rubinstein A. Games with procedurally rational players. American Economic Review 88 (1998) 834-847
35. Papadimitriou C.H., and J N T. The complexity of optimal queuing network control. Mathematics of Operations Research 24 (1999) 293-305
36. Seligman M.E., Maier S.F., and Geer J.H. Alleviation of learned helplessness in the dog. Journal of Abnormal Psychology 73 (1968) 256-262
37. Weber E.U., Shafir S., and Blais A.-R. Predicting risk sensitivity in humans and lower animals: Risk as variance or coefficient of variation. Psychological Review 111 (2004) 430-445
38. Whittle P. Activity allocation in a changing world. Journal of Applied Probability 25A (1988) 287-298
39. White C.C., and Eldeib H.K. Markov Decision Processes with imprecise Transition Probabilities. Operations Research 42 (1994) 739-749
40. Yechiam E., and J R B. Comparison of basic assumptions embedded in learning models for experience-based decision making. Psychonomic Bulletin & Review 12 (2005) 387-402
41. Yechiam E., and Busemeyer J. The effect of forgone payoffs on underweighting small probability events. Journal of Behavioral Decision Making 19 (2006) 1-16