Do learning rates adapt to the distribution of rewards?

Psychonomic Bulletin and Review

Volumn 22, Issue 5, 2015, Pages 1320-1327

  Gershman, Samuel J ^a  

^a MASSACHUSETTS INSTITUTE OF TECHNOLOGY   (United States)

Author keywords

Decision making; Multi armed bandit; Reinforcement learning

Indexed keywords

ADULT; FEMALE; HIGH RISK BEHAVIOR; HUMAN; LEARNING; MALE; MOTIVATION; REINFORCEMENT; REWARD;

ADULT; FEMALE; HUMANS; LEARNING; MALE; MOTIVATION; PROBABILITY LEARNING; REINFORCEMENT (PSYCHOLOGY); REWARD; RISK-TAKING;

EID: 84942370022     PISSN: 10699384     EISSN: 15315320     Source Type: Journal    
DOI: 10.3758/s13423-014-0790-3     Document Type: Article

References (26)

1. Behrens, T.E., Woolrich, M.W., Walton, M.E., Rushworth, M.F. (2007). Learning the value of information in an uncertain world. Nature Neuroscience, 10, 1214–1221.
2. Bromberg-Martin, E.S., Matsumoto, M., Nakahara, H., Hikosaka, O. (2010). Multiple timescales of memory in lateral habenula and dopamine neurons. Neuron, 67, 499–510.
3. Cazé, R.D., & van der Meer, M.A. (2013). Adaptive properties of differential learning rates for positive and negative outcomes. Biological Cybernetics, 107, 711–719.
4. Christakou, A., Gershman, S.J., Niv, Y., Simmons, A., Brammer, M., Rubia, K. (2013). Neural and psychological maturation of decision-making in adolescence and young adulthood. Journal of Cognitive Neuroscience, 25, 1807–1823.
5. Crump, M.J., McDonnell, J.V., Gureckis, T.M. (2013). Evaluating Amazon’s Mechanical Turk as a tool for experimental behavioral research. PLoS One, 8, e57410.
6. Daw, N.D., Kakade, S., Dayan, P. (2002). Opponent interactions between serotonin and dopamine. Neural Networks, 15, 603–616.
7. Daw, N.D., O’Doherty, J.P., Dayan, P., Seymour, B., Dolan, R.J. (2006). Cortical substrates for exploratory decisions in humans. Nature, 441, 876–879.
8. Dayan, P., Kakade, S., Montague, P.R. (2000). Learning and selective attention. Nature Neuroscience, 3, 1218–1223.
9. Doya, K. (2002). Metalearning and neuromodulation. Neural Networks, 15, 495–506.
10. Frank, M.J., Doll, B.B., Oas-Terpstra, J., Moreno, F. (2009). Prefrontal and striatal dopaminergic genes predict individual differences in exploration and exploitation. Nature Neuroscience, 12, 1062–1068.
11. Frank, M.J., Moustafa, A.A., Haughey, H.M., Curran, T., Hutchison, K.E. (2007). Genetic triple dissociation reveals multiple roles for dopamine in reinforcement learning. Proceedings of the National Academy of Sciences, 104, 16311–16316.
12. Frank, M.J., Seeberger, L.C., O’Reilly, R.C. (2004). By carrot or by stick: Cognitive reinforcement learning in Parkinsonism. Science, 306, 1940–1943.
13. MacKay, D.J. (2003). Information Theory, Inference and Learning Algorithms. Cambridge University Press.
14. Mackintosh, N.J. (1975). A theory of attention: Variations in the associability of stimuli with reinforcement. Psychological Review, 82, 276–298.
15. Mihatsch, O., & Neuneier, R. (2002). Risk-sensitive reinforcement learning. Machine Learning, 49, 267–290.
16. Niv, Y., Edlund, J.A., Dayan, P., O’Doherty, J.P. (2012). Neural prediction errors reveal a risk-sensitive reinforcement-learning process in the human brain. The Journal of Neuroscience, 32, 551– 562.
17. Niv, Y., & Schoenbaum, G. (2008). Dialogues on prediction errors. Trends in Cognitive Sciences, 12, 265–272.
18. Pearce, J.M., & Hall, G. (1980). A model for Pavlovian learning: Variations in the effectiveness of conditioned but not of unconditioned stimuli. Psychological Review, 87, 532–552.
19. Rescorla, R.A., & Wagner, A.R. (1972). A theory of Pavlovian conditioning: Variations in the effectiveness of reinforcement and nonreinforcement. In: Black, A., & Prokasy, W. (Eds.), Classical conditioning II: Current research and theory. Appleton-Century-Crofts, New York, (pp. 64–99).
20. Robert, C.P., & Casella, G (2004). Monte Carlo statistical methods. Springer.
21. Rutledge, R.B., Lazzaro, S.C., Lau, B., Myers, C.E., Gluck, M.A., Glimcher, P.W. (2009). Dopaminergic drugs modulate learning rates and perseveration in Parkinson’s patients in a dynamic foraging task. The Journal of Neuroscience, 29, 15104–15114.
22. Schwarz, G. (1978). Estimating the dimension of a model. The Annals of Statistics, 6, 461–464.
23. Stephan, K.E., Penny, W.D., Daunizeau, J., Moran, R.J., Friston, K.J. (2009). Bayesian model selection for group studies. NeuroImage, 46, 1004–1017.
24. Sutton, R., & Barto, A. (1990). Time-derivative models of pavlovian reinforcement. In: Gabriel, M., & Moore, J. (Eds.), Learning and computational neuroscience: Foundations of adaptive networks. MIT Press, (pp. 497–537).
25. Sutton, R.S., & Barto, A.G. (1998). Reinforcement learning: An introduction. MIT Press.
26. Waltz, J.A., Frank, M.J., Robinson, B.M., Gold, J.M. (2007). Selective reinforcement learning deficits in schizophrenia support predictions from computational models of striatal-cortical dysfunction. Biological Psychiatry, 62, 756–764.