Adding prediction risk to the theory of reward learning

Annals of the New York Academy of Sciences

Volumn 1104, Issue , 2007, Pages 135-146

  Preuschoff, Kerstin ^a   Bossaerts, Peter ^a  

^a CALIFORNIA INSTITUTE OF TECHNOLOGY   (United States)

Author keywords

Adaptive encoding; Dopaminergic system; Learning rate; Least squares learning; Prediction risk; Reinforcement learning; Reward anticipation; Uncertainty

Indexed keywords

BRAIN REGION; CONFERENCE PAPER; COVARIANCE; DOPAMINERGIC SYSTEM; HUMAN; LEARNING; LEARNING THEORY; MATHEMATICAL ANALYSIS; MATHEMATICAL COMPUTING; NONHUMAN; REGRESSION ANALYSIS; REINFORCEMENT; REWARD; RISK ASSESSMENT; ANIMAL; BRAIN; BRAIN MAPPING; CONDITIONED REFLEX; HISTOLOGY; METABOLISM; NERVE CELL; PHYSIOLOGY; PROBABILITY; REVIEW; RISK;

PRIMATES;

DOPAMINE;

ANIMALS; ASSOCIATION LEARNING; BRAIN; BRAIN MAPPING; CONDITIONING, CLASSICAL; DOPAMINE; HUMANS; LEARNING; LEAST-SQUARES ANALYSIS; NEURONS; PROBABILITY; REINFORCEMENT (PSYCHOLOGY); REWARD; RISK;

EID: 34447623582     PISSN: 00778923     EISSN: 17496632     Source Type: Book Series    
DOI: 10.1196/annals.1390.005     Document Type: Conference Paper

References (21)

1. SCHULTZ, W. 2004. Neural coding of basic reward terms of animal learning theory, game theory, microeconomics and behavioural ecology. Curr. Opin. Neurobiol. 14: 139-147.
2. MCCLURE, S.M., G.S. BERNS & P.R. MONTAGUE. 2003. Temporal prediction errors in a passive learning task activate human striatum. Neuron 38: 339-346.
3. O'DOHERTY, J.P., P. DAYAN, K. FRISTON, et al. 2003. Temporal difference models and reward-related learning in the human brain. Neuron 38: 329.
4. MONTAGUE, P.R., P. DAYAN & T.J. SEJNOWSKI. 1996. A framework for mesencephalic dopamine systems based on predictive Hebbian learning. J. Neurosci. 16: 1936-1947.
5. MONTAGUE, P.R., S.E. HYMAN & J.D. COHEN. 2004. Computational roles for dopamine in behavioural control. Nature 431: 760.
6. MASANAO, A. 1986. State Space Modeling of Time Series. Springer-Verlag. New York.
7. FIORILLO, C.D., P.N. TOBLER & W. SCHULTZ. 2003. Discrete coding of reward probability and uncertainty by dopamine neurons. Science 299: 1898-1902.
8. PREUSCHOFF, K. 2006. Neural differentiation of expected reward and risk in human subcortical structures. Neuron 51: 381-390.
9. NIV, Y., M.O. DUFF & P. DAYAN. 2005. Dopamine, uncertainty and TD learning. Behav. Brain Funct. 1: 6.
10. FIORILLO, C.D., P.N. TOBLER & W. SCHULTZ. 2005. Evidence that the delay-period activity of dopamine neurons corresponds to reward uncertainty rather than back-propagating TD errors. Behav. Brain Funct. 1: 7.
11. TOBLER, P.N., C.D. FIORILLO & W. SCHULTZ. 2005. Adaptive coding of reward value by dopamine neurons. Science 307: 1642-1645.
12. HOLT, C.A. & S.K. LAURY. 2002. Risk aversion and incentive effects. Am. Eco. Rev. 92: 1644-1655.
13. MCCOY, A.N. & M.L. PLATT. 2005. Risk-sensitive neurons in macaque posterior cingulate cortex. Nat. Neurosci. 8: 1220-1227.
14. FLYNN, F.G. 1999. Anatomy of the insula functional and clinical correlates. Aphasiology 13: 55.
15. HUETTEL, S., A. SONG & G. MCCARTHY. 2005. Decisions under uncertainty: probabilistic context influences activation of prefrontal and parietal cortices. J. Neurosci. 25: 3304-3311.
16. KUHNEN, C.M. & B. KNUTSON. 2005. The neural basis of financial risk taking. Neuron 47: 763-770.
17. MARKOWITZ, H. 1991. Foundations of portfolio theory. J. Finance 46: 469-477.
18. KROLL, Y., H. LEVY & H.M. MARKOWITZ. 1984. Mean-variance versus direct utility maximization. J. Finance 39: 47-61.
19. BOSSAERTS, P. In New Encyclopedia of Neuroscience. Eds.: L. Squire, T. Albright, F. Bloom, F. Gage & N. Spitzer Elsevier. In press.
20. BEHRENS, T., M. WOOLRICH, M. WALTON & M. RUSHWORTH. 2006. Learning the value of information in an uncertain world. Oxford University working paper.
21. YU, A.J. & P. DAYAN. 2003. Expected and unexpected uncertainty: ACh and NE in the neocortex. In Advances in Neural Information Processing Systems 15. MIT Press. Cambridge, MA.