An approximately Bayesian delta-rule model explains the dynamics of belief updating in a changing environment

Journal of Neuroscience

Volumn 30, Issue 37, 2010, Pages 12366-12378

  Nassar, Matthew R ^a   Wilson, Robert C ^b   Heasly, Benjamin ^a   Gold, Joshua I ^a  

^a UNIVERSITY OF PENNSYLVANIA   (United States)

^b PRINCETON UNIVERSITY   (United States)

Author keywords

[No Author keywords available]

Indexed keywords

ADULT; ARTICLE; BAYES THEOREM; BEHAVIOR; DECISION MAKING; ENVIRONMENTAL CHANGE; FEMALE; HEALTH BELIEF; HUMAN; MALE; NORMAL HUMAN; PRIORITY JOURNAL; ADAPTIVE BEHAVIOR; COGNITION; COMPUTER SIMULATION; CULTURAL ANTHROPOLOGY; ENVIRONMENT; EXECUTIVE FUNCTION; NERVE CELL PLASTICITY; NEUROPSYCHOLOGICAL TEST; PHYSIOLOGY; PREDICTIVE VALUE; PSYCHOLOGICAL MODEL; YOUNG ADULT;

ADAPTATION, PSYCHOLOGICAL; ADULT; BAYES THEOREM; COGNITION; COMPUTER SIMULATION; CULTURE; DECISION MAKING; ENVIRONMENT; EXECUTIVE FUNCTION; FEMALE; HUMANS; MALE; MODELS, PSYCHOLOGICAL; NEURONAL PLASTICITY; NEUROPSYCHOLOGICAL TESTS; PREDICTIVE VALUE OF TESTS; YOUNG ADULT;

EID: 77956862944     PISSN: 02706474     EISSN: 15292401     Source Type: Journal    
DOI: 10.1523/JNEUROSCI.0822-10.2010     Document Type: Article

References (24)

1. Adams RP, MacKay DJ (2007) Bayesian online changepoint detection. Cambridge, UK: University of Cambridge Technical Report.
2. Behrens TE, Woolrich MW, Walton ME, Rushworth MF (2007) Learning the value of information in an uncertain world. Nat Neurosci 10:1214-1221.
3. Bruder GE, Keilp JG, Xu H, Shikhman M, Schori E, Gorman JM, Gilliam TC (2005) Catechol-O-methyltransferase (COMT) genotypes and working memory: associations with differing cognitive operations. Biol Psychiatry 58:901-907.
4. Corrado GS, Sugrue LP, Seung HS, Newsome WT (2005) Linear-nonlinear- Poisson models of primate choice dynamics. J Exp Anal Behav 84:581-617.
5. Daw N, O'Doherty J, Dayan P, Seymour B, Dolan R (2006) Cortical substrates for exploratory decisions in humans. Nature 441:876-879.
6. Dayan P, Kakade S, Montague P (2000) Learning and selective attention. Nat Neurosci 3:1218-1223.
7. Debener S, Ullsperger M, Siegel M, Fiehler K, von Cramon DY, Engel AK (2005) Trial-by-trial coupling of concurrent electroencephalogram and functional magnetic resonance imaging identifies the dynamics of performance monitoring. J Neurosci 25:11730-11737.
8. Fearnhead P, Liu Z (2007) On-line inference for multiple changepoint problems. J R Stat Soc Series B Stat Methodol 69:589-605.
9. Hayden BY, Pearson JM, Platt ML (2009) Fictive reward signals in the anterior cingulate cortex. Science 324:948-950.
10. Kennerley SW, Walton ME, Behrens TE, Buckley MJ, Rushworth MF (2006) Optimal decision making and the anterior cingulate cortex. Nat Neurosci 9:940-947.
11. Kennerley SW, Dahmubed AF, Lara AH, Wallis JD (2009) Neurons in the frontal lobe encode the value of multiple decision variables. J Cogn Neurosci 21:1162-1178.
12. Krugel L, Biele G, Mohr P, Li S, Heekeren HR (2009) Genetic variation in dopaminergic neuromodulation influences the ability to rapidly and flexibly adapt decisions. Proc Natl Acad Sci U S A 106:17591-17596.
13. Matsumoto M, Matsumoto K, Abe H, Tanaka K (2007) Medial prefrontal cell activity signaling prediction errors of action values. Nat Neurosci 10:647-656.
14. Niv Y (2009) Reinforcement learning in the brain. J Math Psychol 53:139-154.
15. Pearce JM, Bouton ME (2001) Theories of associative learning in animals. Annu Rev Psychol 52:111-139.
16. Posner MI (2008) Measuring alertness. Ann N Y Acad Sci 1129:193-199.
17. Rushworth MF, Behrens TE (2008) Choice, uncertainty and value in prefrontal and cingulate cortex. Nat Neurosci 11:389-397.
18. Schultz W (1998) Predictive reward signal of dopamine neurons. J Neurophysiol 80:1-27.
19. Steyvers M, Brown S (2006) Prediction and change detection. Adv Neural Inform Processing Systems 18:1281-1288.
20. Sutton RS, Barto AG (1998) Reinforcement learning: an introduction. Cambridge, MA: MIT.
21. Williams RJ (1992) Simple statistical gradient-following algorithms for connectionist reinforcement learning. Machine Learning 8:229-256.
22. Wilson RC, Nassar MR, Gold JI (2010) Bayesian on-line learning of the hazard rate in change-point problems. Neural Comput 22:2452-2476.
23. Yu A, Dayan P (2003) Expected and unexpected uncertainty: ACh and NE in the neocortex. Adv Neural Inform Processing Systems 15:173-180.
24. Yu AJ, Dayan P (2005) Uncertainty, neuromodulation, and attention. Neuron 46:681-692.