Evidence for Model-based Computations in the Human Amygdala during Pavlovian Conditioning

PLoS Computational Biology

Volumn 9, Issue 2, 2013, Pages

  Prévost, Charlotte ^a,b   McNamee, Daniel ^b   Jessup, Ryan K ^b,c   Bossaerts, Peter ^b   O'Doherty, John P ^a,b  

^a TRINITY COLLEGE DUBLIN   (Ireland)

^b CALIFORNIA INSTITUTE OF TECHNOLOGY   (United States)

^c ABILENE CHRISTIAN UNIVERSITY   (United States)

Author keywords

[No Author keywords available]

Indexed keywords

ENCODINGS; FMRI DATA; HIGH RESOLUTION; MODEL FREE; MODEL-BASED ALGORITHMS; MODEL-BASED OPC; MODEL-BASED SYSTEMS; MODEL-FREE ALGORITHMS; SIMPLE STRUCTURES;

INFERENCE ENGINES;

ACCURACY; AMYGDALOID NUCLEUS; ARTICLE; ASSOCIATION; AVERSION; BLINKING; BOLD SIGNAL; BRAIN REGION; BREATHING; COMPUTER MODEL; CONDITIONED REFLEX; CONTROLLED STUDY; FEMALE; FUNCTIONAL MAGNETIC RESONANCE IMAGING; HEART RATE; HUMAN; HUMAN EXPERIMENT; INTERMETHOD COMPARISON; LEARNING ALGORITHM; MYDRIASIS; NORMAL HUMAN; STIMULUS RESPONSE; TASK PERFORMANCE; VISUAL STIMULATION;

EID: 84874778074     PISSN: 1553734X     EISSN: 15537358     Source Type: Journal    
DOI: 10.1371/journal.pcbi.1002918     Document Type: Article

References (46)

1. Doya K, Samejima K, Katagiri K, Kawato M, (2002) Multiple model-based reinforcement learning. Neural Comput 14: 1347-1369.
2. Corrado G, Doya K, (2007) Understanding neural coding through the model-based analysis of decision making. J Neurosci 27: 8178-8180.
3. Dayan P, Daw ND, (2008) Decision theory, reinforcement learning, and the brain. Cogn Affect Behav Neurosci 8: 429-453.
4. Daw ND, Niv Y, Dayan P, (2005) Uncertainty-based competition between prefrontal and dorsolateral striatal systems for behavioral control. Nat Neurosci 8: 1704-1711.
5. Gershman SJ, Blei DM, Niv Y, (2010) Context, learning, and extinction. Psychol Rev 117: 197-209.
6. Dayan P, Kakade S, Montague PR, (2000) Learning and selective attention. Nat Neurosci 3 Suppl: 1218-1223.
7. Glascher J, Daw N, Dayan P, O'Doherty JP, (2010) States versus rewards: dissociable neural prediction error signals underlying model-based and model-free reinforcement learning. Neuron 66: 585-595.
8. Hampton AN, Bossaerts P, O'Doherty JP, (2006) The role of the ventromedial prefrontal cortex in abstract state-based inference during decision making in humans. J Neurosci 26: 8360-8367.
9. Daw ND, Gershman SJ, Seymour B, Dayan P, Dolan RJ, (2011) Model-based influences on humans' choices and striatal prediction errors. Neuron 69: 1204-1215.
10. Courville AC, Daw ND, Touretzky DS, (2006) Bayesian theories of conditioning in a changing world. Trends Cogn Sci 10: 294-300.
11. Rescorla RA (1972) A theory of Pavlovian conditioning: Variations in effectiveness of reinforcement and nonreinforcement. In: Century-Crofts N-YA, editor. Classical conditioning II: Current research and Theory: Black A.H, P.W.H.
12. Pearce JM, Hall G, (1980) A model for Pavlovian learning: variations in the effectiveness of conditioned but not of unconditioned stimuli. Psychol Rev 87: 532-552.
13. Li J, Schiller D, Schoenbaum G, Phelps EA, Daw ND, (2011) Differential roles of human striatum and amygdala in associative learning. Nat Neurosci 14: 1250-1252.
14. Buchel C, Dolan RJ, (2000) Classical fear conditioning in functional neuroimaging. Curr Opin Neurobiol 10: 219-223.
15. Fanselow MS, LeDoux JE, (1999) Why we think plasticity underlying Pavlovian fear conditioning occurs in the basolateral amygdala. Neuron 23: 229-232.
16. Johansen JP, Cain CK, Ostroff LE, LeDoux JE, (2011) Molecular mechanisms of fear learning and memory. Cell 147: 509-524.
17. Delgado MR, Olsson A, Phelps EA, (2006) Extending animal models of fear conditioning to humans. Biol Psychol 73: 39-48.
18. Prevost C, McCabe JA, Jessup RK, Bossaerts P, O'Doherty JP, (2011) Differentiable contributions of human amygdalar subregions in the computations underlying reward and avoidance learning. Eur J Neurosci 34: 134-145.
19. Libby WL Jr, Lacey BC, Lacey JI, (1973) Pupillary and cardiac activity during visual attention. Psychophysiology 10: 270-294.
20. Bradley MM, (2009) Natural selective attention: orienting and emotion. Psychophysiology 46: 1-11.
21. Nicotra A, Critchley HD, Mathias CJ, Dolan RJ, (2006) Emotional and autonomic consequences of spinal cord injury explored using functional brain imaging. Brain 129: 718-728.
22. Bitsios P, Szabadi E, Bradshaw CM, (2004) The fear-inhibited light reflex: importance of the anticipation of an aversive event. Int J Psychophysiol 52: 87-95.
23. Seymour B, Daw N, Dayan P, Singer T, Dolan R, (2007) Differential encoding of losses and gains in the human striatum. J Neurosci 27: 4826-4831.
24. Bray S, Rangel A, Shimojo S, Balleine B, O'Doherty JP, (2008) The neural mechanisms underlying the influence of pavlovian cues on human decision making. J Neurosci 28: 5861-5866.
25. Mai Jk, Paxinos G, Voss T (2008) Atlas of the Human Brain: Elsevier.
26. O'Doherty JP, Hampton A, Kim H, (2007) Model-based fMRI and its application to reward learning and decision making. Ann N Y Acad Sci 1104: 35-53.
27. Roesch MR, Calu DJ, Esber GR, Schoenbaum G, (2010) Neural correlates of variations in event processing during learning in basolateral amygdala. J Neurosci 30: 2464-2471.
28. Murray EA, (2007) The amygdala, reward and emotion. Trends Cogn Sci 11: 489-497.
29. Baxter MG, Murray EA, (2002) The amygdala and reward. Nat Rev Neurosci 3: 563-573.
30. Hampton AN, Adolphs R, Tyszka MJ, O'Doherty JP, (2007) Contributions of the amygdala to reward expectancy and choice signals in human prefrontal cortex. Neuron 55: 545-555.
31. Elliott R, Newman JL, Longe OA, William Deakin JF, (2004) Instrumental responding for rewards is associated with enhanced neuronal response in subcortical reward systems. Neuroimage 21: 984-990.
32. Yacubian J, Glascher J, Schroeder K, Sommer T, Braus DF, et al. (2006) Dissociable systems for gain- and loss-related value predictions and errors of prediction in the human brain. J Neurosci 26: 9530-9537.
33. Seymour B, O'Doherty JP, Koltzenburg M, Wiech K, Frackowiak R, et al. (2005) Opponent appetitive-aversive neural processes underlie predictive learning of pain relief. Nat Neurosci 8: 1234-1240.
34. O'Doherty J, Dayan P, Schultz J, Deichmann R, Friston K, et al. (2004) Dissociable roles of ventral and dorsal striatum in instrumental conditioning. Science 304: 452-454.
35. Gottfried JA, O'Doherty J, Dolan RJ, (2002) Appetitive and aversive olfactory learning in humans studied using event-related functional magnetic resonance imaging. J Neurosci 22: 10829-10837.
36. Gottfried JA, O'Doherty J, Dolan RJ, (2003) Encoding predictive reward value in human amygdala and orbitofrontal cortex. Science 301: 1104-1107.
37. Insausti R, Juottonen K, Soininen H, Insausti AM, Partanen K, et al. (1998) MR volumetric analysis of the human entorhinal, perirhinal, and temporopolar cortices. AJNR Am J Neuroradiol 19: 659-671.
38. Miller MI, Beg MF, Ceritoglu C, Stark C, (2005) Increasing the power of functional maps of the medial temporal lobe by using large deformation diffeomorphic metric mapping. Proc Natl Acad Sci U S A 102: 9685-9690.
39. Yassa MA, Stark CE, (2009) A quantitative evaluation of cross-participant registration techniques for MRI studies of the medial temporal lobe. Neuroimage 44: 319-327.
40. Vercauteren T, Pennec X, Perchant A, Ayache N, (2007) Non-parametric diffeomorphic image registration with the demons algorithm. Med Image Comput Comput Assist Interv 10: 319-326.
41. Thirion JP, (1998) Image matching as a diffusion process: an analogy with Maxwell's demons. Med Image Anal 2: 243-260.
42. Bray S, O'Doherty J, (2007) Neural coding of reward-prediction error signals during classical conditioning with attractive faces. J Neurophysiol 97: 3036-3045.
43. Schwarz (1978) Estimating the dimension of a model. The annals of statistics pp. 461-464.
44. Glover GH, Li TQ, Ress D, (2000) Image-based method for retrospective correction of physiological motion effects in fMRI: RETROICOR. Magn Reson Med 44: 162-167.
45. Stephan KE, Penny WD, Daunizeau J, Moran RJ, Friston KJ, (2009) Bayesian model selection for group studies. Neuroimage 46: 1004-1017.
46. Kriegeskorte N, Simmons WK, Bellgowan PS, Baker CI, (2009) Circular analysis in systems neuroscience: the dangers of double dipping. Nat Neurosci 12: 535-540.