Experimental predictions drawn from a computational model of sign-trackers and goal-trackers

Journal of Physiology Paris

Volumn 109, Issue 1-3, 2015, Pages 78-86

  Lesaint, Florian ^a,b   Sigaud, Olivier ^a,b   Clark, Jeremy J ^c   Flagel, Shelly B ^d,e,f   Khamassi, Mehdi ^a,b  

^a SORBONNE UNIVERSITÉ   (France)

^b UNIVERSITÉ PIERRE ET MARIE CURIE   (France)

^c UNIVERSITY OF WASHINGTON   (United States)

^d UNIVERSITY OF MICHIGAN HEALTH SYSTEM   (United States)

^e UNIVERSITY OF MICHIGAN   (United States)

^f UNIVERSITY OF MICHIGAN   (United States)

Author keywords

Autoshaping; Conditioned approach; Dopamine; Factored representation; Goal tracker; Model based; Model free; Pavlovian conditioning; Reinforcement learning; Sign tracker

Indexed keywords

ARTICLE; BRAIN REGION; CONDITIONING; DOPAMINERGIC ACTIVITY; GOAL TRACKER CONDITIONED RESPONSE; IMPULSIVENESS; MALADJUSTMENT; MATHEMATICAL COMPUTING; MEMORY CONSOLIDATION; NERVE STIMULATION; NUCLEUS ACCUMBENS; PREDICTION; REINFORCEMENT; RELIABILITY; REWARD; SIGN TRACKER CONDITIONED RESPONSE; VALIDATION PROCESS; ANIMAL; ASSOCIATION; BIOLOGICAL MODEL; COMPUTER SIMULATION; CONDITIONED REFLEX; MOTIVATION; PREDICTIVE VALUE; PROBABILITY;

ANIMALS; COMPUTER SIMULATION; CONDITIONING, CLASSICAL; CUES; GOALS; MODELS, NEUROLOGICAL; PREDICTIVE VALUE OF TESTS; PROBABILITY; REWARD;

EID: 84929369050     PISSN: 09284257     EISSN: 17697115     Source Type: Journal    
DOI: 10.1016/j.jphysparis.2014.06.001     Document Type: Article

References (32)

1. Bornstein A.M., Daw N.D. Dissociating hippocampal and striatal contributions to sequential prediction learning. Eur. J. Neurosci. 2012, 35(7):1011-1023.
2. Caluwaerts K., Staffa M., N'Guyen S., Grand C., Dollé L., Favre-Félix A., Girard B., Khamassi M. A biologically inspired meta-control navigation system for the psikharpax rat robot. Bioinspir. Biomimet. 2012, 7(2):025009.
3. Clark J.J., Hollon N.G., Phillips P.E.M. Pavlovian valuation systems in learning and decision making. Curr. Opin. Neurobiol. 2012, 22(6):1054-1061.
4. Danna C.L., Elmer G.I. Disruption of conditioned reward association by typical and atypical antipsychotics. Pharmacol. Biochem. Behav. 2010, 96(1):40-47.
5. Daw N.D. Trial-by-trial data analysis using computational models. Decision Making, Affect, and Learning: Attention and Performance XXIII 2011, vol. 23. Oxford University Press, chap. 1. M.R. Delgado, E.A. Phelps, T.W. Robbins (Eds.).
6. Daw N.D., Niv Y., Dayan P. Uncertainty-based competition between prefrontal and dorsolateral striatal systems for behavioral control. Nat. Neurosci. 2005, 8(12):1704-1711.
7. Daw N.D., Gershman S.J., Seymour B., Dayan P., Dolan R.J. Model-based influences on humans' choices and striatal prediction errors. Neuron 2011, 69(6):1204-1215.
8. Dayan P., Niv Y., Seymour B., Daw N.D. The misbehavior of value and the discipline of the will. Neural Netw. 2006, 19(8):1153-1160.
9. Deb K., Pratap A., Agarwal S., Meyarivan T. A fast and elitist multiobjective genetic algorithm: NSGA-II. IEEE Trans. Evol. Comput. 2002, 6(2):182-197.
10. DiFeliceantonio A.G., Berridge K.C. Which cue to 'want'? Opioid stimulation of central amygdala makes goal-trackers show stronger goal-tracking, just as sign-trackers show stronger sign-tracking. Behav. Brain Res. 2012, 230(2):399-408.
11. Fitzpatrick C.J., Gopalakrishnan S., Cogan E.S., Yager L.M., Meyer P.J., Lovic V., Saunders B.T., Parker C.C., Gonzales N.M., Aryee E. Variation in the form of pavlovian conditioned approach behavior among outbred male sprague-dawley rats from different vendors and colonies: sign-tracking vs. goal-tracking. PloS ONE 2013, 8(10):e75042.
12. Flagel S.B., Watson S.J., Robinson T.E., Akil H. Individual differences in the propensity to approach signals vs. goals promote different adaptations in the dopamine system of rats. Psychopharmacology 2007, 191(3):599-607.
13. Flagel S.B., Akil H., Robinson T.E. Individual differences in the attribution of incentive salience to reward-related cues: implications for addiction. Neuropharmacology 2009, 56:139-148.
14. Flagel S.B., Cameron C.M., Pickup K.N., Watson S.J., Akil H., Robinson T.E. A food predictive cue must be attributed with incentive salience for it to induce c-fos mRNA expression in cortico-striatal-thalamic brain regions. Neuroscience 2011, 196:80-96.
15. Flagel S.B., Clark J.J., Robinson T.E., Mayo L., Czuj A., Willuhn I., Akers C.A., Clinton S.M., Phillips P.E.M., Akil H. A selective role for dopamine in stimulus-reward learning. Nature 2011, 469(7328):53-57.
16. Gallistel C.R., Fairhurst S., Balsam P. The learning curve: implications of a quantitative analysis. Proc. Natl. Acad. Sci. U.S.A. 2004, 101(36):13124-13131.
17. Humphries M.D., Khamassi M., Gurney K. Dopaminergic control of the exploration-exploitation trade-off via the basal ganglia. Front. Neurosci. 2012, 6(9).
18. Huys Q.J.M., Tobler P.N., Hasler G., Flagel S.B. The role of learning-related dopamine signals in addiction vulnerability. Prog. Brain Res. 2014, 211:31-77. http://www.sciencedirect.com/science/article/pii/B9780444634252000039.
19. Khamassi M., Humphries M.D. Integrating cortico-limbic-basal ganglia architectures for learning model-based and model-free navigation strategies. Front. Behav. Neurosci. 2012, 6(79).
20. Lesaint F., Sigaud O., Flagel S.B., Robinson T.E., Khamassi M. Modelling individual differences in the form of pavlovian conditioned approach responses: a dual learning systems approach with factored representations. PLoS Comput. Biol. 2014, 10(2):e1003466.
21. Mahler S.V., Berridge K.C. Which cue to want? Central amygdala opioid activation enhances and focuses incentive salience on a prepotent reward cue. J. Neurosci. 2009, 29(20):6500-6513.
22. McDannald M.A., Lucantonio F., Burke K.A., Niv Y., Schoenbaum G., Striatum Ventral Ventral striatum and orbitofrontal cortex are both required for model-based, but not model-free, reinforcement learning. J. Neurosci. 2011, 31(7):2700-2705.
23. Meyer P.J., Lovic V., Saunders B.T., Yager L.M., Flagel S.B., Morrow J.D., Robinson T.E. Quantifying individual variation in the propensity to attribute incentive salience to reward cues. PLoS ONE 2012, 7(6):e38987.
24. Mouret, J.-B., Doncieux, S., 2010. SFERESv2: evolvin' in the multi-core world. In: WCCI 2010 IEEE World Congress on Computational Intelligence, Congress on Evolutionary Computation (CEC), pp. 4079-4086.
25. Penner M.R., Mizumori S.J.Y. Neural systems analysis of decision making during goal-directed navigation. Prog. Neurobiol. 2012, 96(1):96-135.
26. Robinson T.E., Flagel S.B. Dissociating the predictive and incentive motivational properties of reward-related cues through the study of individual differences. Biol. Psychiatr. 2009, 65(10):869-873.
27. Saunders B.T., Robinson T.E. The role of dopamine in the accumbens core in the expression of Pavlovian-conditioned responses. Eur. J. Neurosci. 2012, 36(4):2521-2532.
28. Saunders B.T., Robinson T.E. Individual variation in resisting temptation: implications for addiction. Neurosci. Biobehav. Rev. 2013, 37(9):1955-1975.
29. Schultz W. Predictive reward signal of dopamine neurons. J. Neurophysiol. 1998, 80:1-27.
30. van der Meer M., Redish A.D. Ventral striatum: a critical look at models of learning and evaluation. Curr. Opin. Neurobiol. 2011, 21(3):387-392.
31. van der Meer M., Johnson A., Schmitzer-Torbert N.C., Redish A.D. Triple dissociation of information processing in dorsal striatum, ventral striatum, and hippocampus on a learned spatial decision task. Neuron 2010, 67(1):25-32.
32. Zhang J., Berridge K.C., Tindell A.J., Smith K.S., Aldridge J.W. A neural computational model of incentive salience. PLoS Comput. Biol. 2009, 5(7):e1000437.