The Curse of Planning: Dissecting Multiple Reinforcement-Learning Systems by Taxing the Central Executive

Psychological Science

Volumn 24, Issue 5, 2013, Pages 751-761

  Otto, A Ross ^a   Gershman, Samuel J ^b   Markman, Arthur B ^a   Daw, Nathaniel D ^c  

^a UNIVERSITY OF TEXAS AT AUSTIN   (United States)

^b PRINCETON UNIVERSITY   (United States)

^c NEW YORK UNIVERSITY   (United States)

Author keywords

cognitive neuroscience; decision making

Indexed keywords

EID: 84877341847     PISSN: 09567976     EISSN: 14679280     Source Type: Journal    
DOI: 10.1177/0956797612463080     Document Type: Article

References (27)

1. Conway A. R. A.,Kane M. J.,Engle R. W.Working memory capacity and its relation to general intelligence.Trends in Cognitive Sciences. 2003;7:547-552.
2. Daw N. D.,Gershman S. J.,Seymour B.,Dayan P.,Dolan R. J.Model-based influences on humans' choices and striatal prediction errors.Neuron. 2011;69:1204-1215.
3. Daw N. D.,Niv Y.,Dayan P.Uncertainty-based competition between prefrontal and dorsolateral striatal systems for behavioral control.Nature Neuroscience. 2005;8:1704-1711.
4. Dayan P.Goal-directed control and its antipodes.Neural Networks. 2009;22:213-219.
5. Dickinson A.Actions and habits: The development of behavioural autonomy.Philosophical Transactions of the Royal Society B: Biological Sciences. 1985;308:67-78.
6. Dickinson A.,Balleine B.Stevens' handbook of experimental psychology: Vol. 3. Learning, motivation, and emotion. 2004). The role of learning in the operation of motivational systems. In GallistelR. (Ed.), Stevens' handbook of experimental psychology: Vol. 3. Learning, motivation, and emotion (3rd ed.). Hoboken, NJ: Wiley.Gallistel R., ed. Hoboken, NJ: Wiley; 2004:.
7. Everitt B. J.,Robbins T. W.Neural systems of reinforcement for drug addiction: From actions to habits to compulsion.Nature Neuroscience. 2005;8:1481-1489.
8. Foerde K.,Knowlton B. J.,Poldrack R. A.Modulation of competing memory systems by distraction.Proceedings of the National Academy of Sciences, USA. 2006;103:11778-11783.
9. Gläscher J.,Daw N.,Dayan P.,O'Doherty J. P.States versus rewards: Dissociable neural prediction error signals underlying model-based and model-free reinforcement learning.Neuron. 2010;66:585-595.
10. Kahneman D.,Frederick S.Heuristics and biases: The psychology of intuitive judgment. Gilovich T.Griffin D.Kahneman D., ed. Cambridge, England: Cambridge University Press; 2002:49-81.
11. KeramatiM.DezfouliA.PirayP. (2011). Speed/accuracy trade-off between the habitual and the goal-directed processes. PLoS Computational Biology, 7(5), e1002055. Retrieved from http://www.ploscompbiol.org/article/info%3Adoi%2F10.1371%2Fjournal.pcbi.1002055.
12. LoewensteinG.O'DonoghueT. (2004). Animal spirits: Affective and deliberative processes in economic behavior (Working Papers No. 04-14). Ithaca, NY: Cornell University Center for Analytic Economics. Retrieved from http://ideas.repec.org/p/ecl/corcae/04-14.html.
13. McClure S. M.,Laibson D. I.,Loewenstein G.,Cohen J. D.Separate neural systems value immediate and delayed monetary rewards.Science. 2004;306:503-507.
14. Miyake A.,Friedman N. P.,Emerson M. J.,Witzki A. H.,Howerter A.,Wager T. D.The unity and diversity of executive functions and their contributions to complex "frontal lobe" tasks: A latent variable analysis.Cognitive Psychology. 2000;41:49-100.
15. Montague P. R.,Dayan P.,Sejnowski T. J.A framework for mesencephalic dopamine systems based on predictive Hebbian learning.The Journal of Neuroscience. 1996;16:1936-1947.
16. Norman D. A.,Shallice T.Consciousness and self-regulation: Advances in research and theory. Davidson R. J.Schwartz G. E.Shapiro D., ed. New York, NY: Plenum; 1986:1-18.
17. O'Doherty J. P.,Dayan P.,Friston K.,Critchley H.,Dolan R. J.Temporal difference models and reward-related learning in the human brain.Neuron. 2003;38:329-337.
18. Otto A. R.,Taylor E. G.,Markman A. B.There are at least two kinds of probability matching: Evidence from a secondary task.Cognition. 2011;118:274-279.
19. Payne J. W.,Bettman J. R.,Johnson E. J.The adaptive decision maker. Cambridge, England: Cambridge University Press; 1993:.
20. Pinheiro J. C.,Bates D. M.Mixed-effects models in S and S-PLUS. New York, NY: Springer; 2000:.
21. Schultz W.,Dayan P.,Montague P. R.A neural substrate of prediction and reward.Science. 1997;275:1593-1599.
22. Sutton R. S.Proceedings of the Seventh International Conference (1990) on Machine Learning. Morgan M. B., ed. San Francisco, CA: Morgan Kaufmann; 1990:216-224.
23. Sutton R. S.,Barto A. G.Reinforcement learning. Cambridge, MA: MIT Press; 1998:.
24. Valentin V. V.,Dickinson A.,O'Doherty J. P.Determining the neural substrates of goal-directed learning in the human brain.Journal of Neuroscience. 2007;27:4019-4026.
25. Waldron E. M.,Ashby F. G.The effects of concurrent task interference on category learning: Evidence for multiple category learning systems.Psychonomic Bulletin & Review. 2001;8:168-176.
26. Yin H. H.,Knowlton B. J.The role of the basal ganglia in habit formation.Nature Reviews Neuroscience. 2006;7:464-476.
27. Zeithamova D.,Maddox W. T.Dual-task interference in perceptual category learning.Memory & Cognition. 2006;34:387-398.