Orbitofrontal and striatal circuits dynamically encode the shift between goal-directed and habitual actions

Nature Communications

Volumn 4, Issue , 2013, Pages

  Gremel, Christina M ^a   Costa, Rui M ^b  

^a NATIONAL INSTITUTE ON ALCOHOL ABUSE AND ALCOHOLISM   (United States)

^b CHAMPALIMAUD CENTRE FOR THE UNKNOWN   (Portugal)

Author keywords

[No Author keywords available]

Indexed keywords

DECISION MAKING; INHIBITION; NEUROLOGY; PARADIGM SHIFT; RODENT;

ANIMAL CELL; ANIMAL EXPERIMENT; ANIMAL TISSUE; ARTICLE; COMPULSION; CONTROLLED STUDY; HABITUAL ADAPTATION; HABITUATION; IN VIVO STUDY; LEARNING; MOUSE; NERVE CELL; NONHUMAN; OPTOGENETICS; ORBITAL CORTEX; REINFORCEMENT; REWARD; STRIATE CORTEX;

EID: 84882406435     PISSN: None     EISSN: 20411723     Source Type: Journal    
DOI: 10.1038/ncomms3264     Document Type: Article

References (56)

1. Adams, C. D. Variations in the sensitivity of instrumental responding to reinforcer devaluation. The Quarterly journal of experimental psychology B, Comparative and physiological psychology 34, 77-98 (1982).
2. Adams, C. &Dickinson, A. Instrumental responding following reinforcer devaluation. Q. J. Exp. Psychol. B 33, 109-121 (1981).
3. Dickinson, A. Actions and habits: the development of behavioural autonomy. Philos. Trans. R. Soc Lond. B. Biol. Sci. 308, 67-78 (1985).
4. Colwill, R. M. &Rescorla, R. A. Postconditioning devaluation of a reinforcer affects instrumental responding. J. Exp. Psychol. Anim. Behav. Process 11, 120-132 (1985).
5. Dias-Ferreira, E. et al. Chronic stress causes frontostriatal reorganization and affects decision-making. Science 325, 621-625 (2009).
6. Balleine, B. W., Delgado, M. R. &Hikosaka, O. The role of the dorsal striatum in reward and decision-making. J. Neurosci. 27, 8161-8165 (2007).
7. Everitt, B. J. &Robbins, T. W. Neural systems of reinforcement for drug addiction: from actions to habits to compulsion. Nat. Neurosci. 8, 1481-1489 (2005).
8. Gillan, C. M. et al. Disruption in the balance between goal-directed behaviour and habit learning in obsessive-compulsive disorder. Am. J. Psychiatry 168, 718-726 (2011).
9. Yin, H. H. &Knowlton, B. J. The role of the basal ganglia in habit formation. Nat. Rev. Neurosci. 7, 464-476 (2006).
10. Balleine, B. W. &O'Doherty, J. P. Human and rodent homologies in action control: corticostriatal determinants of goal-directed and habitual action. Neuropsychopharmacology 35, 48-69 (2009).
11. Liljeholm, M. &O'Doherty, J. P. Contributions of the striatum to learning, motivation, and performance: an associative account. Trends. Cogn. Sci. (Regul. Ed.) 16, 467-475 (2012).
12. Pan, W. X., Mao, T. &Dudman, J. T. Frontiers: inputs to the dorsal striatum of the mouse reflect the parallel circuit architecture of the forebrain. Front. Neuroanat. 4, 147 (2010).
13. Voorn, P., Vanderschuren, L. J. M. J., Groenewegen, H. J., Robbins, T. W. &Pennartz, C. M. A. Putting a spin on the dorsal-ventral divide of the striatum. Trends. Neurosci. 27, 468-474 (2004).
14. McGeorge, A. J. &Faull, R. L. The organization of the projection from the cerebral cortex to the striatum in the rat. Neuroscience 29, 503-537 (1989).
15. Yin, H. H., Ostlund, S. B., Knowlton, B. J. &Balleine, B. W. The role of the dorsomedial striatum in instrumental conditioning. Eur. J. Neurosci. 22, 513-523 (2005).
16. Yin, H. H., Knowlton, B. J. &Balleine, B. W. Lesions of dorsolateral striatum preserve outcome expectancy but disrupt habit formation in instrumental learning. Eur. J. Neurosci. 19, 181-189 (2004).
17. Yin, H. H., Knowlton, B. J. &Balleine, B. W. Inactivation of dorsolateral striatum enhances sensitivity to changes in the action-outcome contingency in instrumental conditioning. Behav. Brain Res. 166, 189-196 (2006).
18. Joel, D., Doljansky, J., Roz, N. &Rehavi, M. Role of the orbital cortex and of the serotonergic system in a rat model of obsessive compulsive disorder. Neuroscience 130, 25-36 (2005).
19. Rotge, J.-Y. et al. Meta-analysis of brain volume changes in obsessivecompulsive disorder. Biol. Psychiatry 65, 75-83 (2009).
20. Atmaca, M. et al. Volumetric MRI assessment of brain regions in patients with refractory obsessive-compulsive disorder. Prog. NeuroPsychopharmacol. Biol. Psychiatry 30, 1051-1057 (2006).
21. Dickinson, A. &Balleine, B. Motivational control of goal-directed action. Anim. Learn. Behav. 22, 1-18 (1994).
22. Derusso, A. L. et al. Instrumental uncertainty as a determinant of behaviour under interval schedules of reinforcement. Front. Integr. Neurosci. 4, 17 (2010).
23. Balleine, B. W. &Ostlund, S. B. Still at the choice-point: action selection and initiation in instrumental conditioning. Ann. N. Y. Acad. Sci. 1104, 147-171 (2007).
24. Hilário, M., Holloway, T., Jin, X. &Costa, R. M. Different dorsal striatum circuits mediate action discrimination and action generalization. Eur. J. Neurosci. 35, 1105-1114 (2012).
25. Walton, M. E., Behrens, T. E. J., Buckley, M. J., Rudebeck, P. H. &Rushworth, M. F. S. Separable learning systems in the macaque brain and the role of orbitofrontal cortex in contingent learning. Neuron 65, 927-939 (2010).
26. Gottfried, J. A., O'doherty, J. &Dolan, R. J. Encoding predictive reward value in human amygdala and orbitofrontal cortex. Science 301, 1104-1107 (2003).
27. Valentin, V. V., Dickinson, A. &O'Doherty, J. P. Determining the neural substrates of goal-directed learning in the human brain. J. Neurosci. 27, 4019-4026 (2007).
28. Tanaka, S. C., Balleine, B. W. &O'Doherty, J. P. Calculating consequences: brain systems that encode the causal effects of actions. J. Neurosci. 28, 6750-6755 (2008).
29. O'Doherty, J. P. Lights, camembert, action! The role of human orbitofrontal cortex in encoding stimuli, rewards, and choices. Ann. N. Y. Acad. Sci. 1121, 254-272 (2007).
30. Schoenbaum, G., Chiba, A. A. &Gallagher, M. Orbitofrontal cortex and basolateral amygdala encode expected outcomes during learning. Nat. Neurosci. 1, 155-159 (1998).
31. Izquierdo, A., Suda, R. &Murray, E. Bilateral orbital prefrontal cortex lesions in rhesus monkeys disrupt choices guided by both reward value and reward contingency. J. Neurosci. 24, 7540 (2004).
32. Rudebeck, P. et al. Frontal cortex subregions play distinct roles in choices between actions and stimuli. J. Neurosci. 28, 13775 (2008).
33. Pickens, C. et al. Different roles for orbitofrontal cortex and basolateral amygdala in a reinforcer devaluation task. J. Neurosci. 23, 11078 (2003).
34. Padoa-Schioppa, C. &Assad, J. A. Neurons in the orbitofrontal cortex encode economic value. Nature 441, 223-226 (2006).
35. Plassmann, H., O'doherty, J. &Rangel, A. Orbitofrontal cortex encodes willingness to pay in everyday economic transactions. J. Neurosci. 27, 9984-9988 (2007).
36. Young, J. J. &Shapiro, M. L. Dynamic coding of goal-directed paths by orbital prefrontal cortex. J. Neurosci. 31, 5989-6000 (2011).
37. Takahashi, Y. K. et al. Expectancy-related changes in firing of dopamine neurons depend on orbitofrontal cortex. Nat. Neurosci. 14, 1590-1597 (2011).
38. Kennerley, S. W., Behrens, T. E. J. &Wallis, J. D. Double dissociation of value computations in orbitofrontal and anterior cingulate neurons. Nat. Neurosci. 14, 1581-1589 (2011).
39. Sul, J. H., Kim, H., Huh, N., Lee, D. &Jung, M. W. ScienceDirect.com- neuron-distinct roles of rodent orbitofrontal and medial prefrontal cortex in decision making. Neuron 66, 449-460 (2010).
40. Hoover, W. B. &Vertes, R. P. Projections of the medial orbital and ventral orbital cortex in the rat. J. Comp. Neurol. 519, 3766-3801 (2011).
41. Schilman, E. A., Uylings, H. B. M., Galis-de Graaf, Y., Joel, D. &Groenewegen, H. J. The orbital cortex in rats topographically projects to central parts of the caudate-putamen complex. Neurosci. Lett. 432, 40-45 (2008).
42. Corbit, L. H. &Balleine, B. W. Double dissociation of basolateral and central amygdala lesions on the general and outcome-specific forms of pavlovianinstrumental transfer. J. Neurosci. 25, 962-970 (2005).
43. Wassum, K. M., Cely, I. C., Balleine, B. W. &Maidment, N. T. Micro-opioid receptor activation in the basolateral amygdala mediates the learning of increases but not decreases in the incentive value of a food reward. J. Neurosci. 31, 1591-1599 (2011).
44. Reynolds, J. N., Hyland, B. I. &Wickens, J. R. A cellular mechanism of reward-related learning. Nature 413, 67-70 (2001).
45. Gourley, S. L., Lee, A. S., Howell, J. L., Pittenger, C. &Taylor, J. R. Dissociable regulation of instrumental action within mouse prefrontal cortex. Eur. J. Neurosci. 32, 1726-1734 (2010).
46. Hilário, M. R., Clouse, E., Yin, H. H. &Costa, R. M. Endocannabinoid signaling is critical for habit formation. Front. Integr. Neurosci. 1 (2007).
47. Jin, X. &Costa, R. M. Start/stop signals emerge in nigrostriatal circuits during sequence learning. Nature 466, 457-462 (2010).
48. Thorn, C. A., Atallah, H., Howe, M. &Graybiel, A. M. Differential dynamics of activity changes in dorsolateral and dorsomedial striatal loops during learning. Neuron 66, 781-795 (2010).
49. Stalnaker, T. A., Calhoon, G. G., Ogawa, M., Roesch, M. R. &Schoenbaum, G. Frontiers: neural correlates of stimulus-response and response-outcome associations in dorsolateral versus dorsomedial striatum. Front. Integr. Neurosci. 4, 12 (2010).
50. Dong, S., Rogan, S. C. &Roth, B. L. Directed molecular evolution of DREADDs: a generic approach to creating next-generation RASSLs. Nat. Protoc. 5, 561-573 (2010).
51. Alexander, G. M. et al. Remote control of neuronal activity in transgenic mice expressing evolved G protein-coupled receptors. Neuron 63, 27-39 (2009).
52. Tye, K. M. et al. Amygdala circuitry mediating reversible and bidirectional control of anxiety. Nature 471, 358-362 (2011).
53. Ostlund, S. B. &Balleine, B. W. Orbitofrontal cortex mediates outcome encoding in Pavlovian but not instrumental conditioning. J. Neurosci. 27, 4819-4825 (2007).
54. Colwill, R. M. &Rescorla, R. A. The Psychology of Learning and MotivationBower, G. (ed.) 55-104 (Academic, 1986).
55. Costa, R. M., Cohen, D. &Nicolelis, M. A. L. Differential corticostriatal plasticity during fast and slow motor skill learning in mice. Curr. Biol. 14, 1124-1134 (2004).
56. Burkhardt, J. M., Jin, X. &Costa, R. M. Dissociable effects of dopamine on neuronal firing rate and synchrony in the dorsal striatum. Front. Integr. Neurosci. 3, 28 (2009).