Habit formation

Dialogues in Clinical Neuroscience

Volumn 18, Issue 1, 2016, Pages 33-43

  Smith, Kyle S ^a   Graybiel, Ann M ^b  

^a Lamont Doherty Earth Observatory   (United States)

^b MASSACHUSETTS INSTITUTE OF TECHNOLOGY   (United States)

Author keywords

Action chunking; Action outcome; Addiction; Basal ganglia; Corticostriatal circuit; Obsessive compulsive disorder; Repetitive behavior; Stimulusresponse

Indexed keywords

ADDICTION; ARTICLE; AUTISM; BEHAVIOR; BRAIN FUNCTION; BRAIN REGION; CONNECTOME; ELECTROENCEPHALOGRAM; FEEDBACK SYSTEM; HABIT; HISTORY; HUMAN; NONHUMAN; OBSESSIVE COMPULSIVE DISORDER; REWARD; ANIMAL; BRAIN; NERVE CELL NETWORK; PHYSIOLOGY;

ANIMALS; BRAIN; HABITS; HUMANS; NERVE NET;

EID: 84964689395     PISSN: 12948322     EISSN: None     Source Type: Journal    
DOI: None     Document Type: Article

References (102)

1. James W. The Principles of Psychology. Vol 1. New York, NY: Cosimo; 1890
2. Hull CL. Principles of Behavior. New York, NY: Appleton-Century Crofts; 1943
3. Thorndike EL. Animal Intelligence: an Experimental Study of the Associative Processes in Animals. New York, NY: Macmillan; 1898
4. Dickinson A. Actions and habits: the development of behavioral autonomy. Philos Trans R Soc Lond B Biol Sci. 1985;308(11 35):67-78
5. Balleine BW., Dickinson A. Goal-directed instrumental action: contingency and incentive learning and their cortical substrates. Neuropharmacology. 1998;37(4-5):407-419
6. Holland PC. Cognitive versus stimulus-response theories of learning. Learn Behav. 2008;36(3):227-241
7. Adams CD. Variations in the sensitivity of instrumental responding to reinforcer devaluation. Quart J Exp Psychol B. 1982;34(2):77-98
8. Balleine BW., O'Doherty JP. Human and rodent homologies in action control: corticostriatal determinants of goal-directed and habitual action. Neuropsychopharmacology. 2010;35(1):48-69
9. Yin HH., Ostlund SB., Balleine BW. Reward-guided learning beyond dopamine in the nucleus accumbens: the integrative functions of corticobasal ganglia networks. Eur J Neurosci. 2008;28(8):1437-1448
10. Killcross S., Coutureau E. Coordination of actions and habits in the medial prefrontal cortex of rats. Cereb Cortex. 2003;13(4):400-408
11. Faure A., Haberland U., Conde F., El Massioui N. Lesion to the nigrostriatal dopamine system disrupts stimulus-response habit formation. J Neurosci. 2005;25(11):2771-2780
12. Smith KS., Graybiel AM. Investigating habits: strategies, technologies and models. Front Behav Neurosci. 2014;8:39
13. Yin HH., Knowlton BJ. The role of the basal ganglia in habit formation. Nat Rev Neurosci. 2006;7(6):464-476
14. Hitchcott PK., Quinn JJ., Taylor JR. Bidirectional modulation of goal-directed actions by prefrontal cortical dopamine. Cereb Cortex. 2007;17(12):2820-2827
15. Gremel CM., Costa RM. Orbitofrontal and striatal circuits dynamically encode the shift between goal-directed and habitual actions. Nat Common. 2013;4:2264
16. Schoenbaum G., Roesch M. Orbitofrontal cortex, associative learning, and expectancies. Neuron. 2005;47(5):633-636
17. Packard MG. Exhumed from thought: basal ganglia and response learning in the plus-maze. Behav Brain Res. 2009;199(1):24-31
18. Packard MG., McGaugh JL. Inactivation of hippocampus or caudate nucleus with lidocaine differentially affects expression of place and response learning. Neurobiol Learn Mem. 1996;65(1):65-72
19. Tolman EC., Ritchie BF., Kalish D. Studies in spatial learning: V. Response learning versus place learning by the non-correction method. J Exp Psychology. 1947;37(4):285-292
20. McDonald RJ., White NM. A triple dissociation of memory systems: hippocampus, amygdala, and dorsal striatum. Behav Neurosci. 1993;107(1):3-22
21. Wang LP., Li F., Wang D., Xie K., Shen X., Tsien JZ. NMDA Receptors in dopaminergic neurons are crucial for habit learning. Neuron. 2011;72(6):1055-1066
22. Lee AS., Duman RS., Pittenger C. A double dissociation revealing bidirectional competition between striatum and hippocampus during learning. Proc Natl Acad Sci U S A. 2008;105(44):17163-17168
23. Jog MS., Kubota Y., Connolly CI., Hillegaart V., Graybiel AM. Building neural representations of habits. Science. 1999;286(5445):1745-1749
24. Thorn CA., Atallah HE., Howe MW., Graybiel AM. Differential dynamics of activity changes in dorsolateral and dorsomedial striatal loops during learning. Neuron. 2010;66(5):781-795
25. Kubota Y., Liu J., Hu D., et al Stable encoding of task structure coexists with flexible coding of task events in sensorimotor striatum. J Neurophysiol. 2009;102(4):2142-2160
26. Smith KS., Graybiel AM. A dual operator view of habitual behavior reflecting cortical and striatal dynamics. Neuron. 2013;79(2):361-374
27. Barnes TD., Kubota Y., Hu D., Jin DZ., Graybiel AM. Activity of striatal neurons reflects dynamic encoding and recoding of procedural memories. Nature. 2005;437(7062):1158-1161
28. Graybiel AM. The basal ganglia and chunking of action repertoires. Neurobiol Learn Mem. 1998;70(1-2):119-136
29. Graybiel AM. Habits, rituals, and the evaluative brain. Annu Rev Neurosci. 2008;31:359-387
30. Miller GA. The magical number seven, plus or minus two: some limits on our capacity for processing information. Psychol Rev. 1956;101(2):343-352
31. Graybiel AM., Grafton ST. The striatum: where skills and habits meet. Cold Spring Harb Perspect Biol. 2015;7(8):a021691
32. Tolman EC. Purposive Behavior in Animals and Men. New York, NY: Century; 1932
33. Muenzinger KF. Vicarious trial and error at a point of choice: I. A general survey of its relation to learning efficiency. Pedagog Semin J Genetic Psychol. 1938;53(1):75-86
34. Tolman EC. Cognitive maps in rats and men. Psychol Rev. 1948;55(4):189-208
35. van der Meer M., Kurth-Nelson Z., Redish AD. Information processing in decision-making systems. Neuroscientist. 2012;18(4):342-359
36. Fujii N., Graybiel AM. Representation of action sequence boundaries by macaque prefrontal cortical neurons. Science. 2003;301(5637):1246-1249
37. Desrochers TM., Amemori K., Graybiel AM. Habit learning by naive macaques is marked by response sharpening of striatal neurons representing the cost and outcome of acquired action sequences. Neuron. 2015;87(4):853-868
38. Wunderlich K., Dayan P., Dolan RJ. Mapping value based planning and extensively trained choice in the human brain. Nat Neurosci. 2012;15(5):786-791
39. Liljeholm M., Dunne S., O'Doherty JP. Differentiating neural systems mediating the acquisition vs. expression of goal-directed and habitual behavioral control. Eur J Neurosci. 2015;41(10):1358-1371
40. Daw ND., Niv Y., Dayan P. Actions, policies, values, and the basal ganglia. In: Bezard E, ed. Recent Breakthroughs in Basal Ganglia Research. Hauppauge, NY: Nova Science Publishers; 2005:91-106
41. Dolan RJ., Dayan P. Goals and habits in the brain. Neuron. 2013;80(2):312-325
42. Regier PS., Amemiya S., Redish AD. Hippocampus and subregions of the dorsal striatum respond differently to a behavioral strategy change on a spatial navigation task. J Neurophysiol. 2015;114(3):1399-1416
43. Aldridge JW., Berridge KC., Rosen AR. Basal ganglia neural mechanisms of natural movement sequences. Can J Physiol Pharmacol. 2004;82(8-9):732-739
44. Cui G., Jun SB., Jin X., et al Concurrent activation of striatal direct and indirect pathways during action initiation. Nature. 2013;494(7436):238-242
45. Jin X., Tecuapetla F., Costa RM. Basal ganglia subcircuits distinctively encode the parsing and concatenation of action sequences. Nat Neurosci. 2014;17(3):423-430
46. Rothwell PE., Hayton SJ., Sun GL., Fuccillo MV., Lim BK., Malenka RC. Input-and output-specific regulation of serial order performance by corticostriatal circuits. Neuron. 2015;88(2):345-356
47. Yu C., Gupta J., Chen JF., Yin HH. Genetic deletion of A2A adenosine receptors in the striatum selectively impairs habit formation. J Neurosci. 2009;29(48):15100-15103
48. Shan Q., Christie MJ., Balleine BW. Plasticity in striatopallidal projection neurons mediates the acquisition of habitual actions. Eur J Neurosci. 2015;42(4):2097-2104
49. Corbit LH., Nie H., Janak PH. Habitual responding for alcohol depends upon both AMPA and D2 receptor signaling in the dorsolateral striatum. Front Behav Neurosci. 2014;8:301
50. O'Hare JK., Ade KK., Sukharnikova T., et al Pathway-specific striatal substrates for habitual behavior. Neuron. 2016;89(3):472-479
51. Jin X., Costa RM. Start/stop signals emerge in nigrostriatal circuits during sequence learning. Nature. 2010;466(7305):457-462
52. Fujimoto H., Hasegawa T., Watanabe D. Neural coding of syntactic structure in learned vocalizations in the songbird. J Neurosci. 2011;31(27):10023-10033
53. Coutureau E., Killcross S. Inactivation of the infralimbic prefrontal cortex reinstates goal-directed responding in overtrained rats. Behav Brain Res. 2003;146(1-2):167-174
54. Smith KS., Virkud A., Deisseroth K., Graybiel AM. Reversible online control of habitual behavior by optogenetic perturbation of medial prefrontal cortex. Proc Natl Acad Sci U S A. 2012;109(46):18932-18937
55. Fenno L., Yizhar O., Deisseroth K. The development and application of optogenetics. Ann Rev Neurosci. 2011;34:389-412
56. Bernstein JG., Boyden ES. Optogenetic tools for analyzing the neural circuits of behavior. Trends Cogn Sci. 2011;15(12):592-600
57. Barker JM., Taylor JR., Chandler LJ. A unifying model of the role of the infralimbic cortex in extinction and habits. Leam Mem. 2014;21(9):441-448
58. Lingawi NW., Balleine BW. Amygdala central nucleus interacts with dorsolateral striatum to regulate the acquisition of habits. J Neurosci. 2012;32(3):1073-1081
59. Stott JJ., Redish AD. A functional difference in information processing between orbitofrontal cortex and ventral striatum during decision-making behaviour. Philos Trans R Soc Lond B Biol Sci. doi:10.1098/rstb.2013.0472. 2014;369(1655)
60. Johnson A., Redish AD. Neural ensembles in CA3 transiently encode paths forward of the animal at a decision point. J Neurosci. 2007;27(45):12176-12189
61. Apicella P., Ljungberg T., Scarnati E., Schultz W. Responses to reward in monkey dorsal and ventral striatum. Exp Brain Research. 1991;85(3):491-500
62. Cromwell HC., Hassani OK., Schultz W. Relative reward processing in primate striatum. Exp Brain Res. 2005;162(4):520-525
63. Hikosaka O., Sakamoto M., Usui S. Functional properties of monkey caudate neurons. III. Activities related to expectation of target and reward. J Neurophysiol. 1989;61(4):814-832
64. Schmitzer-Torbert N., Redish AD. Neuronal activity in the rodent dorsal striatum in sequential navigation: separation of spatial and reward responses on the multiple T task. J Neurophysiol. 2004;91(5):2259-2272
65. Smith KS., Graybiel AM. Habit formation coincides with shifts in reinforcement representations in the sensorimotor striatum. J Neurophysiol. Jan 6. Epub ahead of print. 2016
66. Holland PC., Wheeler DS. Representation-mediated food aversions. In: Reilly S, Schachtman T, eds. Conditioned Taste Aversion: Behavioral and Neural Processes. Oxford, UK: Oxford University Press; 2009:196-225
67. Berridge KC. Wanting and liking: observations from the neuroscience and psychology laboratory. Inquiry (Oslo). 2009;52(4):378
68. Glickman SE., Schiff BB. A biological theory of reinforcement. Psychol Rev. 1967;74(2):81-109
69. Robinson TE., Berridge KC. Addiction. Annu Rev Psychol. 2003;54:25-53
70. Carr H., Watson JB. Orientation in the white rat. J Comp Neurol Psychol. 1908;18(1):27-44
71. de Wit S., Barker RA., Dickinson AD., Cools R. Habitual versus goal-directed action control in Parkinson disease. J Cogn Neurosci. 2011;23(5):1218-1229
72. Desmurget M., Turner RS. Motor sequences and the basal ganglia: kinematics, not habits. J Neurosci. 2010;30(22):7685-7690
73. Seger CA., Spiering BJ. A critical review of habit learning and the basal ganglia. Front Syst Neurosci. 2011;5:66
74. Everitt BJ., Robbins TW. Neural systems of reinforcement for drug addiction: from actions to habits to compulsion. Nat Neurosci. 2005;8(11):1481-1489
75. Dezfouli A., Balleine BW. Habits, action sequences and reinforcement learning. Eur J Neurosci. 2012;35(7):1036-1051
76. Dezfouli A., Lingawi NW., Balleine BW. Habits as action sequences: hierarchical action control and changes in outcome value. Philos Trans R Soc Lond B Biol Sci. doi:10.1098/rstb.2013.0482. 2014;369(1655)
77. Nelson A., Killcross S. Amphetamine exposure enhances habit formation. J Neurosci. 2006;26(14):3805-3812
78. Morgan MJ. Resistance to satiation. Animal Behavior. 1974;22:449-466
79. Stalnaker TA., Calhoon GG., Ogawa M., Roesch MR., Schoenbaum G. Neural correlates of stimulus-response and response-outcome associations in dorsolateral versus dorsomedial striatum. Front Integr Neurosci. 2010;4:12
80. Root DH., Tang CC., Ma S., Pawlak AP., West MO. Absence of cue-evoked firing in rat dorsolateral striatum neurons. Behav Brain Res. 2010;211(1):23-32
81. Everitt BJ., Belin D., Economidou D., Pelloux Y., Dalley JW., Robbins TW. Review. Neural mechanisms underlying the vulnerability to develop compulsive drug-seeking habits and addiction. Philos Trans R Soc Lond B Biol Sci. 2008;363(1507):3125-3135
82. Gillan CM., Robbins TW. Goal-directed learning and obsessive-compulsive disorder. Philos Trans R Soc Lond B Biol Sci. doi:10.1098/rstb.2013.0475. 2014;369(1655)
83. Gillan CM., Apergis-Schoute AM., Morein-Zamir S., et al Functional neuroimaging of avoidance habits in obsessive-compulsive disorder. Am J Psychiatry. 2015;172(3):284-293
84. Ahmari SE., Spellman T., Douglass NL., et al Repeated corticostriatal stimulation generates persistent OCD-like behavior. Science. 2013;340(6137):1234-1239
85. Burguiere E., Monteiro P., Feng G., Graybiel AM. Optogenetic stimulation of lateral orbitofronto-striatal pathway suppresses compulsive behaviors. Science. 2013;340(6137):1243-1246
86. Shepherd GM. Corticostriatal connectivity and its role in disease. Nat Rev Neurosci. 2013;14(4):278-291
87. Di Martino A., Kelly C., Grzadzinski R., et al Aberrant striatal functional connectivity in children with autism. Biol Psychiatry. 2011;69(9):847-856
88. Gruner P., Anticevic A., Lee D., Pittenger C. Arbitration between action strategies in obsessive-compulsive disorder. Neuroscientist. Jan 20. Epub ahead of print. 2015
89. Robinson TE., Berridge KC. The neural basis of drug craving: an incentive-sensitization theory of addiction. Brain Res Brain Res Rev. 1993;18(3):247-291
90. Koob GF. Negative reinforcement in drug addiction: the darkness within. Curr Opin Neurobiol. 2013;23(4):559-563
91. Kalivas PW., Volkow ND. The neural basis of addiction: a pathology of motivation and choice. Am J Psychiatry. 2005;162(8):1403-1413
92. Redish AD., Jensen S., Johnson A. A unified framework for addiction: vulnerabilities in the decision process. Behav Brain Sci. 2008;31(4):415-437; discussion 437-487
93. Voon V., Derbyshire K., Ruck C., et al Disorders of compulsivity: a common bias towards learning habits. Mol Psychiatry. 2015;20(3):345-352
94. Belin D., Jonkman S., Dickinson A., Robbins TW., Everitt BJ. Parallel and interactive learning processes within the basal ganglia: relevance for the understanding of addiction. Behav Brain Res. 2009;199(1):89-102
95. Willuhn I., Burgeno LM., Everitt BJ., Phillips PE. Hierarchical recruitment of phasic dopamine signaling in the striatum during the progression of cocaine use. Proc Natl Acad Sci U S A. 109(50):20703-20708
96. Jonkman S., Pelloux Y., Everitt BJ. Differential roles of the dorsolateral and midlateral striatum in punished cocaine seeking. J Neurosci. 2012;32(13):4645-4650
97. Gillan CM., Papmeyer M., Morein-Zamir S., et al Disruption in the balance between goal-directed behavior and habit learning in obsessive-compulsive disorder. Am J Psychiatry. 2011;168(7):718-726
98. Banca P., Voon V., Vestergaard MD., et al Imbalance in habitual versus goal directed neural systems during symptom provocation in obsessive-compulsive disorder. Brain. 2015;138( pt 3):798-811
99. Dias-Ferreira E., Sousa JC., Melo I., et al Chronic stress causes frontostriatal reorganization and affects decision-making. Science. 2009;325(5940):621-625
100. Braun S., Hauber W. Acute stressor effects on goal-directed action in rats. Learn Mem. 2013;20(12):700-709
101. Schwabe L., Wolf OT. Stress prompts habit behavior in humans. J Neurosci. 2009;29(22):7191-7198
102. Asem JS., Holland PC. Immediate response strategy and shift to place strategy in submerged T-maze. Behav Neurosci. 2013;127(6):854-859