A dual operator view of habitual behavior reflecting cortical and striatal dynamics

Neuron

Volumn 79, Issue 2, 2013, Pages 361-374

  Smith, KyleS ^a   Graybiel, AnnM ^a  

^a MASSACHUSETTS INSTITUTE OF TECHNOLOGY   (United States)

Author keywords

[No Author keywords available]

Indexed keywords

ANIMAL BEHAVIOR; ANIMAL EXPERIMENT; ARTICLE; AVERSION; BRAIN FUNCTION; BRAIN REGION; CONTROLLED STUDY; DORSOLATERAL STRIATUM; HEAD MOVEMENT; INFRALIMBIC NEOCORTEX; LEARNING CURVE; MAZE TEST; NEUROMODULATION; NONHUMAN; PRIORITY JOURNAL; RAT; REWARD; SPIKE WAVE; TASK PERFORMANCE;

ACTION POTENTIALS; ANIMALS; BEHAVIOR, ANIMAL; CEREBRAL CORTEX; CORPUS STRIATUM; HABITS; MAZE LEARNING; RATS;

EID: 84880698628     PISSN: 08966273     EISSN: 10974199     Source Type: Journal    
DOI: 10.1016/j.neuron.2013.05.038     Document Type: Article

References (58)

1. Adams C.D. Variations in the sensitivity of instrumental responding to reinforcer devaluation. Q.J. Exp. Psychol. B 1982, 34:77-98.
2. Aldridge J.W., Berridge K.C., Rosen A.R. Basal ganglia neural mechanisms of natural movement sequences. Can. J. Physiol. Pharmacol. 2004, 82:732-739.
3. Anderson C.T., Sheets P.L., Kiritani T., Shepherd G.M. Sublayer-specific microcircuits of corticospinal and corticostriatal neurons in motor cortex. Nat. Neurosci. 2010, 13:739-744.
4. Aston-Jones G., Cohen J.D. An integrative theory of locus coeruleus-norepinephrine function: adaptive gain and optimal performance. Annu. Rev. Neurosci. 2005, 28:403-450.
5. Atallah H.E., Lopez-Paniagua D., Rudy J.W., O'Reilly R.C. Separate neural substrates for skill learning and performance in the ventral and dorsal striatum. Nat. Neurosci. 2007, 10:126-131.
6. Balleine B.W., Dickinson A. Goal-directed instrumental action: contingency and incentive learning and their cortical substrates. Neuropharmacology 1998, 37:407-419.
7. Balleine B.W., Liljeholm M., Ostlund S.B. The integrative function of the basal ganglia in instrumental conditioning. Behav. Brain Res. 2009, 199:43-52.
8. Barnes T.D., Kubota Y., Hu D., Jin D.Z., Graybiel A.M. Activity of striatal neurons reflects dynamic encoding and recoding of procedural memories. Nature 2005, 437:1158-1161.
9. Berke J.D., Breck J.T., Eichenbaum H. Striatal versus hippocampal representations during win-stay maze performance. J.Neurophysiol. 2009, 101:1575-1587.
10. Brainard M.S., Doupe A.J. What songbirds teach us about learning. Nature 2002, 417:351-358.
11. Carelli R.M., Wolske M., West M.O. Loss of lever press-related firing of rat striatal forelimb neurons after repeated sessions in a lever pressing task. J.Neurosci. 1997, 17:1804-1814.
12. Charlesworth J.D., Warren T.L., Brainard M.S. Covert skill learning in a cortical-basal ganglia circuit. Nature 2012, 486:251-255.
13. Coutureau E., Killcross S. Inactivation of the infralimbic prefrontal cortex reinstates goal-directed responding in overtrained rats. Behav. Brain Res. 2003, 146:167-174.
14. Daw N.D., Niv Y., Dayan P. Actions, policies, values, and the basal ganglia. Recent Breakthroughs in Basal Ganglia Research 2005, 91-106. Nova Science Publishers, Hauppauge. E. Bezard (Ed.).
15. de Wit S., Barker R.A., Dickinson A.D., Cools R. Habitual versus goal-directed action control in Parkinson disease. J.Cogn. Neurosci. 2011, 23:1218-1229.
16. Dickinson A. Actions and habits: the development of behavioral autonomy. Philos. Trans. R. Soc. Lond. B Biol. Sci. 1985, 308:67-78.
17. Douglas R.J., Martin K.A. Neuronal circuits of the neocortex. Annu. Rev. Neurosci. 2004, 27:419-451.
18. Everitt B.J., Robbins T.W. Neural systems of reinforcement for drug addiction: from actions to habits to compulsion. Nat. Neurosci. 2005, 8:1481-1489.
19. Fujii N., Graybiel A.M. Representation of action sequence boundaries by macaque prefrontal cortical neurons. Science 2003, 301:1246-1249.
20. Fujimoto H., Hasegawa T., Watanabe D. Neural coding of syntactic structure in learned vocalizations in the songbird. J.Neurosci. 2011, 31:10023-10033.
21. Garcia J., Ervin F.R. Appetites, aversions, and addictions: a model for visceral memory. Recent Adv. Biol. Psychiatry 1968, 10:284-293.
22. Ghazizadeh A., Ambroggi F., Odean N., Fields H.L. Prefrontal cortex mediates extinction of responding by two distinct neural mechanisms in accumbens shell. J.Neurosci. 2012, 32:726-737.
23. Glickman S.E., Schiff B.B. A biological theory of reinforcement. Psychol. Rev. 1967, 74:81-109.
24. Graybiel A.M. The basal ganglia and chunking of action repertoires. Neurobiol. Learn. Mem. 1998, 70:119-136.
25. Graybiel A.M. Habits, rituals, and the evaluative brain. Annu. Rev. Neurosci. 2008, 31:359-387.
26. Henze D.A., Borhegyi Z., Csicsvari J., Mamiya A., Harris K.D., Buzsáki G. Intracellular features predicted by extracellular recordings in the hippocampus invivo. J.Neurophysiol. 2000, 84:390-400.
27. Hikosaka O., Isoda M. Switching from automatic to controlled behavior: cortico-basal ganglia mechanisms. Trends Cogn. Sci. 2010, 14:154-161.
28. Hitchcott P.K., Quinn J.J., Taylor J.R. Bidirectional modulation of goal-directed actions by prefrontal cortical dopamine. Cereb. Cortex 2007, 17:2820-2827.
29. Holland P.C., Straub J.J. Differential effects of two ways of devaluing the unconditioned stimulus after Pavlovian appetitive conditioning. J.Exp. Psychol. Anim. Behav. Process. 1979, 5:65-78.
30. Holtzheimer P.E., Mayberg H.S. Deep brain stimulation for psychiatric disorders. Annu. Rev. Neurosci. 2011, 34:289-307.
31. Hurley K.M., Herbert H., Moga M.M., Saper C.B. Efferent projections of the infralimbic cortex of the rat. J.Comp. Neurol. 1991, 308:249-276.
32. Hyman S.E., Malenka R.C., Nestler E.J. Neural mechanisms of addiction: the role of reward-related learning and memory. Annu. Rev. Neurosci. 2006, 29:565-598.
33. Jin X., Costa R.M. Start/stop signals emerge in nigrostriatal circuits during sequence learning. Nature 2010, 466:457-462.
34. Jog M.S., Kubota Y., Connolly C.I., Hillegaart V., Graybiel A.M. Building neural representations of habits. Science 1999, 286:1745-1749.
35. Kalivas P.W., Volkow N.D. The neural basis of addiction: a pathology of motivation and choice. Am. J. Psychiatry 2005, 162:1403-1413.
36. Killcross S., Coutureau E. Coordination of actions and habits in the medial prefrontal cortex of rats. Cereb. Cortex 2003, 13:400-408.
37. Kimchi E.Y., Torregrossa M.M., Taylor J.R., Laubach M. Neuronal correlates of instrumental learning in the dorsal striatum. J.Neurophysiol. 2009, 102:475-489.
38. Kubota Y., Liu J., Hu D., DeCoteau W.E., Eden U.T., Smith A.C., Graybiel A.M. Stable encoding of task structure coexists with flexible coding of task events in sensorimotor striatum. J.Neurophysiol. 2009, 102:2142-2160.
39. Lingawi N.W., Balleine B.W. Amygdala central nucleus interacts with dorsolateral striatum to regulate the acquisition of habits. J.Neurosci. 2012, 32:1073-1081.
40. Marder E. Variability, compensation, and modulation in neurons and circuits. Proc. Natl. Acad. Sci. USA 2011, 108(Suppl 3):15542-15548.
41. McGeorge A.J., Faull R.L. The organization of the projection from the cerebral cortex to the striatum in the rat. Neuroscience 1989, 29:503-537.
42. Muenzinger K.F. Vicarious trial and error at a point of choice: I. A general survey of its relation to learning efficiency. J.Genet. Psychol. 1938, 53:75-86.
43. Packard M.G. Exhumed from thought: basal ganglia and response learning in the plus-maze. Behav. Brain Res. 2009, 199:24-31.
44. Peters J., Kalivas P.W., Quirk G.J. Extinction circuits for fear and addiction overlap in prefrontal cortex. Learn. Mem. 2009, 16:279-288.
45. Quirk G.J., Beer J.S. Prefrontal involvement in the regulation of emotion: convergence of rat and human studies. Curr. Opin. Neurobiol. 2006, 16:723-727.
46. Redish A.D., Jensen S., Johnson A., Kurth-Nelson Z. Reconciling reinforcement learning models with behavioral extinction and renewal: implications for addiction, relapse, and problem gambling. Psychol. Rev. 2007, 114:784-805.
47. Redish A.D., Jensen S., Johnson A. A unified framework for addiction: vulnerabilities in the decision process. Behav. Brain Sci. 2008, 31:415-437. discussion 437-487.
48. Rhodes S.E., Killcross S. Lesions of rat infralimbic cortex enhance recovery and reinstatement of an appetitive Pavlovian response. Learn. Mem. 2004, 11:611-616.
49. Rich E.L., Shapiro M. Rat prefrontal cortical neurons selectively code strategy switches. J.Neurosci. 2009, 29:7208-7219.
50. Root D.H., Tang C.C., Ma S., Pawlak A.P., West M.O. Absence of cue-evoked firing in rat dorsolateral striatum neurons. Behav. Brain Res. 2010, 211:23-32.
51. Smith K.S., Virkud A., Deisseroth K., Graybiel A.M. Reversible online control of habitual behavior by optogenetic perturbation of medial prefrontal cortex. Proc. Natl. Acad. Sci. USA 2012, 109:18932-18937.
52. Sutton R.S., Barto A.G. Reinforcement Learning: An Introduction 1998, MIT Press, Cambridge.
53. Tang C., Pawlak A.P., Prokopenko V., West M.O. Changes in activity of the striatum during formation of a motor habit. Eur. J. Neurosci. 2007, 25:1212-1227.
54. Thorn C.A., Atallah H., Howe M., Graybiel A.M. Differential dynamics of activity changes in dorsolateral and dorsomedial striatal loops during learning. Neuron 2010, 66:781-795.
55. Tolman E.C. Cognitive maps in rats and men. Psychol. Rev. 1948, 55:189-208.
56. Tricomi E., Balleine B.W., O'Doherty J.P. A specific role for posterior dorsolateral striatum in human habit learning. Eur. J. Neurosci. 2009, 29:2225-2232.
57. Wunderlich K., Dayan P., Dolan R.J. Mapping value based planning and extensively trained choice in the human brain. Nat. Neurosci. 2012, 15:786-791.
58. Yin H.H., Knowlton B.J. The role of the basal ganglia in habit formation. Nat. Rev. Neurosci. 2006, 7:464-476.