Microstimulation of the human substantia nigra alters reinforcement learning

Journal of Neuroscience

Volumn 34, Issue 20, 2014, Pages 6887-6895

  Ramayya, Ashwin G ^a   Misra, Amrit ^b   Baltuch, Gordon H ^a   Kahana, Michael J ^a  

^a UNIVERSITY OF PENNSYLVANIA   (United States)

^b DREXEL UNIVERSITY   (United States)

Author keywords

Dopamine; Human; Microstimulation; Parkinson's disease; Reinforcement learning; Substantia nigra

Indexed keywords

ADULT; AGED; ARTICLE; BEHAVIOR CHANGE; BRAIN DEPTH STIMULATION; CLINICAL ARTICLE; CONTROLLED STUDY; DOPAMINERGIC ACTIVITY; ELECTROSTIMULATION; FEEDBACK SYSTEM; FEMALE; HUMAN; MALE; MIDDLE AGED; PARKINSON DISEASE; PRIORITY JOURNAL; REINFORCEMENT; REWARD; STATISTICAL MODEL; STIMULUS RESPONSE; SUBSTANTIA NIGRA; TASK PERFORMANCE;

DOPAMINE; HUMAN; MICROSTIMULATION; PARKINSON'S DISEASE; REINFORCEMENT LEARNING; SUBSTANTIA NIGRA;

AGED; DEEP BRAIN STIMULATION; ELECTRIC STIMULATION; FEMALE; HUMANS; MALE; MIDDLE AGED; PARKINSON DISEASE; PROBABILITY LEARNING; REINFORCEMENT (PSYCHOLOGY); SUBSTANTIA NIGRA;

EID: 84900385713     PISSN: 02706474     EISSN: 15292401     Source Type: Journal    
DOI: 10.1523/JNEUROSCI.5445-13.2014     Document Type: Article

References (43)

1. Akaike H (1974) A new look at the statistical model identification. IEEE Transactions on Automatic Control 19:716-723. CrossRef
2. Bayer HM, Glimcher PW (2005) Midbrain dopamine neurons encode a quantitative reward prediction error signal. Neuron 47:129-141. CrossRef Medline
3. Bayer HM, Lau B, GlimcherPW (2007) Statistics of midbrain dopaminergic neuron spike trains in the awake primate. J Neurophysiol 98:1428-1439. CrossRef Medline
4. Clark KL, Armstrong KM, Moore T (2011) Probing neural circuitry and function with electrical microstimulation. Proc Biol Sci 278:1121-1130. CrossRef Medline
5. Cools R, Barker RA, Sahakian BJ, Robbins TW (2001) Enhanced or impaired cognitive function in Parkinson's disease as a function of dopaminergic medication and task demands. Cereb Cortex 11:1136-1143. CrossRef Medline
6. Damier P, Hirsch EC, Agid Y, Graybiel AM (1999) The substantia nigra of the human brain. II. Patterns of loss of dopamine-containing neurons in Parkinson's disease. Brain 122:1437-1448. CrossRef Medline
7. Daw ND, Kakade S, Dayan P (2002) Opponent interactions between serotonin and dopamine. Neural Netw 15:603-616. CrossRef Medline
8. Daw ND, O'Doherty JP, Dayan P, Seymour B, Dolan RJ (2006) Cortical substrates for exploratory decisions in humans. Nature 441:876-879. CrossRef Medline
9. Foerde K, Race E, Verfaellie M, Shohamy D (2013) A role for the medial temporal lobe in feedback-driven learning: evidence from amnesia. J Neurosci 33:5698-5704. CrossRef Medline
10. Frank MJ, Surmeier DJ (2009) Do substantia nigra dopaminergic neurons differentiate between reward and punishment? J Mol Cell Biol 1:15-16. CrossRef Medline
11. Frank MJ, Seeberger LC, O'reilly RC (2004) By carrot or by stick: Cognitive reinforcement learning in parkinsonism. Science 306:1940-1943. CrossRef Medline
12. Frank MJ, Moustafa AA, Haughey HM, Curran T, Hutchison KE (2007) Genetic triple dissociation reveals multiple roles for dopamine in reinforcement learning. Proc Natl Acad SciU S A 104:16311-16316. CrossRef Medline
13. Glimcher PW (2011) Understanding dopamine and reinforcement learning: the dopamine reward prediction error hypothesis. Proc Natl Acad Sci U S A 108:15647-15654. CrossRef Medline
14. Grattan L, Rutledge R, Glimcher P (2011) Program No. 732.12. 2011 Neuroscience Meeting Planner, San Diego, CA: Society for Neuroscience. Online.
15. Haber SN, Fudge JL, McFarland NR (2000) Striatonigrostriatal pathways in primates form an ascending spiral from the shell to the dorsolateral striatum. J Neurosci 20:2369-2382. Medline
16. Henny P, Brown M, Northrop A, Faunes M, Ungless MA, Magill PJ, Bolam JP (2012) Structural correlates of heterogeneous in vivo activity of midbrain dopaminergic neurons. Nat Neurosci 15:613-619. CrossRef Medline
17. Histed MH, Bonin V, Reid RC (2009) Direct activation of sparse, distributed populations of cortical neurons by electrical microstimulation. Neuron 63:508-522. CrossRef Medline
18. Hollerman JR, Grace AA (1990) The effects of dopamine-depleting brain lesions on the electrophysiological activity of rat substantia nigra dopamine neurons. Brain Res 533:203-212. CrossRef Medline
19. Jaggi JL, Umemura A, Hurtig HI, Siderowf AD, Colcher A, Stern MB, Baltuch GH (2004) Bilateral subthalamic stimulation of the subthalamic nucleus in Parkinson's disease: surgical efficacy and prediction of outcome. Stereotact Funct Neurosurg 82:104-114. CrossRef Medline
20. Knowlton BJ, Mangles JA, Squire LR (1996) A neostriatal habit learning system in humans. Science 273:1399-1402. CrossRef Medline
21. Lafreniere-Roula M, Hutchison WD, Lozano AM, Hodaie M, Dostrovsky JO (2009) Microstimulation-induced inhibition as a tool to aid targeting the ventral border of the subthalamic nucleus. J Neurosurg 111:724-728. CrossRef Medline
22. Lau B, Glimcher PW (2008) Value representations in the primate striatum during matching behavior. Neuron 58:451-463. CrossRef Medline
23. Ma SY, Rinne JO, Collan Y, Röyttä M, Rinne UK (1996) A quantitative morphometrical study of neuron degeneration in the substantia nigra in Parkinson's disease. J Neurol Sci 140:40-45. CrossRef Medline
24. Matsumoto M, Hikosaka O (2009) Two types of dopamine neuron distinctly convey positive and negative motivational signals. Nature 459: 837-841. CrossRef Medline
25. Montague PR, Dayan P, Sejnowski TJ (1996) A framework for mesencephalic dopamine systems based on predictive Hebbian learning. J Neurosci 16:1936-1947. Medline
26. Morita K, Morishima M, Sakai K, Kawaguchi Y (2012) Reinforcement learning: computing the temporal difference of values via distinct corticostriatal pathways. Trends Neurosci 35:457-467. CrossRef Medline
27. Niv Y, Daw ND, Joel D, Dayan P (2007) Tonic dopamine: opportunity costs and the control of response vigor. Psychopharmacology 191:507-520. CrossRef Medline
28. O'Doherty J, Dayan P, Schultz J, Deichmann R, Friston K, Dolan RJ (2004) Dissociable roles of ventral and dorsal striatum in instrumental conditioning. Science 304:452-454. CrossRef Medline
29. Pan WX, Brown J, Dudman JT (2013) Neural signals of extinction in the inhibitory microcircuit of the ventral midbrain. Nat Neurosci 16:71-78. CrossRef Medline
30. Quiroga RQ, Reddy L, Kreiman G, Koch C, Fried I (2005) Invariant visual representation by single neurons in the human brain. Nature 435:1102-1107. CrossRef Medline
31. Reynolds JN, Hyland BI, Wickens JR (2001) A cellular mechanism of reward-related learning. Nature 413:67-70. CrossRef Medline
32. Rossion B, Pourtois G (2004) Revisiting Snodgrass and Vanderwart's object set: The role of surface detail in basic-level object recognition. Perception 33:217-236. CrossRef Medline
33. Rutledge RB, Lazzaro SC, Lau B, Myers CE, Gluck MA, GlimcherPW (2009) Dopaminergic drugs modulate learning rates and perseveration in Parkinson's patients in a dynamic foraging task. J Neurosci 29:15104-15114. CrossRef Medline
34. Schultz W, Dayan P, Montague PR (1997) A neural substrate of prediction and reward. Science 275:1593-1599. CrossRef Medline
35. Shiner T, Seymour B, Wunderlich K, Hill C, Bhatia KP, Dayan P, Dolan RJ (2012) Dopamine and performance in a reinforcement learning task: evidence from Parkinson's disease. Brain 135:1871-1883. CrossRef Medline
36. Sugrue LP, Corrado GS, Newsome WT (2005) Choosing the greater of two goods: neural currencies for valuation and decision making. Nat Rev Neurosci 6:363-375. CrossRef Medline
37. Sutton, R. and Barto, A (1990) Time-derivative models of Pavlovian reinforcement. In: Learning and computational neuroscience: foundations of adaptive networks (Gabriel M, Moore J, eds), pp 497-537. Cambridge, MA: MIT.
38. Tepper JM, Martin LP, Anderson DR (1995) GABA-A receptor-mediated inhibition of rat substantia nigra dopaminergic neurons by pars reticulata projection neurons. J Neurosci 15:3092-3103. Medline
39. Tsai HC, Zhang F, Adamantidis A, Stuber GD, Bonci A, de Lecea L, Deisseroth K (2009) Phasic firing in dopaminergic neurons is sufficient for behavioral conditioning. Science 324:1080-1084. CrossRef Medline
40. Ungless MA, Grace AA (2012) Are you or aren't you? Challenges associated with physiologically identifying dopamine neurons. Trends Neurosci 35: 422-430. CrossRef Medline
41. Vandecasteele M, Glowinski J, Venance L (2005) Electrical synapses between dopaminergic neurons of the substantia nigra pars compacta. J Neurosci 25:291-298. CrossRef Medline
42. Zaghloul KA, Blanco JA, Weidemann CT, McGill K, Jaggi JL, Baltuch GH, Kahana MJ (2009) Human substantia nigra neurons encode unexpected financial rewards. Science 323:1496-1499. CrossRef Medline
43. Zigmond MJ, Abercrombie ED, Berger TW, Grace AA, Stricker EM (1990) Compensations after lesions of central dopaminergic neurons: some clinical and basic implications. Trends Neurosci 13:290-296. CrossRef Medline