Decomposing effects of dopaminergic medication in Parkinson's disease on probabilistic action selection - learning or performance?

European Journal of Neuroscience

Volumn 35, Issue 7, 2012, Pages 1144-1151

  Smittenaar, P ^a   Chase, H W ^b   Aarts, E ^c   Nusselein, B ^d   Bloem, B R ^e   Cools, R ^d,f  

^a UNIVERSITY COLLEGE LONDON   (United Kingdom)

^b UNIVERSITY OF PITTSBURGH SCHOOL OF MEDICINE   (United States)

^c UNIVERSITY OF CALIFORNIA   (United States)

^d RADBOUD UNIVERSITY NIJMEGEN   (Netherlands)

^e RADBOUD UNIVERSITY   (Netherlands)

^f RADBOUD UNIVERSITY MEDICAL CENTER   (Netherlands)

Author keywords

Decision making; Dopamine; Reinforcement learning; Valence; Value based performance

Indexed keywords

DOPAMINE; DOPAMINE RECEPTOR STIMULATING AGENT; LEVODOPA;

ACCURACY; ADULT; ARTICLE; AVOIDANCE BEHAVIOR; CLINICAL ARTICLE; CONTROLLED STUDY; DRUG EFFECT; FEMALE; HUMAN; LEARNING; MALE; PARKINSON DISEASE; PERFORMANCE; PRIORITY JOURNAL; PUNISHMENT; RESPONSE TIME; REWARD; TASK PERFORMANCE;

DOPAMINE; DOPAMINE AGENTS; FEMALE; HUMANS; LEARNING; LEVODOPA; MALE; MIDDLE AGED; PARKINSON DISEASE; PROBABILITY; PSYCHOMOTOR PERFORMANCE; REACTION TIME;

EID: 84859359283     PISSN: 0953816X     EISSN: 14609568     Source Type: Journal    
DOI: 10.1111/j.1460-9568.2012.08043.x     Document Type: Article

References (65)

1. Barratt, E.S. (1985) Impulsiveness subtraits: Arousal and information processing. In Spence, J.T. & Itard, C.E. (Eds), Motivation, emotion, and personality. Amsterdam, Elsevier, pp. 137-146.
2. Bayer, H.M. & Glimcher, P.W. (2005) Midbrain dopamine neurons encode a quantitative reward prediction error signal. Neuron, 47, 129-141.
3. Beck, A.T., Ward, C.H., Mendelson, M.L., Mock, J.E. & Erbaugh, J.K. (1961) An inventory for measuring depression. Arch. Gen. Psychiatry, 4, 561-571.
4. Belin, D., Jonkman, S., Dickinson, A., Robbins, T.W. & Everitt, B.J. (2009) Parallel and interactive learning processes within the basal ganglia: relevance for the understanding of addiction. Behav. Brain Res., 199, 89-102.
5. Berridge, K.C. (2007) The debate over dopamine's role in reward: the case for incentive salience. Psychopharmacology, 191, 391-431.
6. Berridge, K.C. & Robinson, T.E. (1998) What is the role of dopamine in reward: hedonic impact, reward learning, or incentive salience? Brain Res. Brain Res. Rev., 28, 309-369.
7. Bodi, N., Keri, S., Nagy, H., Moustafa, A., Myers, C.E., Daw, N., Dibo, G., Takats, A., Bereczki, D. & Gluck, M.A. (2009) Reward-learning and the novelty-seeking personality: a between-and within-subjects study of the effects of dopamine agonists on young Parkinson's patients. Brain, 132, 2385-2395.
8. Bornstein, A.M. & Daw, N.D. (2011) Multiplicity of control in the basal ganglia: computational roles of striatal subregions. Curr. Opin. Neurobiol., 21, 374-380.
9. Boureau, Y.L. & Dayan, P. (2011) Opponency revisited: competition and cooperation between dopamine and serotonin. Neuropsychopharmacology, 36, 74-97.
10. Cools, R. (2006) Dopaminergic modulation of cognitive function-implications for L-DOPA treatment in Parkinson's disease. Neurosci. Biobehav. Rev., 30, 1-23.
11. Cools, R. (2011) Dopaminergic control of the striatum for high-level cognition. Curr. Opin. Neurobiol., 21, 402-407.
12. Cools, R., Barker, R.A., Sahakian, B.J. & Robbins, T.W. (2001) Enhanced or impaired cognitive function in Parkinson's disease as a function of dopaminergic medication and task demands. Cereb. Cortex, 11, 1136-1143.
13. Cools, R., Altamirano, L. & D'Esposito, M. (2006) Reversal learning in Parkinson's disease depends on medication status and outcome valence. Neuropsychologia, 44, 1663-1673.
14. Cools, R., Nakamura, K. & Daw, N.D. (2011) Serotonin and dopamine: unifying affective, activational, and decision functions. Neuropsychopharmacology, 36, 98-113.
15. Crockett, M.J., Clark, L. & Robbins, T.W. (2009) Reconciling the role of serotonin in behavioral inhibition and aversion: acute tryptophan depletion abolishes punishment-induced inhibition in humans. J. Neurosci., 29, 11993-11999.
16. Daw, N.D., Niv, Y. & Dayan, P. (2005) Uncertainty-based competition between prefrontal and dorsolateral striatal systems for behavioral control. Nat. Neurosci., 8, 1704-1711.
17. Dubois, B., Slachevsky, A., Litvan, I. & Pillon, B. (2000) The FAB: a Frontal Assessment Battery at bedside. Neurology, 55, 1621-1626.
18. Fahn, S., Elton, R.L. & Members of the UPDRS Development Committee. (1987) Unified Parkinson's disease rating scale. In: Fahn, S., Marsden, C.D., Calne, D.B. & Goldstein, M. (Eds), Recent Developments in Parkinson's Disease, Vol 2. Macmillan Health Care Information 1987, Florham Park, NJ, pp. 153-163.
19. Folstein, M.F., Folstein, S.E. & McHugh, P.R. (1975) Mini-mental state A practical method for grading the cognitive state of patients for the clinician. J. Psychiatr. Res., 12, 189-198.
20. Frank, M.J. (2005) Dynamic Dopamine modulation in the basal ganglia: a neurocomputational account of cognitive deficits in medicated and nonmedicated parkinsonism. J. Cogn. Neurosci., 17, 51-72.
21. Frank, M.J. (2011) Computational models of motivated action selection in corticostriatal circuits. Curr. Opin. Neurobiol., 21, 381-386.
22. Frank, M.J., Seeberger, L.C. & O'Reilly, R.C. (2004) By carrot or by stick: cognitive reinforcement learning in parkinsonism. Science, 306, 1940-1943.
23. Frank, M.J., Moustafa, A., Haughey, H.M., Curran, T. & Hutchison, K.E. (2007) Genetic triple dissociation reveals multiple roles for dopamine in reinforcement learning. Proc. Nat. Acad. Sci. USA, 104, 16311-16316.
24. Gallistel, C.R. (1974) Note on temporal summation in the reward system. J Comp Physiol Psychol, 87, 870-875.
25. Hassler, R. (1938) Zur Pathologie der Paralysis agitans und des postenzephalitischen Parkinsonismus. J Psychol Neurol, 48, 387-476.
26. Hollerman, J.R. & Schultz, W. (1998) Dopamine neurons report an error in the temporal prediction of reward during learning. Nat. Neurosci., 1, 304-309.
27. Houk, J.C. & Wise, S.P. (1995) Distributed modular architectures linking basal ganglia, cerebellum, and cerebral cortex: their role in planning and controlling action. Cereb. Cortex, 5, 95-110.
28. Howell, D.C. (1997). Statistical Methods for Psychology. Duxbury Press, Belmont, CA.
29. Ikemoto, S. & Panksepp, J. (1999) The role of nucleus accumbens dopamine in motivated behavior: a unifying interpretation with special reference to reward-seeking. Brain Res. Rev., 31, 6-41.
30. Jocham, G., Klein, T.A. & Ullsperger, M. (2011) Dopamine-mediated reinforcement learning signals in the striatum and ventromedial prefrontal cortex underlie value-based choices. J. Neurosci., 31, 1606-1613.
31. Ljungberg, T., Apicella, P. & Schultz, W. (1992) Responses of monkey dopamine neurons during learning of behavioral reactions. J. Neurophysiol., 67, 145-163.
32. Maia, T.V. & Frank, M.J. (2011) From reinforcement learning models to psychiatric and neurological disorders. Nat. Neurosci., 14, 154-162.
33. Montague, P.R., Dayan, P. & Sejnowski, T.J. (1996) A framework for mesencephalic dopamine systems based on predictive Hebbian learning. J. Neurosci., 16, 1936-1947.
34. Montague, P.R., Hyman, S.E. & Cohen, J.D. (2004) Computational roles for dopamine in behavioural control. Nature, 431, 760-767.
35. Moustafa, A., Cohen, M.X., Sherman, S.J. & Frank, M.J. (2008) A role for dopamine in temporal decision making and reward maximization in parkinsonism. J. Neurosci., 28, 12294-12304.
36. Nelson, H.E. & O'Connell, A. (1978) Dementia: the estimation of premorbid intelligence levels using the New Adult Reading Test. Cortex, 14, 234-244.
37. Niv, Y., Joel, D. & Dayan, P. (2006) A normative perspective on motivation. Trends Cogn. Sci., 10, 375-381.
38. Niv, Y., Daw, N.D., Joel, D. & Dayan, P. (2007) Tonic dopamine: opportunity costs and the control of response vigor. Psychopharmacology, 191, 507-520.
39. Palminteri, S., Lebreton, M., Worbe, Y., Grabli, D., Hartmann, A. & Pessiglione, M. (2009) Pharmacological modulation of subliminal learning in Parkinson's and Tourette's syndromes. Proc. Natl. Acad. Sci. USA, 106, 19179-19184.
40. Parkinson, J.A., Olmstead, M.C., Burns, L.H., Robbins, T.W. & Everitt, B.J. (1999) Dissociation in effects of lesions of the nucleus accumbens core and shell on appetitive pavlovian approach behavior and the potentiation of conditioned reinforcement and locomotor activity by D-amphetamine. J. Neurosci., 19, 2401-2411.
41. Paton, J.J., Belova, M.A., Morrison, S.E. & Salzman, C.D. (2006) The primate amygdala represents the positive and negative value of visual stimuli during learning. Nature, 439, 865-870.
42. Poldrack, R.A., Clark, J., Paré-Blagoev, E.J., Shohamy, D., Myers, C. & Gluck, M.A. (2001) Interactive memory systems in the human brain. Nature, 414, 546-550.
43. Robbins, T.W. & Everitt, B.J. (1982) Functional studies of the central catecholamines. Int. Rev. Neurobiol., 23, 303-365.
44. Robbins, T.W. & Everitt, B.J. (1992) Functions of dopamine in the dorsal and ventral striatum. Semin. Neurosci., 4, 119-127.
45. Robbins, T.W. & Everitt, B.J. (2007) A role for mesencephalic dopamine in activation: commentary on Berridge (2006). Psychopharmacology, 191, 433-437.
46. Robbins, T.W., Cador, M., Taylor, J.R. & Everitt, B.J. (1989) Limbic-striatal interactions in reward-related processes. Neurosci. Biobehav. Rev., 13, 155-162.
47. Robinson, S., Sandstrom, S.M., Denenberg, V.H. & Palmiter, R.D. (2005) Distinguishing whether dopamine regulates liking, wanting, and/or learning about rewards. Behav. Neurosci., 119, 5-15.
48. Rutledge, R.B., Lazzaro, S.C., Lau, B., Myers, C.E., Gluck, M.A. & Glimcher, P.W. (2009) Dopaminergic drugs modulate learning rates and perseveration in Parkinson's patients in a dynamic foraging task. J. Neurosci., 29, 15104-15114.
49. Sadeh, T., Shohamy, D., Levy, D.R., Reggev, N. & Maril, A. (2011) Cooperation between the hippocampus and the striatum during episodic encoding. J. Cogn. Neurosci., 23, 1597-1608.
50. Salamone, J.D., Correa, M., Mingote, S.M. & Weber, S.M. (2005) Beyond the reward hypothesis: alternative functions of nucleus accumbens dopamine. Curr. Opin. Pharmacol., 5, 34-41.
51. Salamone, J.D., Correa, M., Farrar, A. & Mingote, S.M. (2007) Effort-related functions of nucleus accumbens dopamine and associated forebrain circuits. Psychopharmacology, 191, 461-482.
52. Salthouse, T.A. (1994) The nature of the influence of speed on adult age differences in cognition. Dev. Psychol., 30, 240-259.
53. Samejima, K., Ueda, Y., Doya, K. & Kimura, M. (2005) Representation of action-specific reward values in the striatum. Science, 310, 1337-1340.
54. Satoh, T., Nakai, S., Sato, T. & Kimura, M. (2003) Correlated coding of motivation and outcome of decision by dopamine neurons. J. Neurosci., 23, 9913-9923.
55. Schmand, B., Bakker, D., Saan, R. & Louman, J. (1991) [The Dutch Reading Test for Adults: a measure of premorbid intelligence level]. Tijdschr. Gerontol. Geriatr., 22, 15-19.
56. Schultz, W., Dayan, P. & Montague, P.R. (1997) A neural substrate of prediction and reward. Science, 275, 1593-1599.
57. Shohamy, D., Myers, C.E., Grossman, S., Sage, J., Gluck, M.A. & Poldrack, R.A. (2004) Cortico-striatal contributions to feedback-based learning: converging data from neuroimaging and neuropsychology. Brain, 127, 851-859.
58. Sutton, R.S. & Barto, A.G. (1998). Reinforcement Learning: An Introduction. MIT Press, Cambridge, MA.
59. Tsai, H.C., Zhang, F., Adamantidis, A., Stuber, G.D., Bonci, A., de Lecea, L. & Deisseroth, K. (2009) Phasic firing in dopaminergic neurons is sufficient for behavioral conditioning. Science, 324, 1080-1084.
60. Voon, V., Pessiglione, M., Brezing, C., Gallea, C., Fernandez, H., Dolan, R.J. & Hallett, M. (2010) Mechanisms underlying dopamine-mediated reward bias in compulsive behaviors. Neuron, 65, 135-142.
61. Waelti, P., Dickinson, A. & Schultz, W. (2001) Dopamine responses comply with basic assumptions of formal learning theory. Nature, 412, 43-48.
62. Wenzelburger, R., Zhang, B., Pohle, S., Klebe, S., Lorenz, D., Herzog, J., Wilms, H., Deuschl, G. & Krack, P. (2002) Force overflow and levodopa-induced dyskinesias in Parkinson's disease. Brain, 125, 871-879.
63. Wilkinson, L., Khan, Z. & Jahanshahi, M. (2009) The role of the basal ganglia and its cortical connections in sequence learning: evidence from implicit and explicit sequence learning in Parkinson's disease. Neuropsychologia, 47, 2564-2573.
64. Wise, R.A. (2004) Dopamine, learning and motivation. Nat. Rev. Neurosci., 5, 483-494.
65. Wyvell, C.L. & Berridge, K.C. (2000) Intra-accumbens amphetamine increases the conditioned incentive salience of sucrose reward: enhancement of reward "wanting" without enhanced "liking" or response reinforcement. J. Neurosci., 20, 8122-8130.