Impulse control disorders in Parkinson's disease are associated with dysfunction in stimulus valuation but not action valuation

Journal of Neuroscience

Volumn 34, Issue 23, 2014, Pages 7814-7824

  Piray, Payam ^a   Zeighami, Yashar ^b   Bahrami, Fariba ^c   Eissa, Abeer M ^d   Hewedi, Doaa H ^d   Moustafa, Ahmed A ^e,f  

^a RADBOUD UNIVERSITY   (Netherlands)

^b MCGILL UNIVERSITY   (Canada)

^c UNIVERSITY OF TEHRAN   (Iran)

^d AIN SHAMS UNIVERSITY   (Egypt)

^e UNIVERSITY OF WESTERN SYDNEY   (Australia)

^f WESTERN SYDNEY UNIVERSITY   (Australia)

Author keywords

Computational modeling; Dopamine; Impulse control disorders; Parkinson's disease; Prediction error; Reinforcement learning

Indexed keywords

DOPAMINE;

ALGORITHM; ARTICLE; BRAIN DEPTH STIMULATION; BRAIN FUNCTION; CLINICAL EVALUATION; CLINICAL TRIAL (TOPIC); COMPUTER MODEL; DISEASE ASSOCIATION; DISEASE SEVERITY; EXPERIMENTAL STUDY; HUMAN; IMPULSE CONTROL DISORDER; INFORMATION PROCESSING; LEARNING; METHODOLOGY; MINI MENTAL STATE EXAMINATION; NERVE STIMULATION; PARKINSON DISEASE; PRIORITY JOURNAL; REINFORCEMENT; SIGNAL TRANSDUCTION; STIMULUS RESPONSE; TASK PERFORMANCE; UNIFIED PARKINSON DISEASE RATING SCALE;

COMPUTATIONAL MODELING; DOPAMINE; IMPULSE CONTROL DISORDERS; PARKINSON'S DISEASE; PREDICTION ERROR; REINFORCEMENT LEARNING;

AGED; ANTIPSYCHOTIC AGENTS; COMPUTER SIMULATION; FEMALE; HUMANS; IMPULSE CONTROL DISORDERS; MALE; MIDDLE AGED; MODELS, PSYCHOLOGICAL; PARKINSON DISEASE; PROBABILITY LEARNING; PUNISHMENT; REINFORCEMENT (PSYCHOLOGY); REWARD; SEVERITY OF ILLNESS INDEX;

EID: 84901797896     PISSN: 02706474     EISSN: 15292401     Source Type: Journal    
DOI: 10.1523/JNEUROSCI.4063-13.2014     Document Type: Article

References (58)

1. Barto AG (1995) Adaptive critic and the basal ganglia. In: Models of information processing in the basal ganglia (Houk JC, Davis JL, Beiser DG, eds), pp 215-232. Cambridge: MIT.
2. Bayer HM, Glimcher PW (2005) Midbrain dopamine neurons encode a quantitative reward prediction error signal. Neuron 47:129-141. CrossRef Medline
3. Beck DC, Carlson GA, Russell AT, Brownfield FE (1987) Use of depression rating instruments in developmentally and educationally delayed adolescents. J Am Acad Child Adolesc Psychiatry 26:97-100. CrossRef Medline
4. Belin D, Mar AC, Dalley JW, Robbins TW, Everitt BJ (2008) High impulsivity predicts the switch to compulsive cocaine-taking. Science 320:1352-1355. CrossRef Medline
5. Belin D, Belin-Rauscent A, Murray JE, Everitt BJ (2013) Addiction: failure of control over maladaptive incentive habits. Curr Opin Neurobiol 23: 564-572. CrossRef Medline
6. Bódi N, Kéri S, Nagy H, Moustafa A, Myers CE, Daw N, Dibó G, Takáts A, Bereczki D, Gluck MA (2009) Reward-learning and the novelty-seeking personality: a between-and within-subjects study of the effects of dopamine agonists on young Parkinson's patients. Brain 132:2385-2395. CrossRef Medline
7. Camerer C, Ho T-H (1999) Experience-weighted attraction learning in games: a unifying approach. Econometrica 67:827-874. CrossRef
8. Cardinal RN, Parkinson JA, Hall J, Everitt BJ (2002) Emotion and motivation: the role of the amygdala, ventral striatum, and prefrontal cortex. Neurosci Biobehav Rev 26:321-352. CrossRef Medline
9. Cools R, Barker RA, Sahakian BJ, Robbins TW (2001) Enhanced or impaired cognitive function in Parkinson's disease as a function of dopaminergic medication and task demands. Cereb Cortex 11:1136-1143. CrossRef Medline
10. Cools R, Barker RA, Sahakian BJ, Robbins TW (2003) L-Dopa medication remediates cognitive inflexibility, but increases impulsivity in patients with Parkinson's disease. Neuropsychologia 41:1431-1441. CrossRef Medline
11. Cools R, Lewis SJ, Clark L, Barker RA, Robbins TW (2007) L-DOPA disrupts activity in the nucleus accumbens during reversal learning in Parkinson's disease. Neuropsychopharmacology 32:180-189. CrossRef Medline
12. Dagher A, Robbins TW (2009) Personality, addiction, dopamine: insights from Parkinson's disease. Neuron 61:502-510. CrossRef Medline
13. Dalley JW, Fryer TD, Brichard L, Robinson ES, Theobald DE, Lääne K, Peña Y, Murphy ER, Shah Y, Probst K, Abakumova I, Aigbirhio FI, Richards HK, Hong Y, Baron JC, Everitt BJ, Robbins TW (2007) Nucleus accumbens D2/3 receptors predict trait impulsivity and cocaine reinforcement. Science 315:1267-1270. CrossRef Medline
14. Daw ND (2011) Trial-by-trial data analysis using computational models. In: Decision making, affect, and learning: attention and performance XXIII (Delgado MR, Phelps EA, Robbins TW, eds), pp 3-38. New York: OUP.
15. Dayan P, Balleine BW (2002) Reward, motivation, and reinforcement learning. Neuron 36:285-298. CrossRef Medline
16. Dempster AP, Laird NM, Rubin DB (1977) Maximum likelihood from incomplete data via the EM algorithm. Journal of the Royal Statistical Society Series B (Methodological) 39:1-38.
17. Everitt BJ, Robbins TW (2005) Neural systems of reinforcement for drug addiction: from actions to habits to compulsion. Nat Neurosci 8:1481-1489. CrossRef Medline
18. Folstein MF, Folstein SE, McHugh PR (1975) "Mini-mental state. " A practical method for grading the cognitive state of patients for the clinician. J Psychiatr Res 12:189-198. CrossRef Medline
19. Frank MJ, Seeberger LC, O'reilly RC (2004) By carrot or by stick: cognitive reinforcement learning in parkinsonism. Science 306:1940-1943. CrossRef Medline
20. Frank MJ, Moustafa AA, Haughey HM, Curran T, Hutchison KE (2007) Genetic triple dissociation reveals multiple roles for dopamine in reinforcement learning. Proc Natl Acad Sci U S A 104:16311-16316. CrossRef Medline
21. Goto Y, Grace AA (2005) Dopaminergic modulation of limbic and cortical drive of nucleus accumbens in goal-directed behavior. Nat Neurosci 8:805-812. CrossRef Medline
22. Grace AA (1991) Phasic versus tonic dopamine release and the modulation of dopamine system responsivity: a hypothesis for the etiology of schizophrenia. Neuroscience 41:1-24. CrossRef Medline
23. Grondman I, Busoniu L, Lopes GAD, Babuska R (2012) A survey of actorcritic reinforcement learning: standard and natural policy gradients. IEEE Transactions on Systems, Man, and Cybernetics, Part C: Applications and Reviews 42:1291-1307.
24. Hart AS, Rutledge RB, Glimcher PW, Phillips PE (2014) Phasic dopamine release in the rat nucleus accumbens symmetrically encodes a reward prediction error term. J Neurosci 34:698-704. CrossRef Medline
25. Hopf FW, Cascini MG, Gordon AS, Diamond I, Bonci A (2003) Cooperative activation of dopamine D1 and D2 receptors increases spike firing of nucleus accumbens neurons via G-protein betagamma subunits. J Neurosci 23:5079-5087. Medline
26. Huys QJ, Cools R, Gölzer M, Friedel E, Heinz A, Dolan RJ, Dayan P (2011a) Disentangling the roles of approach, activation and valence in instrumental and Pavlovian responding. PLo S Comput Biol 7:e1002028. CrossRef Medline
27. Huys QJ, Moutoussis M, Williams J (2011b) Are computational models of any use to psychiatry? Neural Netw 24:544-551. CrossRef Medline
28. Huys QJ, Eshel N, O'Nions E, Sheridan L, Dayan P, Roiser JP (2012) Bonsai trees in your head: how the Pavlovian system sculpts goal-directed choices by pruning decision trees. PLo S Comput Biol 8:e1002410. CrossRef Medline
29. Ikemoto S, Glazier BS, Murphy JM, McBride WJ (1997) Role of dopamine D1 and D2 receptors in the nucleus accumbens in mediating reward. J Neurosci 17:8580-8587. Medline
30. Kass RE, Raftery AE (1995) Bayes factor. Journal of the American Statistical Association 90:773-795. CrossRef
31. Kish SJ, Shannak K, Hornykiewicz O (1988) Uneven pattern of dopamine loss in the striatum of patients with idiopathic Parkinson's disease: pathophysiologic and clinical implications. N Engl J Med 318:876-880. CrossRef Medline
32. Klein TA, Neumann J, Reuter M, Hennig J, von Cramon DY, Ullsperger M (2007) Genetically determined differences in learning from errors. Science 318:1642-1645. CrossRef Medline
33. Lang AE, Fahn S (1989) Assesment of Parkinson's disease. In: Quantification of neurologic deficit (Munsat TL, ed), pp 285-309. Boston: Butterworths.
34. Lau B, Glimcher PW (2005) Dynamic response-by-response models of matching behavior in rhesus monkeys. J Exp Anal Behav 84:555-579. CrossRef Medline
35. MacKay DJC (2003) Information theory, inference, and learning algorithms. Cambridge: Cambridge University.
36. Maia TV, Frank MJ (2011) From reinforcement learning models to psychiatric and neurological disorders. Nat Neurosci 14:154-162. CrossRef Medline
37. Montague PR, Dolan RJ, Friston KJ, Dayan P (2012) Computational psychiatry. Trends Cogn Sci 16:72-80. CrossRef Medline
38. Monterosso J, Piray P, Luo S (2012) Neuroeconomics and the study of addiction. Biol Psychiatry 72:107-112. CrossRef Medline
39. Moustafa AA, Cohen MX, Sherman SJ, Frank MJ (2008) A role for dopamine in temporal decision making and reward maximization in parkinsonism. J Neurosci 28:12294-12304. CrossRef Medline
40. Moustafa AA, Krishna R, Eissa AM, Hewedi DH (2013) Factors underlying probabilistic and deterministic stimulus-response learning performance in medicated and unmedicated patients with Parkinson's disease. Neuropsychology 27:498-510. CrossRef Medline
41. Nader MA, Daunais JB, Moore T, Nader SH, Moore RJ, Smith HR, Friedman DP, Porrino LJ (2002) Effects of cocaine self-administration on striatal dopamine systems in rhesus monkeys: initial and chronic exposure. Neuropsychopharmacology 27:35-46. CrossRef Medline
42. Niv Y, Edlund JA, Dayan P, O'Doherty JP (2012) Neural prediction errors reveal a risk-sensitive reinforcement-learning process in the human brain. J Neurosci 32:551-562. CrossRef Medline
43. O'Doherty J, Dayan P, Schultz J, Deichmann R, Friston K, Dolan RJ (2004) Dissociable roles of ventral and dorsal striatum in instrumental conditioning. Science 304:452-454. CrossRef Medline
44. O'Reilly RC, Frank MJ, Hazy TE, Watz B (2007) PVLV: the primary value and learned value Pavlovian learning algorithm. Behav Neurosci 121:31-49. CrossRef Medline
45. Packard MG, Knowlton BJ (2002) Learning and memory functions of the Basal Ganglia. Annu Rev Neurosci 25:563-593. CrossRef Medline
46. Pessiglione M, Seymour B, Flandin G, Dolan RJ, Frith CD (2006) Dopamine-dependent prediction errors underpin reward-seeking behaviour in humans. Nature 442:1042-1045. CrossRef Medline
47. Piray P, Keramati MM, Dezfouli A, Lucas C, Mokri A (2010) Individual differences in nucleus accumbens dopamine receptors predict development of addiction-like behavior: a computational approach. Neural Comput 22:2334-2368. CrossRef Medline
48. Porrino LJ, Daunais JB, Smith HR, Nader MA (2004) The expanding effects of cocaine: studies in a nonhuman primate model of cocaine selfadministration. Neurosci Biobehav Rev 27:813-820. CrossRef Medline
49. Redish AD, Jensen S, Johnson A. A unified framework for addiction: vulnerabilities in the decision process. Behav Brain Sci 31:415-437, 2008; discussion 437-487.
50. Rutledge RB, Lazzaro SC, Lau B, Myers CE, Gluck MA, Glimcher PW (2009) Dopaminergic drugs modulate learning rates and perseveration in Parkinson's patients in a dynamic foraging task. J Neurosci 29:15104-15114. CrossRef Medline
51. Schultz W, Dayan P, Montague PR (1997) A neural substrate of prediction and reward. Science 275:1593-1599. CrossRef Medline
52. Steeves TD, Miyasaki J, Zurowski M, Lang AE, Pellecchia G, Van Eimeren T, Rusjan P, Houle S, Strafella AP (2009) Increased striatal dopamine release in parkinsonian patients with pathological gambling: a [11C] raclopride PET study. Brain 132:1376-1385. CrossRef Medline
53. Swainson R, Rogers RD, Sahakian BJ, Summers BA, Polkey CE, Robbins TW (2000) Probabilistic learning and reversal deficits in patients with Parkinson's disease or frontal or temporal lobe lesions: possible adverse effects of dopaminergic medication. Neuropsychologia 38:596-612. CrossRef Medline
54. Uttl B (2002) North American Adult Reading Test: age norms, reliability, and validity. J Clin Exp Neuropsychol 24:1123-1137. CrossRef Medline
55. Voon V, Potenza MN, Thomsen T (2007) Medication-related impulse control and repetitive behaviors in Parkinson's disease. Curr Opin Neurol 20:484-492. CrossRef Medline
56. Voon V, Pessiglione M, Brezing C, Gallea C, Fernandez HH, Dolan RJ, Hallett M (2010) Mechanisms underlying dopamine-mediated reward bias in compulsive behaviors. Neuron 65:135-142. CrossRef Medline
57. Weintraub D, Siderowf AD, Potenza MN, Goveas J, Morales KH, Duda JE, Moberg PJ, Stern MB (2006) Dopamine Agonist Use is Associated with Impulse Control Disorders in Parkinson's Disease. Arch Neurol 63:969-973. CrossRef Medline
58. Wunderlich K, Smittenaar P, Dolan RJ (2012) Dopamine enhances modelbased over model-free choice behavior. Neuron 75:418-424. CrossRef Medline