Understanding addiction as a pathological state of multiple decision making processes: A neurocomputational perspective

Computational Neuroscience of Drug Addiction

Volumn , Issue , 2012, Pages 205-233

  Keramati, Mehdi ^a   Dezfouli, Amir ^b   Piray, Payam ^b  

^a ECOLE NORMALE SUPÉRIEURE   (France)

^b TEHRAN UNIVERSITY OF MEDICAL SCIENCES   (Iran)

Author keywords

[No Author keywords available]

Indexed keywords

EID: 84896942262     PISSN: None     EISSN: None     Source Type: Book    
DOI: 10.1007/978-1-4614-0751-5_8     Document Type: Chapter

References (107)

1. Abercrombie ED, Keefe KA, DiFrischia DS, Zigmond MJ (1989) Differential effect of stress on in vivo dopamine release in striatum, nucleus accumbens, and medial frontal cortex. J Neurochem 52:1655-1658
2. Ahmed SH (2004) Addiction as compulsive reward prediction. Science (New York) 306:1901-1902
3. Ahmed SH, Koob GF (2005) Transition to drug addiction: a negative reinforcement model based on an allostatic decrease in reward function. Psychopharmacology 180:473-490
4. Alexander GE, Crutcher MD (1990) Functional architecture of basal ganglia circuits: neural substrates of parallel processing. Trends Neurosci 13:266-271
5. Alexander GE, DeLong MR, Strick PL (1986) Parallel organization of functionally segregated circuits linking basal ganglia and cortex. Annu Rev Neurosci 9:357-381
6. Alexander GE, Crutcher MD, De Long MR (1990) Basal ganglia-thalamocortical circuits: parallel substrates for motor, oculomotor, "prefrontal" and "limbic" functions. Prog Brain Res 85:119-146
7. Almeida PP, Novaes MAFP, Bressan RA, de Lacerda ALT (2008) Executive functioning and cannabis use. Rev Bras Psiquiatr (São Paulo, 1999) 30:69-76
8. American Psychiatric Association (2000) Diagnostic and statistical manual of mental disorders: DSM-IV-TR, 4th edn, Washington, DC
9. Balleine BW, O'Doherty JP (2010) Human and rodent homologies in action control: corticostriatal determinants of goal-directed and habitual action. Neuropsychopharmacology 35:48-69
10. Belin D, Everitt BJ (2008) Cocaine seeking habits depend upon dopamine-dependent serial connectivity linking the ventral with the dorsal striatum. Neuron 57:432-441
11. Belin D, Mar AC, Dalley JW, Robbins TW, Everitt BJ (2008) High impulsivity predicts the switch to compulsive cocaine-taking. Science (New York) 320:1352-1355
12. Belin D, Jonkman S, Dickinson A, Robbins TW, Everitt BJ (2009) Parallel and interactive learning processes within the basal ganglia: relevance for the understanding of addiction. Behav Brain Res 199:89-102
13. Bickel WK, Madden GJ (1999) A comparison of measures of relative reinforcing efficacy and behavioral economics: cigarettes and money in smokers. Behav Pharmacol 10:627-637
14. Bickel WK, Marsch LA (2001) Toward a behavioral economic understanding of drug dependence: delay discounting processes. Addiction (Abingdon) 96:73-86
15. Bubar MJ, Cunningham KA (2008) Prospects for serotonin 5-HT2R pharmacotherapy in psychostimulant abuse. Prog Brain Res 172:319-346
16. Dalley JW, Fryer TD, Brichard L, Robinson ESJ, Theobald DEH et al (2007) Nucleus accumbens d2/3 receptors predict trait impulsivity and cocaine reinforcement. Science (New York) 315:1267-1270
17. Dalley JW, Everitt BJ, Robbins TW (2011) Impulsivity, compulsivity, and top-down cognitive control. Neuron 69(4):680-694
18. Daw ND, Niv Y, Dayan P (2005) Uncertainty-based competition between prefrontal and dorsolateral striatal systems for behavioral control. Nat Neurosci 8:1704-1711
19. Dayan P (2009) Dopamine, reinforcement learning, and addiction. Pharmacopsychiatry 42(1):S56-S65 Suppl
20. Dayan P, Balleine BW (2002) Reward, motivation, and reinforcement learning. Neuron 36:285-298
21. deWit H (2009) Impulsivity as a determinant and consequence of drug use: a review of underlying processes. Addict Biol 14:22-31
22. Deroche-Gamonet V, Belin D, Piazza PV (2004) Evidence for addiction-like behavior in the rat. Science (New York) 305:1014-1017
23. Dezfouli A, Piray P, Keramati MM, Ekhtiari H, Lucas C et al (2009) A neurocomputational model for cocaine addiction. Neural Comput 21:2869-2893
24. Dhonnchadha BAN, Cunningham KA (2008) Serotonergic mechanisms in addiction-related memories. Behav Brain Res 195:39-53
25. Di Chiara G, Imperato A (1988) Drugs abused by humans preferentially increase synaptic dopamine concentrations in the mesolimbic system of freely moving rats. Proc Natl Acad Sci USA 85:5274-5278
26. Dias-Ferreira E, Sousa JC, Melo I, Morgado P, Mesquita AR et al (2009) Chronic stress causes frontostriatal reorganization and affects decision-making. Science (New York) 325:621-625
27. Dickinson A, Balleine BW (2002) The role of learning in motivation. In: Gallistel CR (ed) Steven's handbook of experimental psychology: learning, motivation, and emotion, 3rd edn, vol 3. Wiley, New York, pp 497-533
28. Estes WK (1948) Discriminative conditioning; effects of a pavlovian conditioned stimulus upon a subsequently established operant response. J Exp Psychol 38:173-177
29. Everitt BJ, Robbins TW (2005) Neural systems of reinforcement for drug addiction: from actions to habits to compulsion. Nat Neurosci 8:1481-1489
30. Everitt BJ, Belin D, Economidou D, Pelloux Y, Dalley JW et al (2008) Neural mechanisms underlying the vulnerability to develop compulsive drug-seeking habits and addiction. Philos Trans R Soc Lond B, Biol Sci 363:3125-3135
31. Ford CP, Mark GP, Williams JT (2006) Properties and opioid inhibition of mesolimbic dopamine neurons vary according to target location. J Neurosci 26:2788-2797
32. Frank MJ, Moustafa AA, Haughey HM, Curran T, Hutchison KE (2007) Genetic triple dissociation reveals multiple roles for dopamine in reinforcement learning. Proc Natl Acad Sci USA 104:16311-16316
33. Garavan H, Pankiewicz J, Bloom A, Cho JK, Sperry L et al (2000) Cue-induced cocaine craving: neuroanatomical specificity for drug users and drug stimuli. Am J Psychiatry 157:1789-1798
34. George O, Mandyam CD, Wee S, Koob GF (2008) Extended access to cocaine self-administration produces long-lasting prefrontal cortex-dependent working memory impairments. Neuropsychopharmacology 33:2474-2482
35. Goldstein RZ, Volkow ND (2002) Drug addiction and its underlying neurobiological basis: neuroimaging evidence for the involvement of the frontal cortex. Am J Psychiatry 159:1642-1652
36. Goldstein RZ, Alia-Klein N, Tomasi D, Zhang L, Cottone LA et al (2007) Is decreased prefrontal cortical sensitivity to monetary reward associated with impaired motivation and self-control in cocaine addiction? Am J Psychiatry 164:43-51
37. Haber SN (2003) The primate basal ganglia: parallel and integrative networks. J Chem Neuroanat 26:317-330
38. Haber SN, Fudge JL, McFarland NR (2000) Striatonigrostriatal pathways in primates form an ascending spiral from the shell to the dorsolateral striatum. J Neurosci 20:2369-2382
39. Holland PC (2004) Relations between pavlovian-instrumental transfer and reinforcer devaluation. J Exp Psychol, Anim Behav Processes 30:104-117
40. Hopf FW, Cascini MG, Gordon AS, Diamond I, Bonci A (2003) Cooperative activation of dopamine d1 and d2 receptors increases spike firing of nucleus accumbens neurons via g-protein betagamma subunits. J Neurosci 23:5079-5087
41. Ikemoto S (2007) Dopamine reward circuitry: two projection systems from the ventral midbrain to the nucleus accumbens-olfactory tubercle complex. Brains Res Rev 56:27-78
42. Ikemoto S, Glazier BS, Murphy JM, McBrideWJ (1997) Role of dopamine d1 and d2 receptors in the nucleus accumbens in mediating reward. J Neurosci 17:8580-8587
43. Ito M, Doya K (2009) Validation of decision-making models and analysis of decision variables in the rat basal ganglia. J Neurosci 29:9861-9874
44. Ito R, Dalley JW, Robbins TW, Everitt BJ (2002) Dopamine release in the dorsal striatum during cocaine-seeking behavior under the control of a drug-associated cue. J Neurosci 22:6247-6253
45. Jacobs EA, Bickel WK (1999) Modeling drug consumption in the clinic using simulation procedures: demand for heroin and cigarettes in opioid-dependent outpatients. Exp Clin Psychopharmacol 7:412-426
46. Joel D, Niv Y, Ruppin E (2002) Actor-critic models of the basal ganglia: new anatomical and computational perspectives. Neural Netw 15:535-547
47. Johnson PM, Kenny PJ (2010) Dopamine d2 receptors in addiction-like reward dysfunction and compulsive eating in obese rats. Nat Neurosci 13:635-641
48. Kalivas PW (2009) The glutamate homeostasis hypothesis of addiction. Nat Rev, Neurosci 10:561-572
49. Kalivas PW, McFarland K (2003) Brain circuitry and the reinstatement of cocaine-seeking behavior. Psychopharmacology 168:44-56
50. Kamin L (1969) Predictability, surprise, attention, and conditioning. In: Campbell BA, Church RM (eds) Punishment and aversive behavior. Appleton-Century-Crofts, New York, pp 279-296
51. Keramati M, Dezfouli A, Piray P (2011) Speed-accuracy trade-off between the habitual and the goal-directed processes. PLoS Comput Biol 7(5):1-25
52. Koob GF, Le Moal M (2005a) Plasticity of reward neurocircuitry and the 'dark side' of drug addiction. Nat Neurosci 8:1442-1444
53. Koob GF, Le Moal M (2005b) Neurobiology of addiction. Academic Press, San Diego
54. Logue A, Tobin H, Chelonis J, Wang R, Geary N et al (1992) Cocaine decreases self-control in rats: a preliminary report. Psychopharmacology 109:245-247
55. Lovibond PF (1983) Facilitation of instrumental behavior by a pavlovian appetitive conditioned stimulus. J Exp Psychol, Anim Behav Processes 9:225-247
56. Mackintosh NJ (1974) The psychology of animal learning. Academic Press, London
57. Mahadevan S (1996) Average reward reinforcement learning: foundations, algorithms, and empirical results. Mach Learn 22:159-195
58. Mantz J, Thierry AM, Glowinski J (1989) Effect of noxious tail pinch on the discharge rate of mesocortical and mesolimbic dopamine neurons: selective activation of the mesocortical system. Brain Res 476:377-381
59. Marks KR, Kearns DN, Christensen CJ, Silberberg A, Weiss SJ (2010) Learning that a cocaine reward is smaller than expected: a test of redish's computational model of addiction. Behav Brain Res 212:204-207
60. Martinez D, Slifstein M, Narendran R, Foltin RW, Broft A et al (2009) Dopamine d1 receptors in cocaine dependence measured with PET and the choice to self-administer cocaine. Neuropsychopharmacology 34:1774-1782
61. Montgomery AJ, Mehta MA, Grasby PM (2006) Is psychological stress in man associated with increased striatal dopamine levels?: a [11C]raclopride PET study. Synapse (New York) 60:124-131
62. Morris G, Nevet A, Arkadir D, Vaadia E, Bergman H (2006) Midbrain dopamine neurons encode decisions for future action. Nat Neurosci 9:1057-1063
63. Nader MA, Daunais JB, Moore T, Nader SH, Moore RJ et al (2002) Effects of cocaine selfadministration on striatal dopamine systems in rhesus monkeys: initial and chronic exposure. Neuropsychopharmacology 27:35-46
64. Nader MA, Czoty PW, Gould RW, Riddick NV (2008) Positron emission tomography imaging studies of dopamine receptors in primate models of addiction. Philos Trans R Soc Lond B, Biol Sci 363:3223-3232
65. Nicola SM (2007) The nucleus accumbens as part of a basal ganglia action selection circuit. Psychopharmacology 191:521-550
66. Niv Y (2007) The effects of motivation on habitual instrumental behavior. PhD thesis, The Hebrew University of Jerusalem, Interdisciplinary Center for Neural Computation
67. Niv Y, Daw ND, Joel D, Dayan P (2007) Tonic dopamine: opportunity costs and the control of response vigor. Psychopharmacology 191:507-520
68. O'Brien CP, Ehrman R, Ternes J (1986) Classical conditioning in human opioid dependence. In: Goldberg SR, Stolerman IP (eds) Behavioral analysis of drug dependence, 1st edn. Academic Press, London, pp 329-356
69. O'Doherty J, Dayan P, Schultz J, Deichmann R, Friston K et al (2004) Dissociable roles of ventral and dorsal striatum in instrumental conditioning. Science (New York) 304:452-454
70. Paine TA, Dringenberg HC, Olmstead MC (2003) Effects of chronic cocaine on impulsivity: relation to cortical serotonin mechanisms. Behav Brain Res 147:135-147
71. Panlilio LV, Thorndike EB, Schindler CW (2007) Blocking of conditioning to a cocaine-paired stimulus: testing the hypothesis that cocaine perpetually produces a signal of larger-thanexpected reward. Pharmacol Biochem Behav 86:774-777
72. Pelloux Y, Everitt BJ, Dickinson A (2007) Compulsive drug seeking by rats under punishment: effects of drug taking history. Psychopharmacology 194:127-137
73. Petry NM, Bickel WK (1998) Polydrug abuse in heroin addicts: a behavioral economic analysis. Addiction (Abingdon) 93:321-335
74. Pierce RC, Vanderschuren LJMJ (2010) Kicking the habit: the neural basis of ingrained behaviors in cocaine addiction. Neurosci Biobehav Rev 35:212-219
75. Piray P, Keramati MM, Dezfouli A, Lucas C, Mokri A (2010) Individual differences in nucleus accumbens dopamine receptors predict development of addiction-like behavior: a computational approach. Neural Comput 22:2334-2368
76. Porrino LJ, Daunais JB, Smith HR, Nader MA (2004a) The expanding effects of cocaine: studies in a nonhuman primate model of cocaine self-administration. Neurosci Biobehav Rev 27:813-820
77. Porrino LJ, Lyons D, Smith HR, Daunais JB, Nader MA (2004b) Cocaine self-administration produces a progressive involvement of limbic, association, and sensorimotor striatal domains. J Neurosci 24:3554-3562
78. Potenza MN (2008) The neurobiology of pathological gambling and drug addiction: an overview and new findings. Philos Trans R Soc Lond B, Biol Sci 363:3181-3189
79. Rangel A, Camerer C, Montague PR (2008) A framework for studying the neurobiology of valuebased decision making. Nat Rev, Neurosci 9:545-556
80. Redish AD (2004) Addiction as a computational process gone awry. Science (New York) 306:1944-1947
81. Redish AD, Jensen S, Johnson A, Kurth-Nelson Z (2007) Reconciling reinforcement learning models with behavioral extinction and renewal: implications for addiction, relapse, and problem gambling. Psychol Rev 114:784-805
82. Redish AD, Jensen S, Johnson A (2008) A unified framework for addiction: vulnerabilities in the decision process. Behav Brain Sci 31:415-437; discussion 437-487
83. Reynolds B (2006) A review of delay-discounting research with humans: relations to drug use and gambling. Behav Pharmacol 17:651-667
84. Robinson ESJ, Eagle DM, Economidou D, Theobald DEH, Mar AC et al (2009) Behavioural characterisation of high impulsivity on the 5-choice serial reaction time task: specific deficits in 'waiting' versus 'stopping'. Behav Brain Res 196:310-316
85. Roesch MR, Calu DJ, Schoenbaum G (2007) Dopamine neurons encode the better option in rats deciding between differently delayed or sized rewards. Nat Neurosci 10:1615-1624
86. Roesch MR, Singh T, Brown PL, Mullins SE, Schoenbaum G (2009) Ventral striatal neurons encode the value of the chosen action in rats deciding between differently delayed or sized rewards. J Neurosci 29:13365-13376
87. Rogers RD, Robbins TW (2001) Investigating the neurocognitive deficits associated with chronic drug misuse. Curr Opin Neurobiol 11:250-257
88. Root DH, Fabbricatore AT, Barker DJ, Ma S, Pawlak AP et al (2009) Evidence for habitual and goal-directed behavior following devaluation of cocaine: a multifaceted interpretation of relapse. PLoS ONE 4:e7170
89. Rutledge RB, Lazzaro SC, Lau B, Myers CE, GluckMAet al (2009) Dopaminergic drugs modulate learning rates and perseveration in parkinson's patients in a dynamic foraging task. J Neurosci 29:15104-15114
90. Schultz W, Dayan P, Montague PR (1997) A neural substrate of prediction and reward. Science (New York) 275:1593-1599
91. Shaham Y, Shalev U, Lu L, Wit HD, Stewart J (2003) The reinstatement model of drug relapse: history, methodology and major findings. Psychopharmacology 168:3-20
92. Simon NW, Mendez IA, Setlow B (2007) Cocaine exposure causes long-term increases in impulsive choice. Behav Neurosci 121:543-549
93. Smith AJ, Li M, Becker S, Kapur S (2006) Linking animal models of psychosis to computational models of dopamine function. Neuropsychopharmacology 32:54-66
94. Steeves TDL, Miyasaki J, Zurowski M, Lang AE, Pellecchia G et al (2009) Increased striatal dopamine release in parkinsonian patients with pathological gambling: a [11C] raclopride PET study. Brain 132:1376-1385
95. Stewart J (2008) Psychological and neural mechanisms of relapse. Philos Trans R Soc Lond B, Biol Sci 363:3147-3158
96. Stuber GD, Wightman RM, Carelli RM (2005) Extinction of cocaine self-administration reveals functionally and temporally distinct dopaminergic signals in the nucleus accumbens. Neuron 46:661-669
97. Sutton RS, Barto AG (1998) Reinforcement learning: an introduction. MIT Press, Cambridge
98. Thierry AM, Tassin JP, Blanc G, Glowinski J (1976) Selective activation of mesocortical DA system by stress. Nature 263:242-244
99. Vanderschuren LJMJ, Everitt BJ (2004) Drug seeking becomes compulsive after prolonged cocaine self-administration. Science (New York) 305:1017-1019
100. Volkow ND, Fowler JS, Wang G (2004a) The addicted human brain viewed in the light of imaging studies: brain circuits and treatment strategies. Neuropharmacology 47(1):3-13 Suppl
101. Volkow ND, Fowler JS, Wang G, Swanson JM (2004b) Dopamine in drug abuse and addiction: results from imaging studies and treatment implications. Mol Psychiatry 9:557-569
102. Volkow ND, Wang G, Fowler JS, Telang F (2008) Overlapping neuronal circuits in addiction and obesity: evidence of systems pathology. Philos Trans R Soc Lond B, Biol Sci 363:3191-3200
103. Yin HH, Knowlton BJ (2006) The role of the basal ganglia in habit formation. Nat Rev, Neurosci 7:464-476
104. Yin HH, Knowlton BJ, Balleine BW (2004) Lesions of dorsolateral striatum preserve outcome expectancy but disrupt habit formation in instrumental learning. Eur J Neurosci 19:181-189
105. Yin HH, Ostlund SB, Knowlton BJ, Balleine BW (2005) The role of the dorsomedial striatum in instrumental conditioning. Eur J Neurosci 22:513-523
106. Yin HH, Ostlund SB, Balleine BW (2008) Reward-guided learning beyond dopamine in the nucleus accumbens: the integrative functions of cortico-basal ganglia networks. Eur J Neurosci 28:1437-1448
107. Zapata A, Minney VL, Shippenberg TS (2010) Shift from Goal-Directed to habitual cocaine seeking after prolonged experience in rats. J Neurosci 30:15457-15463