A Neurocomputational Model for Cocaine Addiction

Neural Computation

Volumn 21, Issue 10, 2009, Pages 2869-2893

  Dezfouli, Amir ^a   Piray, Payam ^a   Keramati, Mohammad Mahdi ^b   Ekhtiari, Hamed ^c   Lucas, Caro ^a   Mokri, Azarakhsh ^d  

^a UNIVERSITY OF TEHRAN   (Iran)

^b SHARIF UNIVERSITY OF TECHNOLOGY   (Iran)

^c TEHRAN UNIVERSITY OF MEDICAL SCIENCES   (Iran)

^d TEHRAN UNIVERSITY OF MEDICAL SCIENCES   (Iran)

Author keywords

[No Author keywords available]

Indexed keywords

AVERAGE REWARD; COCAINE ADDICTION; COMPUTATIONAL MODELLING; DOPAMINE; DRUG ADDICTION; NEUROCOMPUTATIONAL MODELS; REWARD SYSTEMS; REWARD THRESHOLDS; SYSTEM SENSITIVITY; TEMPORAL DIFFERENCE REINFORCEMENT LEARNING;

COCAINE; DOPAMINE; DOPAMINE UPTAKE INHIBITOR;

ALGORITHM; ANIMAL; ARTICLE; BRAIN; BRAIN LEVEL; CHEMICALLY INDUCED DISORDER; COCAINE DEPENDENCE; COMPUTER SIMULATION; DECISION MAKING; DISEASE MODEL; DRUG EFFECT; HUMAN; IMPULSIVENESS; LEARNING; METABOLISM; PATHOPHYSIOLOGY; PHYSIOLOGY; REINFORCEMENT; REWARD;

ALGORITHMS; ANIMALS; BRAIN; BRAIN CHEMISTRY; COCAINE; COCAINE-RELATED DISORDERS; COMPUTER SIMULATION; DECISION MAKING; DISEASE MODELS, ANIMAL; DOPAMINE; DOPAMINE UPTAKE INHIBITORS; HUMANS; IMPULSIVE BEHAVIOR; LEARNING; REINFORCEMENT (PSYCHOLOGY); REWARD;

EID: 70449730463     PISSN: 08997667     EISSN: 1530888X     Source Type: Journal    
DOI: 10.1162/neco.2009.10-08-882     Document Type: Article

References (57)

1. Ahmed, S. H. (2004). Neuroscience: Addiction as compulsive reward prediction. Science, 306(5703), 19011902.
2. Ahmed, S. H., & Koob, G. F. (1998). Transition from moderate to excessive drug intake: Change in hedonic set point. Science, 282(5387), 298–300.
3. Ahmed, S. H., & Koob, G. F. (1999). Long-lasting increase in the set point for cocaine self-administration after escalation in rats. Psychopharmacology, 146(3), 303–312.
4. Ahmed, S. H., & Koob, G. F. (2005). Transition to drug addiction: A negative reinforcement model based on an allostatic decrease in reward function. Psychopharmacology, 180, 473–490.
5. American Psychiatric Association. (2000). Diagnostic and statistical manual of mental disorders (4th ed., text rev.). Washington, DC: Author.
6. Aragona, B. J., Cleaveland, N. A., Stuber, G. D., Day, J. J., Carelli, R. M., & Wightman, R. M. (2008). Preferential enhancement of dopamine transmission within the nucleus accumbens shell by cocaine is attributable to a direct increase in phasic dopamine release events. Journal of Neuroscience, 28(35), 8821–8831.
7. Chen, R., Tilley, M. R., Wei, H., Zhou, F., Zhou, F., Ching, S., et al. (2006). Abolished cocaine reward in mice with a cocaine-insensitive dopamine transporter. Proceedings of the National Academy of Sciences, 103(24), 9333–9338.
8. Daruna, J. H., & Barnes, P. A. (1993). A neurodevelopmental view of impulsivity. In W. G. McCown, J. L. Johnson, & M. B. Shure (Eds.), The impulsive client: Theory, research, and treatment (p. 23). Washington, DC: American Psychological Association.
9. Das, T. K., Gosavi, A., Mahadevan, S., & Marchalleck, N. (1999). Solving semi-Markov decision problems using average reward reinforcement learning. Management Science, 45, 560–574.
10. Daw, N. D. (2003). Reinforcement learning models of the dopamine system and their behavioral implications. Unpublished doctoral dissertation, Carnegie Mellon University.
11. Daw, N. D., & Doya, K. (2006). The computational neurobiology of learning and reward. Current Opinion in Neurobiology, 16(2), 199–204.
12. Daw, N. D., Kakade, S., & Dayan, P. (2002). Opponent interactions between serotonin and dopamine. Neural Networks, 15(4–6), 603–616.
13. Daw, N. D., Niv, Y., & Dayan, P. (2005). Uncertainty-based competition between prefrontal and dorsolateral striatal systems for behavioral control. Nature Neuroscience, 8(12), 1704–1711.
14. Daw, N. D., O’Doherty, J. P., Dayan, P., Seymour, B., & Dolan, R. J. (2006). Cortical substrates for exploratory decisions in humans. Nature, 441, 876–879.
15. Daw, N. D., & Touretzky, D. S. (2002). Long-term reward prediction in TD models of the dopamine system. Neural Computation, 14(11), 2567–2583.
16. Deroche-Gamonet, V., Belin, D., & Piazza, P. V. (2004). Evidence for addiction-like behavior in the rat. Science, 305, 1014–1017.
17. Doya, K. (1999). What are the computations of the cerebellum, the basal ganglia and the cerebral cortex? Neural Networks, 12, 961–974.
18. Everitt, B. J., & Robbins, T. W. (2005). Neural systems of reinforcement for drug addiction: From actions to habits to compulsion. Nature Neuroscience, 8(11), 1481– 1489.
19. Garavan, H., Pankiewicz, J., Bloom, A., Cho, J. K., Sperry, L., Ross, T. J., et al. (2000). Cue-induced cocaine craving: Neuroanatomical specificity for drug users and drug stimuli. American Journal of Psychiatry, 157(11), 1789–1798.
20. Goldstein, R. Z., Alia-Klein, N., Tomasi, D., Zhang, L., Cottone, L. A., Maloney, T., et al. (2007). Is decreased prefrontal cortical sensitivity to monetary reward associated with impaired motivation and self-control in cocaine addiction? American Journal of Psychiatry, 164(1), 43–51.
21. Grace, A. A. (1995). The tonic/phasic model of dopamine system regulation: Its relevance for understanding how stimulant abuse can alter basal ganglia function. Drug and Alcohol Dependence, 37(2), 111–129.
22. Grace, A. A. (2000). The tonic/phasic model of dopamine system regulation and its implications for understanding alcohol and psychostimulant craving. Addiction, 95 (Suppl. 2), S119–128.
23. Gutkin, B. S., Dehaene, S., & Changeux, J. (2006). A neurocomputational hypothesis for nicotine addiction. Proceedings of the National Academy of Sciences of the United States of America, 103(4), 1106–1111.
24. Kalivas, P. W., & O’Brien, C. (2007). Drug addiction as a pathology of staged neuroplasticity. Neuropsychopharmacology, 33(1), 166–180.
25. Kalivas, P. W., & Volkow, N. D. (2005). The neural basis of addiction: A pathology of motivation and choice. American Journal of Psychiatry, 162(8), 1403–1413.
26. Kamin, L. (1969). Predictability, surprise, attention, and conditioning. In B. A. Campbell & R. M. Church (Eds.), Punishment and aversive behavior (pp. 279–296). New York: Appleton-Century-Crofts.
27. Kauer, J. A., & Malenka, R. C. (2007). Synaptic plasticity and addiction. Nature Reviews Neuroscience, 8(11), 844–858.
28. Koob, G. F., & Moal, M. L. (2005). Plasticity of reward neurocircuitry and the “dark side” of drug addiction. Nature Neuroscience, 8(11), 1442–1444.
29. Logue, A., Tobin, H., Chelonis, J., Wang, R., Geary, N., & Schachter, S. (1992). Cocaine decreases self-control in rats: A preliminary report. Psychopharmacology, 109(1), 245–247.
30. Mahadevan, S. (1996). Average reward reinforcement learning: Foundations, algorithms, and empirical results. Machine Learning, 22(1), 159–195.
31. Mantsch, J. R., Ho, A., Schlussman, S. D., & Kreek, M. J. (2001). Predictable individual differences in the initiation of cocaine self-administration by rats under extendedaccess conditions are dose-dependent. Psychopharmacology, 157(1), 31–39.
32. Mateo, Y., Lack, C. M., Morgan, D., Roberts, D. C. S., & Jones, S. R. (2005). Reduced dopamine terminal function and insensitivity to cocaine following cocaine binge self-administration and deprivation. Neuropsychopharmacology, 30(8), 1455–1463.
33. Melo, F. S., & Ribeiro, M. I. (2007). Q-learning with linear function approximation. Proceedings of the 20th Annual Conference on Learning Theory (pp. 308–322). Berlin: Springer.
34. Montague, P. R., Dayan, P., & Sejnowski, T. J. (1996). A framework for mesencephalic dopamine systems based on predictive Hebbian learning. Journal of Neuroscience, 16, 1936–1947.
35. Morris, G., Nevet, A., Arkadir, D., Vaadia, E., & Bergman, H. (2006). Midbrain dopamine neurons encode decisions for future action. Nature Neuroscience, 9(8), 1057–1063.
36. Nader,M.A.,Morgan,D.,Gage,H.D.,Nader,S.H.,Calhoun,T.L.,Buchheimer, N., et al. (2006). PET imaging of dopamine D2 receptors during chronic cocaine self-administration in monkeys. Nature Neuroscience, 9(8), 1050–1056.
37. Niv, Y., Daw, N. D., Joel, D., & Dayan, P. (2007). Tonic dopamine: Opportunity costs and the control of response vigor. Psychopharmacology, 191, 507–520.
38. Norman, A. B., & Tsibulsky, V. L. (2006). The compulsion zone: A pharmacological theory of acquired cocaine self-administration. Brain Research, 1116(1), 143–152.
39. Paine, T. A., Dringenberg, H. C., & Olmstead, M. C. (2003). Effects of chronic cocaine on impulsivity: Relation to cortical serotonin mechanisms. Behavioural Brain Research, 147(1–2), 135–147.
40. Panlilio, L. V., Thorndike, E. B., & Schindler, C. W. (2007). Blocking of conditioning to a cocaine-paired stimulus: Testing the hypothesis that cocaine perpetually produces a signal of larger-than-expected reward. Pharmacology, Biochemistry, and Behavior, 86(4), 774–777.
41. Paterson, N. E., & Markou, A. (2003). Increased motivation for self-administered cocaine after escalated cocaine intake. Neuroreport, 14(17), 2229–2232.
42. Pelloux, Y., Everitt, B. J., & Dickinson, A. (2007). Compulsive drug seeking by rats under punishment: Effects of drug taking history. Psychopharmacology, 194, 127– 137.
43. Rangel, A., Camerer, C., & Montague, P. R. (2008). A framework for studying the neurobiology of value-based decision making. Nature Reviews. Neuroscience, 9(7), 545–556.
44. Redish, A. D. (2004). Addiction as a computational process gone awry. Science, 306, 1944–1947.
45. Redish, A. D., Jensen, S., & Johnson, A. (2008). A unified framework for addiction: Vulnerabilities in the decision process. Behavioral and Brain Sciences, 31(4), 415– 487.
46. Redish, A. D., Jensen, S., Johnson, A., & Kurth-Nelson, Z. (2007). Reconciling reinforcement learning models with behavioral extinction and renewal: Implications for addiction, relapse, and problem gambling. Psychological Review, 114(3), 784– 805.
47. Roesch, M. R., Calu, D. J., & Schoenbaum, G. (2007). Dopamine neurons encode the better option in rats deciding between differently delayed or sized rewards. Nature Neuroscience, 10(12), 1615–1624.
48. Schultz, W., Dayan, P., & Montague, P. R. (1997). A neural substrate of prediction and reward. Science, 275, 1593–1599.
49. Simon, N. W., Mendez, I. A., & Setlow, B. (2007). Cocaine exposure causes long-term increases in impulsive choice. Behavioral Neuroscience, 121(3), 543–549.
50. Stuber, G. D., Wightman, R. M., & Carelli, R. M. (2005). Extinction of cocaine selfadministration reveals functionally and temporally distinct dopaminergic signals in the nucleus accumbens. Neuron, 46(4), 661–669.
51. Sutton, R. S., & Barto, A. G. (1998). Reinforcement learning: An introduction.Cambridge, MA: MIT Press.
52. Tilley, M. R., O’Neill, B., Han, D. D., & Gu, H. H. (2009). Cocaine does not produce reward in absence of dopamine transporter inhibition. Neuroreport, 20(1), 9–12.
53. Vanderschuren, L. J. M. J., & Everitt, B. J. (2004). Drug seeking becomes compulsive after prolonged cocaine self-administration. Science, 305, 1017–1019.
54. Volkow, N. D., Fowler, J. S., Wang, G., & Swanson, J. M. (2004). Dopamine in drug abuse and addiction: Results from imaging studies and treatment implications. Molecular Psychiatry, 9(6), 557–569.
55. Volkow, N. D., Fowler, J. S., Wang, G., Swanson, J. M., & Telang, F. (2007). Dopamine in drug abuse and addiction: Results of imaging studies and treatment implications. Archives of Neurology, 64(11), 1575–1579.
56. Volkow, N. D., Wang, G. J., Fowler, J. S., Logan, J., Gatley, S. J., Hitzemann, R., et al. (1997). Decreased striatal dopaminergic responsiveness in detoxified cocainedependent subjects. Nature, 386(6627), 830–833.
57. Watkins, C. (1989). Learning from Delayed Rewards. Unpublished doctoral dissertation, King’s College, Cambridge, U.K.