Dopamine modulates novelty seeking behavior during decision making

Behavioral Neuroscience

Volumn 128, Issue 5, 2014, Pages 556-566

  Costa, Vincent D ^a   Tran, Valery L ^a   Turchi, Janita ^a   Averbeck, Bruno B ^a  

^a NATIONAL INSTITUTE OF MENTAL HEALTH   (United States)

Author keywords

Curiosity; Dopamine; Exploitation; Exploration; Foraging; Impulsivity; Novelty seeking; Reinforcement learning; Reuptake; Uncertainty

Indexed keywords

DOPAMINE; DOPAMINE TRANSPORTER; DOPAMINE UPTAKE INHIBITOR; PIPERAZINE DERIVATIVE; VANOXERINE;

ANIMAL BEHAVIOR; ANIMAL EXPERIMENT; ANIMAL TISSUE; ARTICLE; BEHAVIOR; BEHAVIOR CHANGE; CONTROLLED STUDY; DATA ANALYSIS; DECISION MAKING; HORMONE ACTION; HORMONE DETERMINATION; MALE; MENTAL TASK; NONHUMAN; NOVELTY SEEKING BEHAVIOR; PREDICTION; REWARD; RHESUS MONKEY; STIMULUS RESPONSE; TASK PERFORMANCE; ANIMAL; ANTAGONISTS AND INHIBITORS; DRUG EFFECTS; EXPLORATORY BEHAVIOR; METABOLISM; NEUROPSYCHOLOGICAL TEST; PHYSIOLOGY; PROBABILITY; PSYCHOLOGICAL MODEL; REINFORCEMENT;

ANIMALS; CHOICE BEHAVIOR; DOPAMINE; DOPAMINE PLASMA MEMBRANE TRANSPORT PROTEINS; DOPAMINE UPTAKE INHIBITORS; EXPLORATORY BEHAVIOR; MACACA MULATTA; MALE; MODELS, PSYCHOLOGICAL; NEUROPSYCHOLOGICAL TESTS; PIPERAZINES; PROBABILITY; REINFORCEMENT (PSYCHOLOGY);

EID: 84925742234     PISSN: 07357044     EISSN: 19390084     Source Type: Journal    
DOI: 10.1037/a0037128     Document Type: Article

References (53)

1. Asaad, W. F., & Eskandar, E. N. (2008). A flexible software tool for temporally-precise behavioral control in Matlab. Journal of Neuroscience Methods, 174, 245-258. doi:10.1016/j.jneumeth.2008.07.014
2. Averbeck, B. B., Djamshidian, A., O'Sullivan, S. S., Housden, C. R., Roiser, J. P., & Lees, A. J. (2013). Uncertainty about mapping future actions into rewards may underlie performance on multiple measures of impulsivity in behavioral addiction: Evidence from Parkinson's disease. Behavioral Neuroscience, 127, 245-255. doi:10.1037/a0032079
3. Beeler, J. A., Daw, N., Frazier, C. R. M., & Zhuang, X. X. (2010). Tonic dopamine modulates exploitation of reward learning. Frontiers in Behavioral Neuroscience, 4.
4. Bromberg-Martin, E. S., Matsumoto, M., & Hikosaka, O. (2010). Dopamine in motivational control: Rewarding, aversive, and alerting. Neuron, 68, 815-834. doi:10.1016/j.neuron.2010.11.022
5. Bunzeck, N., Dayan, P., Dolan, R. J., & Duzel, E. (2010). A common mechanism for adaptive scaling of reward and novelty. Human Brain Mapping, 31, 1380-1394. doi:10.1002/hbm.20939
6. Bunzeck, N., & Duzel, E. (2006). Absolute coding of stimulus novelty in the human substantia nigra/VTA. Neuron, 51, 369-379. doi:10.1016/j .neuron.2006.06.021
7. Chang, L., Alicata, D., Ernst, T., & Volkow, N. (2007). Structural and metabolic brain changes in the striatum associated with methamphetamine abuse. Addiction, 102 Suppl, 1, 16-32. doi:10.1111/j.1360-0443 .2006.01782.x
8. Cilia, R., Ko, J. H., Cho, S. S., van Eimeren, T., Marotta, G., Pellecchia, G., . . . Strafella, A. P. (2010). Reduced dopamine transporter density in the ventral striatum of patients with Parkinson's disease and pathological gambling. Neurobiology of Disease, 39, 98-104. doi:10.1016/j.nbd .2010.03.013
9. Cohen, J. D., McClure, S. M., & Yu, A. J. (2007). Should I stay or should I go? How the human brain manages the trade-off between exploitation and exploration. Philosophical Transactions of the Royal Society of London. Series B, Biological Sciences, 362, 933-942. doi:10.1098/rstb .2007.2098
10. Czoty, P. W., Justice, J. B., Jr., & Howell, L. L. (2000). Cocaine-induced changes in extracellular dopamine determined by microdialysis in awake squirrel monkeys. Psychopharmacology (Berl), 148, 299-306. doi: 10.1007/s002130050054
11. Dalley, J. W., Everitt, B. J., & Robbins, T. W. (2011). Impulsivity, compulsivity, and top-down cognitive control. Neuron, 69, 680-694. doi:10.1016/j.neuron.2011.01.020
12. Djamshidian, A., Jha, A., O'Sullivan, S. S., Silveira-Moriyama, L., Jacobson, C., Brown, P., . . . Averbeck, B. B. (2010). Risk and learning in impulsive and nonimpulsive patients with Parkinson's disease. Movement Disorders, 25, 2203-2210. doi:10.1002/mds.23247
13. Djamshidian, A., O'Sullivan, S. S., Wittmann, B. C., Lees, A. J., & Averbeck, B. B. (2011). Novelty seeking behaviour in Parkinson's disease. Neuropsychologia, 49, 2483-2488. doi:10.1016/j.neuropsychologia. 2011.04.026
14. Dulawa, S. C., Grandy, D. K., Low, M. J., Paulus, M. P., & Geyer, M. A. (1999). Dopamine D4 receptor-knock-out mice exhibit reduced exploration of novel stimuli. The Journal of Neuroscience, 19, 9550-9556.
15. Eriksson, O., Langstrom, B., & Josephsson, R. (2011). Assessment of receptor occupancy-over-time of two dopamine transporter inhibitors by [C-11]CIT and target controlled infusion. Upsala Journal of Medical Sciences, 116, 100-106. doi:10.3109/03009734.2011.563878
16. Frank, M. J., Seeberger, L. C., & O'Reilly, R. C. (2004). By carrot or by stick: Cognitive reinforcement learning in parkinsonism. Science, 306, 1940-1943. doi:10.1126/science.1102941
17. Gittins, J. C. (1979). Bandit processes and dynamic allocation indexes. Journal of the Royal Statistical Society Series B-Methodological, 41, 148-177.
18. Glowa, J. R., Wojnicki, F. H. E., Matecka, D., Rice, K. C., & Rothman, R. B. (1995). Effects of dopamine reuptake inhibitors on food- and cocaine-maintained responding: II. Comparisons with other drugs and repeated administrations. Experimental and Clinical Psychopharmacology, 3, 232-239. doi:10.1037/1064-1297.3.3.232
19. Golimbet, V. E., Alfimova, M. V., Gritsenko, I. K., & Ebstein, R. P. (2007). Relationship between dopamine system genes and extraversion and novelty seeking. Neuroscience and Behavioral Physiology, 37, 601-606. doi:10.1007/s11055-007-0058-8
20. Hayden, B. Y., Pearson, J. M., & Platt, M. L. (2011). Neuronal basis of sequential foraging decisions in a patchy environment. Nature Neuroscience, 14, 933-939. doi:10.1038/nn.2856
21. Hazy, T. E., Frank, M. J., & O'Reilly, R. C. (2010). Neural mechanisms of acquired phasic dopamine responses in learning. Neuroscience and Biobehavioral Reviews, 34, 701-720. doi:10.1016/j.neubiorev.2009.11 .019
22. Hills, T. T. (2006). Animal foraging and the evolution of goal-directed cognition. Cognitive Science, 30, 3-41. doi:10.1207/s15516709cog0000_50
23. Horvitz, J. C. (2000). Mesolimbocortical and nigrostriatal dopamine responses to salient non-reward events. Neuroscience, 96, 651-656. doi: 10.1016/S0306-4522(00)00019-1
24. Housden, C. R., O'Sullivan, S. S., Joyce, E. M., Lees, A. J., & Roiser, J. P. (2010). Intact reward learning but elevated delay discounting in Parkinson's disease patients with impulsive-compulsive spectrum behaviors. Neuropsychopharmacology, 35, 2155-2164. doi:10.1038/npp.2010.84
25. Kahlig, K. M., & Galli, A. (2003). Regulation of dopamine transporter function and plasma membrane expression by dopamine, amphetamine, and cocaine. European Journal of Pharmacology, 479, 153-158. doi: 10.1016/j.ejphar.2003.08.065
26. Kakade, S., & Dayan, P. (2002). Dopamine: Generalization and bonuses. Neural Networks, 15(4-6), 549 -559. doi:10.1016/S0893- 6080(02)00048-5
27. Kolling, N., Behrens, T. E., Mars, R. B., & Rushworth, M. F. (2012). Neural mechanisms of foraging. Science, 336, 95-98. doi:10.1126/ science.1216930
28. Kreek, M. J., Nielsen, D. A., Butelman, E. R., & LaForge, K. S. (2005). Genetic influences on impulsivity, risk taking, stress responsivity and vulnerability to drug abuse and addiction. Nature Neuroscience, 8, 1450-1457. doi:10.1038/nn1583
29. Laatikainen, L. M., Sharp, T., Bannerman, D. M., Harrison, P. J., & Tunbridge, E. M. (2012). Modulation of hippocampal dopamine metabolism and hippocampal-dependent cognitive function by catechol-Omethyltransferase inhibition. Journal of Psychopharmacology, 26, 1561-1568. doi:10.1177/0269881112454228
30. Leroy, C., Karila, L., Martinot, J. L., Lukasiewicz, M., Duchesnay, E., Comtat, C., . . . Trichard, C. (2012). Striatal and extrastriatal dopamine transporter in cannabis and tobacco addiction: A high-resolution PET study. Addiction Biology, 17, 981-990. doi:10.1111/j.1369-1600.2011 .00356.x
31. Levin, J. R., Serlin, R. C., & Seaman, M. A. (1994). A controlled, powerful multiple-comparison strategy for several situations. Psychological Bulletin, 115, 153-159. doi:10.1037/0033-2909.115.1.153
32. Lewis, D. A., Melchitzky, D. S., Sesack, S. R., Whitehead, R. E., Auh, S., & Sampson, A. (2001). Dopamine transporter immunoreactivity in monkey cerebral cortex: Regional, laminar, and ultrastructural localization. Journal of Comparative Neurology, 432, 119-136. doi:10.1002/cne .1092
33. Lyons, D., Friedman, D. P., Nader, M. A., & Porrino, L. J. (1996). Cocaine alters cerebral metabolism within the ventral striatum and limbic cortex of monkeys. Journal of Neuroscience, 16, 1230-1238.
34. Mitz, A. R. (2005). A liquid-delivery device that provides precise reward control for neurophysiological and behavioral experiments. Journal of Neuroscience Methods, 148, 19-25. doi:10.1016/j.jneumeth.2005.07 .012.
35. Murray, E. A., & Rudebeck, P. H. (2013). The drive to strive: Goal generation based on current needs. Frontiers in Neuroscience, 7, 112. doi:10.3389/fnins.2013.00112
36. Neter, J., Wasserman, W., & Kutner, M. H. (1990). Applied linear statistical models: Regression, analysis of variance, and experimental designs (3rd ed.). Homewood, IL: Irwin.
37. Niv, Y., Daw, N. D., Joel, D., & Dayan, P. (2007). Tonic dopamine: Opportunity costs and the control of response vigor. Psychopharmacology (Berl), 191, 507-520. doi:10.1007/s00213-006-0502-4
38. Pessiglione, M., Seymour, B., Flandin, G., Dolan, R. J., & Frith, C. D. (2006). Dopamine-dependent prediction errors underpin reward-seeking behaviour in humans. Nature, 442, 1042-1045. doi:10.1038/ nature05051
39. Rangel-Gomez, M., Hickey, C., van Amelsvoort, T., Bet, P., & Meeter, M. (2013). The detection of novelty relies on dopaminergic signaling: Evidence from apomorphine's impact on the novelty N2. Plos One, 8.
40. Reed, P., Mitchell, C., & Nokes, T. (1996). Intrinsic reinforcing properties of putatively neutral stimuli in an instrumental two-lever discrimination task. Animal Learning & Behavior, 24, 38-45. doi:10.3758/ BF03198952
41. Robinson, S., Sandstrom, S. M., Denenberg, V. H., & Palmiter, R. D. (2005). Distinguishing whether dopamine regulates liking, wanting, and/or learning about rewards. Behavioral Neuroscience, 119, 5-15. doi:10.1037/0735-7044.119.1.5
42. Seamans, J. K., & Yang, C. R. (2004). The principal features and mechanisms of dopamine modulation in the prefrontal cortex. Progress in Neurobiology, 74, 1-58. doi:10.1016/j.pneurobio.2004.05.006
43. Søgaard, U., Michalow, J., Butler, B., Laursen, A. L., Ingersen, S. H., Skrumsager, B. K., & Rafaelsen, O. J. (1990). A tolerance study of single and multiple dosing of the selective dopamine uptake inhibitor GBR-12909 in healthy-subjects. International Clinical Psychopharmacology, 5, 237-252. doi:10.1097/00004850-199010000-00001
44. Stephens, D. W., & Krebs, J. R. (1986). Foraging theory. Princeton, NJ: Princeton University Press.
45. Sugrue, L. P., Corrado, G. S., & Newsome, W. T. (2004). Matching behavior and the representation of value in the parietal cortex. Science, 304, 1782-1787. doi:10.1126/science.1094765
46. Tsukada, H., Harada, N., Nishiyama, S., Ohba, H., & Kakiuchi, T. (2000). Dose-response and duration effects of acute administrations of cocaine and GBR12909 on dopamine synthesis and transporter in the conscious monkey brain: PET studies combined with microdialysis. Brain Research, 860, 141-148. doi:10.1016/S0006-8993(00)02057-6
47. Villemagne, V. L., Rothman, R. B., Yokoi, F., Rice, K. C., Matecka, D., Dannals, R. F., & Wong, D. F. (1999). Doses of GBR12909 that suppress cocaine self-administration in non-human primates substantially occupy dopamine transporters as measured by [C-11] WIN35,428 PET scans. Synapse, 32, 44 -50. doi:10.1002/(SICI)1098- 2396(199904)32:1<44::AID-SYN6>3.0.CO;2-9
48. Willenbockel, V., Sadr, J., Fiset, D., Horne, G. O., Gosselin, F., & Tanaka, J. W. (2010). Controlling low-level image properties: The SHINE toolbox. Behavior Research Methods, 42, 671-684. doi:10.3758/BRM.42.3 .671
49. Wills, T. A., Vaccaro, D., & McNamara, G. (1994). Novelty seeking, risk taking, and related constructs as predictors of adolescent substance use: An application of Cloninger's theory. Journal of Substance Abuse, 6, 1-20. doi:10.1016/S0899-3289(94)90039-6
50. Wilson, F. A., & Goldman-Rakic, P. S. (1994). Viewing preferences of rhesus monkeys related to memory for complex pictures, colours and faces. Behavioural Brain Research, 60, 79-89. doi:10.1016/0166- 4328(94)90066-3
51. Wittmann, B. C., Daw, N. D., Seymour, B., & Dolan, R. J. (2008). Striatal activity underlies novelty-based choice in humans. Neuron, 58, 967- 973. doi:10.1016/j.neuron.2008.04.027
52. Zhang, L., Doyon, W. M., Clark, J. J., Phillips, P. E., & Dani, J. A. (2009). Controls of tonic and phasic dopamine transmission in the dorsal and ventral striatum. Molecular Pharmacology, 76, 396-404. doi:10.1124/ mol.109.056317
53. Zhuang, X., Oosting, R. S., Jones, S. R., Gainetdinov, R. R., Miller, G. W., Caron, M. G., & Hen, R. (2001). Hyperactivity and impaired response habituation in hyperdopaminergic mice. Proceedings of the National Academy of Sciences of the United States of America, 98, 1982-1987. doi:10.1073/pnas.98.4.1982