Neuromodulatory systems and their interactions: A review of models, theories, and experiments

Frontiers in Neural Circuits

Volumn 11, Issue , 2017, Pages

  Avery, Michael C ^a   Krichmar, Jeffrey L ^b,c  

^a SALK INSTITUTE FOR BIOLOGICAL STUDIES   (United States)

^b UNIVERSITY OF CALIFORNIA   (United States)

^c Irvine   (United States)

Author keywords

Brain disorders; Computational modeling; Computational neuroscience; Neuromodulation; Neuromodulatory systems

Indexed keywords

ACETYLCHOLINE; DOPAMINE; NORADRENALIN; SEROTONIN; NEUROTRANSMITTER RECEPTOR;

AMYGDALA; ANXIETY; ARTIFICIAL NEURAL NETWORK; ATTENTION; AUTONOMIC NERVOUS SYSTEM; BEHAVIOR; BRAIN DISEASE; CHOLINERGIC SYSTEM; COGNITION; DECISION MAKING; DOPAMINERGIC SYSTEM; DOPAMINERGIC TRANSMISSION; EMOTION; FRONTAL CORTEX; HIPPOCAMPUS; HUMAN; INHIBITION (PSYCHOLOGY); LEARNING; MATHEMATICAL MODEL; MOLECULAR INTERACTION; NEUROMODULATION; NEUROPHYSIOLOGY; NEUROSCIENCE; NEUROTRANSMISSION; NONHUMAN; NORADRENERGIC SYSTEM; PUNISHMENT; REINFORCEMENT; REVIEW; REWARD; SENSORY CORTEX; SEROTONINERGIC SYSTEM; THEORETICAL MODEL; ANIMAL; BIOLOGICAL MODEL; COMPUTER SIMULATION; METABOLISM;

ANIMALS; BRAIN DISEASES; COMPUTER SIMULATION; HUMANS; MODELS, NEUROLOGICAL; RECEPTORS, NEUROTRANSMITTER;

EID: 85041328688     PISSN: 16625110     EISSN: None     Source Type: Journal    
DOI: 10.3389/fncir.2017.00108     Document Type: Review

References (146)

1. Adams, R. A., Stephan, K. E., Brown, H. R., Frith, C. D., and Friston, K. J. (2013). The computational anatomy of psychosis. Front. Psychiatry 4:47. doi: 10.3389/fpsyt.2013.00047
2. Arnsten, A. F. (2009). Stress signalling pathways that impair prefrontal cortex structure and function. Nat. Rev. Neurosci. 10, 410–422. doi: 10.1038/nrn2648
3. Arnsten, A. F., Wang, M. J., and Paspalas, C. D. (2012). Neuromodulation of thought: flexibilities and vulnerabilities in prefrontal cortical network synapses. Neuron 76, 223–239. doi: 10.1016/j.neuron.2012.08.038
4. Asher, D. E., Craig, A. B., Zaldivar, A., Brewer, A. A., and Krichmar, J. L. (2013). A dynamic, embodied paradigm to investigate the role of serotonin in decision-making. Front. Integr. Neurosci. 7:78. doi: 10.3389/fnint.2013.00078
5. Asher, D. E., Zaldivar, A., Barton, B., Brewer, A. A., and Krichmar, J. L. (2012). Reciprocity and retaliation in social games with adaptive agents. IEEE Trans. Auton. Ment. Dev. 4, 226–238. doi: 10.1109/TAMD.2012.2202658
6. Asher, D. E., Zaldivar, A., and Krichmar, J. L. (2010). “Effect of neuromodulation on performance in game playing: a modeling study,” in IEEE 9th International Conference on Development and Learning (Ann Arbor, MI: IEEE Xplore).
7. Aston-Jones, G., and Cohen, J. D. (2005). An integrative theory of locus coeruleus-norepinephrine function: adaptive gain and optimal performance.
8. Annu. Rev. Neurosci. 28, 403–450. doi: 10.1146/annurev.neuro.28.061604. 135709
9. Aston-Jones, G., Rajkowski, J., Kubiak, P., and Alexinsky, T. (1994). Locus coeruleus neurons in monkey are selectively activated by attended cues in a vigilance task. J. Neurosci. 14, 4467–4480.
10. Avery, M. C., Dutt, N., and Krichmar, J. L. (2013). A large-scale neural network model of the influence of neuromodulatory levels on working memory and behavior. Front. Comput. Neurosci. 7:133. doi: 10.3389/fncom.2013.00133
11. Avery, M. C., Dutt, N., and Krichmar, J. L. (2014). Mechanisms underlying the basal forebrain enhancement of top-down and bottom-up attention. Eur. J. Neurosci. 39, 852–865. doi: 10.1111/ejn.12433
12. Avery, M. C., and Krichmar, J. L. (2015). Improper activation of D1 and D2 receptors leads to excess noise in prefrontal cortex. Front. Comput. Neurosci. 9:31. doi: 10.3389/fncom.2015.00031
13. Avery, M. C., Nitz, D. A., Chiba, A. A., and Krichmar, J. L. (2012). Simulation of cholinergic and noradrenergic modulation of behavior in uncertain environments. Front. Comput. Neurosci. 6:5. doi: 10.3389/fncom.2012.00005
14. Balasubramani, P. P., Chakravarthy, S., Ravindran, B., and Moustafa, A. A. (2015). Anetwork model of basal ganglia for understanding the roles of dopamine and serotonin in reward-punishment-risk based decision making. Front. Comput. Neurosci. 9:76. doi: 10.3389/fncom.2015.00076
15. Barker, D. J., Root, D. H., Zhang, S., and Morales, M. (2016). Multiplexed neurochemical signaling by neurons of the ventral tegmental area. J. Chem. Neuroanat. 73, 33–42. doi: 10.1016/j.jchemneu.2015.12.016
16. Barnes, N. M., and Sharp, T. (1999). A review of central 5-HT receptors and their function. Neuropharmacology 38, 1083–1152. doi: 10.1016/S0028-3908(99)00010-6
17. Baxter, M. G., and Chiba, A. A. (1999). Cognitive functions of the basal forebrain. Curr. Opin. Neurobiol. 9, 178–183. doi: 10.1016/S0959-4388(99)80024-5
18. Bédard, C., Wallman, M. J., Pourcher, E., Gould, P. V., Parent, A., and Parent, M. (2011). Serotonin and dopamine striatal innervation in Parkinson’s disease and Huntington’s chorea. Parkinsonism Relat. Disord. 17, 593–598. doi: 10.1016/j.parkreldis.2011.05.012
19. Berridge, C. W., and Waterhouse, B. D. (2003). The locus coeruleus-noradrenergic system: modulation of behavioral state and state-dependent cognitive processes. Brain Res. Rev. 42, 33–84. doi: 10.1016/S0165-0173(03)00143-7
20. Berridge, K. C. (2004). Motivation concepts in behavioral neuroscience. Physiol. Behav. 81, 179–209. doi: 10.1016/j.physbeh.2004.02.004
21. Berridge, K. C. (2012). From prediction error to incentive salience: mesolimbic computation of reward motivation. Eur. J. Neurosci. 35, 1124–1143. doi: 10.1111/j.1460-9568.2012.07990.x
22. Berridge, K. C., andRobinson, T. E. (1998). What is the role of dopamine inreward: hedonic impact, reward learning, or incentive salience? Brain Res. Brain Res. Rev. 28, 309–369. doi: 10.1016/S0165-0173(98)00019-8
23. Björklund, A., and Dunnett, S. B. (2007). Fifty years of dopamine research. Trends Neurosci. 30, 185–187. doi: 10.1016/j.tins.2007.03.004
24. Boureau, Y. L., and Dayan, P. (2011). Opponency revisited: competition and cooperation between dopamine and serotonin. Neuropsychopharmacology 36, 74–97. doi: 10.1038/npp.2010.151
25. Bouret, S., and Sara, S. J. (2005). Network reset: a simplified overarching theory of locus coeruleus noradrenaline function. Trends Neurosci. 28, 574–582. doi: 10.1016/j.tins.2005.09.002
26. Braver, T. S., and Cohen, J. D. (1999). Dopamine, cognitive control, and schizophrenia: the gating model. Prog. Brain Res. 121, 327–349. doi: 10.1016/S0079-6123(08)63082-4
27. Briand, L. A., Gritton, H., Howe, W. M., Young, D. A., and Sarter, M. (2007). Modulators in concert for cognition: modulator interactions in the prefrontal cortex. Prog. Neurobiol. 83, 69–91. doi: 10.1016/j.pneurobio.2007.06.007
28. Bromberg-Martin, E. S., Matsumoto, M., and Hikosaka, O. (2010). Dopamine in motivational control: rewarding, aversive, and alerting. Neuron 68, 815–834. doi: 10.1016/j.neuron.2010.11.022
29. Broussard, J. I., Karelina, K., Sarter, M., and Givens, B. (2009). Cholinergic optimization of cue-evoked parietal activity during challenged attentional performance. Eur. J. Neurosci. 29, 1711–1722. doi: 10.1111/j.1460-9568.2009.06713.x
30. Bucci, D. J., Holland, P. C., and Gallagher, M. (1998). Removal of cholinergic input to rat posterior parietal cortex disrupts incremental processing of conditioned stimuli. J. Neurosci. 18, 8038–8046.
31. Caetano, M., Jin, L. E., Harenberg, L., Stachenfeld, K. L., Arnsten, A. F., and Laubach, M. (2013). Noradrenergic control of error perseveration in medial prefrontal cortex. Front. Integr. Neurosci. 6:125. doi: 10.3389/fnint.2012.00125
32. Cannon, C. M., and Palmiter, R. D. (2003). Reward without dopamine. J. Neurosci. 23, 10827–10831.
33. Cano-Colino, M., Almeida, R., Gomez-Cabrero, D., Artigas, F., and Compte, A. (2014). Serotonin regulates performance nonmonotonically in a spatial working memory network. Cereb. Cortex 24, 2449–2463. doi: 10.1093/cercor/bht096
34. Chen, C., Takahashi, T., Nakagawa, S., Inoue, T., and Kusumi, I. (2015). Reinforcement learning in depression: a review of computational research. Neurosci. Biobehav. Rev. 55, 247–267. doi: 10.1016/j.neubiorev.2015.05.005
35. Chen, N., Sugihara, H., and Sur, M. (2015). Anacetylcholine-activated microcircuit drives temporal dynamics of cortical activity. Nat. Neurosci. 18, 892–902. doi: 10.1038/nn.4002
36. Chiba, A. A., Bucci, D. J., Holland, P. C., and Gallagher, M. (1995). Basal forebrain cholinergic lesions disrupt increments but not decrements in conditioned stimulus processing. J. Neurosci. 15, 7315–7322.
37. Cohen, J. D., McClure, S. M., andYu, A. J. (2007). ShouldI stay or shouldI go? How the human brain manages the trade-off between exploitation and exploration. Philos. Trans. RSoc. Lond. BBiol. Sci. 362, 933–942. doi: 10.1098/rstb.2007.2098
38. Cools, R., Roberts, A. C., and Robbins, T. W. (2008). Serotoninergic regulation of emotional and behavioural control processes. Trends Cogn. Sci. 12, 31–40. doi: 10.1016/j.tics.2007.10.011
39. Corbetta, M., Patel, G., and Shulman, G. L. (2008). The reorienting system of the human brain: from environment to theory of mind. Neuron 58, 306–324. doi: 10.1016/j.neuron.2008.04.017
40. Craske, M. G., and Stein, M. B. (2016). Anxiety. Lancet 388, 3048–3059. doi: 10.1016/S0140-6736(16)30381-6
41. Crockett, M. J., Clark, L., Apergis-Schoute, A. M., Morein-Zamir, S., and Robbins, T. W. (2012). Serotonin modulates the effects of Pavlovian aversive predictions on response vigor. Neuropsychopharmacology 37, 2244–2252. doi: 10.1038/npp.2012.75
42. Crockett, M. J., Clark, L., and Robbins, T. W. (2009). Reconciling the role of serotonin in behavioral inhibition and aversion: acute tryptophan depletion abolishes punishment-induced inhibition in humans. J. Neurosci. 29, 11993–11999. doi: 10.1523/JNEUROSCI.2513-09.2009
43. Crockett, M. J., Clark, L., Tabibnia, G., Lieberman, M. D., and Robbins, T. W. (2008). Serotonin modulates behavioral reactions to unfairness. Science 320:1739. doi: 10.1126/science.1155577
44. Crockett, M. J., Siegel, J. Z., Kurth-Nelson, Z., Ousdal, O. T., Story, G., Frieband, C., et al. (2015). Dissociable effects of serotonin and dopamine on the valuation of harm in moral decision making. Curr. Biol. 25, 1852–1859. doi: 10.1016/j.cub.2015.05.021
45. Dalley, J. W., McGaughy, J., O’Connell, M. T., Cardinal, R. N., Levita, L., and Robbins, T. W. (2001). Distinct changes in cortical acetylcholine and noradrenaline efflux during contingent and noncontingent performance of a visual attentional task. J. Neurosci. 21, 4908–4914.
46. Daw, N. D., Kakade, S., and Dayan, P. (2002). Opponent interactions between serotonin and dopamine. Neural Netw. 15, 603–616. doi: 10.1016/S0893-6080(02)00052-7
47. Deakin, J. F. (2003). Depression and antisocial personality disorder: two contrasting disorders of 5HT function. J. Neural. Transm. 79–93. doi: 10.1007/978-3-7091-6020-6_5
48. Deco, G., and Thiele, A. (2011). Cholinergic control of cortical network interactions enables feedback-mediated attentional modulation. Eur. J. Neurosci. 34, 146–157. doi: 10.1111/j.1460-9568.2011.07749.x
49. de la Fuente-Sandoval, C., León-Ortiz, P., Favila, R., Stephano, S., Mamo, D., Ramírez-Bermúdez, J., et al. (2011). Higher levels of glutamate in the associative-striatum of subjects with prodromal symptoms of schizophrenia and patients with first-episode psychosis. Neuropsychopharmacology 36, 1781–1791. doi: 10.1038/npp.2011.65
50. Disney, A. A., Aoki, C., and Hawken, M. J. (2007). Gain modulation by nicotine in macaque v1. Neuron 56, 701–713. doi: 10.1016/j.neuron.2007.09.034
51. Doya, K. (2002). Metalearning and neuromodulation. Neural Netw. 15, 495–506. doi: 10.1016/S0893-6080(02)00044-8
52. Doya, K. (2008). Modulators of decision making. Nat. Neurosci. 11, 410–416. doi: 10.1038/nn2077
53. Durstewitz, D., and Seamans, J. K. (2008). The dual-state theory of prefrontal cortex dopamine function with relevance to catechol-o-methyltransferase genotypes and schizophrenia. Biol. Psychiatry 64, 739–749. doi: 10.1016/j.biopsych.2008.05.015
54. Fonio, E., Benjamini, Y., and Golani, I. (2009). Freedom of movement and the stability of its unfolding in free exploration of mice. Proc. Natl. Acad. Sci. U.S.A. 106, 21335–21340. doi: 10.1073/pnas.0812513106
55. Friston, K. J., Shiner, T., FitzGerald, T., Galea, J. M., Adams, R., Brown, H., et al. (2012). Dopamine, affordance and active inference. PLoS Comput. Biol. 8:e1002327. doi: 10.1371/journal.pcbi.1002327
56. Gee, S., Ellwood, I., Patel, T., Luongo, F., Deisseroth, K., and Sohal, V. S. (2012). Synaptic activity unmasks dopamine D2 receptor modulation of a specific class of layer V pyramidal neurons in prefrontal cortex. J. Neurosci. 32, 4959–4971. doi: 10.1523/JNEUROSCI.5835-11.2012
57. Goard, M., and Dan, Y. (2009). Basal forebrain activation enhances cortical coding of natural scenes. Nat. Neurosci. 12, 1444–1449. doi: 10.1038/nn.2402
58. Gold, P. W., Machado-Vieira, R., and Pavlatou, M. G. (2015). Clinical and biochemical manifestations of depression: relationto the neurobiology of stress. Neural Plast. 2015:581976. doi: 10.1155/2015/581976
59. Granholm, E., Holden, J., Link, P. C., and McQuaid, J. R. (2014). Randomized clinical trial of cognitive behavioral social skills training for schizophrenia:
60. improvement in functioning and experiential negative symptoms. J. Consult. Clin. Psychol. 82, 1173–1185. doi: 10.1037/a0037098
61. Granholm, E., and Verney, S. P. (2004). Pupillary responses and attentional allocation problems on the backward masking task in schizophrenia. Int. J. Psychophysiol. 52, 37–51. doi: 10.1016/j.ijpsycho.2003.12.004
62. Gremel, C. M., and Costa, R. M. (2013). Orbitofrontal and striatal circuits dynamically encode the shift between goal-directed and habitual actions. Nat. Commun. 4:2264. doi: 10.1038/ncomms3264
63. Harvey, J. A. (2003). Role of the serotonin 5-HT(2A) receptor in learning. Learn. Mem. 10, 355–362. doi: 10.1101/lm.60803
64. Hasselmo, M. E. (1999). Neuromodulation: acetylcholine and memory consolidation. Trends Cogn. Sci. 3, 351–359. doi: 10.1016/S1364-6613(99)01365-0
65. Hasselmo, M. E., and McGaughy, J. (2004). High acetylcholine levels set circuit dynamics for attention and encoding and low acetylcholine levels set dynamics for consolidation. Prog. Brain Res. 145, 207–231. doi: 10.1016/S0079-6123(03)45015-2
66. Hasselmo, M. E., and Stern, C. E. (2006). Mechanisms underlying working memory for novel information. Trends Cogn. Sci. 10, 487–493. doi: 10.1016/j.tics.2006.09.005
67. Hazy, T. E., Frank, M. J., and O’Reilly, R. C. (2006). Banishing the homunculus: making working memory work. Neuroscience 139, 105–118. doi: 10.1016/j.neuroscience.2005.04.067
68. Heisler, L. K., Chu, H. M., Brennan, T. J., Danao, J. A., Bajwa, P., Parsons, L. H., et al. (1998). Elevated anxiety and antidepressant-like responses in serotonin 5-HT1A receptor mutant mice. Proc. Natl. Acad. Sci. U.S.A. 95, 15049–15054. doi: 10.1073/pnas.95.25.15049
69. Hermans, E. J., van Marle, H. J., Ossewaarde, L., Henckens, M. J., Qin, S., van Kesteren, M. T., et al. (2011). Stress-related noradrenergic activity prompts large-scale neural network reconfiguration. Science 334, 1151–1153. doi: 10.1126/science.1209603
70. Herrero, J. L., Roberts, M. J., Delicato, L. S., Gieselmann, M. A., Dayan, P., and Thiele, A. (2008). Acetylcholine contributes through muscarinic receptors to attentional modulation in V1. Nature 454, 1110–1114. doi: 10.1038/nature07141
71. Hong, S., and Hikosaka, O. (2014). Pedunculopontine tegmental nucleus neurons provide reward, sensorimotor, and alerting signals to midbrain dopamine neurons. Neuroscience 282, 139–155. doi: 10.1016/j.neuroscience.2014.07.002
72. ∗) nAChRs: underlying cholinergic mechanisms. Neuropsychopharmacology 35, 1391–1401. doi: 10.1038/npp.2010.9
73. Huys, Q. J., Pizzagalli, D. A., Bogdan, R., and Dayan, P. (2013). Mapping anhedonia onto reinforcement learning: a behavioural meta-analysis. Biol. Mood Anxiety Disord. 3:12. doi: 10.1186/2045-5380-3-12
74. Hyman, S. E., Malenka, R. C., and Nestler, E. J. (2006). Neural mechanisms of addiction: the role of reward-related learning and memory. Annu. Rev. Neurosci. 29, 565–598. doi: 10.1146/annurev.neuro.29.051605.113009
75. Jasinska, A. J., Lowry, C. A., and Burmeister, M. (2012). Serotonintransporter gene, stress and raphe-raphe interactions: a molecular mechanism of depression. Trends Neurosci. 35, 395–402. doi: 10.1016/j.tins.2012.01.001
76. Kapur, S. (2003). Psychosis as a state of aberrant salience: a framework linking biology, phenomenology, and pharmacology in schizophrenia. Am. J. Psychiatry 160, 13–23. doi: 10.1176/appi.ajp.160.1.13
77. Körding, K. P., and Wolpert, D. M. (2006). Bayesian decision theory in sensorimotor control. Trends Cogn. Sci. 10, 319–326. doi: 10.1016/j.tics.2006.05.003
78. Koukouli, F., Rooy, M., Tziotis, D., Sailor, K. A., O’Neill, H. C., Levenga, J., et al. (2017). Nicotine reverses hypofrontality in animal models of addiction and schizophrenia. Nat. Med. 23, 347–354. doi: 10.1038/nm.4274
79. Krichmar, J. L. (2008). The neuromodulatory system - a framework for survival and adaptive behavior in a challenging world. Adapt. Behav. 16, 385–399. doi: 10.1177/1059712308095775
80. Krichmar, J. L. (2013). A neurorobotic platform to test the influence of neuromodulatory signaling on anxious and curious behavior. Front. Neurorobot. 7:1. doi: 10.3389/fnbot.2013.00001
81. Lacroix, L., Spinelli, S., Heidbreder, C. A., and Feldon, J. (2000). Differential role of the medial and lateral prefrontal cortices in fear and anxiety. Behav. Neurosci. 114, 1119–1130. doi: 10.1037/0735-7044.114.6.1119
82. Lavín, C., San Martín, R., and Rosales Jubal, E. (2014). Pupil dilation signals uncertainty and surprise in a learning gambling task. Front. Behav. Neurosci. 7:218. doi: 10.3389/fnbeh.2013.00218
83. Letzkus, J. J., Wolff, S. B., and Lüthi, A. (2015). Disinhibition, a circuit mechanism for associative learning and memory. Neuron 88, 264–276. doi: 10.1016/j.neuron.2015.09.024
84. Lipkind, D., Sakov, A., Kafkafi, N., Elmer, G. I., Benjamini, Y., and Golani, I. (2004). New replicable anxiety-related measures of wall vs. center behavior of mice in the open field. J. Appl. Physiol. 97, 347–359. doi: 10.1152/japplphysiol.00148.2004
85. Loftus, E. F. (1996). Memory distortion and false memory creation. Bull. Am. Acad. Psychiatry Law 24, 281–295.
86. Loh, M., Rolls, E. T., and Deco, G. (2007). A dynamical systems hypothesis of schizophrenia. PLoS Comput. Biol. 3:e228. doi: 10.1371/journal.pcbi.0030228
87. Lubow, R. E., and Weiner, I. (2010). Latent Inhibition: Cognition, Neuroscience, and Applications to Schizophrenia. Cambridge: Cambridge University Press.
88. Maia, T. V., and Frank, M. J. (2011). From reinforcement learning models to psychiatric and neurological disorders. Nat. Neurosci. 14, 154–162. doi: 10.1038/nn.2723
89. Manohar, S. G., and Husain, M. (2015). Reduced pupillary reward sensitivity in Parkinson’s disease. NPJ Parkinsons Dis. 1:15026. doi: 10.1038/npjparkd.2015.26
90. Marcott, P. F., Mamaligas, A. A., and Ford, C. P. (2014). Phasic dopamine release drives rapid activation of striatal D2-receptors. Neuron 84, 164–176. doi: 10.1016/j.neuron.2014.08.058
91. Matsumoto, M., and Hikosaka, O. (2007). Lateral habenula as a source of negative reward signals in dopamine neurons. Nature 447, 1111–1115. doi: 10.1038/nature05860
92. McGinley, M. J., David, S. V., and McCormick, D. A. (2015). Cortical membrane potential signature of optimal states for sensory signal detection. Neuron 87, 179–192. doi: 10.1016/j.neuron.2015.05.038
93. Meneses, A., and Perez-Garcia, G. (2007). 5-HT(1A) receptors and memory. Neurosci. Biobehav. Rev. 31, 705–727. doi: 10.1016/j.neubiorev.2007.02.001
94. Millan, M. J. (2003). The neurobiology and control of anxious states. Prog. Neurobiol. 70, 83–244. doi: 10.1016/S0301-0082(03)00087-X
95. Miyazaki, K., Miyazaki, K. W., and Doya, K. (2011). Activation of dorsal raphe serotonin neurons underlies waiting for delayed rewards. J. Neurosci. 31, 469–479. doi: 10.1523/JNEUROSCI.3714-10.2011
96. Moustafa, A. A., and Gluck, M. A. (2011). A neurocomputational model of dopamine and prefrontal-striatal interactions during multicue category learning by Parkinson patients. J. Cogn. Neurosci. 23, 151–167. doi: 10.1162/jocn.2010.21420
97. Moustafa, A. A., Herzallah, M. M., and Gluck, M. A. (2013). Dissociating the cognitive effects of levodopa versus dopamine agonists in a neurocomputational model of learning in Parkinson’s disease. Neurodegener. Dis. 11, 102–111. doi: 10.1159/000341999
98. Muhammed, K., Manohar, S., Ben Yehuda, M., Chong, T. T., Tofaris, G., Lennox, G., et al. (2016). Reward sensitivity deficits modulated by dopamine are associated with apathy in Parkinson’s disease. Brain 139(Pt 10), 2706–2721. doi: 10.1093/brain/aww188
99. Murty, V. P., Ballard, I. C., and Adcock, R. A. (2017). Hippocampus and prefrontal cortex predict distinct timescales of activation in the human ventral tegmental area. Cereb. Cortex 27, 1660–1669. doi: 10.1093/cercor/bhw005
100. Nakamura, K., Matsumoto, M., and Hikosaka, O. (2008). Reward-dependent modulation of neuronal activity in the primate dorsal raphe nucleus. J. Neurosci. 28, 5331–5343. doi: 10.1523/JNEUROSCI.0021-08.2008
101. Nassar, M. R., Rumsey, K. M., Wilson, R. C., Parikh, K., Heasly, B., and Gold, J. I. (2012). Rational regulation of learning dynamics by pupil-linked arousal systems. Nat. Neurosci. 15, 1040–1046. doi: 10.1038/nn.3130
102. Nelson, C. L., Sarter, M., and Bruno, J. P. (2005). Prefrontal cortical modulation of acetylcholine release in posterior parietal cortex. Neuroscience 132, 347–359. doi: 10.1016/j.neuroscience.2004.12.007
103. Newman, E. L., Gupta, K., Climer, J. R., Monaghan, C. K., and Hasselmo, M. E. (2012). Cholinergic modulation of cognitive processing: insights drawn from computational models. Front. Behav. Neurosci. 6:24. doi: 10.3389/fnbeh.2012.00024
104. Noudoost, B., and Moore, T. (2011). Control of visual cortical signals by prefrontal dopamine. Nature 474, 372–375. doi: 10.1038/nature09995
105. Oros, N., Chiba, A. A., Nitz, D. A., and Krichmar, J. L. (2014). Learning to ignore: a modeling study of a decremental cholinergic pathway and its influence on attention and learning. Learn. Mem. 21, 105–118. doi: 10.1101/lm.032433.113
106. Parikh, V., and Sarter, M. (2008). Cholinergic mediation of attention: contributions of phasic and tonic increases in prefrontal cholinergic activity. Ann. N.Y. Acad. Sci. 1129, 225–235. doi: 10.1196/annals.1417.021
107. Preuschoff, K., Hart, B. M., and Einhäuser, W. (2011). Pupil dilation signals surprise: evidence for noradrenaline’s role in decision making. Front. Neurosci. 5:115. doi: 10.3389/fnins.2011.00115
108. Puig, M. V., and Miller, E. K. (2014). Neural substrates of dopamine d2 receptor modulated executive functions in the monkey prefrontal cortex. Cereb. Cortex 25, 2980–2987. doi: 10.1093/cercor/bhu096
109. Redgrave, P., and Gurney, K. (2006). The short-latency dopamine signal: a role in discovering novel actions? Nat. Rev. Neurosci. 7, 967–975. doi: 10.1038/nrn2022
110. Reimer, J., Froudarakis, E., Cadwell, C. R., Yatsenko, D., Denfield, G. H., and Tolias, A. S. (2014). Pupil fluctuations track fast switching of cortical states during quiet wakefulness. Neuron 84, 355–362. doi: 10.1016/j.neuron.2014.09.033
111. Rolls, E. T., Loh, M., Deco, G., and Winterer, G. (2008). Computational models of schizophrenia and dopamine modulation in the prefrontal cortex. Nat. Rev. Neurosci. 9, 696–709. doi: 10.1038/nrn2462
112. Sara, S. J., Dyon-Laurent, C., and Hervé, A. (1995). Novelty seeking behavior in the rat is dependent upon the integrity of the noradrenergic system. Brain Res. Cogn. 2, 181–187. doi: 10.1016/0926-6410(95)90007-1
113. Sarter, M., Givens, B., and Bruno, J. P. (2001). The cognitive neuroscience of sustained attention: where top-down meets bottom-up. Brain Res. Rev. 35, 146–160. doi: 10.1016/S0165-0173(01)00044-3
114. Sarter, M., Hasselmo, M. E., Bruno, J. P., and Givens, B. (2005). Unraveling the attentional functions of cortical cholinergic inputs: interactions between signal-driven and cognitive modulation of signal detection. Brain Res. Rev. 48, 98–111. doi: 10.1016/j.brainresrev.2004.08.006
115. Schoenbaum, G., Roesch, M. R., Stalnaker, T. A., and Takahashi, Y. K. (2009). A new perspective on the role of the orbitofrontal cortex in adaptive behaviour. Nat. Rev. Neurosci. 10, 885–892. doi: 10.1038/nrn2753
116. Schultz, W. (1997). Dopamine neurons and their role in reward mechanisms. Curr. Opin. Neurobiol. 7, 191–197. doi: 10.1016/S0959-4388(97)80007-4
117. Schultz, W. (2007). Behavioral dopamine signals. Trends Neurosci. 30, 203–210. doi: 10.1016/j.tins.2007.03.007
118. Schultz, W., Dayan, P., and Montague, P. R. (1997). A neural substrate of prediction and reward. Science 275, 1593–1599. doi: 10.1126/science.275.5306.1593
119. Schultz, W., Tremblay, L., and Hollerman, J. R. (2000). Reward processing in primate orbitofrontal cortex and basal ganglia. Cereb. Cortex 10, 272–284. doi: 10.1093/cercor/10.3.272
120. Schweighofer, N., Bertin, M., Shishida, K., Okamoto, Y., Tanaka, S. C., Yamawaki, S., et al. (2008). Low-serotonin levels increase delayed reward discounting in humans. J. Neurosci. 28, 4528–4532. doi: 10.1523/JNEUROSCI.4982-07.2008
121. Sepeta, L., Tsuchiya, N., Davies, M. S., Sigman, M., Bookheimer, S. Y., and Dapretto, M. (2012). Abnormal social reward processing in autism as indexed by pupillary responses to happy faces. J. Neurodev. Disord. 4:17. doi: 10.1186/1866-1955-4-17
122. Seymour, B., Daw, N. D., Roiser, J. P., Dayan, P., and Dolan, R. (2012). Serotonin selectively modulates reward value in human decision-making. J. Neurosci. 32, 5833–5842. doi: 10.1523/JNEUROSCI.0053-12.2012
123. Sillitoe, R. V., and Vogel, M. W. (2008). Desire, disease, and the origins of the dopaminergic system. Schizophr. Bull. 34, 212–219. doi: 10.1093/schbul/sbm170
124. Stern, E. R., Gonzalez, R., Welsh, R. C., and Taylor, S. F. (2010). Updating beliefs for a decision: neural correlates of uncertainty and underconfidence. J. Neurosci. 30, 8032–8041. doi: 10.1523/JNEUROSCI.4729-09.2010
125. Takahashi, Y. K., Roesch, M. R., Wilson, R. C., Toreson, K., O’Donnell, P., Niv, Y., et al. (2011). Expectancy-related changes in firing of dopamine neurons depend on orbitofrontal cortex. Nat. Neurosci. 14, 1590–1597. doi: 10.1038/nn.2957
126. Tanaka, S. C., Schweighofer, N., Asahi, S., Shishida, K., Okamoto, Y., Yamawaki, S., et al. (2007). Serotonin differentially regulates short- and long-term prediction of rewards in the ventral and dorsal striatum. PLoS ONE 2:e1333. doi: 10.1371/journal.pone.0001333
127. Tanaka, S. C., Shishida, K., Schweighofer, N., Okamoto, Y., Yamawaki, S., and Doya, K. (2009). Serotonin affects association of aversive outcomes to past actions. J. Neurosci. 29, 15669–15674. doi: 10.1523/JNEUROSCI.2799-09.2009
128. Tops, M., Russo, S., Boksem, M. A., and Tucker, D. M. (2009). Serotonin: modulator of a drive to withdraw. Brain Cogn. 71, 427–436. doi: 10.1016/j.bandc.2009.03.009
129. Trudeau, L. E. (2004). Glutamate co-transmission as an emerging concept in monoamine neuron function. J. Psychiatry Neurosci. 29, 296–310.
130. Vankov, A., Hervé-Minvielle, A., and Sara, S. J. (1995). Response to novelty and its rapid habituation in locus coeruleus neurons of the freely exploring rat. Eur. J. Neurosci. 7, 1180–1187. doi: 10.1111/j.1460-9568.1995.tb01108.x
131. Vijayraghavan, S., Wang, M., Birnbaum, S. G., Williams, G. V., and Arnsten, A. F. (2007). Inverted-U dopamine D1 receptor actions on prefrontal neurons engaged in working memory. Nat. Neurosci. 10, 376–384. doi: 10.1038/nn1846
132. Wang, M., Ramos, B. P., Paspalas, C. D., Shu, Y., Simen, A., Duque, A., et al. (2007). Alpha2A-adrenoceptors strengthen working memory networks by inhibiting cAMP-HCN channel signaling in prefrontal cortex. Cell 129, 397–410. doi: 10.1016/j.cell.2007.03.015
133. Weisstaub, N. V., Zhou, M., Lira, A., Lambe, E., González-Maeso, J., Hornung, J. P., et al. (2006). Cortical 5-HT2A receptor signaling modulates anxiety-like behaviors in mice. Science 313, 536–540. doi: 10.1126/science.1123432
134. Weng, J., Paslaski, S., Daly, J., VanDam, C., and Brown, J. (2013). Modulation for emergent networks: serotonin and dopamine. Neural Netw. 41, 225–239. doi: 10.1016/j.neunet.2012.11.008
135. Winstanley, C. A., Dalley, J. W., Theobald, D. E., and Robbins, T. W. (2003). Global 5-HT depletion attenuates the ability of amphetamine to decrease impulsive choice on a delay-discounting task in rats. Psychopharmacology 170, 320–331. doi: 10.1007/s00213-003-1546-3
136. Wise, R. A. (2004). Dopamine, learning and motivation. Nat. Rev. Neurosci. 5, 483–494. doi: 10.1038/nrn1406
137. Wixted, J. T., Squire, L. R., Jang, Y., Papesh, M. H., Goldinger, S. D., Kuhn, J. R., et al. (2014). Sparse and distributed coding of episodic memory in neurons of the human hippocampus. Proc. Natl. Acad. Sci. U.S.A. 111, 9621–9626. doi: 10.1073/pnas.1408365111
138. Wong-Lin, K., Joshi, A., Prasad, G., and McGinnity, T. M. (2012). Network properties of a computational model of the dorsal raphe nucleus. Neural Netw. 32, 15–25. doi: 10.1016/j.neunet.2012.02.009
139. Wood, R. M., Rilling, J. K., Sanfey, A. G., Bhagwagar, Z., and Rogers, R. D. (2006). Effects of tryptophan depletion on the performance of an iterated prisoner’s dilemma game in healthy adults. Neuropsychopharmacology 31, 1075–1084. doi: 10.1038/sj.npp.1300932
140. Yu, A. J., and Dayan, P. (2002). Acetylcholine in cortical inference. Neural Netw. 15, 719–730. doi: 10.1016/S0893-6080(02)00058-8
141. Yu, A. J., and Dayan, P. (2005). Uncertainty, neuromodulation, and attention. Neuron 46, 681–692. doi: 10.1016/j.neuron.2005.04.026
142. Zaborszky, L. (2002). The modular organization of brain systems. Basal forebrain: the last frontier. Prog. Brain Res. 136, 359–372. doi: 10.1016/S0079-6123(02)36030-8
143. Zaborszky, L., and Duque, A. (2003). Sleep-wake mechanisms and basal forebrain circuitry. Front. Biosci. 8, D1146–D1169. doi: 10.2741/1112
144. Zaldivar, A., Asher, D. E., and Krichmar, J. L. (2010). “Simulation of how neuromodulation influences cooperative behavior,” in Simulation of Adaptive Behavior: From Animals to Animats, Lecture Notes on Artificial Intelligence (LNAI 6226), eds S. Doncieux, J.-A. Meyer, A. Guillot, and J. Hallam. (Berlin; Heidelberg: Springer-Verlag), 649–660.
145. Zaldivar, A., and Krichmar, J. L. (2013). Interactions between the neuromodulatory systems and the amygdala: exploratory survey using the Allen mouse brain atlas. Brain Struct. Funct. 218, 1513–1530. doi: 10.1007/s00429-012-0473-7
146. Zhou, F. M., Liang, Y., Salas, R., Zhang, L., De Biasi, M., and Dani, J. A. (2005). Corelease of dopamine and serotonin fromstriatal dopamine terminals. Neuron 46, 65–74. doi: 10.1016/j.neuron.2005.02.010