Toward a theoretical role for tonic norepinephrine in the orbitofrontal cortex in facilitating flexible learning

Neuroscience

Volumn 345, Issue , 2017, Pages 124-129

  Sadacca, Brian F ^a   Wikenheiser, Andrew M ^a   Schoenbaum, Geoffrey ^a,b,c  

^a NATIONAL INSTITUTE ON DRUG ABUSE   (United States)

^b UNIVERSITY OF MARYLAND SCHOOL OF MEDICINE   (United States)

^c JOHNS HOPKINS UNIVERSITY SCHOOL OF MEDICINE   (United States)

Author keywords

ensemble states; flexible learning; norepinephrine; orbitofrontal cortex; reinforcement learning

Indexed keywords

NORADRENALIN;

ADRENAL FUNCTION; ANATOMICAL VARIATION; ANIMAL BEHAVIOR; BEHAVIOR CONTROL; BRAIN FUNCTION; BRAIN MAPPING; BRAIN REGION; COGNITIVE FLEXIBILITY; COMPREHENSION; CONCEPTUAL FRAMEWORK; ENVIRONMENTAL CHANGE; HUMAN; LEARNING; LOCUS CERULEUS; ORBITAL CORTEX; REVIEW; REWARD; STRUCTURE ACTIVITY RELATION; UNCERTAINTY; ANIMAL; EXECUTIVE FUNCTION; METABOLISM; PHYSIOLOGY; PREFRONTAL CORTEX;

ANIMALS; EXECUTIVE FUNCTION; LEARNING; NOREPINEPHRINE; PREFRONTAL CORTEX;

EID: 84975780315     PISSN: 03064522     EISSN: 18737544     Source Type: Journal    
DOI: 10.1016/j.neuroscience.2016.04.017     Document Type: Review

References (71)

1. Abeles, M., Bergman, H., Gat, I., Meilijson, I., Seidemann, E., Tishby, N., Vaadia, E., Cortical activity flips among quasi-stationary states. Proc Natl Acad Sci USA 92 (1995), 8616–8620.
2. Agster, K.L., Mejias-Aponte, C.A., Clark, B.D., Waterhouse, B.D., Evidence for a regional specificity in the density and distribution of noradrenergic varicosities in rat cortex. J Comp Neurol 521 (2013), 2195–2207.
3. Arnsten, A.F., Wang, M.J., Paspalas, C.D., Neuromodulation of thought: flexibilities and vulnerabilities in prefrontal cortical network synapses. Neuron 76 (2012), 223–239.
4. Aston-Jones, G., Rajkowski, J., Kubiak, P., Conditioned responses of monkey locus coeruleus neurons anticipate acquisition of discriminative behavior in a vigilance task. Neuroscience 80 (1997), 697–715.
5. Aston-Jones, G., Rajkowski, J., Cohen, J., Role of locus coeruleus in attention and behavioral flexibility. Biol Psychiatry 46 (1999), 1309–1320.
6. Bari, A., Theobald, D.E., Caprioli, D., Mar, A.C., Aidoo-Micah, A., Dalley, J.W., Robbins, T.W., Serotonin modulates sensitivity to reward and negative feedback in a probabilistic reversal learning task in rats. Neuropsychopharmacology 35 (2010), 1290–1301.
7. Berridge, C.W., Waterhouse, B.D., The locus coeruleus-noradrenergic system: modulation of behavioral state and state-dependent cognitive processes. Brain Res Brain Res Rev 42 (2003), 33–84.
8. Bollimunta, A., Totten, D., Ditterich, J., Neural dynamics of choice: single-trial analysis of decision-related activity in parietal cortex. J Neurosci 32 (2012), 12684–12701.
9. Boulougouris, V., Glennon, J.C., Robbins, T.W., Dissociable effects of selective 5-HT2A and 5-HT2C receptor antagonists on serial spatial reversal learning in rats. Neuropsychopharmacology 33 (2008), 2007–2019.
10. Boureau, Y.L., Dayan, P., Opponency revisited: competition and cooperation between dopamine and serotonin. Neuropsychopharmacology 36 (2011), 74–97.
11. Bouret, S., Sara, S.J., Reward expectation, orientation of attention and locus coeruleus-medial frontal cortex interplay during learning. Eur J Neurosci 20 (2004), 791–802.
12. Bouret, S., Sara, S.J., Network reset: a simplified overarching theory of locus coeruleus noradrenaline function. Trends Neurosci 28 (2005), 574–582.
13. Burke, K.A., Takahashi, Y.K., Correll, J., Brown, P.L., Schoenbaum, G., Orbitofrontal inactivation impairs reversal of Pavlovian learning by interfering with ‘disinhibition’ of responding for previously unrewarded cues. Eur J Neurosci 30 (2009), 1941–1946.
14. Chandler, D., Waterhouse, B.D., Evidence for broad versus segregated projections from cholinergic and noradrenergic nuclei to functionally and anatomically discrete subregions of prefrontal cortex. Front Behav Neurosci, 6, 2012, 20.
15. Chandler, D.J., Lamperski, C.S., Waterhouse, B.D., Identification and distribution of projections from monoaminergic and cholinergic nuclei to functionally differentiated subregions of prefrontal cortex. Brain Res 1522 (2013), 38–58.
16. Chandler, D.J., Gao, W.J., Waterhouse, B.D., Heterogeneous organization of the locus coeruleus projections to prefrontal and motor cortices. Proc Natl Acad Sci USA 111 (2014), 6816–6821.
17. Chandler, D.J., Waterhouse, B.D., Gao, W.J., New perspectives on catecholaminergic regulation of executive circuits: evidence for independent modulation of prefrontal functions by midbrain dopaminergic and noradrenergic neurons. Front Neural Circuits, 8, 2014, 53.
18. Clarke, H.F., Dalley, J.W., Crofts, H.S., Robbins, T.W., Roberts, A.C., Cognitive inflexibility after prefrontal serotonin depletion. Science 304 (2004), 878–880.
19. Cohen, J.D., McClure, S.M., Yu, A.J., Should I stay or should I go? How the human brain manages the trade-off between exploitation and exploration. Philos Trans R Soc Lond B Biol Sci 362 (2007), 933–942.
20. Cools, R., Barker, R.A., Sahakian, B.J., Robbins, T.W., Enhanced or impaired cognitive function in Parkinson's disease as a function of dopaminergic medication and task demands. Cereb Cortex 11 (2001), 1136–1143.
21. Courville, A.C., Daw, N.D., Touretzky, D.S., Bayesian theories of conditioning in a changing world. Trends Cogn Sci 10 (2006), 294–300.
22. Crofts, H.S., Dalley, J.W., Collins, P., Van Denderen, J.C., Everitt, B.J., Robbins, T.W., Roberts, A.C., Differential effects of 6-OHDA lesions of the frontal cortex and caudate nucleus on the ability to acquire an attentional set. Cereb Cortex 11 (2001), 1015–1026.
23. Dayan, P., Balleine, B.W., Reward, motivation, and reinforcement learning. Neuron 36 (2002), 285–298.
24. Deco, G., Rolls, E.T., Albantakis, L., Romo, R., Brain mechanisms for perceptual and reward-related decision-making. Prog Neurobiol 103 (2013), 194–213.
25. Doya, K., Modulators of decision making. Nat Neurosci 11 (2008), 410–416.
26. Evers, E.A., Cools, R., Clark, L., van der Veen, F.M., Jolles, J., Sahakian, B.J., Robbins, T.W., Serotonergic modulation of prefrontal cortex during negative feedback in probabilistic reversal learning. Neuropsychopharmacology 30 (2005), 1138–1147.
27. Furr, A., Lapiz-Bluhm, M.D., Morilak, D.A., 5-HT2A receptors in the orbitofrontal cortex facilitate reversal learning and contribute to the beneficial cognitive effects of chronic citalopram treatment in rats. Int J Neuropsychopharmacol 15 (2012), 1295–1305.
28. Gershman, S.J., Blei, D.M., Niv, Y., Context, learning, and extinction. Psychol Rev 117 (2010), 197–209.
29. Gershman, S.J., Jones, C.E., Norman, K.A., Monfils, M.H., Niv, Y., Gradual extinction prevents the return of fear: implications for the discovery of state. Front Behav Neurosci, 7, 2013, 164.
30. Ghods-Sharifi, S., Haluk, D.M., Floresco, S.B., Differential effects of inactivation of the orbitofrontal cortex on strategy set-shifting and reversal learning. Neurobiol Learn Mem 89 (2008), 567–573.
31. Greuel, J.M., Luhmann, H.J., Singer, W., Pharmacological induction of use-dependent receptive field modifications in the visual cortex. Science 242 (1988), 74–77.
32. Hasselmo, M.E., Linster, C., Patil, M., Ma, D., Cekic, M., Noradrenergic suppression of synaptic transmission may influence cortical signal-to-noise ratio. J Neurophysiol 77 (1997), 3326–3339.
33. Hermans, E.J., van Marle, H.J., Ossewaarde, L., Henckens, M.J., Qin, S., van Kesteren, M.T., Schoots, V.C., Cousijn, H., Rijpkema, M., Oostenveld, R., Fernandez, G., Stress-related noradrenergic activity prompts large-scale neural network reconfiguration. Science 334 (2011), 1151–1153.
34. Hyman, S.E., Addiction: a disease of learning and memory. Am J Psychiatry 162 (2005), 1414–1422.
35. Janak, P.H., Corbit, L.H., Deepened extinction following compound stimulus presentation: noradrenergic modulation. Learn Memory 18 (2011), 1–10.
36. Jepma, M., Nieuwenhuis, S., Pupil diameter predicts changes in the exploration-exploitation trade-off: evidence for the adaptive gain theory. J Cogn Neurosci 23 (2011), 1587–1596.
37. Jones, L.M., Fontanini, A., Sadacca, B.F., Miller, P., Katz, D.B., Natural stimuli evoke dynamic sequences of states in sensory cortical ensembles. Proc Natl Acad Sci USA 104 (2007), 18772–18777.
38. Kemere, C., Santhanam, G., Yu, B.M., Afshar, A., Ryu, S.I., Meng, T.H., Shenoy, K.V., Detecting neural-state transitions using hidden Markov models for motor cortical prostheses. J Neurophysiol 100 (2008), 2441–2452.
39. Lara, A.H., Kennerley, S.W., Wallis, J.D., Encoding of gustatory working memory by orbitofrontal neurons. J Neurosci 29 (2009), 765–774.
40. Lucantonio, F., Kambhampati, S., Haney, R.Z., Atalayer, D., Rowland, N.E., Shaham, Y., Schoenbaum, G., Effects of prior cocaine versus morphine or heroin self-administration on extinction learning driven by overexpectation versus omission of reward. Biol Psychiatry 77 (2015), 912–920.
41. Mason, S.T., Fibiger, H., Noradrenaline, fear and extinction. Brain Res 165 (1979), 47–56.
42. McCormick, D.A., Thompson, R.F., Locus coeruleus lesions and resistance to extinction of a classically conditioned response: involvement of the neocortex and hippocampus. Brain Res 245 (1982), 239–249.
43. McGaughy, J., Ross, R.S., Eichenbaum, H., Noradrenergic, but not cholinergic, deafferentation of prefrontal cortex impairs attentional set-shifting. Neuroscience 153 (2008), 63–71.
44. Merlo, A.B., Izquierdo, I., The effect of catecholamines on learning in rats. Med Pharmacol Exp Int J Exp Med 16 (1967), 343–349.
45. Moran, A., Katz, D.B., Sensory cortical population dynamics uniquely track behavior across learning and extinction. J Neurosci 34 (2014), 1248–1257.
46. Mueller, D., Cahill, S.P., Noradrenergic modulation of extinction learning and exposure therapy. Behav Brain Res 208 (2010), 1–11.
47. Nassar, M.R., Rumsey, K.M., Wilson, R.C., Parikh, K., Heasly, B., Gold, J.I., Rational regulation of learning dynamics by pupil-linked arousal systems. Nat Neurosci 15 (2012), 1040–1046.
48. Nomura, M., Effects of bifemelane on discrimination learning of serotonergic-dysfunction rats. Pharmacol Biochem Behav 42 (1992), 721–731.
49. Payzan-LeNestour, E., Dunne, S., Bossaerts, P., O'Doherty, J.P., The neural representation of unexpected uncertainty during value-based decision making. Neuron 79 (2013), 191–201.
50. Ramus, S.J., Eichenbaum, H., Neural correlates of olfactory recognition memory in the rat orbitofrontal cortex. J Neurosci 20 (2000), 8199–8208.
51. Redish, A.D., Jensen, S., Johnson, A., Kurth-Nelson, Z., Reconciling reinforcement learning models with behavioral extinction and renewal: implications for addiction, relapse, and problem gambling. Psychol Rev 114 (2007), 784–805.
52. Ridley, R.M., Haystead, T.A., Baker, H.F., An analysis of visual object reversal learning in the marmoset after amphetamine and haloperidol. Pharmacol Biochem Behav 14 (1981), 345–351.
53. Rudebeck, P.H., Murray, E.A., Amygdala and orbitofrontal cortex lesions differentially influence choices during object reversal learning. J Neurosci 28 (2008), 8338–8343.
54. Schoenbaum, G., Eichenbaum, H., Information coding in the rodent prefrontal cortex. I. Single-neuron activity in orbitofrontal cortex compared with that in pyriform cortex. J Neurophysiol 74 (1995), 733–750.
55. Schoenbaum, G., Chiba, A.A., Gallagher, M., Neural encoding in orbitofrontal cortex and basolateral amygdala during olfactory discrimination learning. J Neurosci 19 (1999), 1876–1884.
56. Schoenbaum, G., Nugent, S.L., Saddoris, M.P., Setlow, B., Orbitofrontal lesions in rats impair reversal but not acquisition of go, no-go odor discriminations. NeuroReport 13 (2002), 885–890.
57. Schoenbaum, G., Saddoris, M.P., Ramus, S.J., Shaham, Y., Setlow, B., Cocaine-experienced rats exhibit learning deficits in a task sensitive to orbitofrontal cortex lesions. Eur J Neurosci 19 (2004), 1997–2002.
58. Schwarz, L.A., Miyamichi, K., Gao, X.J., Beier, K.T., Weissbourd, B., DeLoach, K.E., Ren, J., Ibanes, S., Malenka, R.C., Kremer, E.J., Luo, L., Viral-genetic tracing of the input-output organization of a central noradrenaline circuit. Nature 524 (2015), 88–92.
59. Seidemann, E., Meilijson, I., Abeles, M., Bergman, H., Vaadia, E., Simultaneously recorded single units in the frontal cortex go through sequences of discrete and stable states in monkeys performing a delayed localization task. J Neurosci 16 (1996), 752–768.
60. Seu, E., Lang, A., Rivera, R.J., Jentsch, J.D., Inhibition of the norepinephrine transporter improves behavioral flexibility in rats and monkeys. Psychopharmacology 202 (2009), 505–519.
61. Shea-Brown, E., Gilzenrat, M.S., Cohen, J.D., Optimization of decision making in multilayer networks: the role of locus coeruleus. Neural Comput 20 (2008), 2863–2894.
62. Stalnaker, T.A., Franz, T.M., Singh, T., Schoenbaum, G., Basolateral amygdala lesions abolish orbitofrontal-dependent reversal impairments. Neuron 54 (2007), 51–58.
63. Stalnaker, T.A., Takahashi, Y., Roesch, M.R., Schoenbaum, G., Neural substrates of cognitive inflexibility after chronic cocaine exposure. Neuropharmacology 56:Suppl 1 (2009), 63–72.
64. Stalnaker, T.A., Cooch, N.K., Schoenbaum, G., What the orbitofrontal cortex does not do. Nat Neurosci 18 (2015), 620–627.
65. Sutton RS (1984) Temporal credit assignment in reinforcement learning.
66. Teitelbaum, H., A comparison of effects of orbitofrontal and hippocampal lesions upon discrimination learning and reversal in the cat. Exp Neurol 9 (1964), 452–462.
67. Tervo, D.G., Proskurin, M., Manakov, M., Kabra, M., Vollmer, A., Branson, K., Karpova, A.Y., Behavioral variability through stochastic choice and its gating by anterior cingulate cortex. Cell 159 (2014), 21–32.
68. Vanes, L.D., van Holst, R.J., Jansen, J.M., van den Brink, W., Oosterlaan, J., Goudriaan, A.E., Contingency learning in alcohol dependence and pathological gambling: learning and unlearning reward contingencies. Alcohol Clin Exp Res 38 (2014), 1602–1610.
69. Wallace, J., Jackson, R.K., Shotton, T.L., Munjal, I., McQuade, R., Gartside, S.E., Characterization of electrically evoked field potentials in the medial prefrontal cortex and orbitofrontal cortex of the rat: modulation by monoamines. Eur Neuropsychopharmacol 24 (2014), 321–332.
70. Wilson, R.C., Takahashi, Y.K., Schoenbaum, G., Niv, Y., Orbitofrontal cortex as a cognitive map of task space. Neuron 81 (2014), 267–279.
71. Yu, A.J., Dayan, P., Uncertainty, neuromodulation, and attention. Neuron 46 (2005), 681–692.