The role of the noradrenergic system in the exploration-exploitation trade-off: A psychopharmacological study

Frontiers in Human Neuroscience

Volumn 4, Issue , 2010, Pages

  Jepma, Marieke ^a   te Beek, Erik T ^b   Wagenmakers, Eric Jan ^c   van Gerven, Joop M A ^b   Nieuwenhuis, Sander ^a  

^a LEIDEN UNIVERSITY   (Netherlands)

^b CENTRE FOR HUMAN DRUG RESEARCH   (Netherlands)

^c UNIVERSITY OF AMSTERDAM   (Netherlands)

Author keywords

Cognitive control; Decision making; Exploration; Locus coeruleus; Norepinephrine; Reboxetine

Indexed keywords

CITALOPRAM; NORADRENALIN; PLACEBO; REBOXETINE;

ADAPTIVE GAIN THEORY; ADULT; ARTICLE; AUTONOMIC NERVOUS SYSTEM; BEHAVIOR; BEHAVIOR THEORY; CENTRAL NERVOUS SYSTEM; COGNITION; CONTROLLED STUDY; DECISION MAKING; EXPLOITATIVE BEHAVIOR; EXPLORATORY BEHAVIOR; FEMALE; HUMAN; HUMAN EXPERIMENT; LOCUS CERULEUS; MALE; NORADRENERGIC SYSTEM; PSYCHOPHARMACOLOGY; REINFORCEMENT; REWARD; TASK PERFORMANCE;

EID: 79953000602     PISSN: 16625161     EISSN: 16625161     Source Type: Journal    
DOI: 10.3389/fnhum.2010.00170     Document Type: Article

References (55)

1. Alvarez, V. A., Chow, C. C., Van Bockstaele, E. J., and Williams, J. T. (2002). Frequency-dependent synchrony in locus ceruleus: role of electrotonic coupling. Proc. Natl. Acad. Sci. U.S.A. 99, 4032-4036.
2. Aston-Jones, G. (2004). "Locus coeruleus, A5 and A7 noradrenergic cell groups," In The Rat Nervous System, 3rd Edn. ed. G. Paxinos. (San Diego: Elsevier Academic Press), 259-294.
3. Aston-Jones, G., and Cohen, J. D. (2005). An integrative theory of locus coeruleus- norepinephrine function: adaptive gain and optimal performance. Annu. Rev. Neurosci. 28, 403-450.
4. Aston-Jones, G., Foote, S. L., and Bloom, F. E. (1984). "Anatomy and physiology of locus coeruleus neurons: functional implications," in Frontiers of Clinical Neuroscience: Vol. 2, Norepinephrine, eds M. Ziegler and C. R. Lake (Baltimore: Williams and Wilkins), 92-116.
5. Aston-Jones, G., Rajkowski, J., and Kubiak, P. (1997). Conditioned responses of monkey locus coeruleus neurons anticipate acquisition of discriminative behavior in a vigilance task. Neuroscience 80, 697-715.
6. Aston-Jones, G., Rajkowski, J., Kubiak, P., Valentino, R. J., and Shipley, M. T. (1996). Role of the locus coeruleus in emotional activation. Prog. Brain Res. 107, 379-402.
7. Behrens, T. E., Woolrich, M W., Walton, M. E., and Rushworth, M. F. (2007). Learning the value of information in an uncertain world. Nat. Neurosci. 10, 1214-1221.
8. Berntson, G. G., Sarter, M., and Cacioppo, J. T. (1998). Anxiety and cardiovascular reactivity: the basal forebrain cholinergic link. Behav. Brain Res. 94, 225-248.
9. Berridge, C. W., and Waterhouse, B. D. (2003). The locus coeruleus-noradrenergic system: modulation of behavioral state and state-dependent cognitive processes. Brain Res. Brain Res. Rev. 42, 33-84.
10. Bond, A., and Lader, M. (1974). The use of analogue scales in rating subjective feelings. Br. J. Psychol. 47, 211-218.
11. Borland, R. G., and Nicholson, A. N. (1984). Visual motor co-ordination and dynamic visual acuity. Br. J. Clin. Pharmacol. 18(Suppl. 1), 69S-72S.
12. Bouret, S., and Sara, S. J. (2005). Network reset: a simplified overarching theory of locus coeruleus noradrenaline function. Trends Neurosci. 28, 574-582.
13. Bymaster, F. P., Zhang, W., Carter, P. A., Shaw, J., Chernet, E., Phebus, L., Wong, D. T., and Perry, K. W. (2002). Fluoxetine, but not other selective serotonin uptake inhibitors, increases norepinephrine and dopamine extracellular levels in prefrontal cortex. Psychopharmacology (Berl.) 160, 353-361.
14. Chamberlain, S. R., Müller, U., Blackwell, A. D., Clark, L., Robbins, T. W., and Sahakian, B. J. (2006). Neurochemical modulation of response inhibition and probabilistic learning in humans. Science 311, 861-863.
15. Clayton, E. C., Rajkowski, J., Cohen, J. D., and Aston-Jones, G. (2004). Phasic activation of monkey locus coeruleus neurons by simple decisions in a forced choice task. J. Neurosci. 24, 9914-9920.
16. Cohen, J. D., McClure, S. M., and Yu, A. J. (2007). Should I stay or should I go? How the human brain manages the trade-off between exploitation and exploration. Philos. Trans. R. Soc. Lond. B Biol. Sci. 362, 933-942.
17. Daw, N. D., O'Doherty, J. P., Dayan, P., Seymour, B., and Dolan, R. J. (2006). Cortical substrates for exploratory decisions in humans. Nature 441, 876-479.
18. De Martino, B., Strange, B. A., and Dolan, R. J. (2008). Noradrenergic neuromodulation of human attention for emotional and neutral stimuli. Psychopharmacology (Berl.) 197, 127-136.
19. Devauges, V., and Sara, S. J. (1990). Activation of the noradrenergic system facilitates an attentional shift in the rat. Behav. Brain Res. 39, 19-28.
20. Dostert, P., Benedetti, M. S., and Poggesi, I. (1997). Review of the pharmacokinetics and metabolism of reboxetine, a selective noradrenaline reuptake inhibitor. Eur. Neuropsychopharmacol. 7(Suppl. 1), S23-S35.
21. Doya, K. (2002). Metalearning and neuromodulation. Neural Netw. 15, 495-506.
22. Edwards, D., M. Pellizzoni, C. Breuel, H. P. Berardi, A. Castelli, M. G. Frigerio, E. Poggesi, I. Rocchetti, M. Dubini, A. and Strolin Benedetti, M. (1995). Pharmacokinetics of reboxetine in healthy volunteers. Single oral doses, linearity and plasma protein binding. Biopharm. Drug Dispos. 16, 443-460.
23. Frank, M. J., Doll, B. B., Oas-Terpstra, J., and Moreno, F. (2009). Prefrontal and striatal dopaminergic genes predict individual differences in exploration and exploitation. Nat. Neurosci. 12, 1062-1068.
24. Gilzenrat, M. S., Nieuwenhuis, S., Jepma, M., and Cohen, J. D. (2010). Pupil diameter tracks changes in control state predicted by the adaptive gaintheory of locus coeruleus function. Cogn. Affect Behav. Neurosci. 10, 252-269.
25. Hyttel, J. (1994). Pharmacological characterization of selective serotonin reuptake inhibitors (SSRIs). Int. Clin. Psychopharmacol. 9(Suppl. 1), 19-26.
26. Invernizzi, R. W., and Garattini, S. (2004). Role of presynaptic alpha2-adrenoceptors in antidepressant action: recent findings from microdialysis studies. Prog. Neuropsychopharmacol. Biol. Psychiatry 28, 819-827.
27. Itil, T. M., Menon, G. N., Bozak, M. M., and Itil, K. Z. (1984). CNS effects of citalopram, a new serotonin inhibitor antidepressant (a quantitative pharmaco-electroencephalography study). Prog. Neuropsychopharmacol. Biol. Psychiatry 8, 397-409.
28. Jepma, M., and Nieuwenhuis, S. (in press). Pupil diameter predicts changes in the exploration-exploitation trade-off: Evidence for the adaptive gain theory. J. Cogn. Neurosci.
29. Jouvet, M. (1969). Biogenic amines and the states of sleep. Science 163, 32-41.
30. Kubiak, P., Rajkowski, J., and Aston-Jones, G. (1992). Behavioral performance and sensory responsiveness of LC neurons in a vigilance task varies with tonic LC discharge rate. Soc. Neurosci. Abstr. 18, 538.
31. Lapiz, M. D., Bondi, C. O., and Morilak, D. A. (2007). Chronic treatment with desipramine improves cognitive performance of rats in an attentional setshifting test. Neuropsychopharmacology (Berl. Ger.) 32, 1000-1010.
32. Lapiz, M. D., and Morilak, D. A. (2006). Noradrenergic modulation of cognitive function in rat medial prefrontal cortex as measured by attentional set shifting capability. Neuroscience 137, 1039-1049.
33. Luksys, G., Gerstner, W., and Sandi, C. (2009). Stress, genotype and norepinephrine in the prediction of mouse behavior using reinforcement learning. Nat. Neurosci. 12, 1180-1186.
34. McGaughy, J., Ross, R. S., and Eichenbaum, H. (2008). Noradrenergic, but notcholinergic, deafferentation of prefrontal cortex impairs attentional set-shifting. Neuroscience 153, 63-71.
35. Miskowiak, K., Papadatou-Pastou, M., Cowen, P. J., Goodwin, G. M., Norbury, R., and Harmer, C. J. (2007). Single dose antidepressant administration modulates the neural processing of self-referent personality trait words. Neuroimage 37, 904-911.
36. Newman, L. A., Darling, J., and McGaughy, J. (2008). Atomoxetine reverses attentional deficits produced by noradrenergic deafferentation of medial prefrontal cortex. Psychopharmacology (Berl.) 200, 39-50.
37. Noble, S., and Benfield, P. (1997). Citalopram: a review of its pharmacology, clinical efficacy and tolerability in the treatment of depression. CNS Drugs 8, 410-431.
38. Page, M. E., and Lucki, I. (2002). Effects of acute and chronic reboxetine treatment on stress-induced monoamine efflux in the rat frontal cortex. Neuropsychopharmacology 27, 237-247.
39. Phillips, M. A., Bitsios, P., Szabadi, E., and Bradshaw, C. M. (2000). Comparison of the antidepressants reboxetine, fluvoxamine and amitriptyline upon spontaneous pupillary fluctuations in healthy human volunteers. Psychopharmacology (Berl.) 149, 72-76.
40. Saletu, B., Anderer, P., Saletu-Zyhlarz, G. M., Arnold, O., and Pascual-Marqui, R. D. (2002). Classification and evaluation of the pharmacodynamics of psychotropic drugs by single-lead pharmaco-EEG, EEG mapping and tomography (LORETA). Methods Find Exp. Clin. Pharmacol. 24(Suppl. C), 97S-120S.
41. Schmitt, J. A., Riedel, W. J., Vuurman, E. F., Kruizinga, M., and Ramaekers, J. G. (2002). Modulation of the critical flicker fusion effects of serotonin reuptake inhibitors by concomitant pupillary changes. Psychopharmacology (Berl.) 160, 381-386.
42. Servan-Schreiber, D., Printz, H., and Cohen, J. D. (1990). A network model of catecholamine effects: gain, signalto-noise ratio, and behavior. Science 249, 892-895.
43. Seu, E., Lang, A., Rivera, R. J., and Jentsch, J. D. (2008). Inhibition of the norepinephrine transporter improves behavioral flexibility in rats and monkeys. Psychopharmacology (Berl.) 202, 505-519.
44. Sutton, R. S., and Barto, A. G. (1998). Re inforcement Learning: An Introduction. Cambridge, MA: MIT Press.
45. Szabo, S. T., and Blier, P. (2001). Effect of the selective noradrenergic reuptake inhibitor reboxetine on the firing activity of noradrenaline and serotonin neurons. Eur. J. Neurosci. 13, 2077-2087.
46. Tait, D. S., Brown, V. J., Farovik, A., Theobald, D. E., Dalley, J. W., and Robbins, T. W. (2007). Lesions of the dorsal noradrenergic bundle impair attentional set-shifting in the rat. Eur. J. Neurosci. 25, 3719-3724.
47. Tse, W. S., and Bond, A. J. (2002). Difference in serotonergic and noradrenergic regulation of human social behaviours. Psychopharmacology (Berl.) 159, 216-221.
48. Twa, M. D., Bailey, M. D., Hayes, J., and Bullimore, M. (2004). Estimation of pupil size by digital photography. J. Cataract Refract. Surg. 30, 381-389.
49. Usher, M., Cohen, J. D., Servan-Schreiber, D., Rajkowski, J., and Aston-Jones, G. (1999). The role of locus coeruleus in the regulation of cognitive performance. Science 283, 549-554.
50. Van Steveninck, A. L., Cohen, A. F., and Ward, T. (1989). A microcomputer based system for recording and analysis of smooth pursuit and saccadic eye movements. Br. J. Clin. Pharmacol. 27, 712-713.
51. Van Steveninck, A. L., Gieschke, R., Schoemaker, H. C., Pieters, M. S., Kroon, J. M., Breimer, D. D., and Cohen, A. F. (1993). Pharmacodynamic interactions of diazepam and intravenous alcohol at pseudo steady state. Psychopharmacology (Berl.) 110, 471-478.
52. Van Steveninck, A. L., Schoemaker, H. C., Pieters, M. S., Kroon, R., Breimer, D. D., and Cohen, A. F. (1991). A comparison of the sensitivities of adaptive tracking, eye movement analysis and visual analog lines to the effects of incremental doses of temazepam in healthy volunteers. Clin. Pharmacol. Ther. 50, 172-180.
53. Van Steveninck, A. L., Van Berckel, B. N., Schoemaker, R. C., Breimer, D. D., Van Gerven, J. M., and Cohen, A. F. (1999). The sensitivity of pharmacodynamic tests for the central nervous system effects of drugs on the effects of sleep deprivation. J. Psychopharmacol. 13, 10-17.
54. Wang, L. E., Fink, G. R., Dafotakis, M., and Grefkes, C. (2009). Noradrenergic stimulation and motor performance: differential effects of reboxetine on movement kinematics and visuomotor abilities in healthy human subjects. Neuropsychologia 47, 1302-1312.
55. Yu, A. J., and Dayan, P. (2005). Uncertainty, neuromodulation, and attention. Neuron 46, 681-692.