Focus on the positive: Computational simulations implicate asymmetrical reward prediction error signals in childhood attention-deficit/hyperactivity disorder

Brain Research

Volumn 1365, Issue , 2010, Pages 18-34

  Cockburn, Jeffrey ^a   Holroyd, Clay B ^b  

^a BROWN UNIVERSITY   (United States)

^b UNIVERSITY OF VICTORIA   (Canada)

Author keywords

ADHD; Dopamine; Model; Reinforcement learning; Temporal difference

Indexed keywords

DOPAMINE;

ARTICLE; ATTENTION DEFICIT DISORDER; BEHAVIORAL RESEARCH; BRAIN ASYMMETRY; COMPUTER SIMULATION; DISEASE ASSOCIATION; HYPERACTIVITY; IMPULSIVENESS; LEARNING; MEDICAL ERROR; NONHUMAN; PRIORITY JOURNAL; ANIMAL; CHILD; DISEASE MODEL; HUMAN; NERVE TRACT; PATHOPHYSIOLOGY; PHYSIOLOGY; PSYCHOLOGICAL ASPECT; PSYCHOLOGICAL MODEL; PSYCHOMOTOR PERFORMANCE; RAT; REWARD; SPONTANEOUSLY HYPERTENSIVE RAT; WISTAR KYOTO RAT; BEHAVIOR CHANGE; CHILD BEHAVIOR; CONDITIONING; CONFERENCE PAPER; CONTROLLED STUDY; DOPAMINERGIC SYSTEM; MEASUREMENT ERROR; REINFORCEMENT; SIMULATION;

ANIMALIA; RATTUS;

ANIMALS; ATTENTION DEFICIT DISORDER WITH HYPERACTIVITY; CHILD; COMPUTER SIMULATION; DISEASE MODELS, ANIMAL; DOPAMINE; HUMANS; MODELS, PSYCHOLOGICAL; NEURAL PATHWAYS; PSYCHOMOTOR PERFORMANCE; RATS; RATS, INBRED SHR; RATS, INBRED WKY; REWARD;

EID: 78649506883     PISSN: 00068993     EISSN: None     Source Type: Journal    
DOI: 10.1016/j.brainres.2010.09.065     Document Type: Article

References (87)

1. H. Akaike A new look at the statistical model identification IEEE Trans. Autom. Control 19 1974 716 723
2. B. Alsop Problems with spontaneously hypertensive rats (SHR) as a model of attention-deficit/hyperactivity disorder (AD/HD) J. Neurosci. Meth. 162 2007 42 48 (Pubitemid 46527908)
3. A.F.T. Arnsten Stimulants: therapeutic actions in ADHD Neuropsychopharmacology 31 2006 2376 2383 official publication of the American College of Neuropsychopharmacology
4. R. Barkley ADHD and the Nature of Self-control 1997 The Guilford Press
5. R. Barkley Behavioral inhibition, sustained attention, and executive functions: constructing a unifying theory of ADHD Psychol. Bull. 121 1997 65 94
6. M.a. Bellgrove, K. Domschke, Z. Hawi, A. Kirley, C. Mullins, I.H. Robertson, and M. Gill The methionine allele of the COMT polymorphism impairs prefrontal cognition in children and adolescents with ADHD Exp. Brain Res. 163 2005 352 360 Experimentelle Hirnforschung. Expérimentation cérébrale
7. R. Bogacz, S.M. McClure, J. Li, J.D. Cohen, and P.R. Montague Short-term memory traces for action bias in human reinforcement learning Brain Res. 1153 2007 111 121
8. G. Bush, J. Frazier, S. Rauch, L. Seidman, P. Whalen, M. Jenike, B. Rosen, and J. Biederman Anterior cingulate cortex dysfunction in attention-deficit/hyperactivity disorder revealed by fMRI and the Counting Stroop Biol. Psychiatry 45 1999 1542 1552
9. B. Casey, F. Castellanos, J. Giedd, W. Marsh, S. Hamburger, A. Schubert, Y. Vauss, A. Vaituzis, D. Dickstein, and S. Sarfatti Implication of right frontostriatal circuitry in response inhibition and attention-deficit/ hyperactivity disorder J. Am. Acad. Child Adolesc. Psychiatry 36 1997 374
10. F. Castellanos Toward a pathophysiology of attention-deficit/ hyperactivity disorder Clin. Pediatr. 36 1997 381 393
11. F.X. Castellanos, E.J.S. Sonuga-Barke, M.P. Milham, and R. Tannock Characterizing cognition in ADHD: beyond executive dysfunction Trends Cogn. Sci. 10 2006 117 123
12. C. Catania, T. Sagvolden, and K. Keller Reinforcement schedules: retroactive and proactive effects of reinforcers inserted into fixed-interval performances J. Exp. Anal. Behav. 49 1988 49 73
13. G. Chelune, W. Ferguson, R. Koon, and T. Dickey Frontal lobe disinhibition in attention deficit disorder Child Psychiatry Hum. Dev. 16 1986 221 234
14. D. Coghill, J. Nigg, A. Rothenberger, E. Sonuga-Barke, and R. Tannock Whither causal models in the neuroscience of ADHD? Dev. Sci. 8 2005 105 114
15. M.X. Cohen, and M.J. Frank Neurocomputational models of basal ganglia function in learning, memory and choice Behav. Brain Res. 199 2009 141 156
16. S.J. Cragg, and M.E. Rice DAncing past the DAT at a DA synapse Trends Neurosci. 27 2004 270 277
17. P. Dayan, and T.J. Sejnowski TD() converges with probability 1 Mach. Learn. 14 1994 295 301
18. A.P.A.T.F. DSM-IV DSM-IV: Diagnostic and Statistical Manual of Mental Disorders 1994 American Psychiatric Association Washington DC
19. D.M. Egelman, C. Person, and P.R. Montague A computational role for dopamine delivery in human decision-making J. Cogn. Neurosci. 10 1998 623 630
20. E. Flores-Barrera, B.J. Vizcarra-Chacón, D. Tapia, J. Bargas, and E. Galarraga Different corticostriatal integration in spiny projection neurons from direct and indirect pathways Front. Syst. Neurosci. 4 2010 15
21. S.B. Floresco, A.R. West, B. Ash, H. Moore, and A.A. Grace Afferent modulation of dopamine neuron firing differentially regulates tonic and phasic dopamine transmission Nat. Neurosci. 6 2003 968 973
22. M.J. Frank Dynamic dopamine modulation in the basal ganglia: a neurocomputational account of cognitive deficits in medicated and nonmedicated Parkinsonism J. Cogn. Neurosci. 17 2005 51 72
23. M.J. Frank, A. Santamaria, R.C. O'Reilly, and E. Willcutt Testing computational models of dopamine and noradrenaline dysfunction in attention deficit/hyperactivity disorder Neuropsychopharmacology 32 2007 1583 1599 official publication of the American College of Neuropsychopharmacology
24. C.R. Gallistel, and J. Gibbon Time, rate, and conditioning Psychol. Rev. 107 2000 289 344
25. P. Garris, E. Ciolkowski, P. Pastore, and R.M. Wightman Efflux of dopamine from the synaptic cleft in the nucleus accumbens of the rat brain J. Neurosci. 4 1994 6084 6093
26. P.S. Goldman-Rakic, C. Leranth, M.S. Williams, N. Mons, and M. Geffard Dopamine synaptic complex with pyramidal neurons in primate cerebral cortex Proc. Natl Acad. Sci. USA 86 1989 9015 9019
27. F.G. Gonon Nonlinear relationship between impulse flow and dopamine released by rat midbrain dopaminergic neurons as studied by in vivo electrochemistry Neuroscience 24 1988 19 28
28. E. Gorenstein, C. Mammato, and J. Sandy Performance of inattentive-overactive children on selected measures of prefrontal-type function J. Clin. Psychol. 45 1989 619 632
29. Y. Goto, and A.A. Grace Limbic and cortical information processing in the nucleus accumbens Trends Neurosci. 31 2008 552 558
30. A.A. Grace Phasic versus tonic dopamine release and the modulation of dopamine system responsivity: a hypothesis for the etiology of schizophrenia Neuroscience 41 1991 1 24
31. A.A. Grace The tonic/phasic model of dopamine system regulation: its relevance for understanding how stimulant abuse can alter basal ganglia function Drug Alcohol Depend. 37 1995 111 129
32. A.A. Grace Psychostimulant actions on dopamine and limbic system function: relevance to the pathophysiology and treatment of ADHD Stimulant Drugs and ADHD: Basic and Clinical Neuroscience 2001 Oxford University Press New York 134 157 chapter 5.
33. A.A. Grace, and B.S. Bunney Intracellular and extracellular electrophysiology of nigral dopaminergic neurons-1. Identification and characterization Neuroscience 10 1983 301 315
34. G. Grodzinsky, and R. Diamond Frontal lobe functioning in boys with attention-deficit hyperactivity disorder Dev. Neuropsychol. 8 1992 427-427
35. D.P. Hanes, and J.D. Schall Neural control of voluntary movement initiation Science 274 1996 427 430 (New York, N.Y.)
36. S. Hernández-López, J. Bargas, D.J. Surmeier, a. Reyes, and E. Galarraga D1 receptor activation enhances evoked discharge in neostriatal medium spiny neurons by modulating an L-type Ca2+ conductance J. Neurosci. 17 1997 3334 3342 the official journal of the Society for Neuroscience
37. C.B. Holroyd, T.E. Baker, K.a. Kerns, and U. Müller Electrophysiological evidence of atypical motivation and reward processing in children with attention-deficit hyperactivity disorder Neuropsychologia 46 2008 2234 2242
38. K. Krause The dopamine transporter and neuroimaging in attention deficit hyperactivity disorder Neurosci. Biobehav. Rev. 27 2003 605 613
39. R. Kuczenski, and D.S. Segal Effects of methylphenidate on extracellular dopamine, serotonin, and norepinephrine: comparison with amphetamine J. Neurochem. 68 1997 2032 2037 (Pubitemid 27176139)
40. H. Lou, L. Henriksen, P. Bruhn, H. Borner, and J. Nielsen Striatal dysfunction in attention deficit and hyperkinetic disorder Arch. Neurol. 46 1989 48
41. M. Luman, J. Oosterlaan, and J.a Sergeant The impact of reinforcement contingencies on AD/HD: a review and theoretical appraisal Clin. Psychol. Rev. 25 2005 183 213
42. M. Luman, G. Tripp, and A. Scheres Identifying the neurobiology of altered reinforcement sensitivity in ADHD: a review and research agenda Neurosci. Biobehav. Rev. 34 2010 744 754
43. S.M. McClure, N.D. Daw, and P. Read Montague A computational substrate for incentive salience Trends Neurosci. 26 2003 423 428
44. P. Montague, P. Dayan, and T. Sejnowski A framework for mesencephalic dopamine systems based on predictive Hebbian learning J. Neurosci. 16 1996 1936
45. P.R. Montague, S.E. Hyman, and J.D. Cohen Computational roles for dopamine in behavioural control Nature 431 2004 760 767
46. G. Morris, A. Nevet, D. Arkadir, E. Vaadia, and H. Bergman Midbrain dopamine neurons encode decisions for future action Nat. Neurosci. 9 2006 1057 1063
47. Y. Niv, N.D. Daw, D. Joel, and P. Dayan Tonic dopamine: opportunity costs and the control of response vigor Psychopharmacology 191 2007 507 520
48. P. O'Donnell, and A.A. Grace Synaptic interactions accumbens neurons: input among excitatory afferents to nucleus hippocampal gating of prefrontal cortical J. Neurosci. 15 1995 3622 3639
49. P. Pedarzani, and J.F. Storm Dopamine modulates the slow Ca(2+)-activated K + current IAHP via cyclic AMP-dependent protein kinase in hippocampal neurons J. Neurophysiol. 74 1995 2749 2753 (Pubitemid 26012330)
50. D. Redish Addiction as a computational process gone awry Science 306 2004 1944 1947 (New York, N.Y.)
51. K. Rubia, S. Overmeyer, E. Taylor, M. Brammer, S. Williams, A. Simmons, and E. Bullmore Hypofrontality in attention deficit hyperactivity disorder during higher-order motor control: a study with functional MRI Am. J. Psychiatry 156 1999 891
52. T. Sagvolden Behavioral validation of the spontaneously hypertensive rat (SHR) as an animal model of attention-deficit/hyperactivity disorder (AD/HD) Neurosci. Biobehav. Rev. 24 2000 31 39
53. T. Sagvolden, E.D. Hendley, and S. Knardahl Behavior of hypertensive and hyperactive rat strains: hyperactivity is not unitarily determined Physiol. Behav. 52 1992 49 57
54. T. Sagvolden, H. Aase, P. Zeiner, and D. Berger Altered reinforcement mechanisms in attention-deficit/hyperactivity disorder Behav. Brain Res. 94 1998 61 71
55. T. Sagvolden, E.B. Johansen, H. Aase, and V.A. Russell A dynamic developmental theory of attention-deficit/hyperactivity disorder (ADHD) predominantly hyperactive/impulsive and combined subtypes Behav. Brain Sci. 28 2005 397 419 discussion 419-68
56. T. Sagvolden, V.a. Russell, H. Aase, E.B. Johansen, and M. Farshbaf Rodent models of attention-deficit/hyperactivity disorder Biol. Psychiatry 57 2005 1239 1247
57. S. Sajikumar, and J.U. Frey Late-associativity, synaptic tagging, and the role of dopamine during LTP and LTD Neurobiol. Learn. Mem. 82 2004 12 25
58. L. Scahill, and M. Schwab-Stone Epidemiology of ADHD in school-age children Child Adolesc. Psychiatr. Clin. N. Am. 9 2000 541 555
59. R. Schachar, S. Chen, G. Logan, T. Ornstein, J. Crosbie, A. Ickowicz, and A. Pakulak Evidence for an error monitoring deficit in attention deficit hyperactivity disorder J. Abnorm. Child Psychol. 32 2004 285 293 (Pubitemid 39004756)
60. W.K. Schiffer, N.D. Volkow, J.S. Fowler, D.L. Alexoff, J. Logan, and S.L. Dewey Therapeutic doses of amphetamine or methylphenidate differentially increase synaptic and extracellular dopamine Synapse 59 2006 243 251 (New York, N.Y.)
61. W. Schultz Getting formal with dopamine and reward Neuron 36 2002 241 263
62. W. Schultz, P. Dayan, and P.R. Montague A neural substrate of prediction and reward Science 275 1997 1593 1599 (New York, N.Y.)
63. P. Seeman, and B. Madras Methylphenidate elevates resting dopamine which lowers the impulse-triggered release of dopamine: a hypothesis Behav. Brain Res. 130 2002 79 83
64. J. Sergeant Modeling attention-deficit/hyperactivity disorder: a critical appraisal of the cognitive-energetic model Biol. Psychiatry 57 2005 1248 1255
65. J. Sergeant, and J. Van Der Meere What happens after a hyperactive child commits an error? Psychiatry Res. 24 1988 157 164
66. D. Servan-Schreiber, H. Printz, and J. Cohen A network model of catecholamine effects: gain, signal-to-noise ratio, and behavior Science 249 1990 892
67. A. Smith, and J. Bolam The neural network of the basal ganglia as revealed by the study of synaptic connections of identified neurones Trends Neurosci. 13 1990 259 265
68. M.V. Solanto Neuropsychopharmacological mechanisms of stimulant drug action in attention-deficit hyperactivity disorder: a review and integration Behav. Brain Res. 94 1998 127 152
69. M.S. Sonders, S.J. Zhu, N.R. Zahniser, M.P. Kavanaugh, and S.G. Amara Multiple ionic conductances of the human dopamine transporter: the actions of dopamine and psychostimulants J. Neurosci. 17 1997 960 974 the official journal of the Society for Neuroscience
70. E. Sonuga-Barke Psychological heterogeneity in AD/HD-a dual pathway model of behaviour and cognition Behav. Brain Res. 130 2002 29 36
71. E.J. Sonuga-Barke, E. Taylor, S. Sembi, and J. Smith Hyperactivity and delay aversion-I. The effect of delay on choice J. Child Psychol. Psychiatr. Allied disciplines 33 1992 387 398
72. P. Sutker, and H. Adams Comprehensive Handbook of Psychopathology 3rd edition 2001 Plenum Pub Corp
73. R.S. Sutton, and A.G. Barto Introduction to Reinforcement Learning 1998 MIT Press Cambridge MA, USA
74. J.M. Swanson, P. Flodman, J. Kennedy, Ma. Spence, R. Moyzis, S. Schuck, M. Murias, J. Moriarity, C. Barr, M. Smith, and M. Posner Dopamine genes and ADHD Neurosci. Biobehav. Rev. 24 2000 21 25
75. E. Taylor Syndromes of attention deficit and overactivity Child Adolesc. Psychiatry 3 1994 285 308 Modern approaches
76. G. Tripp, and J.R. Wickens Research review: Dopamine transfer deficit: a neurobiological theory of altered reinforcement mechanisms in ADHD J. Child Psychol. Psychiatry 7 2008 691 704
77. B. Vitiello, J. Severe, L. Greenhill, L. Arnold, H. Abikoff, O. Bukstein, G. Elliott, L. Hechtman, P. Jensen, and S. Hinshaw Methylphenidate dosage for children with ADHD over time under controlled conditions: lessons from the MTA J. Am. Acad. Child Adolesc. Psychiatry 40 2001 188
78. N.D. Volkow, G.J. Wang, J.S. Fowler, S.J. Gatley, J. Logan, Y.S. Ding, R. Hitzemann, and N. Pappas Dopamine transporter occupancies in the human brain induced by therapeutic doses of oral methylphenidate Am. J. Psychiatry 155 1998 1325 1331
79. N.D. Volkow, G.J. Wang, J.S. Fowler, F. Telang, L. Maynard, J. Logan, S.J. Gatley, N. Pappas, C. Wong, P. Vaska, W. Zhu, and J.M. Swanson Evidence that methylphenidate enhances the saliency of a mathematical task by increasing dopamine in the human brain Am. J. Psychiatry 161 2004 1173 1180
80. N.D. Volkow, G.J. Wang, J.S. Fowler, and Y.S. Ding Imaging the effects of methylphenidate on brain dopamine: new model on its therapeutic actions for attention-deficit/hyperactivity disorder Biol. Psychiatry 57 2005 1410 1415
81. I.D. Waldman, D.C. Rowe, a. Abramowitz, S.T. Kozel, J.H. Mohr, S.L. Sherman, H.H. Cleveland, M.L. Sanders, J.M. Gard, and C. Stever Association and linkage of the dopamine transporter gene and attention-deficit hyperactivity disorder in children: heterogeneity owing to diagnostic subtype and severity Am. J. Hum. Genet. 63 1998 1767 1776
82. J. Wickens, A. Begg, and G. Arbuthnott Dopamine reverses the depression of rat corticostriatal synapses which normally follows high-frequency stimulation of cortex in vitro Neuroscience 70 1996 1 5
83. T.E. Wilens, T.J. Spencer, J. Biederman, K. Girard, R. Doyle, J. Prince, D. Polisner, R. Solhkhah, S. Comeau, M.C. Monuteaux, and A. Parekh A controlled clinical trial of bupropion for attention deficit hyperactivity disorder in adults Am. J. Psychiatry 158 2001 282 288
84. T.E. Wilens, S.V. Faraone, and J. Biederman Attention-deficit/ hyperactivity disorder in adults JAMA 292 2004 619 623 the journal of the American Medical Association
85. J. Williams, and P. Dayan Dopamine, learning, and impulsivity: a biological account of attention-deficit/hyperactivity disorder J. Child Adolesc. Psychopharmacol. 15 2005 160 179 discussion 157-9
86. J. Williams, G. Sagvolden, E. Taylor, and T. Sagvolden Dynamic behavioural changes in the spontaneously hyperactive rat: 1. Control by place, timing, and reinforcement rate Behav. Brain Res. 198 2009 273 282
87. L. Zhang, W.M. Doyon, J.J. Clark, P.E.M. Phillips, and J.A. Dani Controls of tonic and phasic dopamine transmission in the dorsal and ventral striatum Mol. Pharmacol. 76 2009 396 404