Reinforcement learning and dopamine in schizophrenia: Dimensions of symptoms or specific features of a disease group?

Frontiers in Psychiatry

Volumn 4, Issue DEC, 2013, Pages

  Deserno, Lorenz ^a,b   Boehme, Rebecca ^b   Heinz, Andreas ^b   Schlagenhauf, Florian ^a,b  

^a MAX PLANCK INSTITUTE FOR HUMAN COGNITIVE AND BRAIN SCIENCES   (Germany)

^b CHARITÉ UNIVERSITÄTSMEDIZIN BERLIN   (Germany)

Author keywords

Aberrant salience; Computational modeling; Dopamine; FMRI; PET imaging; Prediction error; Reinforcement learning; Schizophrenia

Indexed keywords

DOPAMINE;

ADAPTATION; AMYGDALOID NUCLEUS; ANTICIPATION; ASSOCIATION; COGNITIVE DEFECT; DOPAMINE RELEASE; FEEDBACK SYSTEM; FUNCTIONAL MAGNETIC RESONANCE IMAGING; HIPPOCAMPUS; HUMAN; INSTRUMENTAL CONDITIONING; LEARNING; NERVE CELL PLASTICITY; NEUROIMAGING; NEUROTRANSMISSION; PARIETAL CORTEX; POSITRON EMISSION TOMOGRAPHY; PSYCHOSIS; REINFORCEMENT; REVIEW; REWARD; SCHIZOPHRENIA; SYMPTOMATOLOGY; TASK PERFORMANCE;

EID: 84891680236     PISSN: None     EISSN: 16640640     Source Type: Journal    
DOI: 10.3389/fpsyt.2013.00172     Document Type: Review

References (143)

1. Carlsson A. Does dopamine have a role in schizophrenia? Biol Psychiatry (1978) 13:3-21.
2. Abi-Dargham A, Gil R, Krystal J, Baldwin RM, Seibyl JP, Bowers M, et al. Increased striatal dopamine transmission in schizophrenia: confirmation in a second cohort. Am J Psychiatry (1998) 155:761-7.
3. Kumakura Y, Cumming P, Vernaleken I, Buchholz HG, Siessmeier T, Heinz A, et al. Elevated [18F]fluorodopamine turnover in brain of patients with schizophrenia: an [18F]fluorodopa/positron emission tomography study. J Neurosci (2007) 27:8080-7. doi:10.1523/JNEUROSCI.0805-07.2007.
4. Howes OD, Kambeitz J, Kim E, Stahl D, Slifstein M, Abi-Dargham A, et al. The nature of dopamine dysfunction in schizophrenia and what this means for treatment. Arch Gen Psychiatry (2012) 69:776-86. doi:10.1001/archgenpsychiatry.2012.169.
5. Fusar-Poli P, Meyer-Lindenberg A. Striatal presynaptic dopamine in schizophrenia, part II: meta-analysis of [(18)F/(11)C]-DOPA PET studies. Schizophr Bull (2013) 39:33-42. doi:10.1093/schbul/sbr180.
6. Schultz W, Dayan P, Montague PR. A neural substrate of prediction and reward. Science (1997) 275:1593-9. doi:10.1126/science.275.5306.1593.
7. Bayer HM, Glimcher PW. Midbrain dopamine neurons encode a quantitative reward prediction error signal. Neuron (2005) 47:129-41. doi:10.1016/j.neuron.2005.05.020.
8. Steinberg EE, Keiflin R, Boivin JR, Witten IB, Deisseroth K, Janak PH. A causal link between prediction errors, dopamine neurons and learning. Nat Neurosci (2013) 16:966-73. doi:10.1038/nn.3413.
9. Heinz A. Dopaminergic dysfunction in alcoholism and schizophrenia-psychopathological and behavioral correlates. Eur Psychiatry (2002) 17:9-16. doi:10.1016/S0924-9338(02)00628-4.
10. Kapur S. Psychosis as a state of aberrant salience: a framework linking biology, phenomenology, and pharmacology in schizophrenia. Am J Psychiatry (2003) 160:13-23. doi:10.1176/appi.ajp.160.1.13.
11. Gold JM, Waltz JA, Prentice KJ, Morris SE, Heerey EA. Reward processing in schizophrenia: a deficit in the representation of value. Schizophr Bull (2008) 34:835-47. doi:10.1093/schbul/sbn068.
12. Montague PR, Dayan P, Sejnowski TJ. A framework for mesencephalic dopamine systems based on predictive Hebbian learning. J Neurosci (1996) 16:1936-47.
13. Friston KJ, Frith CD. Schizophrenia: a disconnection syndrome? Clin Neurosci (1995) 3:89-97.
14. Friston KJ. The disconnection hypothesis. Schizophr Res (1998) 30:115-25. doi:10.1016/S0920-9964(97)00140-0.
15. Knutson B, Adams CM, Fong GW, Hommer D. Anticipation of increasing monetary reward selectively recruits nucleus accumbens. J Neurosci (2001) 21:RC159.
16. Breiter HC, Aharon I, Kahneman D, Dale A, Shizgal P. Functional imaging of neural responses to expectancy and experience of monetary gains and losses. Neuron (2001) 30:619-39. doi:10.1016/S0896-6273(01)00303-8.
17. Juckel G, Schlagenhauf F, Koslowski M, Wustenberg T, Villringer A, Knutson B, et al. Dysfunction of ventral striatal reward prediction in schizophrenia. Neuroimage (2006) 29:409-16. doi:10.1016/j.neuroimage.2005.07.051.
18. Esslinger C, Englisch S, Inta D, Rausch F, Schirmbeck F, Mier D, et al. Ventral striatal activation during attribution of stimulus saliency and reward anticipation is correlated in unmedicated first episode schizophrenia patients. Schizophr Res (2012) 140:114-21. doi:10.1016/j.schres.2012.06.025.
19. Nielsen MO, Rostrup E, Wulff S, Bak N, Lublin H, Kapur S, et al. Alterations of the brain reward system in antipsychotic naive schizophrenia patients. Biol Psychiatry (2012) 71:898-905. doi:10.1016/j.biopsych.2012.02.007.
20. Juckel G, Schlagenhauf F, Koslowski M, Filonov D, Wustenberg T, Villringer A, et al. Dysfunction of ventral striatal reward prediction in schizophrenic patients treated with typical, not atypical, neuroleptics. Psychopharmacology (Berl) (2006) 187:222-8. doi:10.1007/s00213-006-0405-4.
21. Schlagenhauf F, Juckel G, Koslowski M, Kahnt T, Knutson B, Dembler T, et al. Reward system activation in schizophrenic patients switched from typical neuroleptics to olanzapine. Psychopharmacology (Berl) (2008) 196:673-84. doi:10.1007/s00213-007-1016-4.
22. Nielsen MO, Rostrup E, Wulff S, Bak N, Broberg BV, Lublin H, et al. Improvement of brain reward abnormalities by antipsychotic monotherapy in schizophrenia. Arch Gen Psychiatry (2012) 69:1195-204. doi:10.1001/archgenpsychiatry.2012.847.
23. Walter H, Kammerer H, Frasch K, Spitzer M, Abler B. Altered reward functions in patients on atypical antipsychotic medication in line with the revised dopamine hypothesis of schizophrenia. Psychopharmacology (Berl) (2009) 206:121-32. doi:10.1007/s00213-009-1586-4.
24. Simon JJ, Biller A, Walther S, Roesch-Ely D, Stippich C, Weisbrod M, et al. Neural correlates of reward processing in schizophrenia-relationship to apathy and depression. Schizophr Res (2010) 118:154-61. doi:10.1016/j.schres.2009.11.007.
25. Waltz JA, Schweitzer JB, Ross TJ, Kurup PK, Salmeron BJ, Rose EJ, et al. Abnormal responses to monetary outcomes in cortex, but not in the basal ganglia, in schizophrenia. Neuropsychopharmacology (2010) 35:2427-39. doi:10.1038/npp.2010.126.
26. Knutson B, Gibbs SE. Linking nucleus accumbens dopamine and blood oxygenation. Psychopharmacology (Berl) (2007) 191:813-22. doi:10.1007/s00213-006-0686-7.
27. Schott BH, Minuzzi L, Krebs RM, Elmenhorst D, Lang M, Winz OH, et al. Mesolimbic functional magnetic resonance imaging activations during reward anticipation correlate with reward-related ventral striatal dopamine release. J Neurosci (2008) 28:14311-9. doi:10.1523/JNEUROSCI.2058-08.2008.
28. Knutson B, Bjork JM, Fong GW, Hommer D, Mattay VS, Weinberger DR. Amphetamine modulates human incentive processing. Neuron (2004) 43:261-9. doi:10.1016/j.neuron.2004.06.030.
29. Heinz A, Schlagenhauf F. Dopaminergic dysfunction in schizophrenia: salience attribution revisited. Schizophr Bull (2010) 36:472-85. doi:10.1093/schbul/sbq031.
30. Pankow A, Friedel E, Sterzer P, Seiferth N, Walter H, Heinz A, et al. Altered amygdala activation in schizophrenia patients during emotion processing. Schizophr Res (2013) 150:101-6. doi:10.1016/j.schres.2013.07.015.
31. Schlagenhauf F, Sterzer P, Schmack K, Ballmaier M, Rapp M, Wrase J, et al. Reward feedback alterations in unmedicated schizophrenia patients: relevance for delusions. Biol Psychiatry (2009) 65:1032-9. doi:10.1016/j.biopsych.2008.12.016.
32. Waltz JA, Schweitzer JB, Gold JM, Kurup PK, Ross TJ, Salmeron BJ, et al. Patients with schizophrenia have a reduced neural response to both unpredictable and predictable primary reinforcers. Neuropsychopharmacology (2009) 34:1567-77. doi:10.1038/npp.2008.214.
33. Morris RW, Vercammen A, Lenroot R, Moore L, Langton JM, Short B, et al. Disambiguating ventral striatum fMRI-related bold signal during reward prediction in schizophrenia. Mol Psychiatry (2011) 17:280-9. doi:10.1038/mp.2011.75.
34. Kapur S, Remington G, Jones C, Wilson A, Dasilva J, Houle S, et al. High levels of dopamine D2 receptor occupancy with low-dose haloperidol treatment: a PET study. Am J Psychiatry (1996) 153:948-50.
35. Kapur S, Zipursky R, Jones C, Shammi CS, Remington G, Seeman P. A positron emission tomography study of quetiapine in schizophrenia: a preliminary finding of an antipsychotic effect with only transiently high dopamine D2 receptor occupancy. Arch Gen Psychiatry (2000) 57:553-9. doi:10.1001/archpsyc.57.6.553.
36. Vernaleken I, Kumakura Y, Cumming P, Buchholz HG, Siessmeier T, Stoeter P, et al. Modulation of [18F]fluorodopa (FDOPA) kinetics in the brain of healthy volunteers after acute haloperidol challenge. Neuroimage (2006) 30:1332-9. doi:10.1016/j.neuroimage.2005.11.014.
37. Vernaleken I, Kumakura Y, Buchholz HG, Siessmeier T, Hilgers RD, Bartenstein P, et al. Baseline [18F]-FDOPA kinetics are predictive of haloperidol-induced changes in dopamine turnover and cognitive performance: a positron emission tomography study in healthy subjects. Neuroimage (2008) 40:1222-31. doi:10.1016/j.neuroimage.2007.12.045.
38. Insel TR, Scolnick EM. Cure therapeutics and strategic prevention: raising the bar for mental health research. Mol Psychiatry (2006) 11:11-7. doi:10.1038/sj.mp.4001777.
39. Insel TR, Wang PS. Rethinking mental illness. JAMA (2010) 303:1970-1. doi:10.1001/jama.2010.555.
40. Huys QJ, Moutoussis M, Williams J. Are computational models of any use to psychiatry? Neural Netw (2011) 24:544-51. doi:10.1016/j.neunet.2011.03.001.
41. Montague PR, Dolan RJ, Friston KJ, Dayan P. Computational psychiatry. Trends Cogn Sci (2012) 16:72-80. doi:10.1016/j.tics.2011.11.018.
42. Sutton RS, Barto AG. Reinforcement Learning: An Introduction. Cambridge, MA: MIT Press (1998).
43. Murray GK, Corlett PR, Clark L, Pessiglione M, Blackwell AD, Honey G, et al. Substantia nigra/ventral tegmental reward prediction error disruption in psychosis. Mol Psychiatry (2008) 13(239):267-76. doi:10.1038/sj.mp.4002058.
44. Robinson TE, Berridge KC. The neural basis of drug craving: an incentive-sensitization theory of addiction. Brain Res Brain Res Rev (1993) 18:247-91. doi:10.1016/0165-0173(93)90013-P.
45. King R, Barchas JD, Huberman BA. Chaotic behavior in dopamine neurodynamics. Proc Natl Acad Sci U S A (1984) 81:1244-7. doi:10.1073/pnas.81.4.1244.
46. Grace AA. Phasic versus tonic dopamine release and the modulation of dopamine system responsivity: a hypothesis for the etiology of schizophrenia. Neuroscience (1991) 41:1-24. doi:10.1016/0306-4522(91)90196-U.
47. Frank MJ. Schizophrenia: a computational reinforcement learning perspective. Schizophr Bull (2008) 34:1008-11. doi:10.1093/schbul/sbn123.
48. Smith A, Li M, Becker S, Kapur S. Dopamine, prediction error and associative learning: a model-based account. Network (2006) 17:61-84. doi:10.1080/09548980500361624.
49. Roesch MR, Esber GR, Li J, Daw ND, Schoenbaum G. Surprise! Neural correlates of Pearce-Hall and Rescorla-Wagner coexist within the brain. Eur J Neurosci (2012) 35:1190-200. doi:10.1111/j.1460-9568.2011.07986.x.
50. Metereau E, Dreher JC. Cerebral correlates of salient prediction error for different rewards and punishments. Cereb Cortex (2013) 23:477-87. doi:10.1093/cercor/bhs037.
51. Barch DM, Dowd EC. Goal representations and motivational drive in schizophrenia: the role of prefrontal-striatal interactions. Schizophr Bull (2010) 36:919-34. doi:10.1093/schbul/sbq068.
52. O'Doherty JP, Hampton A, Kim H. Model-based fMRI and its application to reward learning and decision making. Ann N Y Acad Sci (2007) 1104:35-53. doi:10.1196/annals.1390.022.
53. Daw ND. Trial-by-trial data analysis using computational models. In: Phelps EA, Robbins TW, Delgado M editors. In Affect, Learning and Decision Making, Attention and Performance XXIII. New York: Oxford University Press (2009) p. 3-38.
54. Roiser JP, Stephan KE, Den Ouden HE, Barnes TR, Friston KJ, Joyce EM. Do patients with schizophrenia exhibit aberrant salience? Psychol Med (2009) 39:199-209. doi:10.1017/S0033291708003863.
55. Roiser JP, Howes OD, Chaddock CA, Joyce EM, Mcguire P. Neural and behavioral correlates of aberrant salience in individuals at risk for psychosis. Schizophr Bull (2012) 39:1328-36. doi:10.1093/schbul/sbs147.
56. Heerey EA, Gold JM. Patients with schizophrenia demonstrate dissociation between affective experience and motivated behavior. J Abnorm Psychol (2007) 116:268-78. doi:10.1037/0021-843X.116.2.268.
57. Dowd EC, Barch DM. Anhedonia and emotional experience in schizophrenia: neural and behavioral indicators. Biol Psychiatry (2010) 67:902-11. doi:10.1016/j.biopsych.2009.10.020.
58. Ursu S, Kring AM, Gard MG, Minzenberg MJ, Yoon JH, Ragland JD, et al. Prefrontal cortical deficits and impaired cognition-emotion interactions in schizophrenia. Am J Psychiatry (2011) 168:276-85. doi:10.1176/appi.ajp.2010.09081215.
59. Heerey EA, Robinson BM, Mcmahon RP, Gold JM. Delay discounting in schizophrenia. Cogn Neuropsychiatry (2007) 12:213-21. doi:10.1080/13546800601005900.
60. Ahn WY, Rass O, Fridberg DJ, Bishara AJ, Forsyth JK, Breier A, et al. Temporal discounting of rewards in patients with bipolar disorder and schizophrenia. J Abnorm Psychol (2011) 120:911-21. doi:10.1037/a0023333.
61. Heerey EA, Matveeva TM, Gold JM. Imagining the future: degraded representations of future rewards and events in schizophrenia. J Abnorm Psychol (2011) 120:483-9. doi:10.1037/a0021810.
62. Wing VC, Moss TG, Rabin RA, George TP. Effects of cigarette smoking status on delay discounting in schizophrenia and healthy controls. Addict Behav (2012) 37:67-72. doi:10.1016/j.addbeh.2011.08.012.
63. Heerey EA, Bell-Warren KR, Gold JM. Decision-making impairments in the context of intact reward sensitivity in schizophrenia. Biol Psychiatry (2008) 64:62-9. doi:10.1016/j.biopsych.2008.02.015.
64. Gold JM, Waltz JA, Matveeva TM, Kasanova Z, Strauss GP, Herbener ES, et al. Negative symptoms and the failure to represent the expected reward value of actions: behavioral and computational modeling evidence. Arch Gen Psychiatry (2012) 69:129-38. doi:10.1001/archgenpsychiatry.2011.1269.
65. Somlai Z, Moustafa AA, Keri S, Myers CE, Gluck MA. General functioning predicts reward and punishment learning in schizophrenia. Schizophr Res (2011) 127:131-6. doi:10.1016/j.schres.2010.07.028.
66. Yilmaz A, Simsek F, Gonul AS. Reduced reward-related probability learning in schizophrenia patients. Neuropsychiatr Dis Treat (2012) 8:27-34. doi:10.2147/NDT.S26243.
67. Gold JM, Strauss GP, Waltz JA, Robinson BM, Brown JK, Frank MJ. Negative symptoms of schizophrenia are associated with abnormal effort-cost computations. Biol Psychiatry (2013) 74:130-6. doi:10.1016/j.biopsych.2012.12.022.
68. Kasanova Z, Waltz JA, Strauss GP, Frank MJ, Gold JM. Optimizing vs. matching: response strategy in a probabilistic learning task is associated with negative symptoms of schizophrenia. Schizophr Res (2011) 127:215-22. doi:10.1016/j.schres.2010.12.003.
69. Elliott R, Mckenna PJ, Robbins TW, Sahakian BJ. Neuropsychological evidence for frontostriatal dysfunction in schizophrenia. Psychol Med (1995) 25:619-30. doi:10.1017/S0033291700033523.
70. Pantelis C, Barber FZ, Barnes TR, Nelson HE, Owen AM, Robbins TW. Comparison of set-shifting ability in patients with chronic schizophrenia and frontal lobe damage. Schizophr Res (1999) 37:251-70. doi:10.1016/S0920-9964(98)00156-X.
71. Waltz JA, Gold JM. Probabilistic reversal learning impairments in schizophrenia: further evidence of orbitofrontal dysfunction. Schizophr Res (2007) 93:296-303. doi:10.1016/j.schres.2007.03.010.
72. Prentice KJ, Gold JM, Buchanan RW. The Wisconsin card sorting impairment in schizophrenia is evident in the first four trials. Schizophr Res (2008) 106:81-7. doi:10.1016/j.schres.2007.07.015.
73. McKirdy J, Sussmann JE, Hall J, Lawrie SM, Johnstone EC, Mcintosh AM. Set shifting and reversal learning in patients with bipolar disorder or schizophrenia. Psychol Med (2009) 39:1289-93. doi:10.1017/S0033291708004935.
74. Murray GK, Cheng F, Clark L, Barnett JH, Blackwell AD, Fletcher PC, et al. Reinforcement and reversal learning in first-episode psychosis. Schizophr Bull (2008) 34:848-55. doi:10.1093/schbul/sbn078.
75. Leeson VC, Robbins TW, Matheson E, Hutton SB, Ron MA, Barnes TR, et al. Discrimination learning, reversal, and set-shifting in first-episode schizophrenia: stability over six years and specific associations with medication type and disorganization syndrome. Biol Psychiatry (2009) 66:586-93. doi:10.1016/j.biopsych.2009.05.016.
76. Waltz JA, Kasanova Z, Ross TJ, Salmeron BJ, Mcmahon RP, Gold JM, et al. The roles of reward, default, and executive control networks in set-shifting impairments in schizophrenia. PLoS One (2013) 8:e57257. doi:10.1371/journal.pone.0057257.
77. Schlagenhauf F, Huys QJM, Deserno L, Rapp MA, Beck A, Heinze HJ, et al. Striatal dysfunction during reversal learning in unmedicated schizophrenia patients. Neuroimage (2013). doi:10.1016/j.neuroimage.2013.11.034.
78. Hampton AN, Bossaerts P, O'Doherty JP. The role of the ventromedial prefrontal cortex in abstract state-based inference during decision making in humans. J Neurosci (2006) 26:8360-7. doi:10.1523/JNEUROSCI.1010-06.2006.
79. Glascher J, Daw N, Dayan P, O'Doherty JP. States versus rewards: dissociable neural prediction error signals underlying model-based and model-free reinforcement learning. Neuron (2010) 66:585-95. doi:10.1016/j.neuron.2010.04.016.
80. Daw ND, Gershman SJ, Seymour B, Dayan P, Dolan RJ. Model-based influences on humans' choices and striatal prediction errors. Neuron (2011) 69:1204-15. doi:10.1016/j.neuron.2011.02.027.
81. Collins AG, Frank MJ. How much of reinforcement learning is working memory, not reinforcement learning? A behavioral, computational, and neurogenetic analysis. Eur J Neurosci (2012) 35:1024-35. doi:10.1111/j.1460-9568.2011.07980.x.
82. Otto AR, Gershman SJ, Markman AB, Daw ND. The curse of planning: dissecting multiple reinforcement-learning systems by taxing the central executive. Psychol Sci (2013) 24:751-61. doi:10.1177/0956797612463080.
83. Waltz JA, Frank MJ, Robinson BM, Gold JM. Selective reinforcement learning deficits in schizophrenia support predictions from computational models of striatal-cortical dysfunction. Biol Psychiatry (2007) 62:756-64. doi:10.1016/j.biopsych.2006.09.042.
84. Morris SE, Heerey EA, Gold JM, Holroyd CB. Learning-related changes in brain activity following errors and performance feedback in schizophrenia. Schizophr Res (2008) 99:274-85. doi:10.1016/j.schres.2007.08.027.
85. Waltz JA, Frank MJ, Wiecki TV, Gold JM. Altered probabilistic learning and response biases in schizophrenia: behavioral evidence and neurocomputational modeling. Neuropsychology (2011) 25:86-97. doi:10.1037/a0020882.
86. Frank MJ, O'Reilly RC. A mechanistic account of striatal dopamine function in human cognition: psychopharmacological studies with cabergoline and haloperidol. Behav Neurosci (2006) 120:497-517. doi:10.1037/0735-7044.120.3.497.
87. Frank MJ, Seeberger LC, O'Reilly RC. By carrot or by stick: cognitive reinforcement learning in parkinsonism. Science (2004) 306:1940-3. doi:10.1126/science.1102941.
88. Kravitz AV, Tye LD, Kreitzer AC. Distinct roles for direct and indirect pathway striatal neurons in reinforcement. Nat Neurosci (2012) 15:816-8. doi:10.1038/nn.3100.
89. Strauss GP, Frank MJ, Waltz JA, Kasanova Z, Herbener ES, Gold JM. Deficits in positive reinforcement learning and uncertainty-driven exploration are associated with distinct aspects of negative symptoms in schizophrenia. Biol Psychiatry (2011) 69:424-31. doi:10.1016/j.biopsych.2010.10.015.
90. Brodersen KH, Schofield TM, Leff AP, Ong CS, Lomakina EI, Buhmann JM, et al. Generative embedding for model-based classification of fMRI data. PLoS Comput Biol (2011) 7:e1002079. doi:10.1371/journal.pcbi.1002079.
91. Brodersen KH, Deserno L, Schlagenhauf F, Lin Z, Penny WD, Buhmann JM, et al. Dissecting psychiatric spectrum disorders by generative embedding. Neuroimage Clin (2014) 4:98-111. doi:10.1016/j.nicl.2013.11.002.
92. Jensen J, Willeit M, Zipursky RB, Savina I, Smith AJ, Menon M, et al. The formation of abnormal associations in schizophrenia: neural and behavioral evidence. Neuropsychopharmacology (2008) 33:473-9. doi:10.1038/sj.npp.1301437.
93. Romaniuk L, Honey GD, King JR, Whalley HC, Mcintosh AM, Levita L, et al. Midbrain activation during Pavlovian conditioning and delusional symptoms in schizophrenia. Arch Gen Psychiatry (2010) 67:1246-54. doi:10.1001/archgenpsychiatry.2010.169.
94. den Ouden HE, Daunizeau J, Roiser J, Friston KJ, Stephan KE. Striatal prediction error modulates cortical coupling. J Neurosci (2010) 30:3210-9. doi:10.1523/JNEUROSCI.4458-09.2010.
95. Li J, Schiller D, Schoenbaum G, Phelps EA, Daw ND. Differential roles of human striatum and amygdala in associative learning. Nat Neurosci (2011) 14:1250-2. doi:10.1038/nn.2904.
96. Dowd EC, Barch DM. Pavlovian reward prediction and receipt in schizophrenia: relationship to anhedonia. PLoS One (2012) 7:e35622. doi:10.1371/journal.pone.0035622.
97. Gradin VB, Waiter G, O'Connor A, Romaniuk L, Stickle C, Matthews K, et al. Salience network-midbrain dysconnectivity and blunted reward signals in schizophrenia. Psychiatry Res (2013) 211:104-11. doi:10.1016/j.pscychresns.2012.06.003.
98. Diaconescu AO, Jensen J, Wang H, Willeit M, Menon M, Kapur S, et al. Aberrant effective connectivity in schizophrenia patients during appetitive conditioning. Front Hum Neurosci (2011) 4:239. doi:10.3389/fnhum.2010.00239.
99. Wernicke C. Grundrisse der Psychiatrie. Leipzig: Thieme (1906).
100. Stephan KE, Baldeweg T, Friston KJ. Synaptic plasticity and dysconnection in schizophrenia. Biol Psychiatry (2006) 59:929-39. doi:10.1016/j.biopsych.2005.10.005.
101. Stephan KE, Friston KJ, Frith CD. Dysconnection in schizophrenia: from abnormal synaptic plasticity to failures of self-monitoring. Schizophr Bull (2009) 35:509-27. doi:10.1093/schbul/sbn176.
102. Corlett PR, Murray GK, Honey GD, Aitken MR, Shanks DR, Robbins TW, et al. Disrupted prediction error signal in psychosis: evidence for an associative account of delusions. Brain (2007) 130:2387-400. doi:10.1093/brain/awm173.
103. Koch K, Schachtzabel C, Wagner G, Schikora J, Schultz C, Reichenbach JR, et al. Altered activation in association with reward-related trial-and-error learning in patients with schizophrenia. Neuroimage (2010) 50:223-32. doi:10.1016/j.neuroimage.2009.12.031.
104. Koch K, Schachtzabel C, Wagner G, Reichenbach JR, Sauer H, Schlosser R. The neural correlates of reward-related trial-and-error learning: an fMRI study with a probabilistic learning task. Learn Mem (2008) 15:728-32. doi:10.1101/lm.1106408.
105. Weickert TW, Goldberg TE, Callicott JH, Chen Q, Apud JA, Das S, et al. Neural correlates of probabilistic category learning in patients with schizophrenia. J Neurosci (2009) 29:1244-54. doi:10.1523/JNEUROSCI.4341-08.2009.
106. Gradin VB, Kumar P, Waiter G, Ahearn T, Stickle C, Milders M, et al. Expected value and prediction error abnormalities in depression and schizophrenia. Brain (2011) 134:1751-64. doi:10.1093/brain/awr059.
107. Tan HY, Callicott JH, Weinberger DR. Dysfunctional and compensatory prefrontal cortical systems, genes and the pathogenesis of schizophrenia. Cereb Cortex (2007) 17(Suppl 1):i171-81. doi:10.1093/cercor/bhm069.
108. Deserno L, Sterzer P, Wustenberg T, Heinz A, Schlagenhauf F. Reduced prefrontal-parietal effective connectivity and working memory deficits in schizophrenia. J Neurosci (2012) 32:12-20. doi:10.1523/JNEUROSCI.3405-11.2012.
109. Fusar-Poli P, Meyer-Lindenberg A. Striatal presynaptic dopamine in schizophrenia, part I: meta-analysis of dopamine active transporter (DAT) density. Schizophr Bull (2013) 39:22-32. doi:10.1093/schbul/sbr111.
110. Weinberger DR. Implications of normal brain development for the pathogenesis of schizophrenia. Arch Gen Psychiatry (1987) 44:660-9. doi:10.1001/archpsyc.1987.01800190080012.
111. Fusar-Poli P, Borgwardt S, Bechdolf A, Addington J, Riecher-Rossler A, Schultze-Lutter F, et al. The psychosis high-risk state: a comprehensive state-of-the-art review. JAMA Psychiatry (2013) 70:107-20. doi:10.1001/jamapsychiatry.2013.269.
112. Roiser JP, Stephan KE, Den Ouden HE, Friston KJ, Joyce EM. Adaptive and aberrant reward prediction signals in the human brain. Neuroimage (2010) 50:657-64. doi:10.1016/j.neuroimage.2009.11.075.
113. Behrens TE, Woolrich MW, Walton ME, Rushworth MF. Learning the value of information in an uncertain world. Nat Neurosci (2007) 10:1214-21. doi:10.1038/nn1954.
114. Mathys C, Daunizeau J, Friston KJ, Stephan KE. A bayesian foundation for individual learning under uncertainty. Front Hum Neurosci (2011) 5:39. doi:10.3389/fnhum.2011.00039.
115. Stephan KE, Penny WD, Daunizeau J, Moran RJ, Friston KJ. Bayesian model selection for group studies. Neuroimage (2009) 46:1004-17. doi:10.1016/j.neuroimage.2009.03.025.
116. Stephan KE, Penny WD, Moran RJ, Den Ouden HE, Daunizeau J, Friston KJ. Ten simple rules for dynamic causal modeling. Neuroimage (2010) 49:3099-109. doi:10.1016/j.neuroimage.2009.11.015.
117. Pessiglione M, Seymour B, Flandin G, Dolan RJ, Frith CD. Dopamine-dependent prediction errors underpin reward-seeking behaviour in humans. Nature (2006) 442:1042-5. doi:10.1038/nature05051.
118. Menon M, Jensen J, Vitcu I, Graff-Guerrero A, Crawley A, Smith MA, et al. Temporal difference modeling of the blood-oxygen level dependent response during aversive conditioning in humans: effects of dopaminergic modulation. Biol Psychiatry (2007) 62:765-72. doi:10.1016/j.biopsych.2006.10.020.
119. Bernacer J, Corlett PR, Ramachandra P, Mcfarlane B, Turner DC, Clark L, et al. Methamphetamine-induced disruption of frontostriatal reward learning signals: relation to psychotic symptoms. Am J Psychiatry (2013) 170:1326-34. doi:10.1176/appi.ajp.2013.12070978.
120. Cools R, Lewis SJ, Clark L, Barker RA, Robbins TW. l-DOPA disrupts activity in the nucleus accumbens during reversal learning in Parkinson's disease. Neuropsychopharmacology (2007) 32:180-9. doi:10.1038/sj.npp.1301153.
121. Schlagenhauf F, Rapp MA, Huys QJ, Beck A, Wustenberg T, Deserno L, et al. Ventral striatal prediction error signaling is associated with dopamine synthesis capacity and fluid intelligence. Hum Brain Mapp (2013) 34:1490-9. doi:10.1002/hbm.22000.
122. Cumming P, Ase A, Laliberte C, Kuwabara H, Gjedde A. In vivo regulation of DOPA decarboxylase by dopamine receptors in rat brain. J Cereb Blood Flow Metab (1997) 17:1254-60. doi:10.1097/00004647-199711000-00014.
123. Bello EP, Mateo Y, Gelman DM, Noain D, Shin JH, Low MJ, et al. Cocaine supersensitivity and enhanced motivation for reward in mice lacking dopamine D2 autoreceptors. Nat Neurosci (2011) 14:1033-8. doi:10.1038/nn.2862.
124. Buckholtz JW, Treadway MT, Cowan RL, Woodward ND, Benning SD, Li R, et al. Mesolimbic dopamine reward system hypersensitivity in individuals with psychopathic traits. Nat Neurosci (2010) 13:419-21. doi:10.1038/nn.2510.
125. Ito H, Kodaka F, Takahashi H, Takano H, Arakawa R, Shimada H, et al. Relation between presynaptic and postsynaptic dopaminergic functions measured by positron emission tomography: implication of dopaminergic tone. J Neurosci (2011) 31:7886-90. doi:10.1523/JNEUROSCI.6024-10.2011.
126. Cools R, Frank MJ, Gibbs SE, Miyakawa A, Jagust W, D'Esposito M. Striatal dopamine predicts outcome-specific reversal learning and its sensitivity to dopaminergic drug administration. J Neurosci (2009) 29:1538-43. doi:10.1523/JNEUROSCI.4467-08.2009.
127. Goldman-Rakic PS. The physiological approach: functional architecture of working memory and disordered cognition in schizophrenia. Biol Psychiatry (1999) 46:650-61. doi:10.1016/S0006-3223(99)00130-4.
128. Cools R, D'Esposito M. Inverted-U-shaped dopamine actions on human working memory and cognitive control. Biol Psychiatry (2011) 69:e113-25. doi:10.1016/j.biopsych.2011.03.028.
129. Fusar-Poli P, Howes OD, Allen P, Broome M, Valli I, Asselin MC, et al. Abnormal frontostriatal interactions in people with prodromal signs of psychosis: a multimodal imaging study. Arch Gen Psychiatry (2010) 67:683-91. doi:10.1001/archgenpsychiatry.2010.77.
130. Tan HY, Sust S, Buckholtz JW, Mattay VS, Meyer-Lindenberg A, Egan MF, et al. Dysfunctional prefrontal regional specialization and compensation in schizophrenia. Am J Psychiatry (2006) 163:1969-77. doi:10.1176/appi.ajp.163.11.1969.
131. Potkin SG, Turner JA, Brown GG, Mccarthy G, Greve DN, Glover GH, et al. Working memory and DLPFC inefficiency in schizophrenia: the FBIRN study. Schizophr Bull (2009) 35:19-31. doi:10.1093/schbul/sbn162.
132. Diaconescu AO, Menon M, Jensen J, Kapur S, Mcintosh AR. Dopamine-induced changes in neural network patterns supporting aversive conditioning. Brain Res (2010) 1313:143-61. doi:10.1016/j.brainres.2009.11.064.
133. Heinz A, Knable MB, Coppola R, Gorey JG, Jones DW, Lee KS, et al. Psychomotor slowing, negative symptoms and dopamine receptor availability-an IBZM SPECT study in neuroleptic-treated and drug-free schizophrenic patients. Schizophr Res (1998) 31:19-26. doi:10.1016/S0920-9964(98)00003-6.
134. Centonze D, Usiello A, Costa C, Picconi B, Erbs E, Bernardi G, et al. Chronic haloperidol promotes corticostriatal long-term potentiation by targeting dopamine D2L receptors. J Neurosci (2004) 24:8214-22. doi:10.1523/JNEUROSCI.1274-04.2004.
135. Tseng KY, O'Donnell P. Dopamine-glutamate interactions controlling prefrontal cortical pyramidal cell excitability involve multiple signaling mechanisms. J Neurosci (2004) 24:5131-9. doi:10.1523/JNEUROSCI.1021-04.2004.
136. Friston K. A theory of cortical responses. Philos Trans R Soc Lond B Biol Sci (2005) 360:815-36. doi:10.1098/rstb.2005.1622.
137. Braver TS, Cohen JD. Dopamine, cognitive control, and schizophrenia: the gating model. Prog Brain Res (1999) 121:327-49. doi:10.1016/S0079-6123(08)63082-4.
138. Rolls ET, Loh M, Deco G, Winterer G. Computational models of schizophrenia and dopamine modulation in the prefrontal cortex. Nat Rev Neurosci (2008) 9:696-709. doi:10.1038/nrn2462.
139. Schmidt A, Smieskova R, Aston J, Simon A, Allen P, Fusar-Poli P, et al. Brain connectivity abnormalities predating the onset of psychosis: correlation with the effect of medication. JAMA Psychiatry (2013) 70:903-12. doi:10.1001/jamapsychiatry.2013.117.
140. Friston KJ, Harrison L, Penny W. Dynamic causal modelling. Neuroimage (2003) 19:1273-302. doi:10.1016/S1053-8119(03)00202-7.
141. den Ouden HE, Friston KJ, Daw ND, Mcintosh AR, Stephan KE. A dual role for prediction error in associative learning. Cereb Cortex (2009) 19:1175-85. doi:10.1093/cercor/bhn161.
142. Iglesias S, Mathys C, Brodersen KH, Kasper L, Piccirelli M, Den Ouden HE, et al. Hierarchical prediction errors in midbrain and basal forebrain during sensory learning. Neuron (2013) 80:519-30. doi:10.1016/j.neuron.2013.09.009.
143. Li J, Daw ND. Signals in human striatum are appropriate for policy update rather than value prediction. J Neurosci (2011) 31:5504-11. doi:10.1523/JNEUROSCI.6316-10.2011.