Working memory contributions to reinforcement learning impairments in schizophrenia

Journal of Neuroscience

Volumn 34, Issue 41, 2014, Pages 13747-13756

  Collins, Anne G E ^a   Brown, Jaime K ^b   Gold, James M ^b   Waltz, James A ^b   Frank, Michael J ^a  

^a BROWN UNIVERSITY   (United States)

^b UNIVERSITY OF MARYLAND SCHOOL OF MEDICINE   (United States)

Author keywords

Computational model; Reinforcement learning; Schizophrenia; Working memory

Indexed keywords

ADULT; ARTICLE; ASSESSMENT OF HUMANS; BRIEF NEGATIVE SYMPTOM RATING SCALE; CLINICAL ARTICLE; COMPUTER MODEL; CONTROLLED STUDY; DECISION MAKING; DSM-IV; FEMALE; HUMAN; LEARNING DISORDER; MACHINE LEARNING; MALE; MATRICS CONSENSUS COGNITIVE BATTERY; MENTAL CAPACITY; POSITIVE AND NEGATIVE SYNDROME SCALE; PREFRONTAL CORTEX; PRINCIPAL COMPONENT ANALYSIS; PRIORITY JOURNAL; REINFORCEMENT LEARNING; SCALE FOR THE ASSESSMENT OF NEGATIVE SYMPTOMS; SCHIZOPHRENIA; STIMULUS RESPONSE; TASK PERFORMANCE; WORKING MEMORY; COMPLICATION; DOSE RESPONSE; LEARNING CURVE; LEARNING DISORDERS; PHOTOSTIMULATION; PHYSIOLOGY; PSYCHOLOGICAL MODEL; PSYCHOLOGY; PSYCHOMOTOR PERFORMANCE; REACTION TIME; REINFORCEMENT; SHORT TERM MEMORY;

NEUROLEPTIC AGENT;

ADULT; ANTIPSYCHOTIC AGENTS; DOSE-RESPONSE RELATIONSHIP, DRUG; FEMALE; HUMANS; LEARNING CURVE; LEARNING DISORDERS; MALE; MEMORY, SHORT-TERM; MODELS, PSYCHOLOGICAL; PHOTIC STIMULATION; PSYCHOMOTOR PERFORMANCE; REACTION TIME; REINFORCEMENT (PSYCHOLOGY); SCHIZOPHRENIA; SCHIZOPHRENIC PSYCHOLOGY;

EID: 84912082331     PISSN: 02706474     EISSN: 15292401     Source Type: Journal    
DOI: 10.1523/JNEUROSCI.0989-14.2014     Document Type: Article

References (63)

1. Abi-Dargham A, Gil R, Krystal J, Baldwin RM, Seibyl JP, Bowers M, van Dyck CH, Charney DS, Innis RB, Laruelle M (1998) Increased striatal dopamine transmission in schizophrenia: confirmation in a second cohort. Am J Psychiatry 155:761-767. Medline
2. American Psychiatric Association (2000) DSM-IV-TR: diagnostic and statistical manual of mental disorders, text revision. Washington, DC: American Psychiatric Associations.
3. Andreasen N (1983) Scale for the assessment of negative symptoms. srspence.com. Available at http://www.srspence.com/SANS.pdf. Retrieved January 17, 2014.
4. Barch DM, Ceaser A (2012) Cognition in schizophrenia: core psychological and neural mechanisms. Trends Cogn Sci 16:27-34. CrossRef Medline
5. Barch DM, Berman MG, Engle R, Jones JH, Jonides J, Macdonald A, Nee DE, Redick TS, Sponheim SR (2009) CNTRICS final task selection: working memory. Schizophr Bull 35:136-152. CrossRef Medline
6. Beninger RJ, Wasserman J, Zanibbi K, Charbonneau D, Mangels J, Beninger BV (2003) Typical and atypical antipsychotic medications differentially affect two nondeclarative memory tasks in schizophrenic patients: a double dissociation. Schizophr Res 61:281-292. CrossRef Medline
7. Burnham KP, Anderson DR (2002) Model selection and multi-model inference: a practical information-theoretic approach. Springer, New York. E-book available at http://www.citeulike.org/group/7954/article/4425594.
8. Collins AG, Frank MJ (2012) How much of reinforcement learning is working memory, not reinforcement learning? A behavioral, computational, and neurogenetic analysis. Eur J Neurosci 35:1024-1035. CrossRef Medline
9. Collins A, Koechlin E (2012) Reasoning, learning, and creativity: frontal lobe function and human decision-making. PLoS Biol 10:e1001293. CrossRef Medline
10. Collins AG, Frank MJ (2013) Cognitive control over learning: creating, clustering, and generalizing task-set structure. Psychol Rev 120:190-229. CrossRef Medline
11. Corlett PR, Murray GK, Honey GD, Aitken MR, Shanks DR, Robbins TW, Bullmore ET, Dickinson A, Fletcher PC (2007) Disrupted prediction-error signal in psychosis: evidence for an associative account of delusions. Brain 130:2387-2400. CrossRef Medline
12. Daw N (2011) Trial-by-trial data analysis using computational models. In: Decision making, affect, and learning: attention and performance XXIII (Delgado MR, Phelps EA, Robbins TW, eds), 1-26. New York: Oxford UP.
13. Dayan P, Daw ND (2008) Decision theory, reinforcement learning, and the brain. Cogn Affect Behav Neurosci 8:429-453. CrossRef Medline
14. Deserno L, Boehme R, Heinz A, Schlagenhauf F (2013) Reinforcement learning and dopamine in schizophrenia: dimensions of symptoms or specific features of a disease group? Front Psychiatry 4:172. CrossRef Medline
15. Doll BB, et al. (2014) Reduced susceptibility to confirmation bias in schizophrenia. Cogn Affect Behav Neurosci 14:715-728. CrossRef Medline
16. Dowd EC, Barch DM (2012) Pavlovian reward prediction and receipt in schizophrenia: relationship to anhedonia. PloS One 7:e35622. CrossRef Medline
17. First MB, Spitzer RL, Gibbon M, Williams JB (1997) Structured clinical interview for DSM-IV axis I disorders, clinician version (SCID-CV). Washington, DC: American Psychiatric Association.
18. Frank MJ, Seeberger LC, O’reilly RC (2004) By carrot or by stick: cognitive reinforcement learning in parkinsonism. Science 306:1940-1943. CrossRef Medline
19. Frank MJ, Moustafa AA, Haughey HM, Curran T, Hutchison KE (2007) Genetic triple dissociation reveals multiple roles for dopamine in reinforcement learning. Proc Natl Acad Sci U S A 104:16311-16316. CrossRef Medline
20. Gogos JA, Morgan M, Luine V, Santha M, Ogawa S, Pfaff D, Karayiorgou M (1998) Catechol-O-methyltransferase-deficient mice exhibit sexually dimorphic changes in catecholamine levels and behavior. Proc Natl Acad Sci U S A 95:9991-9996. CrossRef Medline
21. Gold JM, Waltz JA, Prentice KJ, Morris SE, Heerey EA (2008) Reward processing in schizophrenia: a deficit in the representation of value. Schizophr Bull 34:835-847. CrossRef Medline
22. Gold JM, Waltz JA, Matveeva TM, Kasanova Z, Strauss GP, Herbener ES, Collins AG, Frank MJ (2012) Negative symptoms and the failure to represent the expected reward value of actions: behavioral and computational modeling evidence. Control 69:129-138. CrossRef Medline
23. Goldberg TE, Weinberger DR, Berman KF, Pliskin NH, Podd MH (1987) Further evidence for dementia of the prefrontal type in schizophrenia? Arch Gen Psychiatry 44:1008-1014. CrossRef Medline
24. Guitart-Masip M, Huys QJ, Fuentemilla L, Dayan P, Duzel E, Dolan RJ (2012) Go and no-go learning in reward and punishment: interactions between affect and effect. Neuroimage 62:154-166. CrossRef Medline
25. Horan WP, Foti D, Hajcak G, Wynn JK, Green MF (2012) Impaired neural response to internal but not external feedback in schizophrenia. Psychol Med 42:1637-1647. CrossRef Medline
26. Howes OD, Kapur S (2009) The dopamine hypothesis of schizophrenia: version III-the final common pathway. Schizophr Bull 35:549-562. CrossRef Medline
27. Howes OD, Kambeitz J, Kim E, Stahl D, Slifstein M, Abi-Dargham A, Kapur S (2012) The nature of dopamine dysfunction in schizophrenia and what this means for treatment. Arch Gen Psychiatry 69:776-786. CrossRef Medline
28. Huotari M, Gogos JA, Karayiorgou M, Koponen O, Forsberg M, Raasmaja A, Hyttinen J, Männistö PT (2002) Brain catecholamine metabolism in catechol-O-methyltransferase (COMT)-deficient mice. Eur J Neurosci 15:246-256. CrossRef Medline
29. Johnson MK, McMahon RP, Robinson BM, Harvey AN, Hahn B, Leonard CJ, Luck SJ, Gold JM (2013) The relationship between working memory capacity and broad measures of cognitive ability in healthy adults and people with schizophrenia. Neuropsychology 27:220-229. CrossRef Medline
30. Just MA, Carpenter PA (1992) A capacity theory of comprehension: individual differences in working memory. Psychol Rev 99:122-149. CrossRef Medline
31. Kapur S, Mizrahi R, Li M (2005) From dopamine to salience to psychosis linking biology, pharmacology and phenomenology of psychosis. Schizophr Res 79:59-68. CrossRef Medline
32. Kim H, Lee D, Shin YM, Chey J (2007) Impaired strategic decision making in schizophrenia. Brain Res 1180:90-100. CrossRef Medline
33. Kirkpatrick B, Strauss GP, Nguyen L, Fischer BA, Daniel DG, Cienfuegos A, Marder SR (2011) The brief negative symptom scale. psychometric properties. Schizophr Bull 37:300-305. CrossRef Medline
34. Koch K, Schachtzabel C, Wagner G, Schikora J, Schultz C, Reichenbach JR, Sauer H, Schlösser RG (2010) Altered activation in association with reward-related trial-and-error learning in patients with schizophrenia. Neuroimage 50:223-232. CrossRef Medline
35. Lee J, Park S (2005) Working memory impairments in schizophrenia: a meta-analysis. J Abnormal Psychol 114:599-611. CrossRef Medline
36. Lobo MK, Cui Y, Ostlund SB, Balleine BW, Yang XW (2007) Genetic control of instrumental conditioning by striatopallidal neuron-specific S1P receptor Gpr6. Nat Neurosci 10:1395-1397. CrossRef Medline
37. Matsumoto M, Weickert CS, Akil M, Lipska BK, Hyde TM, Herman MM, Kleinman JE, Weinberger DR (2003) Catechol O-methyltransferase mRNA expression in human and rat brain: evidence for a role in cortical neuronal function. Neuroscience 116:127-137. CrossRef Medline
38. Montague PR, Dolan RJ, Friston KJ, Dayan P (2012) Computational psychiatry. Trends Cogn Sci 16:72-80. CrossRef Medline
39. Morris SE, Holroyd CB, Mann-Wrobel MC, Gold JM (2011) Dissociation of response and feedback negativity in schizophrenia: electrophysiological and computational evidence for a deficit in the representation of value. Front Hum Neurosci 5:123. CrossRef Medline
40. Murray GK, Corlett PR, Clark L, Pessiglione M, Blackwell AD, Honey G, Jones PB, Bullmore ET, Robbins TW, Fletcher PC (2008a) Substantianigra/ventral tegmental reward prediction error disruption in psychosis. Mol Psychiatry 13:267-276. CrossRef Medline
41. Murray GK, Cheng F, Clark L, Barnett JH, Blackwell AD, Fletcher PC, Robbins TW, Bullmore ET, Jones PB (2008b) Reinforcement and reversal learning in first-episode psychosis. Schizophr Bull 34:848-855. CrossRef Medline
42. Nuechterlein K, Green M (2006) MCCB: matrics consensus cognitive battery. Los Angeles: MATRICS Assessment.
43. O’Doherty JP, Hampton A, Kim H (2007) Model-based fMRI and its application to reward learning and decision making. Ann N Y Acad Sci 1104: 35-53. CrossRef Medline
44. Overall JE, Gorham DR (1962) The brief psychiatric rating scale. Psychol Rep 10:799-812. CrossRef
45. Paulus MP, Frank L, Brown GG, Braff DL (2003) Schizophrenia subjects show intact success-related neural activation but impaired uncertainty processing during decision-making. Neuropsychopharmacology 28:795-806. CrossRef Medline
46. Petrides M (1985) Deficits on conditional associative-learning tasks after frontaland temporal-lobe lesions in man. Neuropsychologia 23:601-614. CrossRef Medline
47. Pfohl B, Blum N, Zimmerman M (1997) Structured interview for DSM-IV personality. Washington, DC: American Psychiatric Publishing.
48. Polli FE, Barton JJ, Thakkar KN, Greve DN, Goff DC, Rauch SL, Manoach DS (2008) Reduced error-related activation in two anterior cingulate circuits is related to impaired performance in schizophrenia. Brain 131:971-986. CrossRef Medline
49. Prentice KJ, Gold JM, Buchanan RW (2008) The Wisconsin Card Sorting impairment in schizophrenia is evident in the first four trials. Schizophr Res 106:81-87. CrossRef Medline
50. Schlagenhauf F, Huys QJ, Deserno L, Rapp MA, Beck A, Heinze HJ, Dolan R, Heinz A (2013) Striatal dysfunction during reversal learning in unmedicated schizophrenia patients. Neuroimage 89:171-180. CrossRef Medline
51. Schwarz G (1978) Estimating the dimension of a model. Ann Stat 6:461-464. CrossRef
52. Shurman B, Horan WP, Nuechterlein KH (2005) Schizophrenia patients demonstrate a distinctive pattern of decision-making impairment on the Iowa Gambling Task. Schizophr Res 72:215-224. CrossRef Medline
53. Slifstein M, Kolachana B, Simpson EH, Tabares P, Cheng B, Duvall M, Frankle WG, Weinberger DR, Laruelle M, Abi-Dargham A (2008) COMT genotype predicts cortical-limbic D1 receptor availability measured with [11C]NNC112 and PET. Mol Psychiatry 13:821-827. CrossRef Medline
54. Somlai Z, Moustafa AA, Kéri S, Myers CE, Gluck MA (2011) General functioning predicts reward and punishment learning in schizophrenia. Schizophr Res 127:131-136. CrossRef Medline
55. Stephan KE, Penny WD, Daunizeau J, Moran RJ, Friston KJ (2009) Bayesian model selection for group studies. Neuroimage 46:1004-1017. CrossRef Medline
56. Strauss GP, Robinson BM, Waltz JA, Frank MJ, Kasanova Z, Herbener ES, Gold JM (2011) Patients with schizophrenia demonstrate inconsistent preference judgments for affective and nonaffective stimuli. Schizophr Bull 37:1295-1304. CrossRef Medline
57. Waltz JA, Gold JM (2007) Probabilistic reversal learning impairments in schizophrenia: further evidence of orbitofrontal dysfunction. Schizophr Res 93:296-303. CrossRef Medline
58. Waltz JA, Frank MJ, Robinson BM, Gold JM (2007) Selective reinforcement learning deficits in schizophrenia support predictions from computational models of striatal-cortical dysfunction. Biol Psychiatry 62:756-764. CrossRef Medline
59. Waltz JA, Schweitzer JB, Gold JM, Kurup PK, Ross TJ, Salmeron BJ, Rose EJ, McClure SM, Stein EA (2009) Patients with schizophrenia have a reduced neural response to both unpredictable and predictable primary reinforcers. Neuropsychopharmacology 34:1567-1577. CrossRef Medline
60. Waltz JA, Frank MJ, Wiecki TV, Gold JM (2011) Altered probabilistic learning and response biases in schizophrenia: behavioral evidence and neurocomputational modeling. Neuropsychology 25:86-97. CrossRef Medline
61. Wechsler D (1999) WASI (Wechsler Adult Scale-reduced). New York: The Psychological Corporation.
62. Wiecki TV, Poland J, Frank MJ (2014) Model-based cognitive neuroscience approaches to computational psychiatry: clustering and classification. Clin Psychol Sci, in press.
63. Ziauddeen H, Murray GK (2010) The relevance of reward pathways for schizophrenia. Curr Opin Psychiatry 23:91-96. CrossRef Medline