Akt1 deficiency modulates reward learning and reward prediction error in mice

Genes, Brain and Behavior

Volumn 11, Issue 2, 2012, Pages 157-169

  Chen, Y C ^a   Chen, Y W ^a   Hsu, Y F ^a   Chang, W T ^a   Hsiao, C K ^a   Min, M Y ^a   Lai, W S ^a  

^a NATIONAL TAIWAN UNIVERSITY   (Taiwan)

Author keywords

Akt1; Bayesian hierarchical model; Dopamine dependent behaviors; Dynamic foraging T maze; Methamphetamine; Mice; Reinforcement learning model; Reward learning; Reward prediction error

Indexed keywords

3,4 DIHYDROXYPHENYLACETIC ACID; DOPAMINE; HOMOVANILLIC ACID; LEVODOPA; METHAMPHETAMINE; PROTEIN KINASE B; SUCROSE;

ANIMAL EXPERIMENT; ANIMAL TISSUE; ARTICLE; AVERSION; CONTROLLED STUDY; CORPUS STRIATUM; DECISION MAKING; ENZYME ACTIVITY; EXPERIMENTAL MODEL; FORAGING; FRONTAL CORTEX; GENE DOSAGE; GENOTYPE; HETEROZYGOTE; HIPPOCAMPUS; LEARNING; LICKING; MALE; MAZE TEST; MESENCEPHALON; METABOLITE; MOUSE; NONHUMAN; PREDICTION; PRIORITY JOURNAL; PROTEIN EXPRESSION; REINFORCEMENT; REWARD; REWARD PREDICTION ERROR; SOMATOSENSORY CORTEX; TASK PERFORMANCE; TASTE PREFERENCE; WILD TYPE;

AMPHETAMINE; ANIMALS; BEHAVIOR, ANIMAL; CORPUS STRIATUM; DOPAMINE; DOPAMINE UPTAKE INHIBITORS; LEARNING; MALE; MICE; MICE, KNOCKOUT; MODELS, NEUROLOGICAL; PROTO-ONCOGENE PROTEINS C-AKT; REWARD;

MUS;

EID: 84863073912     PISSN: 16011848     EISSN: 1601183X     Source Type: Journal    
DOI: 10.1111/j.1601-183X.2011.00759.x     Document Type: Article

References (56)

1. Balleine, B.W., Daw, N. & O'Doherty, J.P. (2008) Multiple forms of value learning and the function of dopamine. In Glimcher, P.W., Fehr, E., Camerer, C. & Poldrack, R.A. (eds), Neuroeconomics: Decision-Making and the Brain. Elsevier, New York.
2. Baum, W. (1974) On two types of deviation from the matching law: bias and undermatching. J Exper Anal Behav 22, 231.
3. Bayer, H.M. & Glimcher, P.W. (2005) Midbrain dopamine neurons encode a quantitative reward prediction error signal. Neuron 47, 129-141.
4. Beaulieu, J.M., Gainetdinov, R.R. & Caron, M.G. (2007) The Akt-GSK-3 signaling cascade in the actions of dopamine. Trend Pharmacol Sci 28, 166-172.
5. Beaulieu, J.M., Gainetdinov, R.R. & Caron, M.G. (2009) Akt/GSK3 Signaling in the action of psychotropic drugs. Annu Rev Pharmacol Toxicol 49, 327-347.
6. Beaulieu, J.M., Sotnikova, T.D., Marion, S., Lefkowitz, R.J., Gainetdinov, R.R. & Caron, M.G. (2005) An Akt/beta-arrestin 2/PP2A signaling complex mediates dopaminergic neurotransmission and behavior. Cell 122, 261-273.
7. Bromberg-Martin, E.S., Matsumoto, M. & Hikosaka, O. (2010) Dopamine in motivational control: rewarding, aversive, and alerting. Neuron 68, 815-834.
8. Caplin, A. & Dean, M. (2008) Axiomatic methods, dopamine and reward prediction error. Curr Opin Neurobiol 18, 197-202.
9. Chen, Y.W. & Lai, W.S. (2011) Behavioral phenotyping of v-akt murine thymoma viral oncogene homolog 1-deficient mice reveals a sex-specific prepulse inhibition deficit in females that can be partially alleviated by glycogen synthase kinase-3 inhibitors but not by antipsychotics. Neuroscience 174, 178-189.
10. Cho, H., Thorvaldsen, J.L., Chu, Q.W., Feng, F. & Birnbaum, M.J. (2001) Akt1/PKB alpha is required for normal growth but dispensable for maintenance of glucose homeostasis in mice. J Biol Chem 276, 38349-38352.
11. Cohen, M.X., Young, J., Baek, J.M., Kessler, C. & Ranganath, C. (2005) Individual differences in extraversion and dopamine genetics predict neural reward responses. Cogn Brain Res 25, 851-861.
12. Corlett, P.R., Murray, G.K., Honey, G.D., Aitken, M.R., Shanks, D.R., Robbins, T.W., Bullmore, E.T., Dickinson, A. & Fletcher, P.C. (2007) Disrupted prediction-error signal in psychosis: evidence for an associative account of delusions. Brain 130, 2387-2400.
13. Emamian, E.S., Hall, D., Birnbaum, M.J., Karayiorgou, M. & Gogos, J.A. (2004) Convergent evidence for impaired AKT1-GSK3 beta signaling in schizophrenia. Nat Genet 36, 131-137.
14. Fletcher, P.C. & Frith, C.D. (2009) Perceiving is believing: a Bayesian approach to explaining the positive symptoms of schizophrenia. Nat Rev Neurosci 10, 48-58.
15. Frank, M.J., Moustafa, A.A., Haughey, H.M., Curran, T. & Hutchison, K.E. (2007) Genetic triple dissociation reveals multiple roles for dopamine in reinforcement learning. Proc Natl Acad Sci U S A 104, 16311-16316.
16. Frank, M.J., Woroch, B.S. & Curran, T. (2005) Error-related negativity predicts reinforcement learning and conflict biases. Neuron 47, 495-501.
17. Garety, P.A., Hemsley, D.R. & Wessely, S. (1991) Reasoning in deluded schizophrenic and paranoid patients. Biases in performance on a probabilistic inference task. J Nerv Mental Dis 179, 194-201.
18. Glimcher, P.W. (2011) Quantification of Behavior Sackler Colloquium: Understanding dopamine and reinforcement learning: the dopamine reward prediction error hypothesis. Proc Natl Acad Sci U S A.
19. Hollerman, J.R. & Schultz, W. (1998) Dopamine neurons report an error in the temporal prediction of reward during learning. Nat Neurosci 1, 304-309.
20. Howes, O.D. & Kapur, S. (2009) The dopamine hypothesis of schizophrenia: version III-the final common pathway. Schizophr Bull 35, 549-562.
21. Jocham, G., Neumann, J., Klein, T.A., Danielmeier, C. & Ullsperger, M. (2009) Adaptive Coding of Action Values in the Human Rostral Cingulate Zone. J Neurosci 29, 7489-7496.
22. Joel, D. & Weiner, I. (2000) The connections of the dopaminergic system with the striatum in rats and primates: an analysis with respect to the functional and compartmental organization of the striatum. Neuroscience 96, 451-474.
23. Jonsson, E.G., Nothen, M.M., Grunhage, F., Farde, L., Nakashima, Y., Propping, P. & Sedvall, G.C. (1999) Polymorphisms in the dopamine D2 receptor gene and their relationships to striatal dopamine receptor density of healthy volunteers. Mol Psychiat 4, 290-296.
24. Juckel, G., Schlagenhauf, F., Koslowski, M., Wustenberg, T., Villringer, A., Knutson, B., Wrase, J. & Heinz, A. (2006) Dysfunction of ventral striatal reward prediction in schizophrenia. Neuroimage 29, 409-416.
25. Kaelbling, L., Littman, M. & Moore, A. (1996) Reinforcement learning: A survey. J Artif Intell Res 4, 102-138.
26. Kapur, S. (2003) Psychosis as a state of aberrant salience: a framework linking biology, phenomenology, and pharmacology in schizophrenia. Am J Psych 160, 13-23.
27. Kapur, S. & Mamo, D. (2003) Half a century of antipsychotics and still a central role for dopamine D-2 receptors. Prog Neuro Psychopharmacol Biol Psychiat 27, 1081-1090.
28. Kapur, S., Mizrahi, R. & Li, M. (2005) From dopamine to salience to psychosis-linking biology, pharmacology and phenomenology of psychosis. Schizophr Res 79, 59-68.
29. Kirsch, P., Reuter, M., Mier, D., Lonsdorf, T., Stark, R., Gallhofer, B., Vaitl, D. & Hennig, J. (2006) Imaging gene-substance interactions: the effect of the DRD2 TaqIA polymorphism and the dopamine agonist bromocriptine on the brain activation during the anticipation of reward. Neurosci Lett 405, 196-201.
30. Klein, T.A., Neumann, J., Reuter, M., Hennig, J., von Cramon, D.Y. & Ullsperger, M. (2007) Genetically determined differences in learning from errors. Science 318, 1642-1645.
31. Knutson, B. & Cooper, J.C. (2005) Functional magnetic resonance imaging of reward prediction. Curr Opin Neurol 18, 411-417.
32. Lai, W.S., Xu, B., Westphal, K., Paterlini, M., Olivier, B., Pavlidis, P., Karayiorgou, M. & Gogos, J. (2006) Akt1 deficiency affects neuronal morphology and predisposes to abnormalities in prefrontal cortex functioning. Proc Natl Acad Sci 103, 16906.
33. Lunn, D.J., Thomas, A., Best, N. & Spiegelhalter, D. (2000) WinBUGS - a Bayesian modelling framework: concepts, structure, and extensibility. Stat Comput 10, 325-337.
34. Matsumoto, M. & Hikosaka, O. (2009a) Representation of negative motivational value in the primate lateral habenula. Nat Neurosci 12, 77-84.
35. Matsumoto, M. & Hikosaka, O. (2009b) Two types of dopamine neuron distinctly convey positive and negative motivational signals. Nature 459, 837-841.
36. McClure, S.M., Berns, G.S. & Montague, P.R. (2003) Temporal prediction errors in a passive learning task activate human striatum. Neuron 38, 339-346.
37. Montague, P.R., Hyman, S.E. & Cohen, J.D. (2004) Computational roles for dopamine in behavioural control. Nature 431, 760-767.
38. Murray, G.K., Corlett, P.R., Clark, L., Pessiglione, M., Blackwell, A.D., Honey, G., Jones, P.B., Bullmore, E.T., Robbins, T.W. & Fletcher, P.C. (2008) Substantia nigra/ventral tegmental reward prediction error disruption in psychosis. Mol Psychiat 13, 267-276.
39. Oyama, K., Hernadi, I., Iijima, T. & Tsutsui, K. (2010) Reward prediction error coding in dorsal striatal neurons. J Neurosci 30, 11447-11457.
40. Roux, S., Froger, C., Porsolt, R.D., Valverde, O. & Maldonado, R. (2003) Place preference test in rodents. Curr Protoc Neurosci 9, 15.
41. Rutledge, R.B., Lazzaro, S.C., Lau, B., Myers, C.E., Gluck, M.A. & Glimcher, P.W. (2009) Dopaminergic drugs modulate learning rates and perseveration in Parkinson's patients in a dynamic foraging task. J Neurosci 29, 15104-15114.
42. Samejima, K., Ueda, Y., Doya, K. & Kimura, M. (2005) Representation of action-specific reward values in the striatum. Science 310, 1337-1340.
43. Schultz, W., Dayan, P. & Montague, P.R. (1997) A neural substrate of prediction and reward. Science 275, 1593-1599.
44. Schultz, W., Tremblay, L. & Hollerman, J.R. (1998) Reward prediction in primate basal ganglia and frontal cortex. Neuropharmacology 37, 421-429.
45. Sul, J.H., Kim, H., Huh, N., Lee, D. & Jung, M.W. (2010) Distinct roles of rodent orbitofrontal and medial prefrontal cortex in decision making. Neuron 66, 449-460.
46. Suri, R.E., Bargas, J. & Arbib, M.A. (2001) Modeling functions of striatal dopamine modulation in learning and planning. Neuroscience 103, 65-85.
47. Sutton, R. & Barto, A. (1998) Reinforcement Learning: An Introduction. MIT Press, Cambridge, M.A.
48. Tan, H.Y., Chen, A.G., Chen, Q., Browne, L.B., Verchinski, B., Kolachana, B., Zhang, F., Apud, J., Callicott, J.H., Mattay, V.S. & Weinberger, D.R. (2011) Epistatic interactions of AKT1 on human medial temporal lobe biology and pharmacogenetic implications. Mol Psychiatry.
49. Tan, H.Y., Nicodemus, K.K., Chen, Q., Li, Z., Brooke, J.K., Honea, R., Kolachana, B.S., Straub, R.E., Meyer-Lindenberg, A., Sei, Y., Mattay, V.S., Callicott, J.H. & Weinberger, D.R. (2008) Genetic variation in AKT1 is linked to dopamine-associated prefrontal cortical structure and function in humans. J Clin Invest 118, 2200-2208.
50. Tzschentke, T.M. (1998) Measuring reward with the conditioned place preference paradigm: a comprehensive review of drug effects, recent progress and new issues. Prog Neurobiol 56, 613-672.
51. Van den Heuvel, D.M. & Pasterkamp, R.J. (2008) Getting connected in the dopamine system. Prog Neurobiol 85, 75-93.
52. Watkins, C. & Dayan, P. (1992) Technical note: Q-learning. Mach Learn 8, 279-292.
53. Wichmann, T., Smith, Y. & Vitek, J.L. (2002) In Factor, S.A. & Weiner, W.J. (eds), Parkinson's Disease: Diagnosis and Clinical Management. Demos Medical Publishing, New York.
54. Wise, R.A. (2004) Dopamine, learning and motivation. Nat Rev Neurosci 5, 483-494.
55. Yin, H.S., Lai, C.C., Tien, T.W., Han, S.K. & Pu, X.L. (2010) Differential changes in cerebellar transmitter content and expression of calcium binding proteins and transcription factors in mouse administered with amphetamine. Neurochem Int 57, 288-296.
56. Zhao, Z., Ksiezak-Reding, H., Riggio, S., Haroutunian, V. & Pasinetti, G.M. (2006) Insulin receptor deficits in schizophrenia and in cellular and animal models of insulin receptor dysfunction. Schizophr Res 84, 1-14.