A functional difference in information processing between orbitofrontal cortex and ventral striatum during decision-making behaviour

Philosophical Transactions of the Royal Society B: Biological Sciences

Volumn 369, Issue 1655, 2014, Pages

  Stott, Jeffrey J ^a   David Redish, A ^a  

^a UNIVERSITY OF MINNESOTA   (United States)

Author keywords

Decision making; Neuroeconomics; Nucleus accumbens; Orbitofrontal cortex; Vicarious trial and error

Indexed keywords

BEHAVIORAL RESPONSE; CELL ORGANELLE; DECISION MAKING; MAGNITUDE; NUMERICAL MODEL; PREDICTION; SIGNALING;

ANIMALIA; RATTUS;

ANIMAL; DECISION MAKING; ELECTROPHYSIOLOGY; FISCHER 344 RAT; INFORMATION PROCESSING; MALE; MAZE TEST; PHYSIOLOGY; PREFRONTAL CORTEX; PROCEDURES; PSYCHOMOTOR PERFORMANCE; RANDOMIZATION; RAT; VENTRAL STRIATUM;

ANIMALS; AUTOMATIC DATA PROCESSING; DECISION MAKING; ELECTROPHYSIOLOGY; MALE; MAZE LEARNING; PREFRONTAL CORTEX; PSYCHOMOTOR PERFORMANCE; RANDOM ALLOCATION; RATS; RATS, INBRED F344; VENTRAL STRIATUM;

EID: 84907487022     PISSN: 09628436     EISSN: 14712970     Source Type: Journal    
DOI: 10.1098/rstb.2013.0472     Document Type: Article

References (71)

1. Daw ND, Doya K. 2006 The computational neurobiology of learning and reward. Curr. Opin. Neurobiol. 16, 199-204. (doi:10.1016/j.conb.2006.03.006)
2. Haber SN, Knutson B. 2010 The reward circuit: linking primate anatomy and human imaging. Neuropsychopharmacology 35, 4–26. (doi:10.1038/npp.2009.129)
3. Kable JW, Glimcher PW. 2009 The neurobiology of decision: consensus and controversy. Neuron 63, 733-745. (doi:10.1016/j.neuron.2009.09.003)
4. Lee D, Seo H, Jung MW. 2012 Neural basis of reinforcement learning and decision making. Annu. Rev. Neurosci. 35, 287-308. (doi:10.1146/annurev-neuro-062111-150512)
5. Rangel A, Camerer C, Montague PR. 2008 A framework for studying the neurobiology of value-based decision making. Nat. Rev. Neurosci. 9, 545-556. (doi:10.1038/nrn2357)
6. van Duuren E, Lankelma J, Pennartz CMA. 2008 Population coding of reward magnitude in the orbitofrontal cortex of the rat. J. Neurosci. 28, 8590-8603. (doi:10.1523/JNEUROSCI5549-072008)
7. Jones JL, Esber GR, McDannald MA, Gruber AJ, Hernandez A, Mirenzi A, Schoenbaum G. 2012 Orbitofrontal cortex supports behavior and learning using inferred but not cached values. Science 338, 953-956. (doi:10.1126/science.1227489)
8. Lansink CS, Goltstein PM, Lankelma JV, Joosten RN, McNaughton BL, Pennartz CM. 2008 Preferential reactivation of motivationally relevant information in the ventral striatum. J. Neurosci. 28, 6372–6382. (doi:10.1523/JNEUROSCI.1054-082008)
9. McDannald MA, Lucantonio F, Burke KA, Niv Y, Schoenbaum G. 2011 Ventral striatum and orbitofrontal cortex are both required for model-based, but not model-free, reinforcement learning. J. Neurosci. 31, 2700-2705. (doi:10.1523/JNEUROSCI.5499-10.2011)
10. van der Meer MA, Redish AD. 2009 Covert expectation-of-reward in rat ventral striatum at decision points. Front. Integr. Neurosci. 3, 1. (doi:10.3389/neuro.07.001.2009)
11. van der Meer MA, Kurth-Nelson Z, Redish AD. 2012 Information processing in decision-making systems. The Neuroscientist 18, 342-359. (doi:10.1177/1073858411435128)
12. Padoa-Schioppa C, Cai X. 2011 The orbitofrontal cortex and the computation of subjective value: consolidated concepts and new perspectives. Ann. NY Acad. Sci. 1239, 130-137. (doi:10.1111/j.1749-6632.2011.06262.x)
13. Steiner AP, Redish AD. 2012 The road not taken: neural correlates of decision making in orbitofrontal cortex. Front. Neurosci. 6, 131. (doi:10.3389/fnins.2012.00131)
14. van Duuren E, van der Plasse G, Lankelma J, Joosten RN, Feenstra MGP, Pennartz CMA. 2009 Single-cell and population coding of expected reward probability in the orbitofrontal cortex of the rat. J. Neurosci. 29, 8965-8976. (doi:10.1523/JNEUROSCI.0005-09.2009)
15. van Duuren E, Escámez FAN, Joosten RN, Visser R, Mulder AB, Pennartz CMA. 2007 Neural coding of reward magnitude in the orbitofrontal cortex of the rat during a five-odor olfactory discrimination task. Learn. Mem. 14, 446-456. (doi:10.1101/lm.546207)
16. Breiter HC, Aharon I, Kahneman D, Dale A, Shizgal P. 2001 Functional imaging of neural responses to expectancy and experience of monetary gains and losses. Neuron 30, 619–639. (doi:10.1016/S0896-6273(01)00303-8)
17. Knutson B, Adams CM, Fong GW, Hommer D. 2001 Anticipation of increasing monetary reward selectively recruits nucleus accumbens. J. Neurosci. 21, RC159.
18. Knutson B, Taylor J, Kaufman M, Peterson R, Glover G. 2005 Distributed neural representation of expected value. J. Neurosci. 25, 4806–4812. (doi:10.1523/JNEUROSCI.0642-05.2005)
19. O’Doherty JP. 2004 Reward representations and reward-related learning in the human brain: insights from neuroimaging. Curr. Opin. Neurobiol. 14, 769-776. (doi:10.1016/j.conb.2004.10.016)
20. Kable JW, Glimcher PW. 2007 The neural correlates of subjective value during intertemporal choice. Nat. Neurosci. 10, 1625-1633. (doi:10.1038/nn2007)
21. Levy I, Snell J, Nelson AJ, Rustichini A, Glimcher PW. 2010 Neural representation of subjective value under risk and ambiguity. J. Neurophysiol. 103, 1036–1047. (doi:10.1152/jn.00853.2009)
22. McClure SM, Laibson DI, Loewenstein G, Cohen JD. 2004 Separate neural systems value immediate and delayed monetary rewards. Science 306, 503–507. (doi:10.1126/science.1100907)
23. Diekhof EK, Kaps L, Falkai P, Gruber O. 2012 The role of the human ventral striatum and the medial orbitofrontal cortex in the representation of reward magnitude: an activation likelihood estimation meta-analysis of neuroimaging studies of passive reward expectancy and outcome processing. Neuropsychologia 50, 1252-1266. (doi:10.1016/j.neuropsychologia.2012.02.007)
24. Levy DJ, Glimcher PW. 2012 The root of all value: a neural common currency for choice. Curr. Opin. Neurobiol. 22, 1027-1038. (doi:10.1016/j.conb.2012.06.001)
25. Papale AE, Stott JJ, Powell NJ, Regier PS, Redish AD. 2012 Interactions between deliberation and delay-discounting in rats. Cogn. Affect. Behav. Neurosci. 12, 513-526. (doi:10.3758/s13415-012-0097-7)
26. Muenzinger KF. 1938 Vicarious trial and error at a point of choice. I. A general survey of its relation to learning efficiency. Pedagog. Semin. J. Genet. Psychol. 53, 75-86. (doi:10.1080/08856559.1938.10533799)
27. Tolman EC. 1939 Prediction of vicarious trial and error by means of the schematic sowbug. Psychol. Rev. 46, 318-336. (doi:10.1037/h0057054)
28. Tolman EC. 1948 Cognitive maps in rats and men. Psychol. Rev. 55, 189-208. (doi:10.1037/h0061626)
29. Johnson A, van der Meer MA, Redish AD. 2007 Integrating hippocampus and striatum in decision-making. Curr. Opin. Neurobiol. 17, 692–697. (doi:10.1016/j.conb.2008.01.003)
30. Nicola SM. 2010 The flexible approach hypothesis: unification of effort and cue-responding hypotheses for the role of nucleus accumbens dopamine in the activation of reward-seeking behavior. J. Neurosci. 30, 16585-16600. (doi:10.1523/JNEUROSCI.3958-10.2010)
31. Johnson A, Redish AD. 2007 Neural ensembles in CA3 transiently encode paths forward of the animal at a decision point. J. Neurosci. 27, 12176–12189. (doi:10.1523/JNEUROSCI.3761-072007)
32. Yin HH, Knowlton BJ, Balleine BW. 2004 Lesions of dorsolateral striatum preserve outcome expectancy but disrupt habit formation in instrumental learning. Eur. J. Neurosci. 19, 181-189. (doi:10.1111/j.1460-9568.2004.03095.x)
33. Yin HH, Knowlton BJ. 2006 The role of the basal ganglia in habit formation. Nat. Rev. Neurosci. 7, 464-476. (doi:10.1038/nrn1919)
34. Graybiel AM. 2008 Habits, rituals, and the evaluative brain. Annu. Rev. Neurosci. 31, 359–387. (doi:10.1146/annurev.neuro.29.051605.112851)
35. Schmitzer-Torbert NC, Redish AD. 2004 Neuronal activity in the rodent dorsal striatum in sequential navigation: separation of spatial and reward responses on the multiple T task. J. Neurophysiol. 91, 2259-2272. (doi:10.1152/jn.006872003)
36. Paxinos G, Watson C. 1998 The rat brain: in stereotaxic coordinates, 4th edn. New York, NY: Academic Press.
37. Kadir SN, Goodman DFM, Harris KD. 2013 High-dimensional cluster analysis with the masked EM algorithm. http://arxiv.org/abs/1309.2848.
38. Hart WE, Goldbaum M, Coˆte´ B, Kube P, Nelson MR. 1999 Measurement and classification of retinal vascular tortuosity. Int. J. Med. Inf. 53, 239–252. (doi:10.1016/S1386-5056(98)00163-4)
39. Janabi-Sharifi F, Hayward V, Chen C. 2000 Discrete-time adaptive windowing for velocity estimation. IEEE Trans. Control Syst. Technol. 8, 1003–1009. (doi:10.1109/87.880606)
40. Masimore B, Schmitzer-Torbert NC, Kakalios J, Redish AD. 2005 Transient striatal gamma local field potentials signal movement initiation in rats. Neuroreport 16, 2021-2024. (doi:10.1097/00001756-200512190-00010)
41. Zhang K, Ginzburg I, McNaughton BL, Sejnowski TJ. 1998 Interpreting neuronal population activity by reconstruction: unified framework with application to hippocampal place cells. J. Neurophysiol. 79, 1017–1044.
42. Jackson JC, Johnson A, Redish AD. 2006 Hippocampal sharp waves and reactivation during awake states depend on repeated sequential experience. J. Neurosci. 26, 12415-12426. (doi:10.1523/JNEUROSCI.4118-06.2006)
43. Lavoie AM, Mizumori SJ. 1994 Spatial, movement-and reward-sensitive discharge by medial ventral striatum neurons of rats. Brain Res. 638, 157–168. (doi:10.1016/0006-8993(94)90645-9)
44. Roesch MR, Stalnaker TA, Schoenbaum G. 2007 Associative encoding in anterior piriform cortex versus orbitofrontal cortex during odor discrimination and reversal learning. Cereb. Cortex 17, 643-652. (doi:10.1093/cercor/bhk009)
45. Schoenbaum G, Chiba AA, Gallagher M. 1998 Orbitofrontal cortex and basolateral amygdala encode expected outcomes during learning. Nat. Neurosci. 1, 155-159. (doi:10.1038/407)
46. Setlow B, Schoenbaum G, Gallagher M. 2003 Neural encoding in ventral striatum during olfactory discrimination learning. Neuron 38, 625–636. (doi:10.1016/S0896-6273(03)00264-2)
47. Rieke F, Warland D, de Ruyter van Steveninck R, Bialek W. 1997 Spikes: exploring the neural code. Cambridge, MA: MIT Press.
48. Gallagher M, McMahan RW, Schoenbaum G. 1999 Orbitofrontal cortex and representation of incentive value in associative learning. J. Neurosci. 19, 6610–6614.
49. Schultz W, Apicella P, Scarnati E, Ljungberg T. 1992 Neuronal activity in monkey ventral striatum related to the expectation of reward. J. Neurosci. 12, 4595–4610.
50. van der Meer MA, Johnson A, Schmitzer-Torbert NC, Redish AD. 2010 Triple dissociation of information processing in dorsal striatum, ventral striatum, and hippocampus on a learned spatial decision task. Neuron 67, 25-32. (doi:10.1016/j.neuron.2010.06.023)
51. Feierstein CE, Quirk MC, Uchida N, Sosulski DL, Mainen ZF. 2006 Representation of spatial goals in rat orbitofrontal cortex. Neuron 51, 495–507. (doi:10.1016/j.neuron.2006.06.032)
52. German PW, Fields HL. 2007 Rat nucleus accumbens neurons persistently encode locations associated with morphine reward. J. Neurophysiol. 97, 2094-2106. (doi:10.1152/jn.00304.2006)
53. Takahashi YK, Roesch MR, Wilson RC, Toreson K, O’Donnell P, Niv Y, Schoenbaum G. 2011 Expectancy-related changes in firing of dopamine neurons depend on orbitofrontal cortex. Nat. Neurosci. 14, 1590-1597. (doi:10.1038/nn.2957)
54. Wilson RC, Takahashi YK, Schoenbaum G, Niv Y. 2014 Orbitofrontal cortex as a cognitive map of task space. Neuron 81, 267-279. (doi:10.1016/j.neuron.2013.11.005)
55. Walton ME, Behrens TE, Noonan MP, Rushworth MF. 2011 Giving credit where credit is due: orbitofrontal cortex and valuation in an uncertain world. Ann. NY Acad. Sci. 1239, 14-24. (doi:10.1111/j.1749-6632.2011.06257.x)
56. Wunderlich K, Rangel A, O’Doherty JP. 2009 Neural computations underlying action-based decision making in the human brain. Proc. Natl Acad. Sci. USA 106, 17199-17204. (doi:10.1073/pnas.0901077106)
57. Larsen T, O’Doherty JP. 2014 Uncovering the spatio-temporal dynamics of value-based decision-making in the human brain: a combined fMRI–EEG study. Phil. Trans. R. Soc. B 369, 20130473. (doi:10.1098/rstb.2013.0473)
58. Padoa-Schioppa C, Assad JA. 2006 Neurons in the orbitofrontal cortex encode economic value. Nature 441, 223-226. (doi:10.1038/nature04676)
59. Tremblay L, Schultz W. 1999 Relative reward preference in primate orbitofrontal cortex. Nature 398, 704-708. (doi:10.1038/19525)
60. van Wingerden M, Vinck M, Tijms V, Ferreira IRS, Jonker AJ, Pennartz CMA. 2012 NMDA receptors control cue-outcome selectivity and plasticity of orbitofrontal firing patterns during associative stimulus-reward learning. Neuron 76, 813–825. (doi:10.1016/j.neuron.2012.09.039)
61. Ostlund SB, Balleine BW. 2007 Orbitofrontal cortex mediates outcome encoding in Pavlovian but not instrumental conditioning. J. Neurosci. 27, 4819–4825. (doi:10.1523/JNEUROSCI5443-06.2007)
62. West EA, DesJardin JT, Gale K, Malkova L. 2011 Transient inactivation of orbitofrontal cortex blocks reinforcer devaluation in macaques. J. Neurosci. 31, 15128-15135. (doi:10.1523/JNEUROSCI.3295-11.2011)
63. Fuchs RA, Evans KA, Parker MP, See RE. 2004 Differential involvement of orbitofrontal cortex subregions in conditioned cue-induced and cocaine- primed reinstatement of cocaine seeking in rats. J. Neurosci. 24, 6600-6610. (doi:10.1523/JNEUROSCI.1924-04.2004)
64. Mar AC, Walker ALJ, Theobald DE, Eagle DM, Robbins TW. 2011 Dissociable effects of lesions to orbitofrontal cortex subregions on impulsive choice in the rat. J. Neurosci. 31, 6398-6404. (doi:10.1523/JNEUROSCI.6620-102011)
65. Stopper CM, Green EB, Floresco SB. 2014 Selective involvement by the medial orbitofrontal cortex in biasing risky, but not impulsive, choice. Cereb. Cortex 24, 154-162. (doi:10.1093/cercor/bhs297)
66. Walton ME, Behrens TEJ, Buckley MJ, Rudebeck PH, Rushworth MFS. 2010 Separable learning systems in the macaque brain and the role of orbitofrontal cortex in contingent learning. Neuron 65, 927–939. (doi:10.1016/j.neuron.2010.02.027)
67. Noonan MP, Walton ME, Behrens TEJ, Sallet J, Buckley MJ, Rushworth MFS. 2010 Separate value comparison and learning mechanisms in macaque medial and lateral orbitofrontal cortex. Proc. Natl Acad. Sci. USA 107, 20547-20552. (doi:10.1073/pnas.1012246107)
68. Noonan MP, Mars RB, Rushworth MFS. 2011 Distinct roles of three frontal cortical areas in reward-guided behavior. J. Neurosci. 31, 14399-14412. (doi:10.1523/JNEUROSCI.6456-10.2011)
69. McDannald MA, Takahashi YK, Lopatina N, Pietras BW, Jones JL, Schoenbaum G. 2012 Model-based learning and the contribution of the orbitofrontal cortex to the model-free world. Eur. J. Neurosci. 35, 991-996. (doi:10.1111/j.1460-9568.2011.07982.x)
70. Schoenbaum G, Roesch MR, Stalnaker TA, Takahashi YK. 2009 A new perspective on the role of the orbitofrontal cortex in adaptive behaviour. Nat. Rev. Neurosci. 10, 885-892. (doi:10.1038/nrn2753)
71. van der Meer MA, Redish AD. 2010 Expectancies in decision making, reinforcement learning, and ventral striatum. Front. Neurosci. 4, 6. (doi:10.3389/neuro.01.006.2010)