Orbitofrontal neurons infer the value and identity of predicted outcomes

Nature Communications

Volumn 5, Issue , 2014, Pages

  Stalnaker, Thomas A ^a   Cooch, Nisha K ^b   McDannald, Michael A ^b   Liu, Tzu Lan ^b   Wied, Heather ^b   Schoenbaum, Geoffrey ^a,b,c  

^a NATIONAL INSTITUTE ON DRUG ABUSE   (United States)

^b UNIVERSITY OF MARYLAND SCHOOL OF MEDICINE   (United States)

^c JOHNS HOPKINS UNIVERSITY SCHOOL OF MEDICINE   (United States)

Author keywords

[No Author keywords available]

Indexed keywords

NEUROLOGY; SIGNAL; SIGNALING;

ANIMAL CELL; ANIMAL EXPERIMENT; ANTICIPATION; ARTICLE; BEHAVIOR; CONTROLLED STUDY; FLAVOR; FORCED CHOICE METHOD; FUZZY SYSTEM; NERVE CELL; NONHUMAN; ORBITAL CORTEX; ORBITOFRONTAL NEURON; RAT; REACTION TIME; REWARD; SINGLE UNIT ACTIVITY; ANIMAL; CYTOLOGY; DECISION MAKING; LEARNING; PHYSIOLOGY; PREFRONTAL CORTEX;

ANIMALS; ANTICIPATION, PSYCHOLOGICAL; CHOICE BEHAVIOR; LEARNING; NEURONS; PREFRONTAL CORTEX; RATS; REWARD;

EID: 84901935293     PISSN: None     EISSN: 20411723     Source Type: Journal    
DOI: 10.1038/ncomms4926     Document Type: Article

References (42)

1. Schoenbaum, G. &Esber, G. How do you (estimate you will) like them apples? Integration as a defining trait of orbitofrontal function. Curr. Opin. Neurobiol. 20, 205-211 (2010).
2. Padoa-Schioppa, C. Neurobiology of economic choice: a goods-based model. Annu. Rev. Neurosci. 34, 333-359 (2011).
3. Wallis, J. D. Orbitofrontal cortex and its contribution to decision-making. Annu. Rev. Neurosci. 30, 31-56 (2007). (Pubitemid 47218750)
4. Schoenbaum, G., Chiba, A. A. &Gallagher, M. Orbitofrontal cortex and basolateral amygdala encode expected outcomes during learning. Nat. Neurosci. 1, 155-159 (1998).
5. Levy, D. J. &Glimcher, P. W. Comparing apples and oranges: Using reward-specific and reward-general subjective value representation in the brain. J. Neurosci. 31, 14693-14707 (2011).
6. Padoa-Schioppa, C. &Assad, J. A. Neurons in orbitofrontal cortex encode economic value. Nature 441, 223-226 (2006).
7. O'Doherty, J., Deichmann, R., Critchley, H. D. &Dolan, R. J. Neural responses during anticipation of a primary taste reward. Neuron 33, 815-826 (2002). (Pubitemid 34219251)
8. Gottfried, J. A., O'Doherty, J. &Dolan, R. J. Encoding predictive reward value in human amygdala and orbitofrontal cortex. Science 301, 1104-1107 (2003). (Pubitemid 37022314)
9. Thorpe, S. J., Rolls, E. T. &Maddison, S. The orbitofrontal cortex: neuronal activity in the behaving monkey. Exp. Brain Res. 49, 93-115 (1983). (Pubitemid 13180756)
10. Tremblay, L. &Schultz, W. Relative reward preference in primate orbitofrontal cortex. Nature 398, 704-708 (1999). (Pubitemid 29197648)
11. Kennerley, S. W., Behrens, T. E. &Wallis, J. D. Double dissociation of value computations in orbitofrontal and anterior cingulate neurons. Nat. Neurosci. 14, 1581-1589 (2011).
12. Sul, J. H., Kim, H., Huh, N., Lee, D. &Jung, M. W. Distinct roles of rodent orbitofrontal and medial prefrontal cortex in decision making. Neuron 66, 449-460 (2010).
13. Gallagher, M., McMahan, R. W. &Schoenbaum, G. Orbitofrontal cortex and representation of incentive value in associative learning. J. Neurosci. 19, 6610-6614 (1999). (Pubitemid 29347459)
14. Roesch, M. R., Taylor, A. R. &Schoenbaum, G. Encoding of time-discounted rewards in orbitofrontal cortex is independent of value representation. Neuron 51, 509-520 (2006). (Pubitemid 44189799)
15. Schoenbaum, G. &Eichenbaum, H. Information coding in the rodent prefrontal cortex. I. Single-neuron activity in orbitofrontal cortex compared with that in pyriform cortex. J. Neurophysiol. 74, 733-750 (1995).
16. Feierstein, C. E., Quirk, M. C., Uchida, N., Sosulski, D. L. &Mainen, Z. F. Representation of spatial goals in rat orbitofrontal cortex. Neuron 51, 495-507 (2006). (Pubitemid 44189800)
17. Furuyashiki, T., Holland, P. C. &Gallagher, M. Rat orbitofrontal cortex separately encodes response and outcome information during performance of goal-directed behavior. J. Neurosci. 28, 5127-5138 (2008).
18. Jones, J. L. et al. Orbitofrontal cortex supports behavior and learning using inferred but not cached values. Science 338, 953-956 (2012).
19. McDannald, M. A., Lucantonio, F., Burke, K. A., Niv, Y. &Schoenbaum, G. Ventral striatum and orbitofrontal cortex are both required for model-based, but not model-free, reinforcement learning. J. Neurosci. 31, 2700-2705 (2011).
20. Gremel, C. M. &Costa, R. M. Orbitofrontal and striatal circuits dynamically encode the shift between goal-directed and habitual actions. Nat. Commun. 4, 2264 (2013).
21. Izquierdo, A. D., Suda, R. K. &Murray, E. A. Bilateral orbital prefrontal cortex lesions in rhesus monkeys disrupt choices guided by both reward value and reward contingency. J. Neurosci. 24, 7540-7548 (2004). (Pubitemid 39166745)
22. Rudebeck, P. H., Saunders, R. C., Prescott, A. T., Chau, L. S. &Murray, E. A. Prefrontal mechanisms of behavioral flexibility, emotion regulation and value updating. Nat. Neurosci. 16, 1140-1145 (2013).
23. West, E. A., DesJardin, J. T., Gale, K. &Malkova, L. Transient inactivation of orbitofrontal cortex blocks reinforcer devaluation in macaques. J. Neurosci. 31, 15128-15135 (2011).
24. Dickinson, A. &Balleine, B. W. Motivational control of goal-directed action. Anim. Learn. Behav. 22, 1-18 (1994). (Pubitemid 24054214)
25. Huys, Q. J. et al. Bonsai trees in your head: how the Pavlovian system sculpts goal-directed choices by pruning decision trees. PLoS Comput. Biol. 8, e1002410 (2012).
26. Daw, N. D., Niv, Y. &Dayan, P. Uncertainty-based competition between prefrontal and dorsolateral striatal systems for behavioral control. Nat. Neurosci. 8, 1704-1711 (2005). (Pubitemid 41683198)
27. Holland, P. C. &Rescorla, R. A. The effects of two ways of devaluing the unconditioned stimulus after first and second-order appetitive conditioning. J. Exp. Psychol. 1, 355-363 (1975).
28. Rolls, E. T. The orbitofrontal cortex. Philos. Trans. R. Soc. Lond B Biol. Sci. 351, 1433-1443 (1996).
29. Bechara, A., Damasio, H., Tranel, D. &Damasio, A. R. Deciding advantageously before knowing the advantageous strategy. Science 275, 1293-1294 (1997).
30. Jones, B. &Mishkin, M. Limbic lesions and the problem of stimulusreinforcement associations. Exp. Neurol. 36, 362-377 (1972).
31. McDannald, M. A. et al.Orbitofrontal neurons acquire responses to 'valueless' Pavlovian cues during unblocking. (in review).
32. Ostlund, S. B. &Balleine, B. W. Orbitofrontal cortex mediates outcome encoding in Pavlovian but not instrumental learning. J. Neurosci. 27, 4819-4825 (2007). (Pubitemid 46699058)
33. Wilson, R. C., Takahashi, Y. K., Schoenbaum, G. &Niv, Y. Orbitofrontal cortex as a cognitive map of task space. Neuron 81, 267-279 (2014).
34. Hornak, J. et al. Reward-related reversal learning after surgical excisions in orbito-frontal or dorsolateral prefrontal cortex in humans. J. Cogn. Neurosci. 16, 463-478 (2004). (Pubitemid 38392404)
35. Schoenbaum, G., Nugent, S., Saddoris, M. P. &Setlow, B. Orbitofrontal lesions in rats impair reversal but not acquisition of go, no-go odor discriminations. Neuroreport 13, 885-890 (2002). (Pubitemid 34492920)
36. Walton, M. E., Behrens, T. E. J., Buckley, M. J., Rudebeck, P. H. &Rushworth, M. F. S. Separable learning systems in teh macaque brain and the role of the orbitofrontal cortex in contingent learning. Neuron 65, 927-939 (2010).
37. Abe, H. &Lee, D. Distributed coding of actual and hypothetical outcomes in the orbital and dorsolateral prefrontal cortex. Neuron 70, 731-741 (2011).
38. Barron, H. C., Dolan, R. J. &Behrens, T. E. Online evaluation of novel choices by simultaneous representation of multiple memories. Nat. Neurosci. 16, 1492-1498 (2013).
39. Wimmer, G. E., Daw, N. D. &Shohamy, D. Generalization of value in reinforcement learning in humans. Eur. J. Neurosci. 35, 1092-1104 (2012).
40. Daw, N. D., Gershman, S. J., Seymour, B., Dayan, P. &Dolan, R. J. Model-based influences on humans' choices and striatal prediction errors. Neuron 69, 1204-1215 (2011).
41. Takahashi, Y. K. et al. Neural estimates of imagined outcomes in the orbitofrontal cortex drive behavior and learning. Neuron 80, 507-518 (2013).
42. Stalnaker, T. A., Calhoon, G., Ogawa, M., Roesch, M. R. &Schoenbaum, G. Neural correlates of stimulus-response and response-outcome associations in dorsolateral versus dorsomedial striatum. Front. Integr. Neurosci. 4, 12 (2010).