Learning theory: A driving force in understanding orbitofrontal function

Neurobiology of Learning and Memory

Volumn 108, Issue , 2014, Pages 22-27

  McDannald, Michael A ^a   Jones, Joshua L ^b   Takahashi, Yuji K ^a   Schoenbaum, Geoffrey ^a  

^a NATIONAL INSTITUTE ON DRUG ABUSE   (United States)

^b UNIVERSITY OF MARYLAND SCHOOL OF MEDICINE   (United States)

Author keywords

Blocking; Devaluation; Learning theory; Orbitofrontal; Overexpectation; Reversal; Sensory preconditioning; Transfer

Indexed keywords

BLOCKING; DEVALUATION; LEARNING THEORY; ORBITOFRONTAL; OVEREXPECTATION; REVERSAL; SENSORY PRECONDITIONING; TRANSFER;

ANIMALS; HUMANS; LEARNING; MODELS, PSYCHOLOGICAL; PREFRONTAL CORTEX; REINFORCEMENT (PSYCHOLOGY); REWARD;

EID: 84893752030     PISSN: 10747427     EISSN: 10959564     Source Type: Journal    
DOI: 10.1016/j.nlm.2013.06.003     Document Type: Review

References (56)

1. Brogden W.J. Sensory pre-conditioning. Journal of Experimental Psychology 1939, 25:323-332.
2. Burke K.A., Franz T.M., Miller D.N., Schoenbaum G. The role of the orbitofrontal cortex in the pursuit of happiness and more specific rewards. Nature 2008, 454:340-344.
3. Burke K.A., Takahashi Y.K., Correll J., Leon Brown P., Schoenbaum G. Orbitofrontal inactivation impairs reversal of Pavlovian learning by interfering with 'disinhibition' of responding for previously unrewarded cues. European Journal of Neuroscience 2009, 30:1941-1946.
4. Chudasama Y., Robbins T.W. Dissociable contributions of the orbitofrontal and infralimbic cortex to Pavlovian autoshaping and discrimination reversal learning: Further evidence for the functional heterogeneity of the rodent frontal cortex. Journal of Neuroscience 2003, 23:8771-8780.
5. Clark J.J., Hollon N.G., Phillips P.E.M. Pavlovian valuation systems in learning and decision making. Current Opinion in Neurobiology 2012, 22:1054-1061.
6. Colwill R.M., Rescorla R.A. Instrumental responding remains sensitive to reinforcer devaluation after extensive training. Journal of Experimental Psychology-Animal Behavior Processes 1985, 11:520-536.
7. Colwill R.M., Rescorla R.A. Postconditioning devaluation of a reinforcer affects instrumental responding. Journal of Experimental Psychology-Animal Behavior Processes 1985, 11:120-132.
8. Corbit L.H., Balleine B.W. Double dissociation of basolateral and central amygdala lesions on the general and outcome-specific forms of Pavlovian-instrumental transfer. Journal of Neuroscience 2005, 25:962-970.
9. Corbit L.H., Balleine B.W. The general and outcome-specific forms of Pavlovian-instrumental transfer are differentially mediated by the nucleus accumbens core and shell. Journal of Neuroscience 2011, 31:11786-11794.
10. Corbit L.H., Janak P.H., Balleine B.W. General and outcome-specific forms of Pavlovian-instrumental transfer: The effect of shifts in motivational state and inactivation of the ventral tegmental area. European Journal of Neuroscience 2007, 26:3141-3149.
11. Delamater A.R. The role of the orbitofrontal cortex in sensory-specific encoding of associations in pavlovian and instrumental conditioning. Annals of the New York Academy of Sciences 2007, 1121:152-173.
12. Dias R., Robbins T.W., Roberts A.C. Dissociation in prefrontal cortex of affective and attentional shifts. Nature 1996, 380:69-72.
13. Eagle D.M., Baunez C., Hutcheson D.M., Lehmann O., Shah A.P., Robbins T.W. Stop-signal reaction-time task performance: Role of prefrontal cortex and subthalamic nucleus. Cerebral Cortex 2008, 18:178-188.
14. Ferrier D. The functions of the brain 1876, GP Putnam's Sons, New York.
15. Fineberg N.A., Potenza M.N., Chamberlain S.R., Berlin H.A., Menzies L., Bechara A., et al. Probing compulsive and impulsive behaviors, from animal models to endophenotypes: A narrative review. Neuropsychopharmacology 2010, 35:591-604.
16. Gallagher M., McMahan R.W., Schoenbaum G. Orbitofrontal cortex and representation of incentive value in associative learning. Journal of Neuroscience 1999, 19:6610-6614.
17. Gottfried J.A., O'Doherty J., Dolan R.J. Encoding predictive reward value in human amygdala and orbitofrontal cortex. Science 2003, 301:1104-1107.
18. Haney R.Z., Calu D.J., Takahashi Y.K., Hughes B.W., Schoenbaum G. Inactivation of the central but not the basolateral nucleus of the amygdala disrupts learning in response to overexpectation of reward. Journal of Neuroscience 2010, 30:2911-2917.
19. Holland P.C. Unblocking in Pavlovian appetitive conditioning. Journal of Experimental Psychology: Animal Behavior Processes 1984, 10:476-497.
20. Holland P.C. Excitation and inhibition in unblocking. Journal of Experimental Psychology: Animal Behavior Processes 1988, 14:261-279.
21. Holland P.C., Gallagher M. Effects of amygdala central nucleus lesions on blocking and unblocking. Behavioral Neuroscience 1993, 107:235-245.
22. Holland P.C., Kenmuir C. Variations in unconditioned stimulus processing in unblocking. Journal of Experimental Psychology: Animal Behavior Processes 2005, 31:155-171.
23. Holland P.C., Rescorla R.A. The effect of two ways of devaluing the unconditioned stimulus after first- and second-order appetitive conditioning. Journal of Experimental Psychology: Animal Behavior Processes 1975, 1:355-363.
24. Holland P.C., Straub J.J. Differential effects of two ways of devaluing the unconditioned stimulus after Pavlovian appetitive conditioning. Journal of Experimental Psychology: Animal Behavior Processes 1979, 5:65-78.
25. Izquierdo A., Jentsch J.D. Reversal learning as a measure of impulsive and compulsive behavior in addictions. Psychopharmacology 2012, 219:607-620.
26. Izquierdo A., Murray E.A. Combined unilateral lesions of the amygdala and orbital prefrontal cortex impair affective processing in rhesus monkeys. Journal of Neurophysiology 2004, 91:2023-2039.
27. Izquierdo A., Murray E.A. Functional interaction of medial mediodorsal thalamic nucleus but not nucleus accumbens with amygdala and orbital prefrontal cortex is essential for adaptive response selection after reinforcer devaluation. Journal of Neuroscience 2010, 30:661-669.
28. Izquierdo A., Suda R.K., Murray E.A. Bilateral orbital prefrontal cortex lesions in rhesus monkeys disrupt choices guided by both reward value and reward contingency. Journal of Neuroscience 2004, 24:7540-7548.
29. Jentsch J.D., Taylor J.R. Impulsivity resulting from frontostriatal dysfunction in drug abuse: Implications for the control of behavior by reward-related stimuli. Psychopharmacology 1999, 146:373-390.
30. Jones J.L., Esber G.R., McDannald M.A., Gruber A.J., Hernandez A., Mirenzi A., et al. Orbitofrontal cortex supports behavior and learning using inferred but not cached values. Science 2012, 338:953-956.
31. Jones B., Mishkin M. Limbic lesions and the problem of stimulus-reinforcement associations. Experimental Neurology 1972, 36:362-377.
32. Kamin L.J. Predictability, surprise, attention, and conditioning. Punishment and aversive behavior 1969, Appleton-Century-Crofts, New York. B.A. Campbell, R.M. Church (Eds.).
33. Levy D.J., Glimcher P.W. Comparing apples and oranges: Using reward-specific and reward-general subjective value representation in the brain. Journal of Neuroscience 2011, 31:14693-14707.
34. Levy D.J., Glimcher P.W. The root of all value: a neural common currency for choice. Current Opinion in Neurobiology 2012, [epub ahead of print].
35. Machado C.J., Bachevalier J. The effects of selective amygdala, orbital frontal cortex or hippocampal formation lesions on reward assessment in nonhuman primates. European Journal of Neuroscience 2007, 25:2885-2904.
36. Man M.S., Clarke H.F., Roberts A.C. The role of the orbitofrontal cortex and medial striatum in the regulation of prepotent responses to food rewards. Cerebral Cortex 2009, 19:899-906.
37. McDannald M.A., Lucantonio F., Burke K.A., Niv Y., Schoenbaum G. Ventral striatum and orbitofrontal cortex are both required for model-based, but not model-free, reinforcement learning. Journal of Neuroscience 2011, 31:2700-2705.
38. McDannald M.A., Takahashi Y.K., Lopatina N., Pietras B.W., Jones J.L., Schoenbaum G. Model-based learning and the contribution of the orbitofrontal cortex to the model-free world. European Journal of Neuroscience 2012, 35:991-996.
39. Murray E.A., O'Doherty J.P., Schoenbaum G. What we know and do not know about the functions of the orbitofrontal cortex after 20years of cross-species studies. Journal of Neuroscience 2007, 27:8166-8169.
40. Ostlund S.B., Balleine B.W. Orbitofrontal cortex mediates outcome encoding in Pavlovian but not instrumental conditioning. Journal of Neuroscience 2007, 27:4819-4825.
41. Padoa-Schioppa C. Neurobiology of economic choice: A goods-based model. Annual Review of Neuroscience 2011, 34:333-359.
42. Padoa-Schioppa C., Assad J.A. Neurons in orbitofrontal cortex encode economic value. Nature 2006, 441:223-226.
43. Pickens C.L., Saddoris M.P., Gallagher M., Holland P.C. Orbitofrontal lesions impair use of cue-outcome associations in a devaluation task. Behavioural Neuroscience 2005, 119:317-322.
44. Pickens C.L., Saddoris M.P., Setlow B., Gallagher M., Holland P.C., Schoenbaum G. Different roles for orbitofrontal cortex and basolateral amygdala in a reinforcer devaluation task. Journal of Neuroscience 2003, 23:11078-11084.
45. Plassmann H., O'Doherty J., Rangel A. Orbitofrontal cortex encodes willingness to pay in everyday economic transactions. Journal of Neuroscience 2007, 27:9984-9988.
46. Rescorla R.A. Reduction in effectiveness of reinforcement after prior excitatory conditioning. Learning and Motivation 1970, 1:372-381.
47. Rescorla R.A. Instrumental responses become associated with reinforcers that differ in one feature. Animal Learning & Behavior 1990, 18:206-211.
48. Rescorla R.A. Learning about qualitatively different outcomes during a blocking procedure. Animal Learning & Behavior 1999, 27:140-151.
49. Rescorla R.A. Spontaneous recovery from overexpectation. Learning Behaviour 2006, 34:13-20.
50. Rescorla R.A. Renewal after overexpectation. Learning Behaviour 2007, 35:19-26.
51. Scarlet J., Delamater A.R., Campese V., Fein M., Wheeler D.S. Differential involvement of the basolateral amygdala and orbitofrontal cortex in the formation of sensory-specific associations in conditioned flavor preference and magazine approach paradigms. European Journal of Neuroscience 2012, 35:1799-1809.
52. Schoenbaum G., Nugent S., Saddoris M.P., Setlow B. Orbitofrontal lesions in rats impair reversal but not acquisition of go, no-go odor discriminations. Neuroreport 2002, 13:885-890.
53. Schoenbaum G., Roesch M.R., Stalnaker T.A., Takahashi Y.K. A new perspective on the role of the orbitofrontal cortex in adaptive behaviour. Nature Reviews Neuroscience 2009, 10:885-892.
54. Takahashi Y.K., Roesch M.R., Stainaker T.A., Haney R.Z., Caiu D.J., Taylor A.R., et al. The orbitofrontal cortex and ventral tegmental area are necessary for learning from unexpected outcomes. Neuron 2009, 62:269-280.
55. Teitelbaum H. A comparison of effects of orbitofrontal and hippocampal lesions upon discrimination learning and reversal in the cat. Experimental Neurology 1964, 9:452-462.
56. West E.A., DesJardin J.T., Gale K., Malkova L. Transient inactivation of orbitofrontal cortex blocks reinforcer devaluation in macaques. Journal of Neuroscience 2011, 31:15128-15135.