The road not taken: Neural correlates of decision making in orbitofrontal cortex

Frontiers in Neuroscience

Volumn , Issue SEP, 2012, Pages

  Steiner, Adam P ^a   Redish, A David ^a  

^a UNIVERSITY OF MINNESOTA   (United States)

Author keywords

Counterfactual; Covert representation of reward; Multiple T; Orbitofrontal cortex; Regret; Vicarious trial and error

Indexed keywords

ANIMAL BEHAVIOR; ANIMAL EXPERIMENT; ARTICLE; BRAIN FUNCTION; CONTROLLED STUDY; DECISION MAKING; NERVE CELL; NEUROANATOMY; NONHUMAN; ORBITAL CORTEX; RAT; REWARD; TASK PERFORMANCE;

EID: 84870942380     PISSN: 16624548     EISSN: 1662453X     Source Type: Journal    
DOI: 10.3389/fnins.2012.00131     Document Type: Article

References (64)

1. Abe, H., and Lee, D. (2011). Distributed coding of actual and hypothetical outcomes in the orbital and dorso-lateral prefrontal cortex. Neuron 70, 731-741.
2. Bechara, A., Damasio, A. R., Dama-sio, H., and Anderson, S. W. (1994). Insensitivity to future consequences following damage to human prefrontal cortex. Cognition 50, 7-15.
3. Bell, D. (1982). Regret in decision making under uncertainty. Oper. Res. 961-981.
4. Bell, D. E. (1985). Disappointment in decision making under uncertainty. Oper. Res. 33,1-27.
5. Blaisdell, A. P., Sawa, K., Leising, K. J., and Waldmann, M. R. (2006). Causal reasoning in rats. Science 311, 1020-1022.
6. Blumenthal, A., Steiner, A., Seeland, K., and Redish, A. D. (2011). Effects of pharmacological manipulations of NMDA-receptors on deliberation in the multiple-T task. Neurobiol. Learn. Mem. 95, 376-384.
7. Camille, N., Coricelli, G., Sallet, J., Pradat-Diehl, P., Duhamel, J., and Sirigu, A. (2004). The involvement of the orbitofrontal cortex in the experience of regret. Science 304, 1167-1170.
8. Chua, H. F., Gonzalez, R., Taylor, S. E, Welsh, R. C., and Liberzon, I. (2009). Decision-related loss: regret and disappointment. Neuroimage 47,2031-2040.
9. Coricelli, G., Critchley, H., Joffily, M., O'Doherty, J., Sirigu, A., and Dolan, R. (2005). Regret and its avoidance: a neuroimaging study of choice behavior. Nat. Neurosci. 8, 1255-1262.
10. Coricelli, G., Dolan, R., and Sirigu, A. (2007). Brain, emotion and decision making: the paradigmatic exampleof regret. Trends Cogn. Sci. 11, 258-265.
11. Damasio, A. (1994). Descartes' Error: Emotion, Reason, and the Human Brain. NewYork: Putnam's Sons.
12. Daw, N. D., Niv, Y., and Dayan, P. (2005). Uncertainty-based competition between prefrontal and dorso-lateral striatal systems for behavioral control. Nat. Neurosci. 8,1704-1711.
13. Daw, N. D., O'Doherty, J. P., Dayan, P., Seymour, B., and Dolan, R. J. (2006). Cortical substrates for exploratory decisions in humans. Nature 441, 876-879.
14. Dias, R., Robbins, T. W., and Roberts, A. C. (1996). Dissociation in prefrontal cortex of affective and attentional shifts. Nature 380, 69-72.
15. Frost, R. (1916). MountainInterval: The Roadnot Taken. New York: Holt.
16. Gallagher, M., McMahan, R., and Schoenbaum, G. (1999). Orbitofrontal cortex and representation of incentive value in associative learning. J. Neurosci. 19, 6610-6614.
17. Gottfried, J., O'Doherty, J., and Dolan, R. (2003). Encoding predictive reward value in human amygdala and orbitofrontal cortex. Science 301, 1104-1107.
18. Gupta, A. S., van der Meer, M. A., Touretzky, D. S., and Redish, A. D. (2010). Hippocampal replay is not a simple function of experience. Neuron 65, 695-705.
19. Hare, T., O'Doherty, J., Camerer, C., Schultz, W., and Rangel, A. (2008). Dissociating the role of the orbitofrontal cortex and the stria-tum in the computation of goal values and prediction errors. J. Neurosci. 28, 5623-5630.
20. Hare, T. A., Camerer, C. F., and Rangel, A. (2009). Self-control in decision-making involves modulation of thevmPFC valuation system. Science 324, 646-648.
21. Janabi-Sharifi, F., Hayward, V., and Chen, C. S. J. (2000). Discrete-time adaptive windowing for velocity estimation. IEEE Trans. Control Syst. Technol. 8,1003-1009.
22. Johnson, A., and Redish, A. D. (2007). Neural ensembles in CA3 transiently encode paths forward of the animal at a decision point. J. Neurosci. 27, 12176-12189.
23. Johnson, A., van der Meer, M. A., and Redish, A. D. (2007). Integrating hippocampus and striatum in decision-making. Curr. Opin. Neurobiol. 17, 692-697.
24. Kepecs, A., Uchida, N., Zariwala, H. A., and Mainen, Z. E (2008). Neural correlates, computation and behavioural impact of decision confidence. Nature 455, 227-231.
25. Krajbich, I., Armel, C., and Rangel, A. (2010). Visual fixations and the computation and comparison of value in simple choice. Nat. Neurosci. 13, 1292-1298.
26. Loomes, G., and Sugden, R. (1982). Regret theory: an alternative theory of rational choice under uncertainty. Econ. J. 92, 805-824.
27. Loomes, G., and Sugden, R. (1986). Disappointment and dynamic consistency in choice under uncertainty. Rev. Econ. Stud. 53,271-282.
28. Lucantonio, F., Stalnaker, T., Shaham, Y., Niv, Y., and Schoenbaum, G. (2012). The impact of orbitofrontal dysfunction on cocaine addiction. Nat. Neurosci. 15, 358-424.
29. Mainen, Z. F., and Kepecs, A. (2009). Neural representation of behavioral outcomes in the orbitofrontal cortex. Curr. Opin. Neurobiol. 19,84-91.
30. McDannald, M. A., Lucantonio, F., Burke, K. A., Niv, Y., and Schoenbaum, G. (2011). Ventral striatumand orbitofrontal cortex are both required for model-based, but not model-free, reinforcement learning. J. Neurosci. 31, 2700-2705.
31. McDannald, M. A., Takahashi, Y K., Lopatina, N., Pietras, B. W., Jones, J. L., and Schoenbaum, G. (2012). Model-based learning and the contribution of the orbitofrontal cortex to the model-free world. Eur. J. Neurosci. 35,991-996.
32. Montague, P., and Lohrenz, T. (2007). To detect and correct: norm violations and their enforcement. Neuron 56, 14-18.
33. Muenzinger, K. E (1938). Vicarious trial and error at a point of choice: I. A general survey of its relation to learning efficiency. Pedagog. Semin. I Genet. Psychol. 53, 75-86.
34. O'Doherty, J. P., Dayan, P., Fris-ton, K., Critchley, H., and Dolan, R. J. (2003). Temporal difference models and reward-related learning in the human brain. Neuron 38, 329-337.
35. Padoa-Schioppa, C., and Assad, J. (2006). Neurons in the orbitofrontal cortex encode economic value. Nature 441, 223-226.
36. Padoa-Schioppa, C., and Assad, J. (2008). The representation of economic value in the orbitofrontal cortex is invariant for changes of menu. Nat. Neurosci. 11,95-102.
37. Papale, A. E., Stott, J. J., Powell, N. J., Regier, P. S., and Redish, A. D. (2012). Interactions between deliberation and delay-discounting in rats. Cogn. Affect. Behav. Neurosci. 12,513-526.
38. Peters, J., and Buchel, C. (2010). Episodic future thinking reduces reward delay discounting through an enhancement of prefrontal-mediotemporal interactions. Neu-ron 66, 138-148.
39. Platt, M. L., and Hayden, B. (2011). Learning: not just the facts, ma'am, but the counterfactuals as well. PLoS Biol. 9, e1001092. doi:10.1371/journal.pbio.1001092
40. Ramus, S., Davis, J., Donahue, R., Discenza, C., and Waite, A. (2007). Interactions between the orbitofrontal cortex and the hip-pocampal memory system during the storage of long-term memory. Ann. N.Y.Acad. Sci. 1121,216-231.
41. Redish, A. D., Jensen,S., Johnson, A., and Kurth-Nelson, Z. (2007). Reconciling reinforcement learning models with behavioral extinction and renewal: implications for addiction, relapse, and problem gambling. Psy-chol. Rev. 114,784-805.
42. Rescorla, R. A., and Wagner, A. R. (1972). "A theory of Pavlovian conditioning: Variations in the effectiveness of reinforcement and non-reinforcement," in Classical Conditioning II, ed. A. H. B. W. E Prokasy (New York: Appleton-Century-Crofts), 64-99.
43. Roesch, M. R., Calu, D. J., Burke, K. A., and Schoenbaum, G. (2007). Should I stay or should I go? Transformation of time-discounted rewards in orbitofrontal cortex and associated brain circuits. Ann. N. Y. Acad. Sci. 1104,21-34.
44. Schmitzer-Torbert, N., and Redish, A. D. (2004). Neuronal activity in the rodent dorsal striatum in sequential navigation: separation of spatial and reward responses on the multiple T task. J. Neurophysiol. 91,2259-2272.
45. Schoenbaum, G., Chiba, A., and Gal-lagher,M. (1998). Orbitofrontal cortex and basolateral amygdala encode expected outcomes during learning. Nat Neurosci. 1,155-159.
46. Schoenbaum, G., and Eichenbaum, H. (1995a). Information coding in the rodent prefrontal cortex. I. Single-neuron activity in orbitofrontal cortex compared with that in pyriform cortex. J. Neurophysiol. 74, 733-750.
47. Schoenbaum, G., and Eichenbaum, H. (1995b). Information coding in the rodent prefrontal cortex. II. ensemble activity in orbitofrontal cortex. J. Neurophysiol. 74, 751-762.
48. Schoenbaum, G., Nugent, L. S., Sad-doris, M. P., and Setlow, B. (2002). Orbitofrontal lesions in rats impair reversal but not acquisition of go, no-go odor discriminations. Neu-roreport 13, 885-890.
49. Schoenbaum, G., Setlow, B., Saddoris, M., and Gallagher, M. (2003). Encoding predicted outcome and acquired value in orbitofrontal cortex during cue sampling depends upon input from basolateral amygdala. Neuron 39, 855-867.
50. Schoenbaum, G., Takahashi, Y., Liu, T.-L., and McDannald, M. (2011). Does the orbitofrontal cortex signal value? Ann. N Y. Acad. Sci. 1239, 87-186.
51. Schultz, W., Dayan, P., and Montague, P. R. (1997). A neural substrate of prediction and reward. Science 275, 1593-1599.
52. Spitzer, M., Fischbacher, U., Herrn-berger, B., Gron, G., and Fehr, E. (2007). The neural signature of social norm compliance. Neuron 56, 185-196.
53. Sul, J. H., Kim, H., Huh, N., Lee, D., and Jung, M.W. (2010). Distinct roles of rodent orbitofrontal and medial prefrontal cortex in decision making. Neuron 66, 449-460.
54. Takahashi, Y. K., Roesch, M. R., Wilson, R. C., Toreson, K., O'Donnell, P., Niv, Y., and Schoenbaum, G. (2011).Expectancy-related changes in firing of dopamine neurons depend on orbitofrontal cortex. Nat. Neurosci. 14, 1590-1597.
55. Tolman, E. C. (1939). Prediction of vicarious trial and error by means of the schematic sowbug. Psychol. Rev. 46, 318-336.
56. Tremblay, L., and Schultz, W (1999). Relative reward preference in primate orbitofrontal cortex. Nature 398, 704-708.
57. van der Meer, M., Kurth-Nelson, Z., and Redish, A. D. (2012). Information processing in decision-making systems. Neuroscientist. 18, 342-359.
58. van der Meer, M., and Redish, A. D. (2009). Covert expectation-of-reward in rat ventral striatum at decision points. Front. Integr. Neurosci. 3:1. doi:10.3389/neuro.07.001.2009
59. van der Meer, M. A., Johnson, A., Schmitzer-Torbert, N. C., and Redish, A. D. (2010). Triple dissociation of information processing in dorsal striatum, ventral striatum, and hippocampus on a learned spatial decision task. Neuron 67,25-32.
60. van Duuren, E., Escámez, F., Joosten, R., Visser, R., Mulder, A., and Pennartz, C. (2007). Neural coding of reward magnitude in the orbitofrontal cortex of the rat during a five-odor olfactory discrimination task. Learn. Mem. 14, 446-456.
61. Van Duuren, E., van der Plasse, G., Lankelma, J., Joosten, R., Feenstra, M., and Pennartz, C. (2009). Single-cell and population coding of expected reward probability in the orbitofrontal cortex of the rat. J. Neurosci. 29, 8965-8976.
62. Wallis, J., and Miller, E. (2003). Neuronal activity in primate dorsolateraland orbital prefrontal cortex during performance of a reward preference task. Eur. J. Neurosci. 18, 2069-2081.
63. Young, J., and Shapiro, M. (2011). The orbitofrontal cortex and response selection. Ann. N Y Acad. Sci. 1239, 25-57.
64. Zhang, K., Ginzburg, I., McNaughton, B., and Sejnowski, T. (1998). Interpreting neuronal population activity by reconstruction: unified framework with application to hippocam-pal place cells. J. Neurophysiol. 79, 1017-1044.