Deciphering decision making: Variation in animal models of effort- and uncertainty-based choice reveals distinct neural circuitries underlying core cognitive processes

Journal of Neuroscience

Volumn 36, Issue 48, 2016, Pages 12069-12079

  Winstanley, Catharine A ^a   Floresco, Stan B ^a  

^a UNIVERSITY OF BRITISH COLUMBIA   (Canada)

Author keywords

[No Author keywords available]

Indexed keywords

AMPHETAMINE; DOPAMINE;

AMBIGUITY; ARTICLE; BIPOLAR MANIA; CONFUSION (UNCERTAINTY); CONNECTOME; CORTICAL AMYGDALA; DECISION MAKING; DELAY DISCOUNTING; DOPAMINERGIC TRANSMISSION; NERVE CONDUCTION; NEUROIMAGING; NONHUMAN; PATHOLOGICAL GAMBLING; PRIORITY JOURNAL; REWARD; RISK ASSESSMENT; SENSORIMOTOR FUNCTION; TASK PERFORMANCE; ANIMAL; ANIMAL BEHAVIOR; ANIMAL MODEL; BRAIN; COGNITION; MOUSE; NERVE CELL NETWORK; PHYSIOLOGY; RAT; SPECIES DIFFERENCE; UNCERTAINTY;

ANIMALS; BEHAVIOR, ANIMAL; BRAIN; CHOICE BEHAVIOR; COGNITION; MICE; MODELS, ANIMAL; NERVE NET; RATS; SPECIES SPECIFICITY; UNCERTAINTY;

EID: 85005992350     PISSN: 02706474     EISSN: 15292401     Source Type: Journal    
DOI: 10.1523/JNEUROSCI.1713-16.2016     Document Type: Article

References (107)

1. Balleine BW, O’Doherty JP (2010) Human and rodent homologies in action control: corticostriatal determinants of goal-directed and habitual action. Neuropsychopharmacology 35:48-69.
2. Bardgett ME, Depenbrock M, Downs N, Points M, Green L (2009) Dopamine modulates effort-based decision-making in rats. Behav Neurosci 123:242-251.
3. Barrus MM, Winstanley CA (2016) Dopamine D3 receptors modulate the ability of win-paired cues to increase risky choice in a rat gambling task. J Neurosci 36:785-794.
4. Barrus MM, Hosking JG, Cocker PJ, Winstanley CA (2016) Inactivation of the orbitofrontal cortex reduces irrational choice on a rodent Betting Task. Neuroscience. Advance online publication. Retrieved Feb. 19, 2016. doi: 10.1016/j.neuroscience.2016.02.028.
5. Bechara A, Damasio H (2002) Decision-making and addiction: I. Impaired activation of somatic states in substance dependent individuals when pondering decisions with negative future consequences. Neuropsychologia 40:1675-1689.
6. Bechara A, Damasio AR, Damasio H, Anderson SW (1994) Insensitivity to future consequences following damage to human prefrontal cortex. Cognition 50:7-15.
7. Bechara A, Damasio H, Damasio AR, Lee GP (1999) Different contributions of the human amygdala and ventromedial prefrontal cortex to decision-making. J Neurosci 19:5473-5481.
8. Blanchard TC, Hayden BY (2015) Monkeys are more patient in a foraging task than in a standard intertemporal choice task. PLoS One 10:e0117057.
9. Blanchard TC, Wilke A, Hayden BY (2014) Hot-hand bias in rhesus mon-keys. J Exp Psychol Anim Learn Cogn 40:280-286.
10. Botvinick MM, Huffstetler S, McGuire JT (2009) Effort discounting in human nucleus accumbens. Cogn Affect Behav Neurosci 9:16-27.
11. Cardinal RN, Howes NJ (2005) Effects of lesions of the nucleus accumbens core on choice between small certain rewards and large uncertain rewards in rats. BMC Neurosci 6:37.
12. Carli M, Robbins TW, Evenden JL, Everitt BJ (1983) Effects of lesions to ascending noradrenergic neurons on performance of a 5-choice serial reaction time task in rats: implications for theories of dorsal noradrenergic bundle function based on selective attention and arousal. Behav Brain Res 9:361-380.
13. Clark L (2010) Decision-making during gambling: an integration of cognitive and psychobiological approaches. Philos Trans R Soc Lond B Biol Sci 365:319-330.
14. Clark L, Iversen SD, Goodwin GM (2001) A neuropsychological investigation of prefrontal cortex involvement in acute mania. Am J Psychiatry 158:1605-1611.
15. Cocker PJ, Winstanley CA (2015) Irrational beliefs, biases and gambling: exploring the role of animal models in elucidating vulnerabilities for the development of pathological gambling. Behav Brain Res 279:259-273.
16. Cocker PJ, Hosking JG, Benoit J, Winstanley CA (2012a) Sensitivity to cog-nitive effort mediates psychostimulant effects on a novel rodent cost/benefit decision-making task. Neuropsychopharmacology 37:1825-1837.
17. Cocker PJ, Dinelle K, Kornelson R, Sossi V, Winstanley CA (2012b) Irratio-nal choice under uncertainty correlates with lower striatal D(2/3) receptor binding in rats. J Neurosci 32:15450-15457.
18. Cousins MS, Wei W, Salamone JD (1994) Pharmacological characterization of performance on a concurrent lever pressing/feeding choice procedure: effects of dopamine antagonist, cholinomimetic, sedative and stimulant drugs. Psychopharmacology (Berl) 116:529-537.
19. Cowen SL, Davis GA, Nitz DA (2012) Anterior cingulate neurons in the rat map anticipated effort and reward to their associated action sequences. J Neurophysiol 107:2393-2407.
20. Croxson PL, Walton ME, O’Reilly JX, Behrens TE, Rushworth MF (2009) Effort-based cost-benefit valuation and the human brain. J Neurosci 29: 4531-4541.
21. Dalley JW, Cardinal RN, Robbins TW (2004) Prefrontal executive and cognitive function in rodents: neural and neurochemical substrates. Neurosci Biobehav Rev 28:771-784.
22. de Visser L, Baars AM, Lavrijsen M, van der Weerd CM, van den Bos R (2011) Decision-making performance is related to levels of anxiety and differential recruitment of frontostriatal areas in male rats. Neuroscience 184:97-106.
23. Evenden JL, Ryan CN (1996) The pharmacology of impulsive behaviour in rats: the effects of drugs on response choice with varying delays of reinforcement. Psychopharmacology 128:161-170.
24. Fellows LK, Farah MJ (2003) Ventromedial frontal cortex mediates affective shifting in humans: evidence from a reversal learning paradigm. Brain 126:1830-1837.
25. Fellows LK, Farah MJ (2005) Different underlying impairments in decision-making following ventromedial and dorsolateral frontal lobe damage in humans. Cereb Cortex 15:58-63.
26. Floresco SB, Ghods-Sharifi S (2007) Amygdala-prefrontal cortical circuitry reg-ulates effort-based decision-making. Cereb Cortex 17:251-260.
27. Floresco SB, Tse MT, Ghods-Sharifi S (2008) Dopaminergic and glutama-tergic regulation of effort-and delay-based decision-making. Neuropsychopharmacology 33:1966-1979.
28. Gee S, Ellwood I, Patel T, Luongo F, Deisseroth K, Sohal VS (2012) Synaptic activity unmasks dopamine D2 receptor modulation of a specific class of layer V pyramidal neurons in prefrontal cortex. J Neurosci 32:4959-4971.
29. Ghods-Sharifi S, Floresco SB (2010) Differential effects on effort discount-ing induced by inactivations of the nucleus accumbens core or shell. Behav Neurosci 124:179-191.
30. Ghods-Sharifi S, St Onge JR, Floresco SB (2009) Fundamental contribution by the basolateral amygdala to different forms of decision-making. J Neurosci 29:5251-5259.
31. Hauber W, Sommer S (2009) Prefrontostriatal circuitry regulates effort-related decision-making. Cereb Cortex 19:2240-2247.
32. Heilbronner SR, Hayden BY (2016) The description-experience gap in risky choice in nonhuman primates. Psychonom Bull Rev 23:593-600.
33. Heilbronner SR, Rodriguez-Romaguera J, Quirk GJ, Groenewegen HJ, Haber SN (2016) Circuit-based corticostriatal homologies between rat and pri-mate. Biol Psychiatry 80:509-521.
34. Holec V, Pirot HL, Euston DR (2014) Not all effort is equal: the role of the anterior cingulate cortex in different forms of effort-reward decisions. Front Behav Neurosci 8:12.
35. Hosking JG, Cocker PJ, Winstanley CA (2014a) Dissociable contributions of anterior cingulate cortex and basolateral amygdala on a rodent cost/benefit decision-making task of cognitive effort. Neuropsychopharmacology 39:1558-1567.
36. Hosking JG, Lam FC, Winstanley CA (2014b) Nicotine increases impulsiv-ity and decreases willingness to exert cognitive effort despite improving attention in “slacker” rats: insights into cholinergic regulation of cost/ benefit decision-making. PLoS One 9:e111580.
37. Hosking JG, Floresco SB, Winstanley CA (2015) Dopamine antagonism decreases willingness to expend physical, but not cognitive, effort: a comparison of two rodentcost/benefitdecision-makingtasks. Neuropsychopharmacology 40:1005-1015.
38. Hosking JG, Cocker PJ, Winstanley CA (2016) Prefrontal cortical inactiva-tions decrease willingness to expend cognitive effort on a rodent cost/benefit decision-making task. Cereb Cortex 26:1529-1538.
39. Huettel SA, Stowe CJ, Gordon EM, Warner BT, Platt ML (2006) Neural signatures of economic preferences for risk and ambiguity. Neuron 49: 765-775.
40. Jollant F, Bellivier F, Leboyer M, Astruc B, Torres S, Verdier R, Castelnau D, Malafosse A, Courtet P (2005) Impaired decision-making in suicide attempters. Am J Psychiatry 162:304-310.
41. Kahneman D (2011) Thinking fast and slow. London: Penguin.
42. Kahneman D, Tversky A (1979) Prospect theory: an analysis of decision under risk. Econometrica 47:263-292.
43. Kennerley SW, Walton ME, Behrens TE, Buckley MJ, Rushworth MF (2006) Optimal decision-making and the anterior cingulate cortex. Nat Neurosci 9:940-947.
44. Kool W, McGuire JT, Rosen ZB, Botvinick MM (2010) Decision-making and the avoidance of cognitive demand. J Exp Psychol Gen 139:665-682.
45. Kravitz AV, Tye LD, Kreitzer AC (2012) Distinct roles for direct and indirect pathway striatal neurons in reinforcement. Nat Neurosci 15:816-818.
46. Kurniawan IT, Seymour B, Talmi D, Yoshida W, Chater N, Dolan RJ (2010) Choosing to make an effort: the role of striatum in signaling physical effort of a chosen action. J Neurophysiol 104:313-321.
47. Larkin JD, Jenni NL, Floresco SB (2016) Modulation of risk/reward decision-making by dopaminergic transmission within the basolateral amygdala. Psychopharmacology (Berl) 233:121-136.
48. Lau B, Glimcher PW (2008) Value representations in the primate striatum during matching behavior. Neuron 58:451-463.
49. Lobo MK, Covington HE 3rd, Chaudhury D, Friedman AK, Sun H, Damez-Werno D, Dietz DM, Zaman S, Koo JW, Kennedy PJ, Mouzon E, Mogri M, Neve RL, Deisseroth K, Han MH, Nestler EJ (2010) Cell type-specific loss of BDNF signaling mimics optogenetic control of cocaine reward. Science 330:385-390.
50. Morris RW, Dezfouli A, Griffiths KR, Balleine BW (2014) Action-value comparisons in the dorsolateral prefrontal cortex control choice between goal-directed actions. Nat Commun 5:4390.
51. Naccache L, Dehaene S, Cohen L, Habert MO, Guichart-Gomez E, Galanaud D, Willer JC (2005) Effortless control: executive attention and conscious feeling of mental effort are dissociable. Neuropsychologia 43:1318-1328.
52. Nowend KL, Arizzi M, Carlson BB, Salamone JD (2001) D1 or D2 antago-nism in nucleus accumbens core or dorsomedial shell suppresses lever pressing for food but leads to compensatory increases in chow consump-tion. Pharmacol Biochem Behav 69:373-382.
53. Orsini CA, Trotta RT, Bizon JL, Setlow B (2015a) Dissociable roles for the basolateral amygdala and orbitofrontal cortex in decision-making under risk of punishment. J Neurosci 35:1368-1379.
54. Orsini CA, Moorman DE, Young JW, Setlow B, Floresco SB (2015b) Neural mechanisms regulating different forms of risk-related deci-sion-making: insights from animal models. Neurosci Biobehav Rev 58:147-167.
55. Orsini CA, Willis ML, Gilbert RJ, Bizon JL, Setlow B (2016) Sex differences in a rat model of risky decision-making. Behav Neurosci 130:50-61.
56. Ostrander S, Cazares VA, Kim C, Cheung S, Gonzalez I, Izquierdo A (2011) Orbitofrontal cortex and basolateral amygdala lesions result in suboptimal and dissociable reward choices on cue-guided effort in rats. Behav Neurosci 125:350-359.
57. Paine TA, Asinof SK, Diehl GW, Frackman A, Leffler J (2013) Medial pre-frontal cortex lesions impair decision-making on a rodent gambling task: reversal by D1 receptor antagonist administration. Behav Brain Res 243: 247-254.
58. Parvizi J, Rangarajan V, Shirer WR, Desai N, Greicius MD (2013) The will to persevere induced by electrical stimulation of the human cingulate gyrus. Neuron 80:1359-1367.
59. Pushparaj A, Kim AS, Musiol M, Zangen A, Daskalakis ZJ, Zack M, Winstan-ley CA, Le Foll B (2015) Differential involvement of the agranular vs granular insular cortex in the acquisition and performance of choice be-havior in a rodent gambling task. Neuropsychopharmacology 40:2832-2842.
60. Randall PA, Pardo M, Nunes EJ, López Cruz L, Vemuri VK, Makriyannis A, Baqi Y, Muller CE, Correa M, Salamone JD (2012) Dopaminergic modulation of effort-related choice behavior as assessed by a progressive ratio chow feeding choice task: pharmacological studies and the role of indi-vidual differences. PLoS One 7:e47934.
61. Redish AD (2016) Vicarious trial and error. Nat Rev Neurosci 17:147-159.
62. Rivalan M, Ahmed SH, Dellu-Hagedorn F (2009) Risk-prone individuals prefer the wrong options on a rat version of the Iowa Gambling Task. Biol Psychiatry 66:743-749.
63. Robbins TW (2002) The 5-choice serial reaction time task: behavioural pharmacology and functional neurochemistry. Psychopharmacology 163:362-380.
64. Rokosik SL, Napier TC (2011) Intracranial self-stimulation as a positive re-inforcer to study impulsivity in a probability discounting paradigm. J Neurosci Methods 198:260-269.
65. Rudebeck PH, Walton ME, Smyth AN, Bannerman DM, Rushworth MF (2006) Separate neural pathways process different decision costs. Nat Neurosci 9:1161-1168.
66. Salamone JD, Steinpreis RE, McCullough LD, Smith P, Grebel D, Mahan K (1991) Haloperidol and nucleus accumbens dopamine depletion suppress lever pressing for food but increase free food consumption in a novel food choice procedure. Psychopharmacology (Berl) 104:515-521.
67. Salamone JD, Cousins MS, Bucher S (1994) Anhedonia or anergia? Effects of haloperidol and nucleus accumbens dopamine depletion on instrumental response selection in a T-maze cost/benefit procedure. Behav Brain Res 65:221-229.
68. Salamone JD, Wisniecki A, Carlson BB, Correa M (2001) Nucleus accum-bens dopamine depletions make animals highly sensitive to high fixed ratio requirements but do not impair primary food reinforcement. Neu-roscience 105:863-870.
69. Salamone JD, Correa M, Farrar A, Mingote SM (2007) Effort-related func-tions of nucleus accumbens dopamine and associated forebrain circuits. Psychopharmacology (Berl) 191:461-482.
70. Samejima K, Ueda Y, Doya K, Kimura M (2005) Representation of actionspecific reward values in the striatum. Science 310:1337-1340.
71. Schweimer J, Hauber W (2005) Involvement of the rat anterior cingulate cortex in control of instrumental responses guided by reward expectancy. Learn Mem 12:334-342.
72. Seong HJ, Carter AG (2012) D1 receptor modulation of action potential firing in a subpopulation of layer 5 pyramidal neurons in the prefrontal cortex. J Neurosci 32:10516-10521.
73. Shafiei N, Gray M, Viau V, Floresco SB (2012) Acute stress induces selective alterations in cost/benefit decision-making. Neuropsychopharmacology 37:2194-2209.
74. Silveira MM, Tremblay M, Winstanley CA (2016a) Dissociable contribu-tions of dorsal and ventral regions of the striatum on a rodent cost/benefit decision-making task requiring cognitive effort. San Diego: Society for Neuroscience.
75. Silveira MM, Adams WK, Morena M, Hill MN, Winstanley CA (2016b) 9-Tetrahydrocannabinol decreases willingness to exert cognitive effort in male rats. J Psychiatry Neurosci 41:150363.
76. Simon NW, Gilbert RJ, Mayse JD, Bizon JL, Setlow B (2009) Balancing risk and reward: a rat model of risky decision-making. Neuropsychopharmacology 34:2208-2217.
77. Simon NW, Montgomery KS, Beas BS, Mitchell MR, LaSarge CL, Mendez IA, Bañuelos C, Vokes CM, Taylor AB, Haberman RP, Bizon JL, Setlow B (2011) Dopaminergic modulation of risky decision-making. J Neurosci 31:17460-17470.
78. Simon NW, Beas BS, Montgomery KS, Haberman RP, Bizon JL, Setlow B (2013) Prefrontal cortical-striatal dopamine receptor mRNA expression predicts distinct forms of impulsivity. Eur J Neurosci 37:1779-1788.
79. Stalnaker TA, Cooch NK, Schoenbaum G (2015) What the orbitofrontal cortex does not do. Nat Neurosci 18:620-627.
80. St Onge JR, Floresco SB (2009) Dopaminergic regulation of risk-based decision-making. Neuropsychopharmacology 34:681-697.
81. St Onge JR, Floresco SB (2010) Prefrontal cortical contribution to risk-based decision-making. Cereb Cortex 20:1816-1828.
82. St Onge JR, Abhari H, Floresco SB (2011) Dissociable contributions by prefrontal D1 and D2 receptors to risk-based decision-making. J Neurosci 31:8625-8633.
83. St Onge JR, Stopper CM, Zahm DS, Floresco SB (2012) Separate prefrontal-subcortical circuits mediate different components of risk-based decision-making. J Neurosci 32:2886-2899.
84. Stopper CM, Floresco SB (2011) Contributions of the nucleus accumbens and its subregions to different aspects of risk-based decision-making. Cogn Affect Behav Neurosci 11:97-112.
85. Stopper CM, Khayambashi S, Floresco SB (2013) Receptor-specific modulation of risk-based decision-making by nucleus accumbens dopamine. Neuropsychopharmacology 38:715-728.
86. Stopper CM, Green EB, Floresco SB (2014) Selective involvement by the medial orbitofrontal cortex in biasing risky, but not impulsive, choice. Cereb Cortex 24:154-162.
87. Suzuki S, Jensen EL, Bossaerts P, O’Doherty JP (2016) Behavioral contagion during learning about another agent’s risk-preferences acts on the neural representation of decision-risk. Proc Natl Acad Sci U S A 113:3755-3760.
88. Treadway MT, Buckholtz JW, Schwartzman AN, Lambert WE, Zald DH (2009) Worth the ‘EEfRT?’ The effort expenditure for rewards task as an objective measure of motivation and anhedonia. PLoS One 4:e6598.
89. Treadway MT, Buckholtz JW, Cowan RL, Woodward ND, Li R, Ansari MS, Baldwin RM, Schwartzman AN, Kessler RM, Zald DH (2012) Dopaminergic mechanisms of individual differences in human effort-based decision-making. J Neurosci 32:6170-6176.
90. Tremblay M, Hosking JG, Winstanley CA (2013) Effects of chronic D-2/3 agonist ropinirole medication on rodent models of gambling behaviour. Mov Disord 28:S218.
91. Tremblay M, Cocker PJ, Hosking JG, Zeeb FD, Rogers RD, Winstanley CA (2014) Dissociable effects of basolateral amygdala lesions on decision-making biases in rats when loss or gain is emphasized. Cogn Affect Behav Neurosci 14:1184-1195.
92. Tremblay M, Silveira MM, Kaur S, Hosking JG, Adams WK, Baunez C, Win-stanley CA (2016) Chronic D2/3 agonist ropinirole treatment increases preference for uncertainty in rats regardless of baseline choice patterns. Eur J Neurosci. Advance online publication. Retrieved July 16, 2016. doi: 10.1111/ejn.13332.
93. Tversky A, Kahneman D (1974) Judgment under uncertainty: heuristics and biases. Science 185:1124-1131.
94. Tversky A, Kahneman D (1981) The framing of decisions and the psychol-ogy of choice. Science 211:453-458.
95. Uylings HB, Groenewegen HJ, Kolb B (2003) Do rats have a prefrontal cortex? Behav Brain Res 146:3-17.
96. van den Bos R, Lasthuis W, den Heijer E, van der Harst J, Spruijt B (2006) Toward a rodent model of the Iowa gambling task. Behav Res Methods 38:470-478.
97. van den Bos R, Jolles J, van der Knaap L, Baars A, de Visser L (2012) Male and female Wistar rats differ in decision-making performance in a rodent version of the Iowa Gambling Task. Behav Brain Res 234:375-379.
98. van Honk J, Hermans EJ, Putman P, Montagne B, Schutter DJ (2002) De-fective somatic markers in sub-clinical psychopathy. Neuroreport 13: 1025-1027.
99. Walton ME, Bannerman DM, Rushworth MF (2002) The role of rat medial frontal cortex in effort-based decision-making. J Neurosci 22:10996-11003.
100. Walton ME, Bannerman DM, Alterescu K, Rushworth MF (2003) Func-tional specialization within medial frontal cortex of the anterior cingulated for evaluating effort-related decisions. J Neurosci 23:6475-6479.
101. Winstanley CA, Balleine BW, Brown JS, Buchel C, Cools R, Durstewitz D, O’Doherty JP, Pennartz CM, Redish AD, Seamans JK, Robbins TW (2012) Search, goals and the brain. In: Cognitive search: evolution, algorithms, and the brain (Todd PM, Hills TT, Robbins TW, eds). Cambridge, MA: Massachusetts Institute of Technology.
102. Yu AJ, Dayan P (2005) Uncertainty, neuromodulation, and attention. Neuron 46:681-692.
103. Zalocusky KA, Ramakrishnan C, Lerner TN, Davidson TJ, Knutson B, Deisseroth K (2016) Nucleus accumbens D2R cells signal prior outcomes and control risky decision-making. Nature 531:642-646.
104. Zeeb FD, Winstanley CA (2011) Lesions of the basolateral amygdala and orbitofrontal cortex differentially affect acquisition and performance of a rodent gambling task. J Neurosci 31:2197-2204.
105. Zeeb FD, Winstanley CA (2013) Functional disconnection of the orbitofrontal cortex and basolateral amygdala impairs acquisition of a rat gambling task and disrupts animals’ ability to alter decision-making behavior after rein-forcer devaluation. J Neurosci 33:6434-6443.
106. Zeeb FD, Robbins TW, Winstanley CA (2009) Serotonergic and dopaminergic modulation of gambling behavior as assessed using a novel rat gambling task. Neuropsychopharmacology 34:2329-2343.
107. Zeeb FD, Baarendse PJ, Vanderschuren LJ, Winstanley CA (2015) Inactivation of the prelimbic or infralimbic cortex impairs decision-making in the rat gambling task. Psychopharmacology 232:4481-4491.