Intersection of effort and risk: Ethological and neurobiological perspectives

Frontiers in Neuroscience

Volumn , Issue 7 NOV, 2013, Pages

  Miller, Mike A ^a   Thomé, Alexander ^a   Cowen, Stephen L ^b  

^a UNIVERSITY OF ARIZONA   (United States)

^b The University of Arizona   (United States)

Author keywords

Anterior cingulate cortex; Decision making; Dopamine; Effort; Foraging; Nucleus accumbens; Risk

Indexed keywords

DOPAMINE;

ANIMAL BEHAVIOR; ANTERIOR CINGULATE; ARTICLE; AVERSION; AVOIDANCE BEHAVIOR; BASOLATERAL AMYGDALA; BOLD SIGNAL; BRAIN CORTEX; DECISION MAKING; DOPAMINE RELEASE; DOPAMINERGIC NERVE CELL; ETHOLOGY; FOOD INTAKE; FORAGING BEHAVIOR; FRONTAL CORTEX; INSULA; LATERAL FRONTAL CORTEX; LATERAL ORBITOFRONTAL CORTEX; MEDIAL PREFRONTAL CORTEX; MIDDLE FRONTAL GYRUS; NEUROBIOLOGY; NONHUMAN; NUCLEUS ACCUMBENS; ORBITAL CORTEX; POSTERIOR CINGULATE; POSTERIOR PARIETAL CORTEX; PREFRONTAL CORTEX; PRELIMBIC CORTEX; REINFORCEMENT; REWARD; RISK; VENTRAL TEGMENTUM; VENTROMEDIAL PREFRONTAL CORTEX;

EID: 84888812423     PISSN: 16624548     EISSN: 1662453X     Source Type: Journal    
DOI: 10.3389/fnins.2013.00208     Document Type: Article

References (149)

1. Amiez, C., Joseph, J. P., and Procyk, E. (2006) Reward encoding in the monkey anterior cingulate cortex. Cereb. Cortex 16, 1040-1055.doi: 10.1093/cercor/bhj046
2. Aron, A. R., Shohamy, D., Clark, J., Myers, C., Gluck, M. A., and Poldrack, R. A. (2004) Human midbrain sensitivity to cognitive feedback and uncertainty during classification learning. J. Neurophys. 92, 1144-1152.doi: 10.1152/jn.01209.2003
3. Barbas, H., and Blatt, G. J. (1995) Topographically specific hippocampal projections target functionally distinct prefrontal areas in the rhesus monkey. Hippocampus 5, 511-533.doi: 10.1002/hipo.450050604
4. Barbas, H., and De Olmos, J. (1990) Projections from the amygdala to basoventral and mediodorsal prefrontal regions in the rhesus monkey. J. Comp. Neurol. 300, 549-571.doi: 10.1002/cne.903000409
5. Bardgett, M. E., Depenbrock, M., Downs, N., Points, M., and Green, L. (2009) Dopamine modulates effort-based decision making in rats. Behav. Neurosci. 123, 242-251.doi: 10.1037/a0014625
6. Bates, J. F., and Goldman-Rakic, P. S. (1993) Prefrontal connections of medial motor areas in the rhesus monkey. J. Comp. Neurol. 336, 211-228.doi: 10.1002/cne.903360205
7. Bateson, M. (2003). "Interval timing and optimal foraging," in Functional and Neural Mechanisms of Interval Timing, ed W. H. Meck (Boca Raton, FL: CRC Press), 113-142.
8. Bautista, L. M., Tinbergen, J., and Kacelnik, A. (2001) To walk or to fly? How birds choose among foraging modes. Proc. Natl. Acad. Sci. U.S.A. 98, 1089-1094.doi: 10.1073/pnas.98.3.1089
9. Baxter, M. G., and Murray, E. A. (2002) The amygdala and reward. Nat. Neurosci. 3, 563-573.doi: 10.1038/nrn875
10. Bechara, A., Damasio, A. R., Damasio, H., and Anderson, S. W. (1994) Insensitivity to future consequences following damage to human prefrontal cortex. Cognition 50, 7-15.doi: 10.1016/0010-0277(94)90018-3
11. Bechara, A., Damasio, H., Damasio, A. R., and Lee, G. P. (1999). Different contributions of the human amygdala and ventromedial prefrontal cortex to decision-making. J. Neurosci. 19, 5473-5481.
12. Behrens, T. E. J., Woolrich, M. W., Walton, M. E., and Rushworth, M. F. S. (2007) Learning the value of information in an uncertain world. Nat. Neurosci. 10, 1214-1221.doi: 10.1038/nn1954
13. Berridge, K. C., and Robinson, T. E. (2003) Parsing reward. Trends. Neurosci. 26, 507-13.doi: 10.1016/S0166-2236(03)00233-9
14. Bissière, S., Humeau, Y., and Lüthi, A. (2003) Dopamine gates LTP induction in lateral amygdala by suppressing feedforward inhibition. Nat. Neurosci. 6, 587-592.doi: 10.1038/nn1058
15. Brito E Abreu, F., and Kacelnik, A. (1999) Energy budgets and risk-sensitive foraging in starlings. Behav. Ecol. 10, 338-345.doi: 10.1093/beheco/10.3.338
16. Burke, C. J., Brünger, C., Kahnt, T., Park, S. Q., and Tobler, P. N. (2013) Neural integration of risk and effort costs by the frontal pole: only upon request. J. Neurosci. 33, 1706-1713.doi: 10.1523/JNEUROSCI.3662-12.2013
17. Caraco, T. (1980) On foraging time allocation in a stochastic environment. Ecology 61, 119-128.doi: 10.2307/1937162
18. Caraco, T. (1981) Energy budgets, risk and foraging preferences in dark-eyed Juncos (Junco hyemalis). Behav. Ecol. Sociobiol. 8, 213-217.doi: 10.1007/BF00299833
19. Caraco, T., Blanckenhorn, W. U., Gregory, G. M., Newman, J. A., Recer, G. M., and Zwicker, S. M. (1990) Risk-sensitivity: ambient temperature affects foraging choice. Anim. Behav. 39, 338-345.doi: 10.1016/S0003-3472(05)80879-6
20. Cardinal, R. N., and Howes, N. J. (2005). Effects of lesions of the nucleus accumbens core on choice between small certain rewards and large uncertain rewards in rats. BMC Neurosci. 6:37. doi: 10.1186/1471-2202-6-37
21. Cerovic, M., d'Isa, R., Tonini, R., and Brambilla, R. (2013) Molecular and cellular mechanisms of dopamine-mediated behavioral plasticity in the striatum. Neurobiol. Learn. Mem. 105, 63-80.doi: 10.1016/j.nlm.2013.06.013
22. Choi, J.-S., and Kim, J. J. (2010) Amygdala regulates risk of predation in rats foraging in a dynamic fear environment. Proc. Natl. Acad. Sci. U.S.A. 107, 21773-21777.doi: 10.1073/pnas.1010079108
23. Christopoulos, G. I., Tobler, P. N., Bossaerts, P., Dolan, R. J., and Schultz, W. (2009) Neural correlates of value, risk, and risk aversion contributing to decision making under risk. J. Neurosci. 29, 12574-12583.doi: 10.1523/JNEUROSCI.2614-09.2009
24. Clement, T. S., Feltus, J. R., Kaiser, D. H., and Zentall, T. R. (2000) "Work ethic" in pigeons: reward value is directly related to the effort or time required to obtain the reward. Psychon. Bull. Rev. 7, 100-106.doi: 10.3758/BF03210727
25. Cousins, M. S. S., Atherton, A., Turner, L., and Salamone, J. D. D. (1996) Nucleus accumbens dopamine depletions alter relative response allocation in a T-maze cost/benefit task. Behav. Brain. Res. 74, 189-197.doi: 10.1016/0166-4328(95)00151-4
26. Cowen, S. L., Davis, G. A., and Nitz, D. A. (2012) Anterior cingulate neurons in the rat map anticipated effort and reward to their associated action sequences. J. Neurophysiol. 107, 2393-2407.doi: 10.1152/jn.01012.2011
27. Cowen, S. L., and McNaughton, B. L. (2007) Selective delay activity in the medial prefrontal cortex of the rat: contribution of sensorimotor information and contingency. J. Neurophysiol. 98, 303-316.doi: 10.1152/jn.00150.2007
28. Craft, B. B., Church, A. C., Rohrbach, C. M., and Bennett, J. M. (2011) The effects of reward quality on risk-sensitivity in Rattus norvegicus. Behav. Process. 88, 44-46.doi: 10.1016/j.beproc.2011.07.002
29. Dang, D., Cunnington, D., and Swieca, J. (2011) The emergence of devastating impulse control disorders during dopamine agonist therapy of the restless legs syndrome. Clin. Neuropharmacol. 34, 66-70.doi: 10.1097/WNF.0b013e31820d6699
30. Day, J. J., Jones, J. L., Wightman, R. M., and Carelli, R. M. (2010) Phasic nucleus accumbens dopamine release encodes effort-and delay-related costs. Biol. Psychiatry 68, 306-309.doi: 10.1016/j.biopsych.2010.03.026
31. Day, J. J., Roitman, M. F., Wightman, R. M., and Carelli, R. M. (2007) Associative learning mediates dynamic shifts in dopamine signaling in the nucleus accumbens. Nat. Neurosci. 10, 1020-1028.doi: 10.1038/nn1923
32. Denk, F., Walton, M. E., Jennings, K. A., Sharp, T., Rushworth, M. F. S., and Bannerman, D. M. (2005) Differential involvement of serotonin and dopamine systems in cost-benefit decisions about delay or effort. J. Psychopharmacol. 179, 587-596.doi: 10.1007/s00213-004-2059-4
33. Dodd, M. L. M., Klos, K. J. K. J., Bower, J. H. J. H., Geda, Y. E., Josephs, K. A., and Ahlskog, J. E. (2005) Pathological gambling caused by drugs used to treat Parkinson disease. Arch. Neural. 62, 1377-1381.doi: 10.1001/archneur.62.9.noc50009
34. Dommett, E., Coizet, V., Blaha, C. D., Martindale, J., Lefebvre, V., Walton, N., et al. (2005) How visual stimuli activate dopaminergic neurons at short latency. Science 307, 1476-1479.doi: 10.1126/science.1107026
35. Dunn, B. D., Dalgleish, T., and Lawrence, A. D. (2006) The somatic marker hypothesis: a critical evaluation. Neurosci. Biobehav. Rev. 30, 239-271.doi: 10.1016/j.neubiorev.2005.07.001
36. Eisenberger, R. (1992) Learned industriousness. Psychol. Rev. 99, 248-267.doi: 10.1037/0033-295X.99.2.248
37. Fiorillo, C. D., Tobler, P. N., and Schultz, W. (2003) Discrete coding of reward probability and uncertainty by dopamine neurons. Science 299, 1898-1902.doi: 10.1126/science.1077349
38. Fitzgerald, T. H. B., Seymour, B., Bach, D. R., and Dolan, R. J. (2010) Differentiable neural substrates for learned and described value and risk. Curr. Biol. 20, 1823-1829.doi: 10.1016/j.cub.2010.08.048
39. Flagel, S. B., Clark, J. J., Robinson, T. E., Mayo, L., Czuj, A., Willuhn, I., et al. (2011) A selective role for dopamine in stimulus-reward learning. Nature 469, 53-57.doi: 10.1038/nature09588
40. Floresco, S. B., and Ghods-Sharifi, S. (2007) Amygdala-prefrontal cortical circuitry regulates effort-based decision making. Cereb. Cortex 17, 251-260.doi: 10.1093/cercor/bhj143
41. Floresco, S. B., Tse, M. T. L., and Ghods-Sharifi, S. (2008) Dopaminergic and glutamatergic regulation of effort-and delay-based decision making. Neuropsychopharmacology 33, 1966-1979.doi: 10.1038/sj.npp.1301565
42. Floresco, S. B., West, A. R., Ash, B., Moore, H., and Grace, A. A. (2003) Afferent modulation of dopamine neuron firing differentially regulates tonic and phasic dopamine transmission. Nat. Neurosci. 6, 968-973.doi: 10.1038/nn1103
43. Freed, C. R., and Yamamoto, B. K. (1985) Regional brain dopamine metabolism: a marker for the speed, direction, and posture of moving animals. Science 229, 62-65.doi: 10.1126/science.4012312
44. Friedrich, A. M., and Zentall, T. R. (2004) Pigeons shift their preference toward locations of food that take more effort to obtain. Behav. Process. 67, 405-415.doi: 10.1016/j.beproc.2004.07.001
45. Gabbott, P. L. A., Warner, T. A., Jays, P. R. L., Salway, P., and Busby, S. J. (2005) Prefrontal cortex in the rat: projections to subcortical autonomic, motor, and limbic centers. J. Comp. Neurol. 492, 145-177.doi: 10.1002/cne.20738
46. Gan, J. O., Walton, M. E., and Phillips, P. E. M. (2010) Dissociable cost and benefit encoding of future rewards by mesolimbic dopamine. Nat. Neurosci. 13, 25-27.doi: 10.1038/nn.2460
47. Gariano, R. F., and Groves, P. M. (1988) Burst firing induced in midbrain dopamine neurons by stimulation of the medial prefrontal and anterior cingulate cortices. Brain Res. 462, 194-198.doi: 10.1016/0006-8993(88)90606-3
48. Ghods-Sharifi, S., and Floresco, S. B. (2010) Differential effects on effort discounting induced by inactivations of the nucleus accumbens core or shell. Behav. Neurosci. 124, 179-191.doi: 10.1037/a0018932
49. Ghods-Sharifi, S., St. Onge, J. R., and Floresco, S. B. (2009) Fundamental contribution by the basolateral amygdala to different forms of decision making. J. Neurosci. 29, 5251-5259.doi: 10.1523/JNEUROSCI.0315-09.2009
50. Glimcher, P. (2002) Decisions, decisions, decisions: choosing a biological science of choice. Neuron 36, 323-332.doi: 10.1016/50898-6273(02)00962-5
51. Goto, Y., and Grace, A. A. (2005) Dopamine-dependent interactions between limbic and prefrontal cortical plasticity in the nucleus accumbens: disruption by cocaine sensitization. Neuron 47, 255-266.doi: 10.1016/j.neuron.2005.06.017
52. Grace, A. A. (1991) Phasic versus tonic dopamine release and the modulation of dopamine system responsivity: a hypothesis for the etiology of schizophrenia. Neuroscience 41, 1-24.doi: 10.1016/0306-4522(91)90196-U
53. Haber, S. N., Kim, K.-S., Mailly, P., and Calzavara, R. (2006) Reward-related cortical inputs define a large striatal region in primates that interface with associative cortical connections, providing a substrate for incentive-based learning. J. Neurosci. 26, 8368-8376.doi: 10.1523/JNEUROSCI.0271-06.2006
54. Hattori, S., Naoi, M., and Nishino, H. (1994) Striatal dopamine turnover during treadmill running in the rat: relation to the speed of running. Brain Res. Bull. 35, 41-49.doi: 10.1016/0361-9230(94)90214-3
55. Hauber, W., and Sommer, S. (2009) Prefrontostriatal circuitry regulates effort-related decision making. Cereb. Cortex 19, 2240-2247.doi: 10.1093/cercor/bhn241
56. Hayden, B. Y., Pearson, J. M., and Platt, M. L. (2011) Neuronal basis of sequential foraging decisions in a patchy environment. Nat. Neruosci. 14, 933-939.doi: 10.1038/nn.2856
57. Hollerman, J. R., and Schultz, W. (1998) Dopamine neurons report an error in the temporal prediction of reward during learning. Nat. Neurosci. 1, 304-309.doi: 10.1038/1124
58. Hoover, W. B., and Vertes, R. P. (2007) Anatomical analysis of afferent projections to the medial prefrontal cortex in the rat. Brain Struct. Funct. 212, 149-179.doi: 10.1007/s00429-007-0150-4
59. Houston, A. (1991) Risk-sensitive foraging theory and operant psychology. J. Exp. Anal. Behav. 3, 585-589.doi: 10.1901/jeab.1991.56-585
60. Howe, M. W., Tierney, P. L., Sandberg, S. G., Phillips, P. E. M., and Graybiel, A. M. (2013) Prolonged dopamine signalling in striatum signals proximity and value of distant rewards. Nature 500, 575-579.doi: 10.1038/nature12475
61. Huettel, S. A., Stowe, C. J., Gordon, E. M., Warner, B. T., and Platt, M. L. (2006) Neural signatures of economic preferences for risk and ambiguity. Neuron 49, 765-775.doi: 10.1016/j.neuron.2006.01.024
62. Ishiwari, K., Weber, S. M., Mingote, S., Correa, M., and Salamone, J. D. (2004) Accumbens dopamine and the regulation of effort in food-seeking behavior: modulation of work output by different ratio or force requirements. Behav. Brain Res. 151, 83-91.doi: 10.1016/j.bbr.2003.08.007
63. Johnson, A. W., and Gallagher, M. (2011) Greater effort boosts the affective taste properties of food. Proc. Biol. Sci. 278, 1450-1456.doi: 10.1098/rspb.2010.1581
64. Kacelnik, A., and Bateson, M. (1996) Risky theories-the effects of variance on foraging decisions. Am. Zool. 36, 402-434.doi: 10.1093/icb/36.4.402
65. Kacelnik, A., and Marsh, B. (2002) Cost can increase preference in starlings. Anim. Behav. 63, 245-250.doi: 10.1006/anbe.2001.1900
66. Kennerley, S. S. W., Dahmubed, A. A. F., Lara, A. H., and Wallis, J. D. (2009) Neurons in the frontal lobe encode the value of multiple decision variables. J. Cogn. Neurosci. 21, 1162-1178.doi: 10.1162/jocn.2009.21100.Neurons
67. Kennerley, S. W., Walton, M. E., Behrens, T. E. J., Buckley, M. J., and Rushworth, M. F. S. (2006) Optimal decision making and the anterior cingulate cortex. Nat. Neurosci. 9, 940-947.doi: 10.1038/nn1724
68. Kirshenbaum, A. P., Szalda-Petree, A. D., and Haddad, N. F. (2003) Increased effort requirements and risk sensitivity: a comparison of delay and magnitude manipulations. Behav. Process. 61, 109-121.doi: 10.1016/S0376-6357(02)00165-1
69. Kolling, N., Behrens, T. E. J., Mars, R. B., and Rushworth, M. F. S. (2012) Neural mechanisms of foraging. Science 336, 95-98.doi: 10.1126/science.1216930
70. Kuhnen, C. M., and Knutson, B. (2005) The neural basis of financial risk taking. Neuron 47, 763-770.doi: 10.1016/j.neuron.2005.08.008
71. Labudda, K., Woermann, F. G., Mertens, M., Pohlmann-Eden, B., Markowitsch, H. J., and Brand, M. (2008) Neural correlates of decision making with explicit information about probabilities and incentives in elderly healthy subjects. Exp. Brain Res. 187, 641-650.doi: 10.1007/s00221-008-1332-x
72. Lawrence, N. S., Jollant, F., O'Daly, O., Zelaya, F., and Phillips, M. L. (2009) Distinct roles of prefrontal cortical subregions in the Iowa Gambling Task. Cereb. Cortex 19, 1134-1143.doi: 10.1093/cercor/bhn154
73. Luppino, G., Matelli, M., Camarda, R. M., Gallese, V., and Rizzolatti, G. (1991) Multiple representations of body movements in mesial area 6 and the adjacent cingulate cortex: an intracortical microstimulation study in the macaque monkey. J. Comp. Neurol. 311, 463-482.doi: 10.1002/cne.903110403
74. Mai, B., and Hauber, W. (2012) Intact risk-based decision making in rats with prefrontal or accumbens dopamine depletion. Cogn. Affect. Behav. Neurosci. 4, 719-729.doi: 10.3758/s13415-012-0115-9
75. Mai, B., Sommer, S., and Hauber, W. (2012) Motivational states influence effort-based decision making in rats: the role of dopamine in the nucleus accumbens. Cogn. Affect. Behav. Neurosci. 12, 74-84.doi: 10.3758/s13415-011-0068-4
76. Maia, T. V., and McClelland, J. L. (2004) A reexamination of the evidence for the somatic marker hypothesis: what participants really know in the Iowa gambling task. Proc. Natl. Acad. Sci. U.S.A. 101, 16075-16080.doi: 10.1073/pnas.0406666101
77. McCoy, A. N., and Platt, M. L. (2005) Risk-sensitive neurons in macaque posterior cingulate cortex. Nat. Neruosci. 8, 1220-1227.doi: 10.1038/nn1523
78. McNamara, J. M., and Houston, A. I. (2009) Integrating function and mechanism. Trends Ecol. Evol. 24, 670-675.doi: 10.1016/j.tree.2009.05.011
79. Meeusen, R., and De Meirleir, K. (1995) Exercise and brain neurotransmission. Sports Med. 20, 160-188.doi: 10.2165/00007256-199520030-00004
80. Meeusen, R., Smolders, I., Sarre, S., de Meirleir, K., Keizer, H., Serneels, M., et al. (1997) Endurance training effects on neurotransmitter release in rat striatum: an in vivo microdialysis study. Acta physiol. Scand. 159, 335-341.doi: 10.1046/j.1365-201X.1997.00118.x
81. Narayanan, N. S., Horst, N. K., and Laubach, M. (2006) Reversible inactivations of rat medial prefrontal cortex impair the ability to wait for a stimulus. Neuroscience 139, 865-876.doi: 10.1016/j.neuroscience.2005.11.072
82. Narayanan, N. S., and Laubach, M. (2006) Top-down control of motor cortex ensembles by dorsomedial prefrontal cortex. Neuron 52, 921-931.doi: 10.1016/j.neuron.2006.10.021
83. Niv, Y., Daw, N. D., and Dayan, P. (2005). "How fast to work: response vigor, motivation and tonic dopamine," in Proceedings of the Neural Information Processing Systems (NIPS), eds Y. Weiss, B. Scholkopf and J. Platt (MIT Press), 1019-1026.
84. Niv, Y., Daw, N. D., Joel, D., and Dayan, P. (2007) Tonic dopamine: opportunity costs and the control of response vigor. J. Psychopharmacol. 191, 507-520.doi: 10.1007/s00213-006-0502-4
85. Norbury, A., Manohar, S., Rogers, R. D., and Husain, M. (2013) Dopamine modulates risk-taking as a function of baseline sensation-seeking trait. J. Neurosci. 33, 12982-12986.doi: 10.1523/JNEUROSCI.5587-12.2013
86. Oades, R. D., and Halliday, G. M. (1987). Ventral tegmental (A10) system: neurobiology. 1. Anatomy and connectivity. Brain Res. 434, 117-165.
87. Ostlund, S. B., Wassum, K. M., Murphy, N. P., Balleine, B. W., and Maidment, N. T. (2011) Extracellular dopamine levels in striatal subregions track shifts in motivation and response cost during instrumental conditioning. J. Neurosci. 31, 200-207.doi: 10.1523/JNEUROSCI.4759-10.2011
88. Ostrander, S., Cazares, V. A., Kim, C., Cheung, S., Gonzalez, I., and Izquierdo, A. (2011) Orbitofrontal cortex and basolateral amygdala lesions result in suboptimal and dissociable reward choices on cue-guided effort in rats. Behav. Neurosci. 125, 350-359.doi: 10.1037/a0023574
89. Owesson-White, C. A., Roitman, M. F., Sombers, L. A., Belle, A. M., Keithley, R. B., Peele, J. L., et al. (2012) Sources contributing to the average extracellular concentration of dopamine in the nucleus accumbens. J. Neurochem. 121, 252-262.doi: 10.1111/j.1471-4159.2012.07677.x
90. Pais-Vieira, M., Lima, D., and Galhardo, V. (2007) Orbitofrontal cortex lesions disrupt risk assessment in a novel serial decision-making task for rats. Neuroscience 145, 225-231.doi: 10.1016/j.neuroscience.2006.11.058
91. Palij, P., Bull, D. R., Sheehan, M. J., Millar, J., Stamford, J., Kruk, Z. L., et al. (1990) Presynaptic regulation of dopamine release in corpus striatum monitored in vitro in real time by fast cyclic voltammetry. Brain Res. 509, 172-174.doi: 10.1016/0006-8993(90)90329-A
92. Pearson, J. M., Hayden, B. Y., Raghavachari, S., and Platt, M. L. (2009) Neurons in posterior cingulate cortex signal exploratory decisions in a dynamic multioption choice task. Curr. Biol. 19, 1532-1537.doi: 10.1016/j.cub.2009.07.048
93. Picard, N., and Strick, P. L. (1996) Motor areas of the medial wall: a review of their location and functional activation. Cereb. Cortex 6, 342-353.doi: 10.1093/cercor/6.3.342
94. Platt, M. M. L., and Huettel, S. S. A. (2008) Risky business: the neuroeconomics of decision making under uncertainty. Nat. Neurosci. 11, 398-403.doi: 10.1038/nn2062
95. Preuschoff, K., Bossaerts, P., and Quartz, S. R. (2006) Neural differentiation of expected reward and risk in human subcortical structures. Neuron 51, 381-390.doi: 10.1016/j.neuron.2006.06.024
96. Preuschoff, K., Quartz, S. R., and Bossaerts, P. (2008) Human insula activation reflects risk prediction errors as well as risk. J. Neurosci. 28, 2745-2752.doi: 10.1523/JNEUROSCI.4286-07.2008
97. Preuss, T. (1995) Do rats have prefrontal cortex? The Rose-Woolsey-Akert program reconsidered. J. Cogn. Neurosci. 7, 1-24.doi: 10.1162/jocn.1995.7.1.1
98. Rivalan, M., Coutureau, E., Fitoussi, A., and Dellu-Hagedorn, F. (2011). Inter-individual decision-making differences in the effects of cingulate, orbitofrontal, and prelimbic cortex lesions in a rat gambling task. Front. Behav. Neurosci. 5:22. doi: 10.3389/fnbeh.2011.00022
99. Rudebeck, P. H., Walton, M. E., Smyth, A. N., Bannerman, D. M., and Rushworth, M. F. S. (2006) Separate neural pathways process different decision costs. Nat. Neurosci. 9, 1161-1168.doi: 10.1038/nn1756
100. Rudorf, S., Preuschoff, K., and Weber, B. (2012) Neural correlates of anticipation risk reflect risk preferences. J. Neurosci. 32, 16683-16692.doi: 10.1523/JNEUROSCI.4235-11.2012
101. Rushworth, M. F. S., and Behrens, T. E. J. (2008) Choice, uncertainty and value in prefrontal and cingulate cortex. Nat. Neurosci. 11, 389-397.doi: 10.1038/nn2066
102. Salamone, J. (1994) The involvement of nucleus accumbens dopamine in appetitive and aversive motivation. Behav. Brain Res. 61, 117-133.doi: 10.1016/0166-4328(94)90153-8
103. Salamone, J., Correa, M., Farrar, A., and Mingote, S. (2007) Effort-related functions of nucleus accumbens dopamine and associated forebrain circuits. Psychopharmacology 191, 461-482.doi: 10.1007/s00213-006-0668-9
104. Salamone, J. D., Correa, M., Nunes, E. J., Randall, P. A., and Pardo, M. (2012) The behavioral pharmacology of effort-related choice behavior: dopamine, adenosine and beyond. J. Exp. Anal. Behav. 97, 125-146.doi: 10.1901/jeab.2012.97-125
105. Salamone, J. D., Cousins, M. S., and Bucher, S. (1994) Anhedonia or anergia? Effects of haloperidol and nucleus accumbens dopamine depletion on instrumental response selection in a T-maze cost/benefit procedure. Behav. Brain Res. 65, 221-229.doi: 10.1016/j.jneumeth.2007.03.007
106. Santiago, M., and Westerink, B. H. (1991) Characterization and pharmacological responsiveness of dopamine release recorded by microdialysis in the substantia nigra of conscious rats. J. Neurochem. 57, 738-747.doi: 10.1111/j.1471-4159.1991.tb08214.x
107. Schultz, W. (2007) Multiple dopamine functions at different time courses. Annu. Rev. Neurosci. 30, 259-288.doi: 10.1146/annurev.neuro.28.061604.135722
108. Schultz, W., Dayan, P., and Montague, P. R. (1997) A neural substrate of prediction and reward. Science 275, 1593-1599.doi: 10.1126/science.275.5306.1593
109. Schultz, W., O'Neill, M., Tobler, P. N., and Kobayashi, S. (2011) Neuronal signals for reward risk in frontal cortex. Ann. N.Y. Acad. Sci. 1239, 109-117.doi: 10.1111/j.1749-6632.2011.06256.x
110. Schweimer, J., and Hauber, W. (2006) Dopamine D1 receptors in the anterior cingulate cortex regulate effort-based decision making. Learn. Mem. 13, 777-782.doi: 10.1101/lm.409306
111. Sesack, S. R., Deutch, A. Y., Roth, R. H., and Bunney, B. S. (1989) Topographical organization of the efferent projections of the medial prefrontal cortex in the rat: an anterograde tract-tracing study with Phaseolus vulgaris leucoagglutinin. J. Comp. Neurol. 290, 213-242.doi: 10.1002/cne.902900205
112. Shenhav, A., Botvinick, M. M., and Cohen, J. D. (2013) The expected value of control: an integrative theory of anterior cingulate cortex function. Neuron 79, 217-240.doi: 10.1016/j.neuron.2013.07.007
113. Shima, K., and Tanji, J. (1998) Role for cingulate motor area cells in voluntary movement selection based on reward. Science 282, 1335-1338.doi: 10.1126/science.282.5392.1335
114. Simon, N. W., Montgomery, K. S., Beas, B. S., Mitchell, M. R., LaSarge, C. L., Mendez, I. A., et al. (2011) Dopaminergic modulation of risky decision-making. J. Neurosci. 31, 17460-17470.doi: 10.1523/JNEUROSCI.3772-11.2011
115. Singer, R. A., Berry, L. M., and Zentall, T. R. (2007) Preference for a stimulus that follows a relatively aversive event: contrast or delay reduction? J. Exp. Anal. Behav. 87, 275-285.doi: 10.1901/jeab.2007.39-06
116. Smith, B. W., Mitchell, D. G. V., Hardin, M. G., Jazbec, S., Fridberg, D., Blair, R. J. R., et al. (2009) Neural substrates of reward magnitude, probability, and risk during a wheel of fortune decision-making task. Neuroimage 44, 600-609.doi: 10.1016/j.neuroimage.2008.08.016
117. Steinberg, E. E., Keiflin, R., Boivin, J. R., Witten, I. B., Deisseroth, K., and Janak, P. H. (2013) A causal link between prediction errors, dopamine neurons and learning. Nat. Neurosci. 16, 966-973.doi: 10.1038/nn.3413
118. Stephens, D. (1981) The logic of risk-sensitive foraging preferences. Anim. Behav. 29, 628-629.doi: 10.1016/s0003-3472(81)80128-5
119. Stephens, D., Brown, J., and Ydenberg, R. (2007). Foraging: Behavior and Ecology. Chicago, IL: University of Chicago Press.
120. Stephens, D. W. (2008) Decision ecology: foraging and the ecology of animal decision making. Cogn. Affect. Behav. Neurosci. 8, 475-484.doi: 10.3758/CABN.8.4.475
121. Stephens, D. W., and Krebs, J. R. (1987). Foraging Theory. Princeton, NJ: Princeton University Press.
122. St. Onge, J. R., Abhari, H., and Floresco, S. B. (2011) Dissociable contributions by prefrontal D1 and D2 receptors to risk-based decision making. J. Neurosci. 31, 8625-8633.doi: 10.1523/JNEUROSCI.1020-11.2011
123. St. Onge, J. R., Ahn, S., Phillips, A. G., and Floresco, S. B. (2012a). Dynamic fluctuations in dopamine efflux in the prefrontal cortex and nucleus accumbens during risk-based decision making. J. Neurosci. 32, 16880-16891. doi: 10.1523/JNEUROSCI.3807-12.2012
124. St. Onge, J. R., Stopper, C. M., Zahm, D. S., and Floresco, S. B. (2012b). Separate prefrontal-subcortical circuits mediate different components of risk-based decision making. J. Neurosci. 32, 2886-2899. doi: 10.1523/JNEUROSCI.5625-11.2012
125. St. Onge, J. R., and Floresco, S. B. (2009) Dopaminergic modulation of risk-based decision making. Neuropsychopharmacology 34, 681-697.doi: 10.1038/npp.2008.121
126. St. Onge, J. R., and Floresco, S. B. (2010) Prefrontal cortical contribution to risk-based decision making. Cereb. Cortex 20, 1816-1828.doi: 10.1093/cercor/bhp250
127. Stopper, C. M., Khayambashi, S., and Floresco, S. B. (2013) Receptor-specific modulation of risk-based decision making by nucleus accumbens dopamine. Neuropsychopharmacology 38, 715-728.doi: 10.1038/npp.2012.240
128. Timmerman, W., De Vries, J. B., and Westerink, B. H. (1990) Effects of D-2 agonists on the release of dopamine: localization of the mechanism of action. Naunyn Schmiedebergs Arch. Pharmacol. 342, 650-654.doi: 10.1007/BF00175707
129. Tobler, P. N., Christopoulos, G. I., Doherty, J. P. O., Dolan, R. J., Schultz, W., and O'Doherty, J. P. (2009) Risk-dependent reward value signal in human prefrontal cortex. Proc. Natl. Acad. Sci. U.S.A. 106, 7185-7190.doi: 10.1073/pnas.0809599106
130. Tobler, P. N. P., Fiorillo, C. D. C. C. D., and Schultz, W. (2005) Adaptive coding of reward value by dopamine neurons. Science 307, 1642-1645.doi: 10.1126/science.1105370
131. Uylings, H. (2003) Do rats have a prefrontal cortex? Behav. Brain Res. 146, 3-17.doi: 10.1016/j.bbr.2003.09.028
132. Vogt, B. A., and Paxinos, G. (2012). Cytoarchitecture of mouse and rat cingulate cortex with human homologies. Brain Struct. Funct. doi: 10.1007/s00429-012-0493-3. [Epub ahead of print].
133. Voorn, P., Vanderschuren, L. J. M. J., Groenewegen, H. J., Robbins, T. W., and Pennartz, C. M. A. (2004) Putting a spin on the dorsal-ventral divide of the striatum. Trends Neurosci. 27, 468-474.doi: 10.1016/j.tins.2004.06.006
134. Waelti, P., Dickinson, A., and Schultz, W. (2001) Dopamine responses comply with basic assumptions of formal learning theory. Nature 412, 43-48.doi: 10.1038/35083500
135. Wahlstrom, D., Collins, P., White, T., and Luciana, M. (2010) Developmental changes in dopamine neurotransmission in adolescence: behavioral implications and issues in assessment. Brain Cogn. 72, 146-159.doi: 10.1016/j.bandc.2009.10.013
136. Walton, M., and Croxson, P. (2005) The mesocortical dopamine projection to anterior cingulate cortex plays no role in guiding effort-related decisions. Behav. Neurosci. 119, 323-328.doi: 10.1037/0735-7044.119.1.323
137. Walton, M. E., Bannerman, D. M., Alterescu, K., and Rushworth, M. F. S. (2003). Functional specialization within medial frontal cortex of the anterior cingulate for evaluating effort-related decisions. J. Neurosci. 23, 6475-6479.
138. Walton, M. E., Bannerman, D. M., and Rushworth, M. F. S. (2002). The role of rat medial frontal cortex in effort-based decision making. J. Neurosci. 22, 10996-1003.
139. Walton, M. E., Groves, J., Jennings, K. A., Croxson, P. L., Sharp, T., Rushworth, M. F. S., et al. (2009) Comparing the role of the anterior cingulate cortex and 6-hydroxydopamine nucleus accumbens lesions on operant effort-based decision making. Euro. J. Neurosci. 29, 1678-1691.doi: 10.1111/j.1460-9568.2009.06726.x
140. Walton, M. E., Kennerley, S. W., Bannerman, D. M., Phillips, P. E. M., and Rushworth, M. F. S. (2006) Weighing up the benefits of work: behavioral and neural analyses of effort-related decision making. Neural Netw. 19, 1302-1314.doi: 10.1016/j.neunet.2006.03.005
141. Wanat, M. J., Kuhnen, C. M., and Phillips, P. E. M. (2010) Delays conferred by escalating costs modulate dopamine release to rewards but not their predictors. J. Neurosci. 30, 12020-12027.doi: 10.1523/JNEUROSCI.2691-10.2010
142. Weber, B. J., and Huettel, S. A. (2008) The neural substrates of probabilistic and intertemporal decision making. Brain Res. 1234, 104-115.doi: 10.1016/j.brainres.2008.07.105
143. Wise, R. (2004) Dopamine, learning and motivation. Nat. Rev. Neurosci. 5, 483-494.doi: 10.1038/nrn1406
144. Zack, M., and Poulos, C. X. (2004) Amphetamine primes motivation to gamble and gambling-related semantic networks in problem gamblers. Neuropsychopharmacology 29, 195-207.doi: 10.1038/sj.npp.1300333
145. Zack, M., and Poulos, C. X. (2007) A D2 antagonist enhances the rewarding and priming effects of a gambling episode in pathological gamblers. Neuropsychopharmacology 32, 1678-1686.doi: 10.1038/sj.npp.1301295
146. Zahm, D., and Brog, J. (1992) On the significance of subterritories in the "accumbens" part of the rat ventral striatum. Neuroscience 50, 751-767.doi: 10.1016/0306-4522(92)90202-D
147. Zeeb, F. D., Robbins, T. W., and Winstanley, C. A. (2009) Serotonergic and dopaminergic modulation of gambling behavior as assessed using a novel rat gambling task. Neuropsychopharmacology 34, 2329-2343.doi: 10.1038/npp.2009.62
148. Zeeb, F. D., and Winstanley, C. A. (2011) Lesions of the basolateral amygdala and orbitofrontal cortex differentially affect acquisition and performance of a rodent gambling task. J. Neurosci. 31, 2197-2204.doi: 10.1523/JNEUROSCI.5597-10.2011
149. Zentall, T. R. (2010) Justification of effort by humans and pigeons: cognitive dissonance or contrast? Curr. Dir. Psychol. Sci. 19, 296-300.doi: 10.1177/0963721410383381