Coding of reward probability and risk by single neurons in animals

Frontiers in Neuroscience

Volumn , Issue OCT, 2011, Pages

  Burke, Christopher J ^a   Tobler, Philippe N ^a  

^a UNIVERSITY OF ZURICH   (Switzerland)

Author keywords

Basal ganglia; Dopamine; Neuroeconomics; Orbitofrontal cortex; Uncertainty

Indexed keywords

ADAPTIVE BEHAVIOR; ANTERIOR CINGULATE; ARTICLE; BASAL GANGLION; DECISION MAKING; DOPAMINERGIC ACTIVITY; DOPAMINERGIC NERVE CELL; FRONTAL CORTEX; FUNCTIONAL MAGNETIC RESONANCE IMAGING; GLOBUS PALLIDUS; HABENULA; HUMAN; LATENT PERIOD; MESENCEPHALON; NERVE CELL; NEUROPHYSIOLOGY; NONHUMAN; ORBITAL CORTEX; PARIETAL CORTEX; PEDUNCULOPONTINE TEGMENTAL NUCLEUS; POSTERIOR CINGULATE; PREFRONTAL CORTEX; PROBABILITY; PUTAMEN; REWARD; RISK; SIMULATION; SUBSTANTIA NIGRA PARS COMPACTA; SUBSTANTIA NIGRA PARS RETICULATA; TASK PERFORMANCE;

EID: 84862174376     PISSN: 16624548     EISSN: 1662453X     Source Type: Journal    
DOI: 10.3389/fnins.2011.00121     Document Type: Article

References (55)

1. Amiez, C., Joseph, J. P., and Procyk, E. (2006). Reward encoding in the monkey anterior cingulate cortex. Cereb. Cortex 16,1040-1055.
2. Apicella, P., Deffains, M., Ravel, S., and Legallet, E. (2009). Tonically active neurons in the striatum differentiate between reward delivery and omission of expected reward in a probabilistic task context. Eur. J. Neurosci. 30,515-526.
3. Apicella, P., Ravel, S., Deffains, M., and Legallet, E. (2011). The role of striatal tonically active neurons in reward prediction error signaling during instrumental task performance.J. Neurosci. 31, 1507-1515.
4. Arkadir, D., Morris, G., Vaadia, E., and Bergman, H. (2004). Independent coding of movement direction and reward prediction by single pallidal neurons. J. Neurosci. 24, 10047-10056.
5. Barbas, H., and De Olmos, J. (1990). Projections from the amygdala to basoventral and mediodorsal pre-frontal regions in the rhesus monkey. J. Comp. Neurol. 300, 549-571.
6. Belova, M. A., Paton, J. J., Morrison, S. E., and Salzman, C. D. (2007). Expectation modulates neural responses to pleasant and aversive stimuli in primate amygdala. Neuron 55, 970-984.
7. Bermudez, M. A., and Schultz, W. (2010a). Responses of amygdala neurons to positive reward predicting stimuli depend on background reward (contingency) rather than stimulus-reward pairing (contiguity). J. Neurophysiol. 103, 1158-1170.
8. Bermudez, M. A., and Schultz, W. (2010b). Reward magnitude coding in primate amygdala neurons. J. Neurophysiol. 104, 3424-3432.
9. Boorman, E. D., and Sallet, J. (2009). Mean-variance or prospect theory? The nature of value representations in the human brain. J. Neurosci. 29, 1876-1879.
10. Braun, D. A., Nagengast, A. J., and Wolpert, D. M. (2011). Risk-sensitivity in sensorimotor control. Front. Hum. Neurosci. 5:1. doi:10.3389/fnhum.2011.00001
11. Bromberg-Martin, E. S., Matsumoto, M., Hong, S., and Hikosaka, O. (2010). A pallidus-habenula-dopamine pathway signals inferred stimulus values. J. Neurophysiol. 104, 1068-1076.
12. Burke, C. J., and Tobler, P. N. (2011). Reward skewness coding in the insula independent of probability and loss. J. Neurophysiol. doi:10.1152/jn.00471.2011 [Epub ahead of print].
13. Cavada, C., Company, T., Tejedor, J., Cruz Rizzolo R, J., and Reinoso Suarez, F. (2000). The anatomical connections of the macaque monkey orbitofrontal cortex. A review. Cereb. Cortex 10, 220-242.
14. Dorris, M. C., and Glimcher, P. W. (2004). Activity in posterior parietal cortex is correlated with the subjective desireability of an action. Neuron 44, 365-378.
15. Ellsberg, D. (1961). Risk, ambiguity, and the savage axioms. Q. J. Econ. 75, 643-669.
16. Fiorillo, C. D., Tobler, P. N., and Schultz, W. (2003). Discrete coding of reward probability and uncertainty by dopamine neurons. Science 299,1898-1902.
17. Goldberg, M. E., Colby, C. L., and Duhamel, J. R. (1990). Representation of visuomotor space in the parietal lobe of the monkey. Cold Spring Harb. Symp. Quant. Biol. 55, 729-739.
18. Haber, S. N. (2003). The primate basal ganglia: parallel and integra-tive networks. J. Chem. Neuroanat. 26,317-330.
19. Hassani, O. K., Cromwell, H. C., and Schultz, W. (2001). Influence of expectation of different rewards on behavior-related neuronal activity in the striatum. J. Neurophysiol. 85, 2477-2489.
20. Hayden, B. Y., Heilbronner, S. R., Pearson, J. M., and Platt, M. L. (2011). Surprise signals in anterior cingulate cortex: neuronal encoding of unsigned reward prediction errors driving adjustment in behavior. J. Neurosci. 31,4178-4187.
21. Herrnstein, R. J., and Vaughn, W. J. (1980). "The allocation of individual behavior," in Limits to Action ed. J. E. R. Staddon (New York: Academic), 143-176.
22. Herry, C., Bach, D. R., Esposito, F., Di Salle, F., Perrig, W. J., Scheffler, K., Lüthi, A., and Seifritz, E. (2007). Processing of temporal unpredictability in human and animal amygdala. J. Neurosci. 27, 5958-5966.
23. Hong, S., and Hikosaka, O. (2008). The globus pallidus sends reward-related signals to the lateral habenula. Neuron 60, 720-729.
24. Hong, S., Jhou, T. C., Smith, M., Saleem, K. S., and Hikosaka, O. (2011). Negative reward signals from the lateral habenula to dopamine neurons are mediated by rostromedial tegmental nucleus in primates. J. Neurosci. 31, 11457-11471.
25. Jhou, T. C., Fields, H. L., Baxter, M. G., Saper, C. B., and Holland, P. C. (2009). The rostromedial tegmental nucleus (RMTg), a GABAergic afferent to midbrain dopamine neurons, encodes aversive stimuli and inhibits motor responses. Neuron 61, 786-800.
26. Joshua, M., Adler, A., Rosin, B., Vaadia, E., and Bergman, H. (2009). Encoding of probabilistic rewarding and aversive events by pallidal and nigral neurons. J. Neurophysiol. 101, 758-772.
27. Kawagoe, R., Takikawa, Y., and Hikosaka, O. (1998). Expectation of reward modulates cognitive signals in the basal ganglia. Nat. Neurosci. 1,411-416.
28. Kennerley, S. W., Dahmubed, A. F., Lara, A. H., and Wallis, J. D. (2009). Neurons in the frontal lobe encode the value of multiple decision variables. J. Cogn. Neurosci. 21, 1162-1178.
29. Kepecs, A., Uchida, N., Zariwala, H., and Mainen, Z. (2008). Neural correlates, computation and behavioral impact of decision confidence. Nature 455, 227-231.
30. Knight, F. H. (1921). Risk, Uncertainty, and Profit. New York: Houghton Mifflin.
31. Lau, B., and Glimcher, P. W. (2008). Value representations in the primate striatum during matching behavior. Neuron 58, 451-463.
32. Long, A. B., Kuhn, C. M., and Platt, M. L. (2009). Serotonin shapes risky decision making in monkeys. Soc. Cogn. Affect. Neurosci. 4, 346-356.
33. Matsumoto, M., and Hikosaka, O. (2009). Representation of negative motivational value in the primate lateral habenula. Nat. Neurosci. 12, 77-84.
34. McCoy, A. N., and Platt, M. L. (2005). Risk-sensitive neurons in macaque posterior cingulate cortex. Nat. Neurosci. 8,1220-1227.
35. Morris, G., Arkadir, D., Nevet, A., Vaadia, E., and Bergman, H. (2004). Coincident but distinct messages of midbrain dopamine and striatal tonically active neurons. Neuron 43, 133-143.
36. Musallam, S., Corneil, B. D., Greger, B., Scherberger, H., and Andersen, R. A. (2004). Cognitive control signals for neural prosthetics. Science 305, 258-262.
37. O'Neill, M., and Schultz, W. (2010). Coding of reward risk by orbitofrontal neurons is mostly distinct from coding of reward value. Neuron 68, 789-800.
38. Oyama, K., Hernádi, I., Iijima, T., and Tsutsui, K. (2010). Reward prediction error coding in dorsal striatal neurons. J. Neurosci. 30, 11447-11457.
39. Platt, M. L., and Glimcher, P. W. (1999). Neural correlates of decision variables in parietal cortex. Nature 400, 233-238.
40. Quirk, G. J., Repa, C., and LeDoux, J. E. (1995). Fear conditioning enhances short-latency auditory responses of lateral amygdala neurons: parallel recordings in the freely behaving rat. Neuron 15, 1029-1039.
41. Real, L., and Caraco, T. (1986). Risk and foraging in stochastic environments. Annu. Rev. Ecol. Syst. 17, 371-390.
42. Rescorla, R. A., and Wagner, A. R. (1972). "A theory of Pavlov-ian conditioning: variations in the effectiveness of reinforcement and nonreinforcement," in Classical Conditioning II: Current Research and Theory, eds A. H. Black and W. E Prokasy (New York: Appleton-Century-Crofts), 64-99.
43. Roitman, J. D., and Roitman, M. F. (2010). Risk-preference differentiates orbitofrontal cortex responses to freely chosen reward outcomes. Eur. I Neurosci. 31, 1492-1500.
44. Rosati, A. G., and Hare, B. (2011). Chimpanzees and bonobos distinguish between risk and ambiguity. Biol. Lett. 7, 15-18.
45. Schultz, W. (1998). Predictive reward signal of dopamine neurons. J. Neu-rophysiol. 80, 1-27.
46. Schultz, W. (2010). Dopamine signals for reward value and risk: basic and recent data. Behav. Brain Funct. 6,24.
47. Schultz, W., Dayan, P., and Montague, P. R. (1997). A neural substrate of prediction and reward. Science 275, 1593-1599.
48. Snyder, L. H., Battista, A. P., and Andersen, R. A. (1997). Coding of intention in the posterior parietal cortex. Nature 386, 167-170.
49. Stephens, D. W., and Krebs, J. R. (1986). Foraging Theory. Princeton, NJ: Princeton University Press.
50. Sugrue, L. P., Corrado, G. S., and Newsome, W. T. (2004). Matching behavior and the representation of value in the parietal cortex. Science 304, 1782-1787.
51. Tobler, P. N., Christopoulos, G. I., O'Doherty, J. P., Dolan, R. J., and Schultz, W. (2008). Neuronal distortions of reward probability without choice. J. Neurosci. 28, 11703-11711.
52. Tsai, H. C., Zhang, F., Adaman-tidis, A., Stuber, G. D., Bonci, A., de Lecea, L., and Deisseroth, K. (2009). Phasic firing in dopaminer-gic neurons is sufficient for behavioral conditioning. Science 324, 1080-1084.
53. van Duuren, E., van der Plasse, G., Lankelma, J., Joosten, R. N., Feen-stra, M. G., and Pennartz, C. M. (2009). Single-cell and population coding of expected reward probability in the orbitofrontal cortex of the rat. J. Neurosci. 29, 8965-8976.
54. Watanabe, M. (1996). Reward expectancy in primate prefrontal neurons. Nature 382, 629-632.
55. Wu, S. W., Delgado, M. R., and Maloney, L. T. (2011). The neural correlates of subjective utility of monetary outcome and probability weight in economic and in motor decision under risk. J. Neurosci. 31, 8822-8831.