Different varieties of uncertainty in human decision-making

Frontiers in Neuroscience

Volumn , Issue JUN, 2012, Pages

  Bland, Amy R ^a   Schaefer, Alexandre ^b  

^a UNIVERSITY OF MANCHESTER   (United Kingdom)

^b DURHAM UNIVERSITY   (United Kingdom)

Author keywords

Decision making; Uncertainty; Unexpected uncertainty; Volatility

Indexed keywords

ADAPTIVE BEHAVIOR; AMYGDALOID NUCLEUS; BAYES THEOREM; BAYESIAN LEARNING; CONFLICT; DECISION MAKING; ELECTROENCEPHALOGRAM; ELECTROPHYSIOLOGY; ENVIRONMENTAL CHANGE; EXECUTIVE FUNCTION; FUNCTIONAL MAGNETIC RESONANCE IMAGING; HUMAN; LEARNING; NERVE CELL NETWORK; NEUROIMAGING; NEUROMODULATION; PREFRONTAL CORTEX; REVIEW; UNCERTAINTY; WORKING MEMORY;

EID: 84862663195     PISSN: 16624548     EISSN: 1662453X     Source Type: Journal    
DOI: 10.3389/fnins.2012.00085     Document Type: Review

References (89)

1. Aston-Jones, G., and Cohen, J. D. (2005). An integrative theory of locus coeruleus-norepinephrine function: adaptive gain and optimal performance. Annu. Rev. Neurosci. 28, 403-450.
2. Aston-Jones, G., Rajkowski, J., and Kubiak, P. (1997). Conditioned responses of monkey locus coeruleus neurons anticipate acquisition of discriminative behavior in a vigilance task. Neuroscience 80, 697-715
3. Aston-Jones, G., Rajkowski, J., Lu, W., Zhu, Y., Cohen, J. D., and Morecraft, R. J. (2002). Prominent projections from the orbital prefrontal cortex to the locus coeruleus in monkey. Soc. Neurosci. Abstr. 28, 86-89
4. Avery, M. C., Nitz, D. A., Chiba, A. A., and Krichmar, J. L. (2012). Simulation of cholinergic and noradrenergic modulation of behavior in uncertain environments. Front. Comput. Neurosci. 6:5. doi:10.3389/fncom.2012.0000
5. Awh, E., and Jonides, J. (2001). Overlapping mechanisms of attention and spatial working memory. Trends Cogn. Sci. (Regul. Ed.) 5, 119-126
6. Baddeley, A. (1992). Working memory. Science 255, 556-559
7. Behrens, T. E. J., Woolrich, M. W., Walton, M. E., and Rushworth, M. F. S. (2007). Learning the value of information in an uncertain world. Nat. Neurosci. 10, 1214-1221
8. Bland, A. R., and Schaefer, A. (2011). Electrophysiological correlates of decision making under varying levels of uncertainty. Brain Res. 1417, 55-66
9. Botvinick, M., Nystrom, L. E., Fissell, K., Carter, C. S., and Cohen, J. D. (1999). Conflict monitoring versus selection-for-action in anterior cin-gulate cortex. Nature 402, 179-181
10. Botvinick, M. M., Braver, T. S., Barch, D. M., Carter, C. S., and Cohen, J. D. (2001). Conflict monitoring and cognitive control. Psychol. Rev. 108, 624-652
11. Bouret, S., and Sara, S. J. (2004). Reward expectation, orientation of attention and locus coeruleus-medial frontal cortex interplay during learning. Eur. J. Neurosci. 20, 791-802
12. Bouret, S., and Sara, S. J. (2005). Network reset: a simplified overarching theory of locus coeruleus noradren-aline function. Trends Neurosci. 28, 574-582
13. Braver, T. S., Gray, J. R., and Burgess, G. C. (2007). " Explaining the many varieties of working memory variation: dual mechanisms of cognitive control," in Variation in Working Memory, eds C. R. A. Andrew, J. Christopher, J. M. Kane, M. Akira, and T. N. John (New York: Oxford University Press), 76-106.
14. Braver, T. S., Paxton, J. L., Locke, H. S., and Barch, D. M. (2009). Flexible neural mechanisms of cognitive control within human prefrontal cortex. Proc. Natl. Acad. Sci. U.S.A. 106, 7351-7356
15. Braver, T. S., Reynolds, J. R., and Donaldson, D. I. (2003). Neural mechanisms of transient and sustained cognitive control during task switching. Neuron 39, 713-726
16. Brown, J. W., and Braver, T. S. (2005). Learned predictions of error likelihood in the anterior cingulate cortex. Science 307, 1118-1121
17. Carter, C. S., Braver, T. S., Barch, D. M., Botvinick, M. M., Noll, D., and Cohen, J. D. (1998). Anterior cingu-late cortex, error detection, and the online monitoring of performance. Science 280, 747-749
18. Cavanagh, J. F., Figueroa, C. M., Cohen, M. X., and Frank, M. J. (2011). Frontal theta reflects uncertainty and unexpectedness during exploration and exploitation. Cereb. Cortex. doi: 10.1093/cercor/bhr332
19. Cohen, J. D., McClure, S. M., andYu,A. J. (2007a). Should I stay or should I go? How the human brain manages the trade-off between exploitation and exploration. Philos. Trans. R. Soc. Lond. B Biol. Sci. 362, 933-942
20. Cohen, M. X., Elger, C. E., and Ran-ganath, C. (2007b). Reward expectation modulates feedback-related negativity and EEG spectra. Neu-roimage 35, 968-978
21. Courville, A. C., Daw, N. D., and Touret-zky, D. S. (2006). Bayesian theories of conditioning in a changing world. Trends Cogn. Sci. (Regul. Ed.) 10, 294-300
22. Critchley, H. D., Mathias, C. J., and Dolan, R. J. (2001). Neural activity in the human brain relating to uncertainty and arousal during anticipation. Neuron 29, 537-545
23. Dalley, J. W., McGaughy, J., O'Connell, M. T., Cardinal, R. N., Levita, L., and Robbins, T. W. (2001). Distinct changes in cortical acetylcholine and noradrenaline efflux during contingent and noncontingent performance of a visual attentional task. J. Neurosci. 21, 4908-4914
24. Daw, N. D., O'Doherty, J. P., Dayan, P., Seymour, B., and Dolan, R. J. (2006). Cortical substrates for exploratory decisions in humans. Nature 441, 876-879
25. Dayan, P., and Yu, A. (2002). " ACh, uncertainty, and cortical inference," in Advances in Neural Information Processing Systems 14, Vols. 1 and 2, eds T. G. Dietterich, S. Becker, and Z. Ghahramani (MIT Press), 189-196
26. Dehaene, S., Posner, M. I., and Tucker, D. M. (1994). Localization of a neural system for error detection and compensation. Psychol. Sci. 5, 303-305.
27. di Pellegrino, G., Ciaramelli, E., and Làdavas, E. (2007). The regulation of cognitive control following rostral anterior cingulate cortex lesion in humans. J. Cogn. Neurosci. 19, 275-286
28. Donchin, E. (1981). Surprise!... Surprise? Psychophysiology 18, 493-513
29. Donchin, E., and Coles, M. G. (1988). Is the P300 component a manifestation of context updating? Behav. Brain. Sci. 11, 357-427
30. Doya, K. (2008). Modulators of decision making. Nat. Neurosci. 11, 410-416
31. Duncan-Johnson, C. C., and Donchin, E. (1977). On quantifying surprise: the variation of event-related potentials with subjective probability. Psy-chophysiology 14, 456-467
32. Duncan-Johnson, C. C., and Donchin, E. (1982). The P300 component of the event-related brain potential as an index of information processing. Biol. Psychol. 14, 1-52
33. Egner, T. (2009). Prefrontal cortex and cognitive control: motivating functional hierarchies. Nat. Neurosci. 12, 821-822
34. Egner, T., and Hirsch, J. (2005). Cognitive control mechanisms resolve conflict through cortical amplification of task-relevant information. Nat. Neurosci. 8, 1784-1790
35. Elliott, R., and Dolan, R. J. (1998). Activation of different anterior cingu-late foci in association with hypothesis testing and response selection. Neuroimage 8, 17-29
36. Fiorillo, C. D., Tobler, P. N., and Schultz, W. (2003). Discrete coding of reward probability and uncertainty by dopamine neurons. Science 299, 1898-1902
37. Frank, M. J., Doll, B. B., Oas-Terpstra, J., and Moreno, F. (2009). Prefrontal and striatal dopaminergic genes predict individual differences in exploration and exploitation. Nat. Neu-rosci. 12, 1062-1068
38. Gehring, W. J., Coles, M. G. H., Meyer, D. E., and Donchin, E. (1990). The error-related negativity: an event-related brain potential accompanying errors. Psychophysiology 27, s34
39. Gilzenrat, M., Nieuwenhuis, S., Jepma, M., and Cohen, J. (2010). Pupil diameter tracks changes in control state predicted by the adaptive gain theory of locus coeruleus function. Cogn. Affect. Behav. Neurosci. 10, 252-269
40. Goel, V., and Dolan, R. J. (2000). Anatomical segregation of component processes in an inductive inference task. J. Cogn. Neurosci. 12, 110-119
41. Hollerman,J. R., and Schultz,W.(1998). Dopamine neurons report an error in the temporal prediction of reward during learning. Nat. Neurosci. 1, 304-309
42. Hsu, M., Bhatt, M., Adolphs, R., Tranel, D., and Camerer, C. F. (2005). Neural systems responding to degrees of uncertainty in human decision-making. Science 310, 1680-1683
43. Huettel, S., Stowe, J., Gordon, E., Warner, B., and Platt, M. (2006). Neural signatures of economic preferences for risk and ambiguity. Neuron 49, 765-775
44. Huettel, S. A., Song, A. W., and McCarthy, G. (2005). Decisions under uncertainty: probabilistic context influences activation of prefrontal and parietal cortices. J. Neurosci. 25, 3304-3311
45. Kennerley,S. W.,Walton, M. E., Behrens, T. E. J., Buckley, M. J., and Rush-worth, M. F. S. (2006). Optimal decision making and the anterior cingulate cortex. Nat. Neurosci. 9, 940-947
46. Kerns, J. G., Cohen, J. D., MacDonald, A. W., Cho, R. Y., Stenger, V. A., and Carter, C. S. (2004). Anterior cingulate conflict monitoring and adjustments in control. Science 303, 1023-1026
47. Knutson, B., Taylor, J., Kaufman, M., Peterson, R., and Glover, G. (2005). Distributed neural representation of expected value. J. Neurosci. 25, 4806-4812
48. Koechlin, E., Ody, C., and Kouneiher, F. (2003). The architecture of cognitive control in the human prefrontal cortex. Science 302, 1181-1185
49. Koechlin, E., and Summerfield, C. (2007). An information theoretical approach to prefrontal executive function. Trends Cogn. Sci. (Regul. Ed.) 11, 229-235
50. Kouneiher, F., Charron, S., and Koech-lin, E. (2009). Motivation and cognitive control in the human prefrontal cortex. Nat. Neurosci. 12, 939-945
51. Krain, A. L., Wilson, A. M., Arbuckle, R., Castellanos, F. X., and Milham, M. P. (2006). Distinct neural mechanisms of risk and ambiguity: a meta-analysis of decision-making. Neu-roimage 32, 477-484
52. Krugel, L. K., Biele, G., Mohr, P. N. C., Li, S.-C., and Heekeren,H. R. (2009). Genetic variation in dopaminer-gic neuromodulation influences the ability to rapidly and flexibly adapt decisions. Proc. Natl. Acad. Sci. U.S.A. 106, 17951-17956
53. Mansouri, F. A., Tanaka, K., and Buckley, M. J. (2009). Conflict-induced behavioural adjustment: a clue to the executive functions of the pre-frontal cortex. Nat. Rev. Neurosci. 10, 141-152.
54. Mathys, C., Daunizeau, J., Fris-ton, K. J., and Stephan, K. E. (2011). A Bayesian foundation for individual learning under uncertainty. Front. Hum. Neurosci. 5:39. doi:10.3389/fnhum.2011.00039
55. McClure, S. M., Gilzenrat, M. S., and Cohen, J. D. (2006). " An exploration - exploitation model based on norepinephrine and dopamine activity," in Advances in Neural Information Processing Systems, Vol. 18, eds Y. Weiss, B. Schölkopf, and J. Platt (Cambridge: MIT Press), 867-874
56. Mushtaq, F., Bland, A. R., and Schae-fer, A. (2011). Uncertainty and cognitive control. Front. Psychol. 2:249. doi:10.3389/fpsyg.2011.00249
57. Nassar, M. R., Wilson, R. C., Heasly, B., and Gold, J. I. (2010). An approximately bayesian delta-rule model explains the dynamics of belief updating in a changing environment. J. Neurosci. 30, 12366-12378
58. Nieuwenhuis, S., Yeung, N., van den Wildenberg, W., and Ridderinkhof, K. (2003). Electrophysiological correlates of anterior cingulate function in a go/no-go task: effects of response conflict and trial type frequency. Cogn. Affect. Behav. Neu-rosci. 3, 17-26
59. Nieuwenhuis, S. (2011). " Learning, the P3, and the locus coeruleus-norepinephrine system," in Neural Basis of Motivational and Cognitive Control, eds R. B. Mars, J. Sallet, M. F. S. Rushworth, and N.Yeung (Oxford University Press), 209-222
60. Owen, A. M., McMillan, K. M., Laird, A. R., and Bullmore, E. (2005). N-back working memory paradigm: a meta-analysis of normative functional neuroimaging studies. Hum. Brain Mapp. 25, 46-59
61. Paulus, M. P., Feinstein, J. S., Simmons, A., and Stein, M. B. (2004). Anterior cingulate activation in high trait anxious subjects is related to altered error processing during decision making. Biol. Psychiatry 55, 1179-1187
62. Paulus, M. P., Hozack, N., Frank, L., and Brown, G. G. (2002). Error rate and outcome predictability affect neural activation in prefrontal cortex and anterior cingulate during decision-making. Neuroimage 15, 836-846
63. Payzan-LeNestour, E., and Bossaerts, P. (2011). Risk, Unexpected uncertainty, and estimation uncertainty: Bayesian learning in unstable settings. PLoS Comput. Biol. 7, e1001048. doi:10.1371/journal.pcbi.1001048
64. Phillips, J. M., McAlonan, K., Robb, W. G. K., and Brown, V. J. (2000). Cholinergic neurotransmis-sion influences covert orientation of visuospatial attention in the rat. Psychopharmacology 150, 112-116
65. Polezzi, D., Lotto, L., Daum, I., Sar-tori, G., and Rumiati, R. (2008). Predicting outcomes of decisions in the brain. Behav. Brain Res. 187, 116-122
66. Polich, J. (1990). Probability and inter-stimulus interval effects on the P300 from auditory stimuli. Int. J. Psy-chophysiol. 10, 163-170
67. Preuschoff, K., Bossaerts, P., and Quartz, S. R. (2006). Neural differentiation of expected reward and risk in human subcortical structures. Neuron 51, 381-390
68. Preuschoff, K., Hart, B. M., and Einhauser, W. (2011). Pupil dilation signals surprise: evidence for noradrenaline's role in decision making. Front. Neurosci. 5:115. doi:10.3389/fnins.2011.00115
69. Rajkowski, J., Kubiak, P., and Aston-Jones, G. (1993). Correlations between locus coeruleus (LC) neural activity, pupil diameter and behavior in monkey support a role of LC in attention. Abstr. Soc. Neurosci. 19, 974
70. Riba, J., Rodríguez-Fornells, A., Morte, A., Münte, T. F., and Barbanoj, M. J. (2005). Noradrenergic stimulation enhances human action monitoring. J. Neurosci. 25, 4370-4374
71. Ridderinkhof, K. R., van den Wilden-berg, W. P. M., Segalowitz, S. J., and Carter, C. S. (2004). Neurocogni-tive mechanisms of cognitive control: the role of prefrontal cortex in action selection, response inhibition, performance monitoring, and reward-based learning. Brain Cogn. 56, 129-140
72. Rushworth, M. F., and Behrens, T. E. (2008). Choice, uncertainty and value in prefrontal and cingu-late cortex. Nat. Neurosci. 11, 389-397
73. Sara, S. J. (2009). The locus coeruleus and noradrenergic modulation of cognition. Nat. Rev. Neurosci. 10, 211-223
74. Sarter, M., Givens, B., and Bruno, J. P. (2001). The cognitive neuroscience of sustained attention: where top-down meets bottom-up. Brain Res. Rev. 35, 146-160
75. Sarter, M., and Parikh, V. (2005). Choline transporters, cholinergic transmission and cognition. Nat. Rev. Neurosci. 6, 48-56
76. Seymour, B., Daw, N., Dayan, P., Singer, T., and Dolan, R. (2007). Differential encoding of losses and gains in the human striatum. J. Neurosci. 27, 4826-4831
77. Stern, E. R., Gonzalez, R., Welsh, R. C., and Taylor, S. F. (2010). Updating beliefs for a decision: neural correlates of uncertainty and underconfidence. J. Neurosci. 30, 8032-8041
78. Sutton, R. S., and Barto, A. G. (1998). Reinforcement Learning: An Introduction. Cambridge: MIT Press
79. Tobler, P. N., O'Doherty, J. P., Dolan, R. J., and Schultz, W. (2007). Reward value coding distinct from risk attitude-related uncertainty coding in human reward systems. J. Neuro-physiol. 97, 1621-1632
80. van Veen, V., and Carter, C. S. (2002). The anterior cingulate as a conflict monitor: fMRI and ERP studies. Physiol. Behav. 77, 477-482
81. van Wouwe, N. C., Band, G. P. H., and Ridderinkhof, K. R. (2011). Positive affect modulates flexibility and evaluative control. J. Cogn. Neurosci. 23, 524-539
82. Volz, K. G., Schubotz, R. I., and von Cra-mon, D. Y. (2003). Predicting events of varying probability: uncertainty investigated by fMRI. Neuroimage 19, 271-280
83. Witte, E. A., Davidson, M. C., and Mar-rocco, R. T. (1997). Effects of altering brain cholinergic activity on covert orienting of attention: comparison of monkey and human performance. Psychopharmacology (Berl.) 132, 324-334
84. Yeung, N., Botvinick, M., and Cohen, J. D. (2004). The neural basis of error detection: conflict monitoring and the error-related negativity. Psychol. Rev. 111, 931-959
85. Yeung, N., and Cohen, J. D. (2006). The impact of cognitive deficits on conflict monitoring. Psychol. Sci. 17, 164-171
86. Yu, A. J. (2007). Adaptive behavior: humans act as bayesian learners. Curr. Biol. 17, R977-R980
87. Yu, A. J., and Dayan, P. (2003). Expected and unexpected uncertainty: ACh and NE in the neocor-tex. Adv. Neural Inf. Proc. Syst. 15, 157-164
88. Yu, A. J., and Dayan, P. (2005). Uncertainty, neuromodulation, and attention. Neuron 46, 681-692
89. Yu, A. J. (ed). (2011). " Uncertainty and Neuromodulation," in Neuroscience of Decision Making. (New York: Psychology Press).