Optimality and some of its discontents: Successes and shortcomings of existing models for binary decisions

Topics in Cognitive Science

Volumn 6, Issue 2, 2014, Pages 258-278

  Holmes, Philip ^a,b   Cohen, Jonathan D ^b  

^a Princeton University   (United States)

^b PRINCETON UNIVERSITY   (United States)

Author keywords

Accumulator; Cognitive control; Costs; Decision making; Drift diffusion process; Exploitation; Exploration; Optimality; Robustness; Speed accuracy trade off

Indexed keywords

ALGORITHM; ARTIFICIAL NEURAL NETWORK; COGNITION; DECISION MAKING; ENVIRONMENT; HUMAN; PHYSIOLOGY; REWARD; THEORETICAL MODEL;

ALGORITHMS; CHOICE BEHAVIOR; COGNITION; DECISION MAKING; ENVIRONMENT; HUMANS; MODELS, THEORETICAL; NEURAL NETWORKS (COMPUTER); REWARD;

EID: 84899652691     PISSN: 17568757     EISSN: 17568765     Source Type: Journal    
DOI: 10.1111/tops.12084     Document Type: Article

References (85)

1. Aston-Jones, G., & Cohen, J. D. (2005). An integrative theory of locus coeruleus-norepinephrine function: Adaptive gain and optimal performance. Annual Review of Neuroscience, 28, 403-450.
2. Balci, F., Simen, P., Niyogi, R., Saxe, A., Holmes, P., & Cohen, J. D. (2011). Acquisition of decision making criteria: Reward rate ultimately beats accuracy. Attention, Perception and Psychophysics, 73(2), 640-657.
3. Ben-Haim, Y. (2006). Information gap decision theory: Decisions under severe uncertainty (2nd ed.). New York: Academic Press.
4. Bogacz, R., Brown, E., Moehlis, J., Holmes, P. & Cohen, J. D. (2006). The physics of optimal decision making: A formal analysis of models of performance in two alternative forced choice tasks. Psychological Review, 113(4), 700-765.
5. Bogacz, R., Hu, P., Cohen, J. D., & Holmes, P. (2010). Do humans produce the speed-accuracy tradeoff that maximizes reward rate? Quarterly Journal of Experimental Psychology, 63, 863-891.
6. Bohil, C. J., & Maddox, W. T. (2003). On the generality of optimal versus objective classifier feedback effects on decision criterion learning in perceptual categorization. Memory and Cognition, 31(2), 181-198.
7. Botvinick, M., Braver, T., Barch, D., Carter, C., & Cohen, J. D. (2001). Conflict monitoring and cognitive control. Psychological Review, 108(3), 625-652.
8. Botvinick, M. M. and Cohen, J. D. (2014). The computational and neural basis of cognitive control: Charted territory and new frontiers. Cognitive Science (in press).
9. Botvinick, M. M., Huffstetler, S., & McGuire, J. (2009). Effort discounting in human nucleus accumbens. Cognitive, Affective and Behavioral Neuroscience, 9, 16-27.
10. Britten, K. H., Shadlen, M. N., Newsome, W. T., & Movshon, J. A. (1993). Responses of neurons in macaque MT to stochastic motion signals. Visual Neuroscience, 10, 1157-1169.
11. Brown, J. W. & Braver, T. S. (2005). Learned predictions of error likelihood in the anterior cingulate cortex. Science, 307(5712), 1118-1121.
12. Brown, E., Gao, J., Holmes, P., Bogacz, R., Gilzenrat, M. and Cohen, J. D. (2005). Simple networks that optimize decisions. International Journal of Bifurcation and Chaos in Applied Sciences and Engineering, 15(3), 803-826.
13. Buhusi, C. V., & Meck, W. H. (2005). What makes us tick? Functional and neural mechanisms of interval timing. Nature, 6, 755-765.
14. Busemeyer, J. R. & Townsend, J. T. (1993). Decision field theory: A dynamic-cognitive approach to decision making in an uncertain environment. Psychological Review, 100, 432-459.
15. Cho, R. Y., Nystrom, L. E., Brown, E. T., Jones, A. D., Braver, T. S., Holmes, P., & Cohen, J. D. (2002). Mechanisms underlying dependencies of performance on stimulus history in a two-alternative forced-choice task. Cognitive, Affective and Behavioral Neuroscience, 2(4), 283-299.
16. Cohen, J. D., Dunbar, K., & McClelland, J. L. (1990). On the control of automatic processes: A parallel distributed processing model of the Stroop effect. Psychological Review, 97(3), 332-361.
17. Cohen, J. D., McClure, S. M., Gilzenrat, M., & Aston-Jones, G. (2005). Dopamine-norepinephrine interactions: Exploitation vs. exploration. Neuropsychopharmacology, 30, S1, S28.
18. Cohen, J. D., McClure, S. M., & Yu, A. J. (2007). Should I stay or should I go? How the human brain manages the tradeoff between exploitation and exploration. Philosophical Transactions of the Royal Society of London. Series B., 362(1481), 933-942.
19. Cohen, J. D., Servan-Schreiber, D., & McClelland, J. L. (1992). A parallel distributed processing approach to automaticity. The American Journal of Psychology, 105, 239-269.
20. Dutilh, G., Vandekerckhove, J., Tuerlinckx, F., & Wagenmakers, E. J. (2009). A diffusion model decomposition of the practice effect. Psychonomic Bulletin and Review, 16(6), 1026-1036.
21. Eckhoff, P., Wong, K.-F., & Holmes, P. (2009). Optimality and robustness of a biophysical decision-making model under norepinephrine modulation. Journal of Neuroscience, 29(13), 4301-4311.
22. Eckhoff, P., Wong, K.-F., & Holmes, P. (2011). Dimension reduction and dynamics of a spiking neuron model for decision making under neuromodulation. SIAM Journal on Applied Dynamical Systems, 10(1), 148-188.
23. Einhäuser, W., Stout, J., Koch, C., & Carter, O. L. (2008). Pupil dilation reflects perceptual selection and predicts subsequent stability in perceptual rivalry. Proceedings of the National Academy of Sciences of the United States of America, 105, 1704-1709.
24. Eriksen, B. A. & Eriksen, C. W. (1974). Effects of noise letters upon the identification of target letters in a non-search task. Perception and Psychophysics, 16, 143-149.
25. Feng, S., Holmes, P., Rorie, A., & Newsome, W. T. (2009). Can monkeys choose optimally when faced with noisy stimuli and unequal rewards? PLoS Computational Biology, 5(2), e1000284.
26. Gao, J., Tortell, R., & McClelland, J. L. (2011). Dynamic integration of reward and stimulus information in perceptual decision-making. PLoS ONE, 6(3), e16749.
27. Gao, J., Wong-Lin, K. F., Holmes, P., Simen, P., & Cohen, J. D. (2009). Sequential effects in two-choice reaction time tasks: Decomposition and synthesis of mechanisms. Neural Computation, 21(9), 2407-2436.
28. Gibbon, J. (1977). Scalar expectancy theory and Weber's law in animal timing. Psychological Review, 84(3), 279-325.
29. Gilzenrat, M. S., Nieuwenhuis, S., & Cohen, J. D. (2010). Pupil diameter tracks changes in control state predicted by the adaptive gain theory of locus coeruleus function. Cognitive, Affective and Behavioral Neuroscience, 10, 252-269.
30. Gold, J. I. & Shadlen, M. N. (2001). Neural computations that underlie decisions about sensory stimuli. Trends in Cognitive Sciences, 5(1), 10-16.
31. Gold, J. I., & Shadlen, M. N. (2002). Banburismus and the brain: Decoding the relationship between sensory stimuli, decisions, and reward. Neuron, 36, 299-308.
32. Goldfarb, S., Wong-Lin, K. F., Schwemmer, M., Leonard, N. E., & Holmes, P. (2012). Can post-error dynamics explain sequential reaction time patterns? Frontiers in Psychology, 3, 213.
33. Gratton, G., Coles, M. G., Sirevaag, E. J., Eriksen, C. W., & Donchin, E. (1988). Pre- and poststimulus activation of response channels: A psychophysiological analysis. Journal of Experimental Psychology. Human Perception and Performance, 14, 331-344.
34. Grossberg, S. (1988). Nonlinear neural networks: principles, mechanisms, and architectures. Neural Networks, 1, 17-61.
35. Hanks, T. D., Mazurek, M. E., Kiani, R., Hopp, E., & Shadlen, M. N. (2011). Elapsed decision time affects the weighting of prior probability in a perceptual decision task. Journal of Neuroscience, 31(17), 6339-6352.
36. Henik, A., Bibi, U., Yanai, M., & Tzelgov, J. (1997). The Stroop effect is largest during first trials. Abstracts of the Psychonomic Society.
37. Holroyd, C. B., & Coles, M. G. H. (2002). The neural basis of human error processing: Reinforcement learning, dopamine, and the error-related negativity. Psychological Review, 109(4), 679-709.
38. Jepma, M., & Nieuwenhuis, S. (2011). Pupil diameter predicts changes in the exploration-exploitation trade-off: Evidence for the adaptive gain theory. Journal of Cognitive Neuroscience, 23, 1587-1596.
39. Jones, A. D., Cho, R. Y., Nystrom, L. E., Cohen, J. D., Brown, E. T., & Braver, T. S. (2002). A computational model of anterior cingulate function in speeded response tasks: Effects of frequency, sequence, and conflict. Cognitive, Affective and Behavioral Neuroscience, 2(4), 300-317.
40. Kiani, R. and Shadlen, M. N. (2009). Representation of confidence associated with a decision by neurons in the parietal cortex. Science, 324(5928), 759-764.
41. Laming, D. (1979). Choice reaction performance following an error. Acta Psychologica, 43(3), 199-224.
42. Laming, D. R. J. (1968). Information theory of choice-reaction times. New York: Academic Press.
43. Lindsay, D. S., & Jacoby, L. L. (1994). Stroop process dissociations: The relationship between facilitation and interference. Journal of Experimental Psychology: Human Perception and Performance, 20, 219-234.
44. Liu, Y., Holmes, P., & Cohen, J. D. (2008). A neural network model of the Eriksen task: Reduction, analysis, and data fitting. Neural Computation, 20(2), 345-373.
45. Liu, Y. S., Yu, A. J., & Holmes, P. (2009). Dynamical analysis of Bayesian inference models for the Eriksen task. Neural Computation, 21(6), 1520-1553.
46. Logan, G. D., & Zbrodoff, N. J. (1979). When it helps to be misled: Facilitative effects of increasing the frequency of conflicting stimuli in a Stroop-like task. Memory and Cognition, 7(3), 166-174.
47. Maddox, W. T., & Bohil, C. J. (1998). Base-rate and payoff effects in multidimensional perceptual categorization. Journal of Experimental Psychology: Learning, Memory and Cognition, 24(6), 1459-1482.
48. Mazurek, M. E., Roitman, J. D., Ditterich, J., & Shadlen, M. N. (2003). A role for neural integrators in perceptual decision making. Cerebral Cortex, 13(11), 891-898.
49. McClure, S. M., Gilzenrat, M. S., & Cohen, J. D. (2006). An exploration-exploitation model based on norepinephrine and dopamine activity. In Y. Weiss, B. Sholkopf, & J. C. Platt (Eds.), Advances in neural information processing systems 18 (pp. 867-874). Cambridge, MA: MIT Press.
50. Miller, K. D., & Fumarola, F. (2012). Mathematical equivalence of two common forms of firing rate models of neural networks. Neural Computation, 24, 25-31.
51. Murphy, P. R., Robertson, I. H., Balsters, J. H., & O'Connell, R. G. (2011). Pupillometry and P3 index the locus coeruleus noradrenergic arousal function in humans. Psychophysiology, 48, 1532-1543.
52. Myung, I. J., & Busemeyer, J. R. (1989). Criterion learning in a deferred decision making task. The American Journal of Psychology, 102(1), 1-16.
53. Petrov, A. A., Van Horn, N. M., & Ratcliff, R. (2011). Dissociable perceptual-learning mechanisms revealed by diffusion-model analysis. Psychonomic Bulletin and Review, 18(3), 490-497.
54. Posner, M. I., & Snyder, C. R. R. (1975). Attention and cognitive control. In R. Solso (Ed.), Information Processing and Cognition: The Loyola Symposium (pp. 55-85) . Hillsdale, NJ: Lawrence Erlbaum.
55. Rabbitt, P. M. A. (1966). Errors and error-correction in choice-response tasks. Journal of Experimental Psychology, 71, 264-272.
56. Rakitin, B. C., Hinton, S. C., Penney, T. B., Malapani, C., Gibbon, J., & Meck, W. H. (1998). Scalar expectancy theory and peak interval timing in humans. Journal of Experimental Psychology: Animal Behavior Processes, 24, 1-19.
57. Ratcliff, R. (1978). A theory of memory retrieval. Psychological Review, 85(1), 59-108.
58. Ratcliff, R., Cherian, A., & Segraves, M. A. (2003). A comparison of macaque behavior and superior colliculus neuronal activity to predictions from models of two choice decisions. Journal of Neurophysiology, 90, 1392-1407.
59. Ratcliff, R., Hasegawa, Y. T., Hasegawa, R. P., Smith, P. L., & Segraves, M. A. (2006). Dual-diffusion model for single-cell recording data from the superior colliculus in a brightness-discrimination task. Journal of Neurophysiology, 97, 1756-1774.
60. Ratcliff, R., Van Zandt, T., & McKoon, G. (1999). Connectionist and diffusion models of reaction time. Psychological Review, 106(2), 261-300.
61. Roitman, J., & Shadlen, M. (2002). Response of neurons in the lateral intraparietal area during a combined visual discrimination reaction time task. Journal of Neuroscience, 22(21), 9475-9489.
62. Rorie, A., Gao, J., McClelland, J. L., & Newsome, W. T. (2010). Integration of sensory and reward information during perceptual decision-making in lateral intraparietal cortex (LIP). PLoS ONE, 5(2), e9308.
63. Roxin, A., & Ledberg, A. (2008). Neurobiological models of two-choice decision making can be reduced to a one-dimensional nonlinear diffusion equation. PLoS Computational Biology, 4(3), e1000046.
64. Rumelhart, D. E., & McClelland, J. L. (1986). Parallel distributed processing: Explorations in the microstructure of cognition. Cambridge, MA: MIT Press.
65. Schall, J. D. (2001). Neural basis of deciding, choosing and acting. Nature Reviews. Neuroscience, 2, 33-42.
66. Servan-Schreiber, D., Printz, H., & Cohen, J. D. (1990). A network model of catecholamine effects: Gain, signal-to-noise ratio, and behavior. Science, 249, 892-895.
67. Shadlen, M. N., & Newsome, W. T. (2001). Neural basis of a perceptual decision in the parietal cortex (area LIP) of the rhesus monkey. Journal of Neurophysiology, 86, 1916-1936.
68. Shea-Brown, E., Gilzenrat, M., & Cohen, J. D. (2008). Optimization of decision making in multilayer networks: The role of locus coeruleus. Neural Computation, 20, 2863-2894.
69. Simen, P., Cohen, J. D., & Holmes, P. (2006). Rapid decision threshold modulation by reward rate in a neural network. Neural Networks, 19, 1013-1026.
70. Simen, P., Contreras, D., Buck, C., Hu, P., Holmes, P., & Cohen, J. D. (2009). Reward rate optimization in two-alternative decision making: Empirical tests of theoretical predictions. Journal of Experimental Psychology: Human Perception and Performance, 35(6), 1865-1897.
71. Smith, P. L., & Ratcliff, R. (2004). Psychology and neurobiology of simple decisions. Trends in Neurosciences, 27(3), 161-168.
72. Stone, M. (1960). Models for choice-reaction time. Psychometrika, 25, 251-260.
73. Todd, M. T., Botvinick, M. M., Schwemmer, M. A., Cohen, J. D., & Dayan, P. (2011). Normative analysis of task switching. Abstract No. 194.21, Society for Neuroscience Meeting.
74. Usher, M., Cohen, J. D., Servan-Schreiber, D., Rajkowsky, J., & Aston-Jones, G. (1999). The role of locus coeruleus in the regulation of cognitive performance. Science, 283, 549-554.
75. Usher, M., & McClelland, J. L. (2001). On the time course of perceptual choice: The leaky competing accumulator model. Psychological Review, 108(3), 550-592.
76. van Ravenzwaaij, D., Mulder, M. J., Tuerlinckx, F., & Wagenmakers, E-J. (2012). Do the dynamics of prior information depend on task context? An analysis of optimal performance and an empirical test. Frontiers in Cognitive Neuroscience, 3, 132.
77. van Ravenzwaaij, D., der Maas, H. L. J., & Wagenmakers, E. J. (2012). Optimal decision making in neural inhibition models. Psychological Review, 119(1), 201-215.
78. Wald, A., & Wolfowitz, J. (1948). Optimum character of the sequential probability ratio test. The Annals of Mathematical Statistics, 19, 326-339.
79. Wong, K. F., & Wang, X.-J. (2006). A recurrent network mechanism of time integration in perceptual decisions. Journal of Neuroscience, 26(4), 1314-1328.
80. Yang, T., Hanks, T. D., Mazurek, M., McKinley, M., Palmer, J., & Shadlen, M. N. (2005). Incorporating prior probability into decision making in the face of uncertain reliability of evidence. Abstract No. 621.4, Society for Neuroscience Meeting, http://sfn.scholarone.com.
81. Yeung, N., Botvinick, M. M., & Cohen, J. D. (2004). The neural basis of error detection: Conflict monitoring and the error-related negativity. Psychological Review, 111(4), 931-959.
82. Yu, A. J., & Cohen, J. D. (2009). Sequential effects: Superstition or rational behavior? In J. C. Platt, D. Koller, Y. Singer, and S. Roweis (Eds.), Advances in neural information processing systems 21 (pp. 1873-1880). Cambridge, MA: MIT Press.
83. Yu, A. J., & Dayan, P. (2005). Uncertainty, neuromodulation, and attention. Neuron, 46, 681-692.
84. Yu, A., Dayan, P., & Cohen, J. D. (2009). Dynamics of attentional selection under conflict: Toward a rational Bayesian account. Journal of Experimental Psychology: Human Perception and Performance, 35(3), 700-717.
85. Zacksenhouse, M., Bogacz, R., & Holmes, P. (2010). Robust versus optimal strategies for two-alternative forced choice tasks. Journal of Mathematical Psychology, 54, 230-246.