Reward Rate Optimization in Two-Alternative Decision Making: Empirical Tests of Theoretical Predictions

Journal of Experimental Psychology: Human Perception and Performance

Volumn 35, Issue 6, 2009, Pages 1865-1897

  Simen, Patrick ^a   Contreras, David ^b   Buck, Cara ^c   Hu, Peter ^d   Holmes, Philip ^e   Cohen, Jonathan D ^a  

^a PRINCETON UNIVERSITY   (United States)

^b UNIVERSITY OF GRANADA   (Spain)

^c UNIVERSITY OF CALIFORNIA   (United States)

^d UNIVERSITY OF CHICAGO   (United States)

^e Princeton University   (United States)

Author keywords

decision making; diffusion; optimization; response time; reward rate

Indexed keywords

ARTICLE; DECISION MAKING; DIFFERENTIAL THRESHOLD; HUMAN; LEARNING; PSYCHOLOGICAL MODEL; PSYCHOMOTOR PERFORMANCE; REACTION TIME; REINFORCEMENT; REWARD;

DECISION MAKING; DIFFERENTIAL THRESHOLD; HUMANS; MODELS, PSYCHOLOGICAL; PROBABILITY LEARNING; PSYCHOMOTOR PERFORMANCE; REACTION TIME; REINFORCEMENT SCHEDULE; REWARD;

EID: 73649097270     PISSN: 00961523     EISSN: None     Source Type: Journal    
DOI: 10.1037/a0016926     Document Type: Article

References (84)

1. Akaike H. A new look at the statistical model identification. IEEE Transactions on Automatic Control 19 (1974) 716-723
2. Audley R.J., and Pike A.R. Some alternative stochastic models of choice. British Journal of Mathematical and Statistical Psychology 18 (1965) 207-225
3. Bogacz R., Brown E., Moehlis J., Holmes P., and Cohen J.D. The physics of optimal decision making: A formal analysis of models of performance in two-alternative forced choice tasks. Psychological Review 113 (2006) 700-765
4. Bogacz, R., Hu, P., Cohen, J., & Holmes, P. (in press). Do humans produce the speed-accuracy tradeoff that maximizes reward rate? Quarterly Journal of Experimental Psychology
5. Brainard D.H. The psychophysics toolbox. Spatial Vision 10 (1997) 433-436
6. Britten K.H., Shadlen M.N., Newsome W.T., and Movshon J.A. The analysis of visual motion: A comparison of neuronal and psychophysical performance. Journal of Neuroscience 12 (1992) 4745-4765
7. Busemeyer J.R., and Townsend J.T. Fundamental derivations from decision field theory. Mathematical Social Sciences 23 (1992) 255-282
8. Busemeyer J.R., and Townsend J.T. Decision field theory: A dynamic-cognitive approach to decision making in an uncertain environment. Psychological Review 100 (1993) 432-459
9. Carpenter R.H.S., and Williams M.L.L. Neural computation of log likelihood in control of saccadic eye movements. Nature 377 (1995) 59-62
10. Edwards W. Costs and payoffs are instructions. Psychological Review 68 (1961) 275-284
11. Edwards W. Optimal strategies for seeking information: Models for statistics, choice reaction times, and human information processing. Journal of Mathematical Psychology 2 (1965) 312-329
12. Efron B., and Tibshirani R.J. An introduction to the bootstrap (1993), Chapman and Hall, New York
13. Estes W.K., and Maddox W.T. Risks of drawing inferences about cognitive processes from model fits to individual versus average performance. Psychonomic Bulletin and Review 12 (2005) 403-408
14. Feller W. An introduction to probability theory and its applications. 3rd ed. (1968), Wiley, New York
15. Fitts P. Cognitive aspects of information processing: III. Set for speed versus accuracy. Journal of Experimental Psychology 71 (1966) 849-857
16. Freedman D., and Diaconis P. On the maximum deviation between the histogram and the underlying density. Zeitschrift fur Wahrscheinlichkeitstheorie und Verwandte Gebiete 58 (1981) 139-167
17. Gardiner C.W. Handbook of stochastic methods. 3rd ed. (2004), Springer-Verlag, New York
18. Garrett H. A study of the relation of accuracy to speed. Archives of Psychology 56 (1922) 1-105
19. Gill P.E., Murray W., Saunders M.A., and Wright M.H. User's guide for NPSOL 5.0: A FORTRAN package for nonlinear programming (Tech. Rep. SOL 86-1.) (1998), Stanford University, Systems Optimization Laboratory, Stanford, CA
20. Gold J.I., and Shadlen M.N. Neural computations that underlie decisions about sensory stimuli. Trends in Cognitive Science 5 (2001) 10-16
21. Gold J.I., and Shadlen M.N. Banburismus and the brain: Decoding the relationship between sensory stimuli, decisions, and reward. Neuron 36 (2002) 299-308
22. Green D.M., and Swets J.A. Signal detection theory and psychophysics (1966), Wiley, New York
23. Grice G.R. Stimulus intensity and response evocation. Psychological Review 75 (1968) 359-373
24. Hanes D.P., and Schall J.D. Neural control of voluntary movement initiation. Science 274 (1996) 427-430
25. Herrnstein R.J. In: Rachlin H., and Laibson D.I. (Eds). The matching law: Papers in psychology and economics (1997), Harvard University Press, Cambridge, MA
26. Jentzsch I., and Dudschig C. Why do we slow down after an error? Mechanisms underlying the effects of posterror slowing. The Quarterly Journal of Experimental Psychology 62 (2009) 209-218
27. LaBerge D. A recruitment theory of simple behavior. Psychometrika 27 (1962) 375-396
28. Lagarias J.C., Reeds J.A., Wright M.H., and Wright P.E. Convergence properties of the Nelder-Mead Simplex method in low dimensions. SIAM Journal on Optimization 9 (1998) 112-147
29. Laming D.R.J. Information theory of choice reaction time (1968), Wiley, New York
30. Link S.W. The relative judgment theory of two choice response time. Journal of Mathematical Psychology 12 (1975) 114-135
31. Link S.W., and Heath R.A. A sequential theory of psychological discrimination. Psychometrika 40 (1975) 77-105
32. Luce R.D. Response times: Their role in inferring elementary mental organization (1986), Oxford University Press, New York
33. Maddox W.T., and Bohil C.J. Base rate and payoff effects in multidimensional perceptual categorization. Journal of Experimental Psychology: Learning, Memory and Cognition 24 (1998) 1459-1482
34. McKinnon K. Convergence of the Nelder-Mead Simplex method to a nonstationary point. SIAM Journal on Optimization 9 (1998) 148-158
35. Myung I.J., and Busemeyer J.R. Criterion learning in a deferred decision making task. American Journal of Psychology 102 (1989) 1-16
36. Nelder J., and Mead R. A simplex method for function minimization. Computer Journal 7 (1965) 308-313
37. Ollman R.T. Fast guesses in choice-reaction time. Psychonomic Science 6 (1966) 155-156
38. Pachella R., and Pew R. Speed-accuracy tradeoff in reaction time: Effect of discrete criterion times. Journal of Experimental Psychology 76 (1968) 19-24
39. Pacut A. Some properties of threshold models of reaction latency. Biological Cybernetics 28 (1977) 63-72
40. Palmer J., Huk A.C., and Shadlen M.N. The effect of stimulus strength on the speed and accuracy of a perceptual decision. Vision Research 5 (2005) 376-404
41. Pelli D.G. The videotoolbox software for visual psychophysics: Transforming numbers into movies. Spatial Vision 10 (1997) 437-442
42. Rabbitt P. Psychological refractory delay and response-stimulus interval in serial, choice-response tasks. In: Koster W. (Ed). Attention and performance II (1969), North-Holland, Amsterdam 195-219
43. Rabbitt P., and Vyas S. An elementary preliminary taxonomy of errors in choice reaction time tasks. Acta Psychologica 33 (1970) 56-76
44. Rapoport A., and Burkheimer G.J. Models for deferred decision making. Journal of Mathematical Psychology 8 (1971) 508-538
45. Ratcliff R. A theory of memory retrieval. Psychological Review 85 (1978) 59-108
46. Ratcliff R. Group reaction time distributions and an analysis of distribution statistics. Psychological Bulletin 86 (1979) 446-461
47. Ratcliff R. Theoretical interpretations of the speed and accuracy of positive and negative responses. Psychological Review 92 (1985) 212-225
48. Ratcliff R. Diffusion and random walk processes. In: Smelser N.J., and Baltes P.B. (Eds). International encyclopedia of the social and behavioral sciences 6 (2001), Elsevier, Oxford 3668-3673
49. Ratcliff R., Cherian A., and Segraves M.A. A comparison of macaque behavior and superior colliculus neuronal activity to predictions from models of two choice decisions. Journal of Neurophysiology 90 (2003) 1392-1407
50. Ratcliff R., and McKoon G. The diffusion decision model: Theory and data for two-choice decision tasks. Neural Computation 20 (2008) 873-922
51. Ratcliff R., and Rouder J.N. Modeling response times for two-choice decisions. Psychological Science 9 (1998) 347-356
52. Ratcliff R., and Rouder J.N. A diffusion model account of masking in two-choice letter identification. Journal of Experimental Psychology: Human Perception and Performance 26 (2000) 127-140
53. Ratcliff R., and Smith P.L. A comparison of sequential sampling models for two-choice reaction time. Psychological Review 111 (2004) 333-367
54. Ratcliff R., Thapar A., Gomez P., and McKoon G. A diffusion model analysis of the effects of aging in the lexical-decision task. Psychology and Aging 19 (2004) 278-289
55. Ratcliff R., and Tuerlinckx F. Estimating parameters of the diffusion model: Approaches to dealing with contaminant reaction times and parameter variability. Psychonomic Bulletin and Review 9 (2002) 438-481
56. Ratcliff R., Van Zandt T., and McKoon G. Connectionist and diffusion models of reaction time. Psychological Review 106 (1999) 261-300
57. Roitman J.D., and Shadlen M.N. Response of neurons in the lateral intraparietal area during a combined visual discrimination reaction time task. Journal of Neuroscience 22 (2002) 9475-9489
58. Rouder J.N., and Speckman P.L. An evaluation of the vincentizing method of forming group-level response time distributions. Psychonomic Bulletin and Review 11 (2004) 419-427
59. Schall J.D. Neural basis of deciding, choosing and acting. Nature Reviews Neuroscience 2 (2001) 33-42
60. Schouten J., and Bekker A. Reaction time and accuracy. Acta Psychologica 27 (1967) 143-153
61. Shadlen M.N., Britten K.H., Newsome W.T., and Movshon J.A. A computational analysis of the relationship between neuronal and behavioral responses to visual motion. Journal of Neuroscience 16 (1996) 1486-1510
62. Shadlen M.N., and Newsome W.T. Neural basis of a perceptual decision in the parietal cortex (area LIP) of the rhesus monkey. Journal of Neurophysiology 86 (2001) 1916-1936
63. Simen P., Cohen J.D., and Holmes P. Rapid decision threshold modulation by reward rate in a neural network. Neural Networks 19 (2006) 1013-1026
64. Smith P.L., and Ratcliff R. Psychology and neurobiology of simple decisions. Trends in Neuroscience 27 (2004) 161-168
65. Smith P.L., and Vickers D. Modeling evidence accumulation with partial loss in expanded judgment. Journal of Experimental Psychology: Human Perception and Performance 15 (1989) 797-815
66. Sommer W., Leuthold H., and Soetens E. Covert signs of expectancy in serial reaction time tasks revealed by event-related potentials. Perception and Psychophysics 61 (1999) 342-352
67. Spieler D.H., Balota D.A., and Faust M.E. Stroop performance in healthy younger and older adults and in individuals with dementia of the Alzheimer's type. Journal of Experimental Psychology: Human Perception and Performance 22 (1996) 461-479
68. Sternberg S. Separate modifiability, mental modules, and the use of pure and composite measures to reveal them. Acta Psychologica 106 (2001) 147-246
69. Stone M. Models for choice reaction time. Psychometrika 25 (1960) 251-260
70. Tanner W., and Swets J. A decision-making theory of visual detection. Psychological Review 61 (1954) 401-409
71. Tuerlinckx F. The efficient computation of the cumulative distribution and probability density functions in the diffusion model. Behavioral Research Methods, Instruments and Computers 36 (2004) 702-716
72. Tuerlinckx F., Maris E., Ratcliff R., and De Boeck P. A comparison of four methods for simulating the diffusion process. Behavioral Research Methods, Instruments and Computers 33 (2001) 443-456
73. Usher M., and McClelland J.L. The time course of perceptual choice: The leaky, competing accumulator model. Psychological Review 108 (2001) 550-592
74. Vandekerckhove J., and Tuerlinckx F. The diffusion model analysis toolbox [computer software and manual] (2007). http://ppw.kuleuven.be/okp/dmatoolbox/ Retrieved July 21, 2009, from http://ppw.kuleuven.be/okp/dmatoolbox/
75. Vandekerckhove J., and Tuerlinckx F. Fitting the Ratcliff diffusion model to experimental data. Psychonomic Bulletin and Review 14 (2007) 1011-1026
76. Van Zandt T. How to fit a response time distribution. Psychonomic Bulletin and Review 7 (2000) 424-465
77. Vickers D. Evidence for an accumulator model of psychophysical discrimination. Ergonomics 13 (1970) 37-58
78. Voss A., Rothermund K., and Voss J. Interpreting the parameters of the diffusion model: An empirical validation. Memory and Cognition 32 (2004) 1206-1220
79. Wald A. Sequential tests of statistical hypotheses. The Annals of Mathematical Statistics 16 (1945) 117-186
80. Wald A., and Wolfowitz J. Optimum character of the sequential probability ratio test. Annals of Mathematical Statistics 19 (1948) 326-339
81. Wickelgren W. Speed-accuracy tradeoff and information processing dynamics. Acta Psychologica 41 (1977) 67-85
82. Yang, T., Hanks, T., Mazurek, M. E., McKinley, M., Palmer, J., & Shadlen, M. N. (2005). Incorporating prior probability into decision-making in the face of uncertain reliability of evidence. Presented at the annual meeting of the Society for Neuroscience, Washington, DC.
83. Yellott J.I. Correction for guessing and the speed-accuracy tradeoff in choice reaction time. Journal of Mathematical Psychology 8 (1971) 159-199
84. Zacksenhouse, M., Holmes, P., & Bogacz, R. (2007). Robust versus optimal strategies for two-alternative forced choice tasks. Manuscript submitted for publication.