Comparing time-accuracy curves: Beyond goodness-of-fit measures

Psychonomic Bulletin and Review

Volumn 16, Issue 1, 2009, Pages 190-203

  Liu, Charles C ^a,b   Smith, PhiliP L ^a  

^a UNIVERSITY OF MELBOURNE   (Australia)

^b MONASH UNIVERSITY   (Australia)

Author keywords

[No Author keywords available]

Indexed keywords

ARTICLE; ASSOCIATION; ATTENTION; BAYES THEOREM; EYE FIXATION; HUMAN; ORIENTATION; PATTERN RECOGNITION; PERCEPTIVE DISCRIMINATION; PSYCHOMOTOR PERFORMANCE; PSYCHOPHYSICS; REACTION TIME; STATISTICAL MODEL;

ATTENTION; BAYES THEOREM; CUES; DISCRIMINATION (PSYCHOLOGY); FIXATION, OCULAR; HUMANS; MODELS, STATISTICAL; ORIENTATION; PATTERN RECOGNITION, VISUAL; PSYCHOMOTOR PERFORMANCE; PSYCHOPHYSICS; REACTION TIME;

EID: 62649174295     PISSN: 10699384     EISSN: 15315320     Source Type: Journal    
DOI: 10.3758/PBR.16.1.190     Document Type: Article

References (84)

1. Akaike, H. (1973). Information theory and an extension of the maxi mum likelihood principle. In B. N. Petrov & F. Caski (Eds.), Proceed ings of the Second International Symposium on Information Theory (pp. 267-281). Budapest: Akademiai Kiado.
2. Anderson-Sprecher, R. (1994). Model comparisons and R2. American Statistician, 48, 113-117.
3. Batchelder, W. H., & Riefer, D. M. (1990). Multinomial processing models of source monitoring. Psychological Review, 97, 548-564.
4. Bozdogan, H. (2000). Akaike's information criterion and recent devel opments in information complexity. Journal of Mathematical Psychology, 44, 62-91.
5. Brown, S., & Heathcote, A. (2003). Averaging learning curves across and within participants. Behavior Research Methods, Instruments, & Computers, 35, 11-21.
6. Burnham, K. P., & Anderson, D. R. (2002). Model selection and mul timodel inference: A practical information-theoretic approach. New York: Springer.
7. Busemeyer, J. R., & Wang, Y.-M. (2000). Model comparisons and model selections based on generalization criterion methodology. Jour nal of Mathematical Psychology, 44, 171-189.
8. Carrasco, M., Giordano, A. M., & McElree, B. (2004). Temporal performance fields: Visual and attentional factors. Vision Research, 44, 1351-1365.
9. Carrasco, M., Giordano, A. M., & McElree, B. (2006). Attention speeds processing across eccentricity: Feature and conjunctive searches. Vision Research, 46, 2028-2040.
10. Carrasco, M., & McElree, B. (2001). Covert attention accelerates the rate of visual information processing. Proceedings of the National Academy of Sciences, 98, 5363-5367.
11. Chechile, R. A. (1977). Likelihood and posterior identification: Impli cations for mathematical psychology. British Journal of Mathematical & Statistical Psychology, 30, 177-184.
12. Davison, A. C., & Hinkley, D. V. (1997). Bootstrap methods and their applications. Cambridge: Cambridge University Press.
13. Dosher, B. A. (1979). Empirical approaches to information processing: Speed-accuracy trade-off functions or reaction time. Acta Psycho logica, 43, 347-359.
14. Dosher, B. A. (1981). The effect of delay and interference: A speed accuracy study. Cognitive Psychology, 13, 551-582.
15. Dosher, B. A. (1982). Sentence size, network distance and sentence retrieval. Journal of Experimental Psychology: Learning, Memory, & Cognition, 8, 173-207.
16. Dosher, B. A. (1984). Degree of learning and retrieval speed: Study time and multiple exposures. Journal of Experimental Psychology: Learning, Memory, & Cognition, 10, 541-574.
17. Dosher, B. A., Han, S. M., & Lu, Z. L. (2004). Parallel processing in vi sual search asymmetry. Journal of Experimental Psychology: Human Perception & Performance, 30, 3-27.
18. Efron, B., & Tibshirani, R. J. (1993). An introduction to the bootstrap. New York: Chapman & Hall.
19. Estes, W. K., & Maddox, W. T. (2005). Risks of drawing inferences about cognitive processes from model fits to individual versus average performance. Psychonomic Bulletin & Review, 12, 403-408.
20. Gill, J. (2002). Bayesian methods: A social and behavioral sciences approach. Boca Raton, FL: Chapman & Hall/CRC.
21. Glover, S., & Dixon, P. (2004). Likelihood ratios: A simple and flex ible statistic for empirical psychologists. Psychonomic Bulletin & Review, 11, 791-806.
22. Gourevitch, V., & Galanter, E. (1967). A significance test for one parameter isosensitivity functions. Psychometrika, 32, 25-33.
23. Gronlund, S. D., & Ratcliff, R. (1989). Time course of item and asso ciative information: Implications for global memory models. Journal of Experimental Psychology: Learning, Memory, & Cognition, 15, 846-858.
24. Heathcote, A., Brown, S. D., & Mewhort, D. J. K. (2000). The power law repealed: The case for an exponential law of practice. Psy chonomic Bulletin & Review, 7, 185-207.
25. Huber, D. E. (2006). Computer simulations of the ROUSE model: An analytic simulation technique and a comparison between the error variance-covariance and bootstrap methods for estimating parameter confidence. Behavior Research Methods, 38, 557-568.
26. Kass, R. E. (1993). Bayes factors in practice. Journal of the Royal Sta tistical Society, 42, 551-560.
27. Kass, R. E., & Raftery, A. E. (1995). Bayes factors. Journal of the American Statistical Association, 90, 773-795.
28. Kuha, J. (2004). AIC and BIC: Comparisons of assumptions and perfor mance. Sociological Methods & Research, 33, 188-229.
29. Kuss, M., Jäkel, F., & Wichmann, F. A. (2005). Bayesian inference for psychometric functions. Journal of Vision, 5, 478-492.
30. Lee, M. D. (2004). A Bayesian analysis of retention functions. Journal of Mathematical Psychology, 48, 310-321.
31. Lee, M. D. (2008). Three case studies in the Bayesian analysis of cogni tive models. Psychonomic Bulletin & Review, 15, 1-15.
32. Lee, M. D., & Wagenmakers, E.-J. (2005). Bayesian statistical infer ence in psychology: Comment on Trafimow (2003). Psychological Review, 112, 662-668.
33. Lee, M. D., & Webb, M. R. (2005). Modeling individual differences in cognition. Psychonomic Bulletin & Review, 12, 605-621.
34. Link, W. A. (1992). The wave theory of difference and similarity. Hills dale, NJ: Erlbaum.
35. Link, W. A., & Barker, R. J. (2006). Model weights and the foundations of multimodel inference. Ecology, 87, 2626-2635.
36. Liu, C. C., & Aitkin, M. (2008). Bayes factors: Prior sensitivity and model generalizability. Journal of Mathematical Psychology, 52, 362-375.
37. Lunn, D. J., Thomas, A., Best, N., & Spiegelhalter, D. (2000). WinBUGS-a Bayesian modeling framework: Concepts, structure, and extensibility. Statistics & Computing, 10, 325-337.
38. McElree, B. (1996). Accessing short-term memory with semantic and phonological information: A time-course analysis. Memory & Cogni tion, 24, 173-187.
39. McElree, B. (1998). Attended and nonattended states in working mem ory: Accessing categorized structures. Journal of Memory & Lan guage, 38, 225-252.
40. McElree, B. (2001). Working memory and focal attention. Journal of Experimental Psychology: Learning, Memory, & Cognition, 27, 817-835.
41. McElree, B., & Dosher, B. A. (1989). Serial position and set size in short-term memory: The time course of recognition. Journal of Experimental Psychology: General, 118, 346-373.
42. McElree, B., & Dosher, B. A. (1993). Serial retrieval processes in the recovery of order information. Journal of Experimental Psychology: General, 122, 291-315.
43. McElree, B., & Griffith, T. (1995). Syntactic and thematic process ing in sentence comprehension: Evidence for a temporal dissociation. Journal of Experimental Psychology: Learning, Memory, & Cognition, 21, 134-157.
44. Miller, J. (1996). The sampling distribution of d'. Perception & Psy chophysics, 58, 65-72.
45. Mulligan, N., & Hirshman, E. (1995). Speed-accuracy trade-offs and the dual process model of recognition memory. Journal of Memory & Language, 34, 1-18.
46. Myung, I. J. (2000). The importance of complexity in model selection. Journal of Mathematical Psychology, 44, 190-204.
47. Myung, I. J. (2003). Tutorial on maximum likelihood estimation. Jour nal of Mathematical Psychology, 47, 90-100.
48. Myung, I. J., Kim, C., & Pitt, M. A. (2000). Toward an explanation of the power law artifact: Insights from response surface analysis. Memory & Cognition, 28, 832-840.
49. Myung, I. J., & Pitt, M. A. (1997). Applying Occam's razor in model ing cognition: A Bayesian approach. Psychonomic Bulletin & Review, 4, 79-95.
50. Nikolic, D., & Gronlund, S. D. (2002). A tandem random walk model of the SAT paradigm: Response times and accumulation of evidence. British Journal of Mathematical & Statistical Psychology, 55, 263-288.
51. Öztekin, I., & McElree, B. (2007). Proactive interference slows rec ognition by eliminating fast assessments of familiarity. Journal of Memory & Language, 57, 126-149.
52. Penny, W. D., Stephan, K. E., Mechelli, A., & Friston, K. J. (2004). Comparing dynamic causal models. NeuroImage, 22, 1157-1172.
53. Pitt, M. A., & Myung, I. J. (2002). When a good fit can be bad. Trends in Cognitive Sciences, 6, 421-425.
54. Pitt, M. A., Myung, I. J., & Zhang, S. (2002). Toward a method of selecting among computational models of cognition. Psychological Review, 109, 472-491.
55. Raftery, A. E. (1995). Bayesian model selection in social research. Sociological Methodology, 25, 111-163.
56. Ratcliff, R. (1978). A theory of memory retrieval. Psychological Review, 85, 59-108.
57. Ratcliff, R. (2006). Modeling response signal and response time data. Cognitive Psychology, 53, 195-237.
58. Ratcliff, R., & Smith, P. L. (2004). A comparison of sequential sam pling models for two-choice reaction time. Psychological Review, 111, 333-367.
59. Reed, A. V. (1973). Speed-accuracy trade-off in recognition memory. Science, 181, 574-576.
60. Reed, A. V. (1976). List length and the time course of recognition in human memory. Memory & Cognition, 4, 16-30.
61. Rouder, J. N., & Lu, J. (2005). An introduction to Bayesian hierar chical models with an application in the theory of signal detection. Psychonomic Bulletin & Review, 12, 573-604.
62. Rouder, J. N., Lu, J., Speckman, P., Sun, D., & Jiang, Y. (2005). A hierarchical model for estimating response time distributions. Psy chonomic Bulletin & Review, 12, 195-223.
63. Rubin, D. B., & Wenzel, A. (1996). One hundred years of forgetting: A quantitative description of retention. Psychological Review, 103, 734-760.
64. Schwarz, G. (1978). Estimating the dimension of a model. Annals of Statistics, 6, 461-464.
65. Smith, B. J. (2005). Bayesian Output Analysis Program (BOA), Version 1.1.5 [Online]. Iowa City: University of Iowa. Available at www.public-health.uiowa.edu/boa.
66. Smith, P. L. (2000). Stochastic dynamic models of response time and accuracy: A foundational primer. Journal of Mathematical Psychol ogy, 44, 408-463.
67. Smith, P. L., Ratcliff, R., & Wolfgang, B. J. (2004). Attention ori enting and the time course of perceptual decisions: Response time distributions with masked and unmasked displays. Vision Research, 44, 1297-1320.
68. Smith, P. L., & Wolfgang, B. J. (2004). The attentional dynamics of masked detection. Journal of Experimental Psychology: Human Perception & Performance, 30, 119-136.
69. Smith, P. L., Wolfgang, B. J., & Sinclair, A. J. (2004). Mask dependent attentional cuing effects in visual signal detection: The psychometric function for contrast. Perception & Psychophysics, 66, 1056-1075.
70. Snodgrass, J. G., & Corwin, J. (1988). Pragmatics of measuring rec ognition memory: Applications to dementia and amnesia. Journal of Experimental Psychology: General, 117, 34-50.
71. Stephan, K. E., & Penny, W. D. (2007). Dynamic causal models and Bayesian selection. In K. Friston, J. Ashburner, S. Kiebel, T. Nichols, & W. Penny (Eds.), Statistical parametric mapping: The analysis of functional brain images (pp. 577-585). Amsterdam: Elsevier.
72. Thomas, R. D. (2006). Processing time predictions of current models of perception in the classic additive factors paradigm. Journal of Math ematical Psychology, 50, 441-455.
73. Usher, M., & McClelland, J. L. (2001). The time course of percep tual choice: The leaky, competing accumulator model. Psychological Review, 108, 550-592.
74. Vandekerckhove, J., & Tuerlinckx, F. (2008). Diffusion model anal ysis with MATLAB: A DMAT primer. Behavior Research Methods, 40, 61-72.
75. Verguts, T., & Storms, G. (2004). Assessing the informational value of parameter estimates in cognitive models. Behavior Research Methods, Instruments, & Computers, 36, 1-10.
76. Voss, A., & Voss, J. (2007). Fast-dm: A free program for efficient diffu sion model analysis. Behavior Research Methods, 39, 767-775.
77. Wagenmakers, E.-J. (2007). A practical solution to the pervasive prob lems of p values. Psychonomic Bulletin & Review, 14, 779-804.
78. Wagenmakers, E.-J., & Farrell, S. (2004). AIC model selection using Akaike weights. Psychonomic Bulletin & Review, 11, 192-196.
79. Wagenmakers, E.-J., Ratcliff, R., Gomez, P., & Iverson, G. J. (2004). Assessing model mimicry using the parametric bootstrap. Journal of Mathematical Psychology, 48, 28-50.
80. Wasserman, L. (2000). Bayesian model selection and model averaging. Journal of Mathematical Psychology, 44, 92-107.
81. Weakliem, D. L. (1999). A critique of the Bayesian information cri terion for model selection. Sociological Methods & Research, 27, 359-397.
82. Wichmann, F. A., & Hill, N. J. (2001). The psychometric function: II. Bootstrap-based confidence intervals and sampling. Perception & Psychophysics, 63, 1314-1329.
83. Wickelgren, W. A. (1977). Speed-accuracy trade-off and information processing dynamics. Acta Psychologica, 41, 67-85.
84. Wickelgren, W. A., & Corbett, A. T. (1977). Associate interference and retrieval dynamics in yes-no recall and recognition. Journal of Experimental Psychology: Human Learning & Memory, 3, 189-202.