A Multivariate multilevel approach to the modeling of accuracy and speed of test takers

Psychometrika

Volumn 74, Issue 1, 2009, Pages 21-48

  Klein Entink, R H ^a   Fox, J P ^a   Van Der Linden, W J ^a  

^a UNIVERSITY OF TWENTE   (Netherlands)

Author keywords

Accuracy; IRT; Response times; Speed

Indexed keywords

EID: 61449153050     PISSN: 00333123     EISSN: None     Source Type: Journal    
DOI: 10.1007/s11336-008-9075-y     Document Type: Article

References (61)

1. Akaike, H. (1973). Information theory and an extension of the maximum likelihood principle. In B.N. Petrov & F. Csaki (Eds.), 2nd international symposium on information theory (pp. 267-281). Budapest: Akademiai Kiado.
2. J.H. Albert 1992 Bayesian estimation of normal ogive item response curves using Gibbs sampling Journal of Educational Statistics 17 251 269
3. J. Barnard R. McCullogh X.-L. Meng 2000 Modelling covariance matrices in terms of standard deviations and correlations, with application to shrinkage Statistica Sinica 10 1281 1311
4. P. Becker 1999 Beyond the big five Personality and Individual Differences 26 511 30
5. A.A. Beguin C.A.W. Glas 2001 MCMC estimation and some model-fit analysis of multidimensional IRT models Psychometrika 66 541 562
6. J.O. Berger M. Delampady 1987 Testing precise hypotheses Statistical Science 2 317 352
7. Boscardin, W.J., & Zhang, X. (2004). Modeling the covariance and correlation matrix of repeated measures. In A. Gelman & X.-L. Meng (Eds.), Applied Bayesian modeling and causal inference from incomplete-data perspectives (pp. 215-226). New York: Wiley.
8. E.T. Bradlow H. Wainer X. Wang 1999 A Bayesian random effects model for testlets Psychometrika 64 153 168
9. B. Bridgeman F. Cline 2004 Effects of differentially time-consuming tests on computer-adaptive test scores Journal of Educational Measurement 41 137 148
10. W.J. Browne 2006 MCMC algorithms for constrained variance matrices Computational Statistics & Data Analysis 50 1655 1677
11. S. Chib E. Greenberg 1998 Analysis of multivariate probit models Biometrika 85 347 361
12. M. DeIorio C.P. Robert 2002 Discussion of Spiegelhalter et al Journal of the Royal Statistical Society, Series B 64 629 630
13. J.M. Dickey 1971 The weighted likelihood ratio, linear hypothesis on normal location parameters The Annals of Mathematical Statistics 42 204 223
14. J.-P. Fox 2005 Multilevel IRT using dichotomous and polytomous items British Journal of Mathematical and Statistical Psychology 58 145 172
15. J.-P. Fox C.A.W. Glas 2001 Bayesian estimation of a multilevel IRT model using Gibbs sampling Psychometrika 66 269 286
16. A.E. Gelfand A.F.M. Smith 1990 Sampling-based approaches to calculating marginal densities Journal of the American Statistical Association 85 398 409
17. Gelman, A., Carlin, J.B., Stern, H.S., & Rubin, D.B. (2004). Bayesian data analysis (2nd ed.). New York: Chapman & Hall/CRC.
18. S. Geman D. Geman 1984 Stochastic relaxation, Gibbs distributions, and the Bayesian restoration of images IEEE Transactions on Pattern Analysis and Machine Intelligence 6 721 741
19. C.A.W. Glas W.J. van der Linden 2003 Computerized adaptive testing with item cloning Applied Psychological Measurement 23 249 263
20. Goldstein, H. (2003). Multilevel statistical models (3rd ed.). London: Arnold.
21. D.A. Harville 1977 Maximum likelihood approaches to variance component estimation and related problems Journal of the American Statistical Association 72 320 338
22. Johnson, V.E., & Albert, J.H. (1999). Ordinal data modeling.
23. R.E. Kass A.E. Raftery 1995 Bayes factors Journal of the American Statistical Association 90 773 795
24. M. Kennedy 1930 Speed as a personality trait Journal of Social Psychology 1 286 298
25. N.M. Laird J.H. Ware 1982 Random-effects models for longitudinal data Biometrics 38 963 974
26. M. Lavine M.J. Schervish 1999 Bayes factors: What they are and what they are not The American Statistician 53 119 122
27. Lee, P.M. (2004). Bayesian statistics, an introduction (3rd ed.). New York: Arnold.
28. Luce, D.R. (1986). Response times: Their role in inferring elementary mental organization. New York: Oxford University Press.
29. R.R. McCrea P.T. Costa 1997 Personality trait structure as a human universal American Psychologist 52 509 516
30. R.E. McCulloch P.E. Rossi 1994 An exact likelihood analysis of the multinomial probit model Journal of Econometrics 64 207 240
31. R.E. McCulloch N.G. Polson P.E. Rossi 2000 A Bayesian analysis of the multinomial probit model with fully identified parameters Journal of Econometrics 99 173 193
32. G.J. Mellenbergh 1994 A unidimensional latent trait model for continuous item responses Multivariate Behavioral Research 29 223 236
33. M.A. Newton A.E. Raftery 1994 Approximate Bayesian inference with the weighted likelhood bootstrap Journal of the Royal Statistical Society Series B 56 3 58
34. R.J. Patz B.W. Junker 1999 A straightforward approach to Markov chain Monte Carlo methods for item response models Journal of Educational and Behavioral Statistics 24 146 178
35. S. Rabe-Hesketh A. Skrondal 2001 Parameterization of multivariate random effects models for categorical data Biometrics 57 1256 1264
36. G. Reinsel 1983 Some results on multivariate autoregressive index models Biometrika 70 145 156
37. J. Rost 1990 Rasch models in latent classes: An integration of two approaches to item analysis Applied Psychological Measurement 14 271 282
38. J.L. Schafer R.C. Yucel 2002 Computational strategies for multivariate linear mixed-effects models with missing values Journal of Computational and Graphical Statistics 11 437 457
39. D.L. Schnipke D.J. Scrams 1997 Modeling item response times with a two-state mixture model: A new method for measuring speededness Journal of Educational Measurement 34 213 232
40. Schnipke, D.L., & Scrams, D.J. (2002). Exploring issues of examinee behavior: Insights gained from response-time analyses. In C. Mills, M.P.J. Fremer, & W. Ward (Eds.), Computer-based testing: Building the foundation for future assessments (pp. 237-266). Mahwah: Lawrence Erlbaum Associates.
41. G. Schwarz 1978 Estimating the dimension of a model The Annals of Statistics 6 461 464
42. Searle, S.R., Casella, G., & McCulloch, C.E. (1992). Variance components. New York: Wiley.
43. A. Shah N. Laird D. Schoenfeld 1997 Random-effects model for multiple characteristics with possibly missing data Journal of the American Statistical Association 92 775 779
44. J.Q. Shi S.Y. Lee 1998 Bayesian sampling based approach for factor analysis models with continuous and polytomous data British Journal of Mathematical and Statistical Psychology 51 233 252
45. S. Sinharay 2005 Assessing fit of unidimensional item response theory models using a Bayesian approach Journal of Educational Measurement 42 375 394
46. S. Sinharay H.S. Stern 2002 On the sensitivity of Bayes factors to the prior distributions The American Statistician 56 196 201
47. S. Sinharay M.S. Johnson H.S. Stern 2006 Posterior predictive assessment of item response theory models Applied Psychological Measurement 30 298 321
48. Snijders, T.A.B., & Bosker, R.J. (1999). Multilevel analysis, an introduction to basic and advanced multilevel modeling. London: Sage Publishers.
49. D.J. Spiegelhalter N.G. Best B.P. Carlin A. van der Linde 2002 Bayesian measures of model complexity and fit Journal of the Royal Statistical Society Series B 64 583 639
50. M.W. Tate 1948 Individual differences in speed of response in mental test materials of varying degrees of difficulty Educational and Psychological Measurement 8 353 374
51. Thissen, D. (1983). Timed testing: An approach using item response theory. In D. Weiss (Ed.), New horizons in testing: Latent trait test theory and computerized adaptive testing (pp. 179-203). New York: Academic Press.
52. F. Vaida S. Blanchard 2005 Conditional Akaike information for mixed-effects models Biometrika 92 351 370
53. W.J. van der Linden 2006 A lognormal model for response times on test items Journal of Educational and Behavioural Statistics 31 181 204
54. W.J. van der Linden 2007 Using response times for item selection in adaptive testing Journal of Educational and Behavioral Statistics 10.3102/1076998607302626
55. W.J. van der Linden 2008 A hierarchical framework for modeling speed and accuracy on test items Psychometrika 72 287 308
56. W.J. van der Linden F. Guo 2008 Bayesian procedures for identifying aberrant response-time patterns in adaptive testing Psychometrika 10.1007/S11336-007-9046-8
57. W.J. van der Linden K. Breithaupt S.C. Chuah Y. Zang 2007 Detecting differential speededness in multistage testing Journal of Educational Measurement 44 117 130
58. I. Verdinelli L. Wasserman 1995 Computing Bayes factors using a generalization of the Savage-Dickey density ratio Journal of the American Statistical Association 90 614 618
59. Verhelst, N., Verstralen, H., & Jansen, M. (1997). Models for time-limit tests. In W.J. van der Linden & R.K. Hambleton (Eds.), Handbook of modern item response theory (pp. 169-185). New York: Springer.
60. S.L. Wise X.J. Kong 2005 Response time effort: A new measure of examinee motivation in computer-based tests Applied Measurement in Education 18 163 183
61. Wise, S.L., Kong, X.J., & Pastor, D.A. (2007). Understanding correlates of rapid-guessing behavior in low-stakes testing: Implications for test development and measurement practice. Paper presented at the 2007 anual meeting of the National Council on Measurement in Education, Chicago, IL.