Posterior Predictive Model Checking for Multidimensionality in Item Response Theory

Applied Psychological Measurement

Volumn 33, Issue 7, 2009, Pages 519-537

  Levy, Roy ^a   Mislevy, Robert J ^b   Sinharay, Sandip ^c  

^a ARIZONA STATE UNIVERSITY   (United States)

^b UNIVERSITY OF MARYLAND   (United States)

^c EDUCATIONAL TESTING SERVICE   (United States)

Author keywords

Conditional covariance theory; Item response theory; Local independence; Multidimensionality; Posterior predictive model checking

Indexed keywords

EID: 70350053095     PISSN: 01466216     EISSN: 15523497     Source Type: Journal    
DOI: 10.1177/0146621608329504     Document Type: Article

References (30)

1. Agresti, A. (2002). Categorical data analysis (2nd ed.). New York: Wiley.
2. Berkhof, J., van Mechelen, I., & Gelman, A. (2004). Enhancing the performance of a posterior predictive check (IAP Statistics Network Tech. Rep. 0350). Louvain-la-Neuve, Belgium: Interuniversity Attraction Pole, Katholieke Universiteit Leuven.
3. Bolt, D., & Stout, W. (1996). Differential item functioning: Its multidimensional model and resulting SIBT-EST detection procedure. Behaviormetrika, 23, 67-95.
4. Bradlow, E.T., Wainer, H., & Wang, X. (1999). A Bayesian random effects model for testlets. Psychometrika, 64, 153-168.
5. Chen, W., & Thissen, D. (1997). Local dependence indexes for item pairs using item response theory. Journal of Educational and Behavioral Statistics, 22, 265-289.
6. Donoghue, J.R., & Hombo, C.M. (2003, April). A corrected asymptotic distribution of an IRT fit measure that accounts for the effects of item parameter estimation. Paper presented at the annual meeting of the National Council on Measurement in Education, Chicago.
7. Fu, J., Bolt, D.M., & Li, Y. (2005, April). Evaluating item fit for a polytomous fusion model using posterior predictive checks. Paper presented at the annual meeting of the National Council on Measurement in Education, Montreal, Quebec, Canada.
8. Gelman, A., Meng, X.L., & Stern, H. (1996). Posterior predictive assessment of model fitness via realized discrepancies. Statistica Sinica, 6, 733-807.
9. Habing, B., & Roussos, L.A. (2003). On the need for local item dependence. Psychometrika, 68, 435-451.
10. Hoijtink, H. (2001). Conditional independence and differential item functioning in the two-parameter logistic model. In A. Boomsma, M. A. J. van Duijn, & T. A. B. Snijders (Eds.), Essays in item response theory (pp. 109-130). New York: Springer-Verlag.
11. Janssen, R., Tuerlinckx, F., Meulders, M., & De Boeck, P. (2000). A hierarchical IRT model for criterion-referenced measurement. Journal of Educational and Behavioral Statistics, 25, 285-306.
12. Levy, R., Mislevy, R.J., & Sinharay, S. (2006, April). Posterior predictive model checking for multidimensionality in item response theory. Paper presented at the annual meeting of the National Council on Measurement in Education, San Francisco.
13. McDonald, R.P., & Mok, M.M.C. (1995). Goodness of fit in item response models. Multivariate Behavioral Research, 30, 23-40.
14. Meng, X.L. (1994). Posterior predictive p-values. Annals of Statistics, 22, 1142-1160.
15. Murdoch, D.J., & Chow, E.D. (1996). A graphical display of large correlation matrices. American Statistician, 50, 178-180.
16. Patz, R.J., & Junker B.W. (1999). A straightforward approach to Markov chain Monte Carlo methods for item response models. Journal of Educational and Behavioral Statistics, 24, 146-178.
17. Reckase, M.D. (1997). A linear logistic multidimensional model. In W. J. van der Linden & R. K. Hambleton (Eds.), Handbook of modern item response theory (pp. 271-286). New York: Springer-Verlag.
18. Reckase, M.D., & McKinley, R.L. (1991). The discriminating power of test items that measure more than one ability. Applied Psychological Measurement, 15, 361-373.
19. Robins, J.M., van der Vaart, A., & Ventura, V. (2000). The asymptotic distribution of P values in composite null models. Journal of the American Statistical Association, 95, 1143-1172.
20. Rubin, D.B. (1984). Bayesianly justifiable and relevant frequency calculations for the applied statistician. Annals of Statistics, 12, 1151-1172.
21. Rubin, D.B. (1996). Comment: On posterior predictive p-values. Statistica Sinica, 6, 787-792.
22. Sinharay, S. (2005). Assessing fit of unidimensional item response theory models using a Bayesian approach. Journal of Educational Measurement, 42, 375-394.
23. Sinharay, S. (2006). Bayesian item fit analysis for unidimensional item response theory models. British Journal of Mathematical and Statistical Psychology, 59, 429-449.
24. Sinharay, S., & Johnson, M. (2003, October). Simulation studies applying posterior predictive model checking for assessing fit of the common item response theory models (ETS RR-03-28). Princeton, NJ: Educational Testing Service.
25. Sinharay, S., Johnson, M., & Stern, H.S. (2006). Posterior predictive assessment of item response theory models. Applied Psychological Measurement, 30, 298-321.
26. Sinharay, S., & Stern, H.S. (2003). Posterior predictive model checking in hierarchical models. Journal of Statistical Planning and Inference, 111, 209-221.
27. Stern, H.S. (2000). Comment. Journal of the American Statistical Association, 95, 1157-1159.
28. Stout, W., Habing, B., Douglas, J., Kim, H.R., Roussos, L., & Zhang, J. (1996). Conditional covariance-based nonparametric multidimensionality assessment. Applied Psychological Measurement, 20, 331-354.
29. Yen, W. (1993). Scaling performance assessments: Strategies for managing local item dependence. Journal of Educational Measurement, 30, 187-213.
30. Zhang, J., & Stout, W. (1999). Conditional covariance structure of generalized compensatory multidimensional items. Psychometrika, 64, 129-152.