A generalized dimensionality discrepancy measure for dimensionality assessment in multidimensional item response theory

British Journal of Mathematical and Statistical Psychology

Volumn 64, Issue 2, 2011, Pages 208-232

  Levy, Roy ^a   Svetina, Dubravka ^a  

^a ARIZONA STATE UNIVERSITY   (United States)

Author keywords

[No Author keywords available]

Indexed keywords

ARTICLE; COMPUTER GRAPHICS; EDUCATION; HUMAN; MATHEMATICAL COMPUTING; MONTE CARLO METHOD; NONPARAMETRIC TEST; PSYCHOLOGIC TEST; PSYCHOMETRY; REPRODUCIBILITY; STATISTICAL MODEL; STATISTICS;

COMPUTER GRAPHICS; EDUCATIONAL MEASUREMENT; HUMANS; MATHEMATICAL COMPUTING; MODELS, STATISTICAL; MONTE CARLO METHOD; PSYCHOLOGICAL TESTS; PSYCHOMETRICS; REPRODUCIBILITY OF RESULTS; STATISTICS, NONPARAMETRIC;

EID: 79954623616     PISSN: 00071102     EISSN: 20448317     Source Type: Journal    
DOI: 10.1348/000711010X500483     Document Type: Article

References (52)

1. Allen, N. L., Carlson, J. E., & Zelenak, C. A. (1999). The NAEP 1996 technical report. Washington, DC National Center for Education Statistics.
2. Bayarri, M. J., & Berger, J. O. (2000a). P values for composite null models. Journal of the American Statistical Association, 95, 1127-1142. doi.
3. Bayarri, M. J., & Berger, J. O. (2000b). Rejoinder. Journal of the American Statistical Association, 95, 1168-1170. doi.
4. Béguin, A. A., & Glas, C. A. W. (2001). MCMC estimation and some model-fit analysis of multidimensional IRT models. Psychometrika, 66, 541-562. doi.
5. Bolt, D. M., & Lall, V. F. (2003). Estimation of compensatory and noncompensatory multidimensional item response models using Markov chain Monte Carlo. Applied Psychological Measurement, 27, 395-414. doi.
6. Brooks, S. P., & Gelman, A. (1998). General methods for monitoring convergence of iterative simulations. Journal of Computational and Graphical Statistics, 7, 434-455. doi.
7. Chen, W., & Thissen, D. (1997). Local dependence indexes for item pairs using item response theory. Journal of Educational and Behavioral Statistics, 22, 265-289.
8. Clinton, J., Jackman, S., & Rivers, D. (2004). The statistical analysis of role call data. American Political Science Review, 98, 355-370. doi.
9. Embretson, S. E. (1997). Multicomponent response models. In W. J. van der Linden & R. K. Hambleton (Eds.), Handbook of modern item response theory (pp. 305-321). New York Springer.
10. Finch, H., & Habing, B. (2005). Comparison of NOHARM and DETECT in item cluster recovery: Counting dimensions and allocating items. Journal of Educational Measurement, 42(2), 149-169. doi.
11. Finch, H., & Habing, B. (2007). Performance of DIMTEST- and NOHARM-based statistics for testing unidimensionality. Applied Psychological Measurement, 31, 292-307. doi.
12. Fraser, C., & McDonald, R. P. (1988). NOHARM: Least squares item factor analysis. Multivariate Behavioral Research, 23, 267-269. doi.
13. Gelman, A. (2003). A Bayesian formulation of exploratory data analysis and goodness-of-fit testing. International Statistical Review, 71, 369-382.
14. Gelman, A. (2007). Comment: Bayesian checking of the second levels of hierarchical models. Statistical Science, 22, 349-352. doi.
15. Gelman, A., Meng, X. L., & Stern, H. (1996). Posterior predictive assessment of model fitness via realized discrepancies. Statistica Sinica, 6, 733-807.
16. Gessaroli, M. E., & De Champlain, A. F. (1996). Using an approximate chi-square statistic to test the number of dimensions underlying the responses to a set of items. Journal of Educational Measurement, 33, 157-192. doi.
17. Gessaroli, M. E., De Champlain, A. F., & Folske, J. C. (1997). Assessing dimensionality using a likelihood-ratio chi-square test based on a non-linear factor analysis of item response data. Paper presented at the Annual Meeting of the National Council on Measurement in Education, Chicago, March.
18. Gill, J. (2007). Bayesian methods: A social and behavioral sciences approach. (2nd ed.). Boca Raton, FL Chapman & Hall/CRC.
19. Habing, B., & Roussos, L. A. (2003). On the need for local item dependence. Psychometrika, 68, 435-451. doi.
20. Hattie, J. (1984). An empirical study of various indices for determining unidimensionality. Multivariate Behavioral Research, 19, 49-78. doi.
21. Hattie, J., Krakowski, K., Rogers, H. J., & Swaminathan, H. (1996). An assessment of Stout's index of essential unidimensionality. Applied Psychological Measurement, 20, 1-14. doi.
22. Hjort, N. L., Dahl, F. A., & Steinbakk, G. H. (2006). Post-processing posterior predictive p values. Journal of the American Statistical Association, 101, 1157-1174. doi.
23. Hoijtink, H. (2001). Conditional independence and differential item functioning in the two-parameter logistic model. In A. Boomsma, M. A. J. van Duijn, & T. A. B. Snijders (Eds.), Essays in item response theory (pp. 109-129). New York Springer.
24. Ip, E. H. (2001). Testing for local dependency in dichotomous and polytomous item response models. Psychometrika, 66, 109-132. doi.
25. Jackman, S. (2008). pscl: Classes and methods for R developed in the political science computational laboratory, Stanford University. Department of Political Science, Stanford University, Stanford, CA. R package version 1.03. Retrieved from
26. Levy, R. (2010). Posterior predictive model checking for conjunctive multidimensionality in item response theory. Journal of Educational and Behavioral Statistics. Advance online publication.
27. Levy, R., Mislevy, R. J., & Sinharay, S. (2009). Posterior predictive model checking for multidimensionality in item response theory. Applied Psychological Measurement, 33, 519-537. doi.
28. Lord, F. M. (1980). Applications of item response theory to practical testing problems. Hillsdale, NJ Erlbaum.
29. McDonald, R. P. (1997). Normal-ogive multidimensional model. In W. J., van der Linden & R. K. Hambleton (Eds.), Handbook of modern item response theory (pp. 257-269). New York Springer.
30. McDonald, R. P., & Mok, M. M. C. (1995). Goodness of fit in item response models. Multivariate Behavioral Research, 30, 23-40. doi.
31. Meng, X. L. (1994). Posterior predictive p-values. Annals of Statistics, 22, 1142-1160. doi.
32. Nandakumar, R. (1993). Simultaneous DIF amplification and cancellation: Shealy-Stout's test for DIF. Journal of Educational Measurement, 30, 293-311. doi.
33. Nandakumar, R., & Stout, W. F. (1993). Refinement of Stout's procedure for assessing latent trait dimensionality. Journal of Educational Statistics, 18, 41-68. doi.
34. Nandakumar, R., Yu, F., Li, H., & Stout, W. (1998). Assessing unidimensionality of polytomous data. Applied Psychological Measurement, 22, 99-115. doi.
35. Reckase, M. D. (1997). A linear logistic multidimensional model. In W. J. van der Linden & R. K. Hambleton (Eds.), Handbook of modern item response theory (pp. 271-286). New York Springer.
36. Robins, J. M., van der Vaart, A., & Ventura, V. (2000). The asymptotic distribution of P values in composite null models. Journal of the American Statistical Association, 95, 1143-1172. doi.
37. Rubin, D. B. (1984). Bayesianly justifiable and relevant frequency calculations for the applied statistician. Annals of Statistics, 12, 1151-1172. doi.
38. Sinharay, S. (2005). Assessing fit of unidimensional item response theory models using a Bayesian approach. Journal of Educational Measurement, 42, 375-394. doi.
39. Sinharay, S. (2006). Bayesian item fit analysis for unidimensional item response theory models. British Journal of Mathematical and Statistical Psychology, 59, 429-449. doi.
40. Sinharay, S., Johnson, M., & Stern, H. S. (2006). Posterior predictive assessment of item response theory models. Applied Psychological Measurement, 30, 298-321. doi.
41. Spiegelhalter, D. J., Thomas, A., Best, N. G., & Lunn, D. (2007). WinBUGS user manual: Version 1.4.3. Cambridge MRC Biostatistics Unit. Retrived from
42. Stern, H. S. (2000). Comment. Journal of the American Statistical Association, 95, 1157-1159. doi.
43. Stout, W. (1987). A nonparametric approach for assessing latent trait unidimensionality. Psychometrika, 52, 589-617. doi.
44. Stout, W. (1990). A new item response theory modeling approach with applications to unidimensionality assessment and ability estimation. Psychometrika, 55, 293-325. doi.
45. Stout, W., Habing, B., Douglas, J., Kim, H. R., Roussos, L., & Zhang, J. (1996). Conditional covariance-based nonparametric multidimensionality assessment. Applied Psychological Measurement, 20, 331-354. doi.
46. Swaminathan, H., Hambleton, R. K., & Rogers, H. J. (2007). Assessing the fit of item response models. In C. R. Rao & S. Sinharay (Eds.), Handbook of statistics (Vol. 26, pp. 683-718). Amsterdam Elsevier/North-Holland.
47. Wei, H., & Mislevy, R. J. (2007). Multidimensionality in the NAEP science assessment. Paper presented at the Annual Meeting of the National Council on Measurement in Education, Chicago, April.
48. Yao, L., & Boughton, K. A. (2007). A multidimensional item response modeling approach for improving subscale proficiency estimation and classification. Applied Psychological Measurement, 31, 83-105. doi.
49. Yen, W. (1984). Effects of local item dependence on the fit and equating performance of the three-parameter logistic model. Applied Psychological Measurement, 21, 93-111.
50. Zhang, J. (2007). Conditional covariance theory and DETECT for polytomous items. Psychometrika, 72, 69-91. doi.
51. Zhang, J., & Stout, W. (1999a). Conditional covariance structure of generalized compensatory multidimensional items. Psychometrika, 64, 129-152. doi.
52. Zhang, J., & Stout, W. (1999b). The theoretical DETECT index of dimensionality and its application to approximate simple structure. Psychometrika, 64, 213-249. doi.