Assessing fit of unidimensional item response theory models using a bayesian approach

Journal of Educational Measurement

Volumn 42, Issue 4, 2005, Pages 375-394

  Sinharay, Sandip ^a  

^a EDUCATIONAL TESTING SERVICE   (United States)

Author keywords

[No Author keywords available]

Indexed keywords

EID: 30344469096     PISSN: 00220655     EISSN: None     Source Type: Journal    
DOI: 10.1111/j.1745-3984.2005.00021.x     Document Type: Article

References (53)

1. Bayarri, S., & Berger, J. (2000). P-values for composite null models. Journal of the American Statistical Association, 95, 1127-1142.
2. Beguin, A. A., & Glas, C. A. W. (2001). MCMC estimation and some model-fit analysis of multidimensional IRT models. Psychometrika, 66(4), 541-562.
3. Bejar, I.I. (2002). Generative testing: From conception to implementation. In S. H. Irvine & P. C. Kyllonen (Eds.), Item generation for test development. Mahwah, NJ: Lawrence Erlbaum Associates.
4. Bolt, D. M., Cohen, A. S., & Wollack, J. A. (2002). Item parameter estimation under conditions of test speededness: Application of a mixture Rasch model with ordinal constraints. Journal of Educational Measurement, 39(4), 331-348.
5. Boughton, K., Larkin, K., & Yamamoto, K. (2004). Modeling differential speededness using a hybrid psychometric approach. Paper presented at the annual meeting of the American Educational Research Association, San Diego, CA.
6. Bradlow, E. T., Wainer, H., & Wang, X. (1999). A Bayesian random effects model for testlets. Psychometrika, 64, 153-168.
7. Casella, G., & Berger, R. L. (1987). Reconciling Bayesian and frequentist evidence in the one-sided testing problem. Journal of the American Statistical Association, 82, 106-111.
8. Chen, W., & Thissen, D. (1997). Local dependence indexes for item pairs using item response theory. Journal of Educational and Behavioral Statistics, 22(3), 265-289.
9. Ferrando, P. J., & Lorenzo-seva, U. (2001). Checking the appropriateness of item response theory models by predicting the distribution of observed scores: The program EP-fit. Educational and Psychological Measurement, 61(5), 895-902.
10. Fox, J. P., & Glas, C. A. W. (2001). Bayesian estimation of a multilevel IRT model using Gibbs sampling. Psychometrika, 66, 269-286.
11. Fox, J. P., & Glas, C. A. W. (2003). Bayesian modeling of measurement error in predictor variables using item response theory. Psychometrika, 68, 169-191.
12. Gelman, A., Carlin, J. B., Stern, H. S., & Rubin, D. B. (2003). Bayesian data analysis. New York: Chapman & Hall.
13. Gelman, A., Meng, X. L., & Stern, H. S. (1996). Posterior predictive assessment of model fitness via realized discrepancies. Statistica Sinica, 6, 733-807.
14. Glas, C. A. W., & Meijer, R. R. (2003). A Bayesian approach to person fit analysis in item response theory models. Applied Psychological Measurement, 27(3), 217-233.
15. Glas, C. A. W., & Suarez-Falcon, J. C. (2003). A comparison of item-fit statistics for the three-parameter logistic model. Applied Psychological Measurement, 27(2), 87-106.
16. Glas, C. A. W., & van der Linden, W. J. (2003). Computerized adaptive testing with item cloning. Applied Psychological Measurement, 27(4), 247-261.
17. Guttman, I. (1967). The use of the concept of a future observation in goodness-of-fit problems. Journal of the Royal Statistical Society B, 29, 83-100.
18. Hambleton, R. K. (1989). Principles and selected applications of item response theory. In R. L. Linn (Ed.), Educational measurement (3rd ed., pp. 143-200). New York: Macmillan.
19. Hambleton, R. K., & Han, N. (2004). Assessing the fit of IRT models: Some approaches and graphical displays. Paper presented at the annual meeting of the National Council on Measurement in Education, San Diego, CA.
20. Hoijtink, H. (2001). Conditional independence and differential item functioning in the two-parameter logistic model. In A. Boomsma, M. A. J. van Duijn, & T. A. B. Snijders (Eds.), Essays in item response theory. New York: Springer-Verlag.
21. Holland, P. W. (1985). On the study of differential item performance without IRT. Proceedings of the 27th annual conference of the Military Testing Association, Vol. I (pp. 282-287). San Diego, CA: Navy Personnel Research and Development Center.
22. Irvine, S. H., & Kyllonen, P. C. (Eds.) (2002). Item generation for test development. Mahwah, NJ: Lawrence Erlbaum Associates.
23. Janssen, R., Tuerlinckx, F., Meulders, M., & De Boeck, P. (2000). A hierarchical IRT model for criterion-referenced measurement. Journal of Educational and Behavioral Statistics, 25(3), 285-306.
24. Lewis, C. (2001). Expected response functions. In A. Boomsma, M. A. J. van Duijn, & T. A. B. Snijders (Eds.), Essays on item response theory. New York: Springer-Verlag.
25. Lord, F. M. (1953). The relation of test score to the latent trait underlying the test. Educational and Psychological Measurement, 13, 517-548.
26. Lord, F. M. (1980). Applications of item response theory to practical testing problems. Hillsdale, NJ: Erlbaum.
27. McDonald, R. P. (1999). Test theory; A unified treatment. Mahwah, NJ: Lawrence Erlbaum.
28. Morimoto, Y., & Tatsuoka, K. K. (1983). Analysis of misconceptions in fraction problems: Interactive diagnostic system on the PLATO system. Proceedings of ICMI-JSME Regional Conference on Mathematics in Education, Tokyo, Japan.
29. Orlando, M., & Thissen, D. (2000). Likelihood-based item-fit indices for dichotomous item response theory models. Applied Psychological Measurement, 24, 50-64.
30. Patz, R., & Junker, B. (1999a). A straightforward approach to Markov chain Monte Carlo methods for item response models. Journal of Educational and Behavioral Statistics, 24, 146-178.
31. Patz, R., & Junker, B. (1999b). Applications and extensions of MCMC in IRT: Multiple item types, missing data, and rated responses. Journal of Educational and Behavioral Statistics, 24, 342-366.
32. Reiser, M. (1996). Analysis of residuals for the multinomial item response models. Psychometrika, 61(3), 509-528.
33. Ripley, B. D. (2001). The {R} project in statistical computing. MSOR Connections. The newsletter of the LTSN Maths, Stats & OR Network, 1(1), 23-25.
34. Rubin, D. B. (1984). Bayesianly justifiable and relevant frequency calculations for the applied statistician. Annals of Statistics, 12, 1151-1172.
35. Sandene, B., Horkay, N., Bennett, R., Allen, N., Braswell, J., Kaplan, B., and Oranje, A. (2005). Online Assessment in Mathematics and Writing: Reports from the NAEP Technology-based Assessment Project, Research, and Development Series (NCES 2005-457). U.S. Department of Education, National Center for Education Statistics. Washington, DC: U.S. Government Printing Office.
36. Sinharay, S. (2003a). Bayesian item fit analysis for dichotomous item response theory models. Princeton, NJ, ETS. Retrieved November 1, 2004, from http://www.ets.org/research/newpubs.html, ETS RR-03-34.
37. Sinharay, S. (2003b). Practical applications of posterior predictive model checking for assessing fit of common item response theory models. Princeton, NJ, ETS. Retrieved November 1, 2004, from http://www.ets.org/ research/newpubs.html, ETS RR-03-33.
38. Sinharay, S. (in press) Assessing fit of Bayesian networks using the posterior predictive model checking method. To appear in Journal of Educational and Behavioral Statistics.
39. Sinharay, S., & Johnson, M. S. (2003). Simulation studies applying posterior predictive model checking for assessing fit of the common item response theory models. Manuscript in preparation. A preliminary version Retrieved November 1, 2004, from http://www.ets.org/research/newpubs.html.
40. Sinharay, S., Johnson, M. S., & Williamson, D. M. (2003). Calibrating item families and summarizing the results using family expected response functions. Journal of Educational and Behavioral Statistics, 28(4), 295-313.
41. Spiegelhalter, D. J., Thomas, A., & Best N. G. (1999). WinBUGS version 1.2 user manual [Computer Software manual]. Cambridge, UK: MRC Biostatistics Unit.
42. Stern, H. (2000). Comments on "P-values for composite null models." Journal of the American Statistical Association, 95, 1157-1160.
43. Stone, C. A., & Zhang, B. (2003). Assessing goodness of fit of item response theory models: A comparison of traditional and alternative procedures. Journal of Educational Measurement, 40(4), 331-352.
44. Stout, W. F. (1987). A nonparametric approach for assessing latent trait dimensionality. Psychometrika, 52, 589-617.
45. Tatsuoka, K. K. (1983). Rule space: An approach for dealing with misconceptions based on item response theory. Journal of Educational Measurement, 20(4), 345-354.
46. Tatsuoka, K. K. (1984). Analysis of errors in fraction addition and subtraction problems. Urbana, IL: University of Illinois, Computer-Based Education Research (NIE Final Rep. for Grant No. NIE-G-81-002).
47. Tatsuoka, K. K. (1990). Toward an integration of item response theory and cognitive error diagnosis. In N. Frederiksen, R. Glaser, A. Lesgold, & M. G. Shafto (Eds.), Diagnostic monitoring of skill and knowledge acquisition (pp. 453-488). Hillsdale, NJ: Erlbaum.
48. Tatsuoka, K. K., Linn, R. L., Tatsuoka, M. M., & Yamamoto, K. (1988). Differential item functioning resulting from the use of different solution strategies. Journal of Educational Measurement, 25(4), 301-319.
49. Tsutakawa, R. K., & Johnson, J. C. (1990). The effect of uncertainty of item parameter estimation on ability estimates. Psychometrika, 55, 371-390.
50. van der Linden, W. J., & Hambleton, R. K. (1997). Handbook of modern item response theory. New York: Springer.
51. Wollack, J. A., Cohen, A. S., & Wells, C. S. (2003). A method for maintaining scale stability in the presence of test speededness. Journal of Educational Measurement, 40(4), 307-330.
52. Yamamoto, K. (1989). HYBRID model of IRT and latent class models. Princeton, NJ: ETS (RR-89-41).
53. Yen, W. (1993). Scaling performance assessments: Strategies for managing local item dependence. Journal of Educational Measurement, 30, 187-213.