A Hierarchical IRT Model for Criterion-Referenced Measurement

Journal of Educational and Behavioral Statistics

Volumn 25, Issue 3, 2000, Pages 285-306

  Janssen, Rianne ^a   Tuerlinckx, Francis ^a   Meulders, Michel ^a   De Boeck, Paul ^a  

^a UNIVERSITY OF LEUVEN   (Belgium)

Author keywords

Criterion referenced measurement; Gibbs sampler; Hierarchical irt; Posterior predictive checks; Standard setting

Indexed keywords

EID: 0346487418     PISSN: 10769986     EISSN: None     Source Type: Journal    
DOI: 10.3102/10769986025003285     Document Type: Article

References (48)

1. Albert, J. H. (1992). Bayesian estimation of normal ogive item response curves using Gibbs sampling. Journal of Educational Statistics, 17, 251-269.
2. Angoff, W. H. (1971). Scales, norms, and equivalent scores. In R. L. Thorndike (Ed.), Educational measurement (2nd ed., pp. 508-600). Washington, DC: American Council on Education.
3. Bergan, J. R., & Stone, C. A. (1985). Latent class models for knowledge domains. Psychological Bulletin, 98, 166-184.
4. Berk, R. A. (1980). A framework for methodological advances in criterion-referenced testing. Applied Psychological Measurement, 4, 563-573.
5. Berk, R. A. (1986). A consumer's guide to setting performance standards on criterion-referenced tests. Review of Educational Research, 56, 137-172.
6. Berk, R. A. (1996). Standard setting: The next generation (where few psychometricians have gone before!). Applied Measurement in Education, 9, 215-235.
7. Bradlow, E. T., Wainer, H., & Wang, X. (1996). A Bayesian random effects model for testlets. Unpublished research report.
8. Chang, L., Dziuban, C. D., Hynes, M. C., & Olson, A. H. (1996). Does a standard reflect minimal competency of examinees or judge competency? Applied Measurement in Education, 9, 161-173.
9. Cizek, G. J. (1993). Reconsidering standards and criteria. Journal of Educational Measurement, 30, 93-106.
10. Cizek, G. J. (1996a). Setting passing scores. Educational Measurement: Issues and Practice, 15(2), 20-31.
11. Cizek, G. J. (1996b). Standard-setting guidelines. Educational Measurement: Issues and Practice, 15(1), 13-21.
12. De Boeck, P., Daems, F., Meulders, M., & Rymenams, R. (1997). Ontwikkeling van een toets voor de eindtermen begrijpend lezen [Construction of a test for the educational target of reading comprehension]. Leuven/Antwerp (Belgium): University of Leuven/ University of Antwerp.
13. Fischer, G. H. (1983). Logistic latent trait models with linear constraints. Psychometrika, 48, 3-26.
14. Gelman, A., Carlin, J. B., Stern, H. S. & Rubin, D. B. (1995). Bayesian data analysis. New York: Chapman & Hall.
15. Gelman, A., & Meng, X. L. (1996). Model checking and model improvement. In W. R. Gilks, S. Richardson, & D. J. Spiegelhalter (Eds.), Practical Markov Chain Monte Carlo (pp. 189-201). New York: Chapman & Hall.
16. Gelman, A., Meng, X. L., & Stern, H. S. (1996). Posterior predictive assessment of model fitness via realized discrepancies (with discussion). Statistica Sinica, 6, 733-807.
17. Gelman, A., & Rubin, D. B. (1992). Inference from iterative simulation using multiple sequences (with discussion). Statistical Science, 7, 457-511.
18. Gilks, W., Richardson, S. & Spiegelhalter, D. (Eds.) (1996). Practical Markov Chain Monte Carlo. New York: Chapman & Hall.
19. Glas, C.A.W. (1988). The derivation of some tests for the Rasch model from the multinomial distribution. Psychometrika, 53, 526-546.
20. Glass, G. V. (1978). Standards and criteria. Journal of Educational Measurement, 15, 237-261.
21. Hambleton, R. K. (1980). Test score validity and standard-setting methods. In R. A. Berk (Ed.), Criterion-referenced measurement: The state of the art (pp. 80-123). Baltimore: John Hopkins University Press.
22. Hambleton, R. K. (1982). Advances in criterion-referenced testing technology. In C. R. Reynolds & T. B. Gutkin (Eds.), The handbook of school psychology (pp. 351-379). New York: Wiley.
23. Hambleton, R. K., & Novick, M. R. (1973). Toward an integration of theory and method for criterion-referenced tests. Journal of Educational Measurement, 10, 159-170.
24. Huynh, H. (1976). Statistical consideration of mastery scores. Psychometrika, 41, 65-78.
25. Hoijtink, H., & Molenaar, I. W. (1997). A multidimensional item response model: Constrained latent class analysis using the Gibbs sampler and posterior predictive checks. Psychometrika, 62, 171-189.
26. Jaeger, R. M. (1989). Certification of student competence. In R. L. Linn (Ed.), Educational measurement (3rd ed., pp. 485-514). New York: Macmillan.
27. Kane, M. (1994). Validating the performance standards associated with passing scores. Review of Educational Research, 64, 425-461.
28. Linn, R. L. (1980). Issues of validity for criterion-referenced measures. Applied Psychological Measurement, 4, 547-561.
29. Livingston, S. A., & Zieky, M. J. (1982). Passing scores. Princeton, NJ: Educational Testing Service.
30. Lord, F. M., & Novick, M. R. (1968). Statistical theories of mental test scores. Reading: Addison-Wesley.
31. Macready, G. B., & Dayton, C. M. (1980). The nature and use of state mastery models. Applied Psychological Measurement, 4, 493-516.
32. Meng, X.-L. (1994). Posterior predictive p-values. Annals of Statistics, 22, 1142-1160.
33. Meskauskas, J. A. (1976). Evaluation models for criterion-referenced testing: Views regarding mastery in standard setting. Review of Educational Research, 45, 133-158.
34. Mislevy, R. J., Wingersky, M. S., & Sheehan, K. M. (1994). Dealing with uncertainty about item parameters: Expected response functions (ETS Research Report 04204). Princeton, NJ: Educational Testing Service.
35. Nedelsky, L. (1954). Absolute grading standards for objective tests. Educational and Psychological Measurement, 14, 3-19.
36. Norcini, J. J. & Shea, J. A. (1997). The credibility and comparability of standards. Applied Measurement in Education, 10, 39-59.
37. Patz, R. J. (1996). Markov Chain Monte Carlo methods for item response theory models with applications for NAEP. Unpublished dissertation, Department of Statistics, Carnegie Mellon University, Pittsburgh, Pennsylvania.
38. Plake, B. S., Impara, J. C., & Potenza, M. T. (1994). Content specificity of expert judgments in a standard-setting study. Journal of Educational Measurement, 31, 339-347.
39. Plake, B. S., & Melican, G. J. (1989). Effects of item context on intrajudge consistency of expert judgments via the Nedelsky standard setting method. Educational and Psychological Measurement, 49, 45-51.
40. Rubin, D. B. (1984). Bayesianly justifiable and relevant frequency calculations for the applied statistician. Annals of Statistics, 12, 1151-1172.
41. Shepard, L. (1980). Standard setting issues and methods. Applied Psychological Measurement, 4, 447-467.
42. Shepard, L. A. (1994, October). Implications for standard setting of the NAE evaluation of NAEP achievement levels. Paper presented at the Joint Conference on Standard Setting for Large-scale Assessments. National Assessment Governing Board and National Center for Education Statistics, Washinghton, DC.
43. Spiegelhalter, D. J., Best, N. G., Gilks, W. R., & Innip, H. (1996). Hepatitis B: A case study in MCMC methods. In W. R. Gilks, S. Richardson, & D. J. Spiegelhalter (Eds.), Markov Chain Monte Carlo in practice (pp. 21-43). New York: Chapman & Hall.
44. Tanner, M. A. (1996). Tools for statistical inference: Methods for the exploration of posterior distributions and likelihood functions (3rd ed.). New York: Springer.
45. van der Linden, W. J. (1982). A latent trait method for determining intrajudge inconsistency in the Angoff and Nedelsky techniques of standard setting. Journal of Educational Measurement, 19, 295-308.
46. Wilson, D., Wood, R., & Gibbons, R. (1991). TESTFACT: Test scoring, item statistics, and item factor analysis. Chicago, IL: Scientific Software Inc.
47. Zellner, A., & Min, C. (1995). Gibbs sampler convergence criteria. Journal of the American Statistical Association, 90, 921-927.
48. Zimowsky, M. R., Muraki, E., Mislevy, R. J., & Bock, R. D. (1994). Bimain™ 2: Multiple-group IRT analysis and test maintenance for binary items [Computer program]. Chicago, IL: Scientific Software International.