Modeling randomness in judging rating scales with a random-effects rating scale model

Journal of Educational Measurement

Volumn 43, Issue 4, 2006, Pages 335-353

  Wang, Wen Chung ^a   Wilson, Mark ^b   Shih, Ching Lin ^a,c  

^a NATIONAL CHUNG CHENG UNIVERSITY   (Taiwan)

^b UNIVERSITY OF CALIFORNIA   (United States)

^c NATIONAL TAICHUNG UNIVERSITY OF EDUCATION   (Taiwan)

Author keywords

[No Author keywords available]

Indexed keywords

EID: 33845383286     PISSN: 00220655     EISSN: 17453984     Source Type: Journal    
DOI: 10.1111/j.1745-3984.2006.00020.x     Document Type: Article

References (54)

1. Adams, R. J., Wilson, M. R., & Wang, W-C. (1997). The multidimensional random coefficients multinomial logit model. Applied Psychological Measurement, 21, 1-23.
2. Adams, R. J., Wilson, M. R., & Wu, M. (1997). Multilevel item response models: An approach to errors in variable regression. Journal of Educational and Behavioral Statistics, 22, 47-76.
3. Andrich, D. (1978). A rating formulation for ordered response categories. Psychometrika, 43, 561-573.
4. Andrich, D. (1988). Rasch models for measurement. Newbury Park, CA: Sage.
5. Biggs, J. B., & Collis, K. F. (1982). Evaluating the quality of learning: The SOLO taxonomy. New York: Academic Press.
6. Bock, R. D. (1972). Estimating item parameters and latent ability when responses are scored in two or more nominal categories. Psychometrika, 37, 29-51.
7. Bock, R. D., & Aitkin, M. (1981). Marginal maximum likelihood estimation of item parameters: Application of an EM algorithm. Psychometrika, 46, 443-459.
8. Bock, R. D., & Mislevy, R. J. (1982). Adaptive EAP estimation of ability in a microcomputer environment. Applied Psychological Measurement, 6, 431-444.
9. Bond, T., & Fox, C. (2001). Applying the Rasch model: Fundamental measurement in the human sciences. Mahwah, NJ: Erlbaum.
10. Bradlow, E. T., Wainer, H., & Wang, X. (1999). A Bayesian random effects model for testlets. Psychometrika, 64, 153-168.
11. De Boeck, P., & Wilson, M. R. (Eds.) (2004). Explanatory item response models: A generalized linear and nonlinear approach. New York: Springer-Verlag.
12. Embretson, S. E., & Reise, S. P. (2000). Item response theory for psychologists. Mahwah, NJ: Erlbaum.
13. Fox, J.-P., & Glas, C. A. W. (2001). Bayesian estimation of a multilevel IRT model using Gibbs sampling. Psycliometrika, 66, 269-286.
14. Lord, F. M. (1980). Applications of item response theory to practical testing problems. Hillsdale, NJ: Lawrence Erlbaum.
15. Masters, G. N. (1982). A Rasch model for partial credit scoring. Psychometrika, 47, 149-174.
16. McCullagh, P., & Nelder, J. A. (1989). Generalized linear models (2nd ed.). London: Chapman & Hall.
17. McCulloch, C. E., & Searle, S. R. (2001). Generalized, linear, and mixed models. New York: Wiley.
18. Mislevy, R. J., Beaton, A. E., Kaplan, B., & Sheehan, K. M. (1992). Estimating population characteristics from sparse matrix samples of item responses. Journal of Educational Measurement, 29, 133-161.
19. Mislevy, R. J., & Bock, R. D. (1989). A hierarchical item-response model for educational testing. In R. D. Bock (Ed.), Multilevel analysis of educational data (pp. 57-74). San Diego, CA: Academic Press.
20. Muraki, E. (1992). A generalized partial credit model: Application of an EM-algorithm. Applied Psychological Measurement, 16, 159-176.
21. Nelder, J. A., & Wedderburn, R. W. M. (1972). Generalized linear models. Journal of the Royal Statistical Society A, 135, 370-384.
22. Patz, R. J., & Junker, B. W. (1999). Applications and extensions of MCMC in IRT: Multiple item types, missing data, and rated responses. Journal of Educational and Behavioral Statistics, 24, 342-366.
23. Rasch, G. (1960). Probabilistic models for some intelligence and attainment tests. Copenhagen, Denmark: Danmarks Paedagogische Institut.
24. Rijmen, F., & De Boeck, P. (2002). The random weights linear logistic test model. Applied Psychological Measurement, 26, 269-283.
25. Rijmen, F., Tuerlinckx, F., De Boeck, P., & Kuppens, P. (2003). A nonlinear mixed model framework for item response theory. Psychological Methods, 8, 185-205.
26. Rosenbaum, P. R. (1988). Item bundles. Psychometrika, 53, 349-359.
27. Samejima, F. (1969). Estimation of latent ability using a response pattern of graded scores. Psychometrika Monograph Supplement, 17, 1-100.
28. SAS Institute (1999). The NLMIXED procedure [Computer software]. Cary, NC: Author.
29. Scholten, I., Keeves, J. P., & Lawson, M. J. (2002). Validation of a free response test of deep learning about the normal swallowing process. Higher Education, 44, 233-255.
30. Sireci, S. G., Thissen, D., & Wainer, H. (1991). On the reliability of testlet-based tests. Journal of Educational Measurement, 28, 237-247.
31. Smith, E. V., Jr., & Smith, R. M. (Eds.) (2004). Introduction to Rasch Measurement. Maple Grove, MN: JAM Press.
32. Stram, D. O., & Lee, J. W. (1994). Variance components testing in the longitudinal mixed-effects model. Biometrics, 50, 1171-1177.
33. Stram, D. O., & Lee, J. W. (1995). Correction to: Variance components testing in the longitudinal mixed-effects model. Biometrics, 51, 1196.
34. Volodin, N., & Adams, R. J. (1995). Identifying and estimating a D-dimensional Rasch model. Paper presented at the International Objective Measurement Workshop. University of California at Berkeley, California.
35. Wainer, H. (1995). Precision and differential item functioning on a testlet-based test: The 1991 Law School Admissions Test as an example. Applied Measurement in Education, 8, 157-186.
36. Wainer, H., & Kiely, G. (1987). Item clusters and computerized adaptive testing: A case for testlets. Journal of Educational Measurement, 24, 185-202.
37. Wainer, H., & Lukhele, R. (1997). How reliable are TOEFL scores? Educational and Psychological Measurement, 57, 749-766.
38. Wainer, H., & Thissen, D. (1996). How is reliability related to the quality of test scores? What is the effect of local dependence on reliability? Educational Measurement: Issues and Practice, 15(1), 22-29.
39. Wainer, H., & Wang, X. (2000). Using a new statistical model for testlets to score TOEFL. Journal of Educational Measurement, 37, 203-220.
40. Wainer, H., Bradlow, E. T., & Du, Z. (2000). Testlet response theory: An analog for the 3PL model useful in testlet-based adaptive testing. In W. van der Linden & C. A. W. Glas (Eds.), Computerized adaptive testing: Theory and practice (pp. 245-269). London: Kluwer.
41. Wang, W.-C., & Wilson, M. R. (2005a). Exploring local item dependence using a random-effects facet model. Applied Psychological Measurement, 29, 296-318.
42. Wang, W.-C., & Wilson, M. R. (2005b). The Rasch testlet model. Applied Psychological Measurement, 29, 126-149.
43. Wang, X., Bradlow, E. T., & Wainer, H. (2002). A general Bayesian model for testlets: Theory and applications. Applied Psychological Measurement, 26, 109-128.
44. Wilson, M. R. (1992). The partial order model: An extension of the partial credit model. Applied Psychological Measurement, 16, 309-425.
45. Wilson, M. R., & Adams, R. J. (1995). Rasch models for item bundles. Psychometrika, 60, 181-208.
46. Wilson, M. R., & Iventosch, L. (1988). Using the partial credit model to investigate responses to structured subtests. Applied Measurement in Education, 1, 319-334.
47. Wilson, M. R., & Sloane, K. (2000). From principles to practice: An embedded assessment system. Applied Measurement in Education, 13, 181-208.
48. Wolfinger, R. D., & SAS Institute (n.d.). Fitting nonlinear mixed models with the new NLMIXED procedure. Retrieved August 17, 2003, from http://support.sas.com/rnd/app/papers/nlmixedsugi.pdf.
49. Wright, B. D. (1999). Model selection: Rating scale or partial credit? Rasch Measurement Transactions, 12(3), 641-642.
50. Wright, B. D., & Masters, G. N. (1982). Rating scale analysis. Chicago: MESA Press.
51. Wright, B. D., Linacre, J. M., Gustafson, J.-E., & Martin-Löf, P. (1994). Reasonable mean-square fit values. Rasch Measurement Transactions, 8(3), 370.
52. Wu, J., Yin, M.-C., Lee, K.-C., & Hu, S.-C. (2001). Prevention and control of school violence in Taiwan. (NSC Research Report No. NSC89-2420-H006-001-QBS). Taipei, Taiwan: The National Science Council.
53. Wu, M. L., Adams, R. J., & Wilson, M. R. (1998). ConQuest [Computer software and manual]. Camberwell, Victoria, Australia: Australian Council for Educational Research.
54. Yen, W. (1993). Scaling performance assessment: Strategies for managing local item dependence. Journal of Educational Measurement, 30, 187-213.