Gain score in item response theory as an effect size measure

Educational and Psychological Measurement

Volumn 64, Issue 5, 2004, Pages 758-780

  Wang, Wen Chung ^a   Chyi In, Wu ^b  

^a NATIONAL CHUNG CHENG UNIVERSITY   (Taiwan)

^b ACADEMIA SINICA   (Taiwan)

Author keywords

Change measurement; Item response theory; Modifiability; Multidimensional item response model; Rasch model

Indexed keywords

EID: 6344278847     PISSN: 00131644     EISSN: None     Source Type: Journal    
DOI: 10.1177/0013164404264118     Document Type: Article

References (78)

1. Adams, R. J., & Wilson, M. R. (1996). Formulating the Rasch models as a mixed coefficients multinomial logit. In G. Engelhard & M. R. Wilson (Eds.), Objective measurement: Theory into practice (Vol. 3, pp. 143-166). Norwood, NJ: Ablex.
2. Adams, R. J., Wilson, M. R., & Wang, W.-C. (1997). The multidimensional random coefficients multinomial logit model. Applied Psychological Measurement, 21, 1-23.
3. American Psychological Association. (1994). Publication manual of the American Psychological Association (4th ed.). Washington, DC: Author.
4. American Psychological Association. (2001). Publication manual of the American Psychological Association (5th ed.). Washington, DC: Author.
5. Andersen, E. B. (1985). Estimating latent correlations between repeated testings. Psychometrika, 50, 3-16.
6. Andrich, D. (1978). A rating formulation for ordered response categories. Psychometrika, 43, 561-573.
7. Andrich, D. (1988). Rasch models for measurement. Newbury Park, CA: Sage.
8. Baugh, F. (2002). Correcting effect sizes for score reliability: A reminder that measurement and substantive issues are linked inextricably. Educational and Psychological Measurement, 62, 254-263.
9. Béguin, A. A., & Glas, C. A. W. (2001). MCMC estimation and model-fit analysis of multidimensional IRT models. Psychometrika, 66, 541-562.
10. Bereiter, C. (1963). Some persisting dilemmas in the measurement of change. In C. W. Harris (Ed.), Problems in measuring change. Madison: University of Wisconsin Press.
11. Birnbaum, A. (1968). Some latent trait models and their use in inferring an examinees ability. In F. M. Lord & M. R. Novick (Eds.), Statistical theories of mental test scores. Reading, MA: Addison-Wesley.
12. Bock, R. D., & Aitkin, M. (1981). Marginal maximum likelihood estimation of item parameters: Application of an EM algorithm. Psychometrika, 46, 443-459.
13. Bock, R. D., Gibbons, R., & Muraki, E. J. (1988). Full information item factor analysis. Applied Psychological Measurement, 12, 261-280.
14. Bock, R. D., & Lieberman, M. (1970). Fitting a response model for n dichotomously scored items. Psychometrika, 35, 179-197.
15. Bock, R. D., & Mislevy, R. J. (1982). Adaptive EAP estimation of ability in a microcomputer environment. Applied Psychological Measurement, 6, 431-444.
16. Bock, R. D., Muraki, E., & Pfeiffenberger, W. (1988). Item pool maintenance in the presence of item parameter drift. Journal of Educational Measurement, 25, 275-285.
17. Bond, T. G., & Fox, C. M. (2001). Applying the Rasch model: Fundamental measurement in the human sciences. Mahwah, NJ: Lawrence Erlbaum.
18. Capraro, R. M., & Capraro, M. M. (2002). Treatments of effect sizes and statistical significance tests in textbooks. Educational and Psychological Measurement, 62, 771-782.
19. Cohen, J. (1969). Statistical power analysis for the behavioral sciences. Hillsdale, NJ: Lawrence Erlbaum.
20. Cohen, J. (1988). Statistical power analysis for the behavioral sciences (2nd ed.). New York: Academic Press.
21. Collins, L. M., & Horn, J. L. (1991). Best methods for the analysis of change. Washington, DC: American Psychological Association.
22. Cribbie, R. A., & Jamieson, J. (2000). Structural equation models and the regression bias for measuring correlates of change. Educational and Psychological Measurement, 60, 893-907.
23. Cronbach, L. J., & Furby, L. (1970). How we should measure change or should we? Psychological Bulletin, 74, 68-80.
24. DeMars, C. (2001). Group difference based on IRT scores: Does the model matter? Educational and Psychological Measurement, 61, 60-70.
25. Dugard, P., & Todman, J. (1995). Analysis of pre-test post-test control group designs in educational research. Educational Psychology, 15, 181-198.
26. Edwards, J. R. (1993). On the use of polynomial regression equations as an alternative to difference scores in organizational research. Academy of Management Journal, 36, 1577-1613.
27. Edwards, J. R. (1995). Alternatives to difference scores as dependent variables in the study of congruence in organizational research. Organizational Behavior and Human Decision Processes, 64, 307-324.
28. Efron, B. (1979). Bootstrap methods: Another look at the jackknife. Annals of Statistics, 7, 1-26.
29. Efron, B., & Tibshirani, R. J. (1993). An introduction to the bootstrap. New York: Chapman & Hall.
30. Eliason, S. R. (1993). Maximum likelihood estimation: Logic and practice. Newbury Park, CA: Sage.
31. Embretson, S. E. (1991). A multidimensional latent trait model for measuring learning and change. Psychometrika, 56, 495-515.
32. Embretson, S. E. (1993). Psychometric models for learning and cognitive processes. In N. Frederiksen, R. J. Mislevy, & I. I. Bejar (Eds.), Test theory for a new generation of tests (pp. 125-150). Hillsdale, NJ: Lawrence Erlbaum.
33. Embretson, S. E., & Reise, S. P. (2000). Item response theory for psychologists. Mahwah, NJ: Lawrence Erlbaum.
34. Fischer, G. H. (1995). Derivation of the Rasch model. In G. H. Fischer & I. W. Molenaar (Eds.), Rasch models: Foundations, recent developments, and applications (pp. 15-38). New York: Springer-Verlag.
35. Fischer, G. H. (2003). The precision of gain scores under an item response theory perspective: A comparison of asymptotic and exact conditional inference about change. Applied Psychological Measurement, 27, 3-26.
36. Fischer, G. H., & Pazer, P. (1991). An extension of the rating scale model with an application to the measurement of change. Psychometrika, 56, 637-651.
37. Fischer, G. H., & Ponocny, I. (1994). An extension of the partial credit 7odel with an application to the measurement of change. Psychometrika, 59, 177-192.
38. Fraser, C. (1988). NOHARM: A Fortran program for fitting unidimensional and multidimensional normal ogive models of latent trait theory [Computer software]. Armidale, New South Wales, Australia: University of New England, Centre for Behavioral Studies.
39. Geenen, R., & van de Vijver, F. J. R. (1993). A simple test of the law of initial values. Psychophysiology, 30, 525-530.
40. Hake, R. R. (1998). Interactive-engagement versus traditional methods: A six-thousand-student survey of mechanics test data for introductory physics courses. American Journal of Physics, 66, 64-74.
41. Henson, R. K. (2001). Understanding internal consistency reliability estimates: A conceptual primer on coefficient alpha. Measurement and Evaluation in Counseling and Development, 34, 177-189.
42. Holland, P. W., & Wainer, H. (Eds.). (1993). Differential item functioning. Hillsdale, NJ: Lawrence Erlbaum.
43. Jamieson, J. (1994). Measurement of change and the law of initial values: A computer simulation study. Educational and Psychological Measurement, 55, 38-46.
44. Lee, P. M. (1997). Bayesian statistics: An introduction (2nd ed.). New York: John Wiley.
45. Lord, F. M. (1956). The measurement of growth. Educational and Psychological Measurement, 16, 421-437.
46. Lord, F. M. (1980). Applications of item response theory to practical testing problems. Hillsdale, NJ: Lawrence Erlbaum.
47. Lord, F. M., & Novick, M. R. (1968). Statistical theories of mental test scores. Reading, MA: Addison-Wesley.
48. Malgady, R. G., & Colon-Malgady, G. (1991). Comparing the reliability of difference scores and residuals in analysis of covariance. Educational and Psychological Measurement, 51, 803-807.
49. Masters, G. N. (1982). A Rasch model for partial credit scoring. Psychometrika, 47, 149-174.
50. Maxwell, S. E., & Delaney, H. D. (1990). Designing experiments and analyzing data: A model comparison perspective. Belmont, CA: Wadsworth.
51. May, K., & Nicewander, W. A. (1998). Measuring change conventionally and adaptively. Educational and Psychological Measurement, 58, 882-897.
52. McDonald, R. P. (1982). Linear versus nonlinear models in latent trait theory. Applied Psychological Measurement, 6, 379-396.
53. McKinley, R. L., & Reckase, M. D. (1983). MAXLOG: A computer program for the estimation of the parameters of a multidimensional logistic model. Behavior Research Methods and Instrumentation, 15, 389-390.
54. Mislevy, R. J., Beaton, A. E., Kaplan, B., & Sheehan, K. M. (1992). Estimating population characteristics from sparse matrix samples of item responses. Journal of Educational Measurement, 29, 133-161.
55. Overall, J. E., & Tonidandel, S. (2002). Measuring change in controlled longitudinal studies. British Journal of Mathematical and Statistical Psychology, 55, 109-124.
56. Perline, R., Wright, B. D., & Wainer, H. (1977). The Rasch model as additive conjoint measurement. Applied Psychological Measurement, 3, 237-255.
57. Rasch, G. (1960). Probabilistic models for some intelligent and attainment tests. Copenhagen, Denmark: Institute of Educational Research.
58. Reckase, M. D. (1985). The difficulty of test items that measure more than one dimension. Applied Psychological Measurement, 9, 401-412.
59. Rogosa, D., & Willett, J. B. (1983). Demonstrating the reliability of the difference score. Journal of Educational Measurement, 20, 335-343.
60. Samejima, F. (1969). Estimation of latent ability using a response pattern of graded scores. Psychometrika Monograph, No. 17.
61. SAS Institute. (1999). SAS OnlineDoc (Version 8) [Computer software manual on CD-ROM]. Cary, NC: Author.
62. Scheiblechner, H. (1999). Additive conjoint isotonic probabilistic models. Psychometrika, 64, 295-316.
63. Shapiro, S. S., & Wilk, M. B. (1965). An analysis of variance test for normality (complete samples). Biometrika, 52, 591-611.
64. Thompson, B., & Snyder, P. A. (1998). Statistical significance and reliability analyses in recent JCD research articles. Journal of Counseling and Development, 76, 431-436.
65. Thompson, B., & Vacha-Haase, T. (2000). Psychometrics is datametrics: The test is not reliable. Educational and Psychological Measurement, 60, 174-195.
66. Vacha-Haase, T., Ness, C., Nilsson, J., & Reetz, D. (1999). Practices regarding reporting of reliability coefficients: A review of three journals. Journal of Experimental Education, 67, 335-341.
67. Wang, W.-C., & Chen, H.-C. (2004). The standardized mean difference within the framework of item response theory. Educational and Psychological Measurement, 64, 201-223.
68. Wang, W.-C., Wilson, M. R., & Adams, R. J. (1998). Measuring individual differences in change with Rasch models. Journal of Outcome Measurement, 2, 240-265.
69. Wilkinson, L., & the APA Task Force on Statistical Inference. (1999). Statistical methods in psychology journals: Guidelines and explanations. American Psychologist, 54, 594-604.
70. Williams, R. H., & Zimmerman, D. W. (1996). Are simple gain scores obsolete? Applied Psychological Measurement, 20, 59-69.
71. Williams, R. H., & Zimmerman, D. W. (1999). Nonindependence of parameters of the validity and reliability of gain scores. Perceptual & Motor Skills, 88, 679-682.
72. Wilson, D. T., Wood, R., & Gibbons, R. (1991). TESTFACT: Test scoring, item statistics, and item factor analysis [Computer software]. Mooresville, IN: Scientific Software.
73. Wolfinger, R. D., & SAS Institute. (n.d.). Fitting nonlinear mixed models with the new NLMIXED procedure. Retrieved August 17, 2003, from http://support.sas.com/rnd/app/papers/nlmixedsugi. pdf.
74. Wright, B. D., & Stone, M. H. (1979). Best test design. Chicago: MESA Press.
75. Wu, C.-I. (1999). The etiology of adolescents substance abuse: A social learning model (Tech. Rep. No. DOH88-HR-621). Taipei, Taiwan: Academia Sinica.
76. Wu, M., Adams, R. J., & Wilson, M. R. (1998). ACER ConQuest: Generalised item response modeling software [Computer software]. Camberwell, Victoria: Australian Council for Educational Research.
77. Zimmerman, D. W., & Williams, R. H. (1982). Gain scores in research can be highly reliable. Journal of Educational Measurement, 19, 149-154.
78. Zimmerman, D. W., & Williams, R. H. (1998). Reliability of gain scores under realistic assumptions about properties of pre-test and post-test scores. British Journal of Mathematical and Statistical Psychology, 51, 343-351.