Scaling, Linking, and Reporting a Periodic Assessment System

Journal of Educational Measurement

Volumn 49, Issue 2, 2012, Pages 148-166

  Mislevy, Robert J ^a   Zwick, Rebecca ^a  

^a EDUCATIONAL TESTING SERVICE   (United States)

Author keywords

[No Author keywords available]

Indexed keywords

EID: 84862901893     PISSN: 00220655     EISSN: 17453984     Source Type: Journal    
DOI: 10.1111/j.1745-3984.2012.00166.x     Document Type: Article

References (52)

1. Adams, R. J., & Wu, M. L. (Eds.) (2002). P1SA 2000 technical report. Paris : OECD.
2. Beaton, A. E. (1987). The NAEP 1983/84 technical report (NAEP Report 15-TR-20). Princeton, NJ: Educational Testing Service.
3. Bejar, I. I., & Graf, E. A. (2010). Updating the duplex design for test-based accountability the twenty-first century. Measurement: Interdisciplinary Research & Perspectives, 8, 110-129.
4. Bock, R. D. (1993). Domareferenced reporting large scale educational assessments. Paper commissioned by the National Academy of Education for the Capstone Report of the NAE Technical Review Panel on State/NAEP Assessment.
5. Bock, R. D., & Mislevy, R. J. (1981). An item response curve model for matrix-sampling data: The California grade three assessment. New Directions for Testing and Measurement, 10, 65-90.
6. Carstens, R. & Hastedt, D. (2010, July). The effect of not using plausible values when they should be: an illustration using TIMSS 2007 grade 8 mathematics data. Paper presented at the 4th IEA International Research Conference (IRC-2010) at the University of Gothenburg. Sweden Retrieved November 15, 2011, from
7. Cho, S.-J., Cohen, A. S., Kim, S.-H., & Bottge, B. (2010). Latent transition analysis with a mixture IRT measurement model. Applied Psychological Measurement, 34, 583-604.
8. Cohen, J., & Jiang, T. (1999). Comparison of partially measured latent traits across nominal subgroups. Journal of the American Statistical Association, 94, 1035-1044.
9. Cronbach, L. J. (1995). A valedictory: Reflections on 60 years educational testing. Washington, DC : National Academies Press.
10. Embretson, S. E. (1991). A multidimensional latent trait model for measuring learning and change. Psychometrika, 56, 495-516.
11. Fischer, G. H. (1983). Logistic latent trait models with linear constraints. Psychometrika, 48, 3-26.
12. Forsyth, R., Hambleton, R., Linn, Mislevy, R., & Yen, W. (1996). Design/feasibility team: Report to the National Assessment Governing Board. Washington, DC : National Assessment Governing Board.
13. Gibbons, R. D., & Hedeker, D. R. (1992). Full-information item bi-factor analysis. Psychometrika, 57, 423-436.
14. Haladyna, T. M., & Roid, G. H. (1981). The role of instructional sensitivity the empirical review of criterion-referenced test items. Journal of Educational Measurement, 18, 39-53.
15. Holland, P. W., & Rosenbaum, P. R. (1986). Conditional association and unidimensionality monotone latent variable models. Annals of Statistics, 14, 1523-1543.
16. Johnson, M. S., & Jenkins . F. (2005). A Bayesian hierarchical model for large-scale educational surveys: An application to the National Assessment of Educational Progress (ETS Research Report No. RR-04-38). Princeton, NJ : Educational Testing Service.
17. Lord, F. M. (1959). Statistical inference about true scores. Psychometrika, 24, 1-18.
18. Lord, F. M. (1976). A study of item bias using characteristic curve theory. Accessed November 20, 2011, at
19. Mazzeo, J., Kulick, E., Tay-Lim, B., & Perie, M. (2006). Technical report for the 2000 market basket study mathematics. ETS-NAEP Report No. 06-T01. Princeton, NJ : Educational Testing Service.
20. Mislevy, R. J. (1985). Estimation of latent group effects. Journal of the American Statistical Association, 80, 993-997.
21. Mislevy, R. J. (1991). Randomization-based inference about latent variables from complex samples. Psychometrika, 56, 177-196.
22. Mislevy, R. J. (1998). Implications of market-basket reporting for achievement level setting. Applied Measurement Education, 11, 49-63.
23. Mislevy, R. J. (2003). Evidentiary relationships among data-gathering methods and reporting scales surveys of educational achievement (CSE Technical Report No. 595). Los Angeles, CA: The National Center for Research on Evaluation, Standards, Student Testing (CRESST), Center for Studies Education, UCLA.
24. Mislevy, R. J., Beaton, A. E., Kaplan, B., & Sheehan, K. M. (1992). Estimating population characteristics from sparse matrix samples of item responses. Journal of Educational Measurement, 29, 133-161.
25. Mullis, I. V. S., Martin, M. O., & Foy, P. (2008). TIMSS 2007 international mathematics report: Findings from IEA's trends international mathematics and science study at the fourth and eighth grades. Chestnut Hill, MA : Boston College.
26. Muthén, B. (1994). Instructionally sensitive psychometrics: Applications to the second international mathematics study. I. Westbury, C. Ethington, L. Sosniak, & D. Baker (Eds.), search of more effective mathematics education: Examining data from the IEA second international mathematics study (pp. 293-324). Norwood, NJ : Ablex.
27. Muthén, B., Kao, C.-F., & Burstein, L. (1991). Instructional sensitivity mathematics achievement test items: Applications of a new IRT-based detection technique. Journal of Educational Measurement, 28, 1-22.
28. Partnership for Readiness for College and Careers (PARCC). (2010). Application for the Race to the Top Comprehensive Assessment Systems Competition. Accessed November 20, 2011, at
29. Petroski, H. (1985). To engineer is human: The role of failure successful design. New York : St. Martin's Press.
30. Polikoff, M. S. (2010). Instructional sensitivity as a psychometric property of assessments. Educational Measurement: Issues and Practice, 29(4), 3-14.
31. Reckase, M. D. (2009). Multidimensional item response theory. New York, NY : Springer.
32. Rijmen, F., De Boeck, P., & van der Maas, H. (2005). An IRT model with a parameter-driven process for change. Psychometrika, 70, 651-669.
33. Rubin, D. B. (1977). Formalizing subjective notions about the effect of nonrespondents sample surveys. Journal of the American Statistical Association, 72, 538-543.
34. Rubin, D. B. (1987). Multiple imputation for nonresponse surveys. New York, NY : Wiley.
35. Simon, H. A. (2001). The sciences of the artificial (3rd ed.). Cambridge, MA : MIT Press.
36. SMARTER Balanced Assessment Consortium. (2010). Race to the Top assessment program application for new grants. Accessed November 20, 2011, at
37. Suppes, P., & Zanotti, M. (1981). When are probabilistic explanations possible Synthese, 48, 191-199.
38. Tatsuoka, K. K., Linn, R. L., Tatsuoka, M. M., & Yamamoto, K. (1988). Differential item functioning resulting from the use of different solution strategies. Journal of Educational Measurement, 25, 301-319.
39. Thomas, N. (1993). Asymptotic corrections for multivariate posterior moments with factored likelihood functions. Journal of Computational and Graphical Statistics, 2, 309-322.
40. Thomas, N. (2000). Assessing model sensitivity of the imputation methods used the National Assessment of Educational Progress. Journal of Educational and Behavioral Statistics, 25, 351-371.
41. U.S. Department of Education. (2010). Overview information; Race to the Top Fund Assessment Program; Notice inviting applications for new awards for fiscal year (FY) 2010. Federal Register, 75, 18171-18185.
42. von Davier, M., Gonzalez, E., & Mislevy, R. J. (2009). What are plausible values and why are they useful? IERI Monograph Series. Issues and Methodologies Large-Scale Assessments, 2, 9-36.
43. von Davier, M., Sinharay, S., Oranje, A., & Beaton, A. E. (2006). Statistical procedures used the National Assessment of Educational Progress (NAEP): Recent developments and future directions. C. R. Rao & S. Sinharay (Eds.), Handbook of statistics: Vol. 26. Psychometrics (pp. 1039-1056). Amsterdam, The Netherlands : Elsevier.
44. von Davier, M., Xu, X., & Carstensen, C. H. (2011). Measuring growth a longitudinal large-scale assessment with a general latent variable model. Psychometrika, 76, 318-336.
45. Wijnstra, J. M., & Eggen, T. J. H. M. (1987). Design of the Dutch national assessment program education. PPON Werkdocument nr 4. Arnhem, The Netherlands: Cito.
46. Wilson, M. R. (1989). Saltus: A psychometric model of discontinuity cognitive development. Psychological Bulletin, 105, 276-289.
47. Wilson, M. R., & McGuire, L.W. (2010, September). Estimating the latent growth item response model. Paper presented at the International Conference on Outcome Measurement, Bethesda, MD .
48. Wingersky, M., Kaplan, B. A., & Beaton, A. E. (1987). Joint estimation procedures. A. E. Beaton (Ed.), The NAEP 1983/84 Technical Report (NAEP Report 15-TR-20) (pp. 285-292). Princeton, NJ: Educational Testing Service.
49. Wise, L. L. (2011). Picking up the pieces: Aggregating results from through-course assessments. Princeton, NJ : Center for K - 12 Assessment & Performance Management, Educational Testing Service.
50. Wu, M. L. (2005). The role of plausible values large-scale surveys. Studies Educational Evaluation, 31, 114-128.
51. Zwick, R. (1991). Effects of item order and context on estimation of NAEP reading proficiency. Educational Measurement: Issues and Practice, 10(3), 10-16.
52. Zwick, R. (2010). Measurement issues state achievement comparisons (ETS Research Report No. RR-10-19). Princeton, NJ : Educational Testing Service.