International large-scale assessment data: Issues in secondary analysis and reporting

Educational Researcher

Volumn 39, Issue 2, 2010, Pages 142-151

  Rutkowski, Leslie ^a   Gonzalez, Eugenio ^b   Joncas, Marc ^c   von Davier, Matthias ^b  

^a INDIANA UNIVERSITY   (United States)

^b EDUCATIONAL TESTING SERVICE   (United States)

^c STATISTICS CANADA   (Canada)

Author keywords

Assessment; International education studies; Statistics

Indexed keywords

EID: 77951601196     PISSN: 0013189X     EISSN: 1935102X     Source Type: Journal    
DOI: 10.3102/0013189X10363170     Document Type: Article

References (42)

1. AM Statistical Software [Manual]. (2009). Washington, DC: American Institutes for Research.
2. Asparouhov, T., & Muthén, B. (2006). Multilevel modelling of complex survey data. Proceedings of the Survey Research Methods Section, American Statistical Association 2006, 2718-2
3. Beaton, A., & Johnson, E. (1992). Overview of the scaling methodology used in the national assessment. Journal of Educational Measurement, 29, 163-175.
4. Brewer, K., & Hanif, M. (1983). Sampling with unequal probabilities, lecture notes in statistics (Vol. 15). New York: Springer-Verlag.
5. Chantala, K., & Suchindran, C. (2006). Adjusting for unequal selection probability in multilevel models: A comparison of software packages. Proceedings of the Survey Research Methods Section, American Statistical Association 2006, 2815-2824.
6. Foy, P., & Kennedy, A. (2008). PIRLS 2006 user guide for the international database. Chestnut Hill, MA: TIMSS & PIRLS International Study Center, Boston College.
7. Foy, P., & Olson, J. (Eds.). (2009). TIMSS 2007 international database and user guide. Chestnut Hill, MA: TIMSS & PIRLS International Study Center, Boston College.
8. IDB Analyzer (Version 2) [Computer software and manual]. (2009). Hamburg, Germany: International Association for the Evaluation of Educational Achievement.
9. Johnson, E., & Rust, K. (1992). Population inferences and variance estimation for NAEP data. Journal of Educational Statistics, 17, 175-190.
10. Judkins, D. (1990). Fay's method for variance estimation. Journal of Official Statistics, 6, 223-239.
11. Martin, M., Mullis, I., & Kennedy, A. (Eds.). (2007). PIRLS 2006 technical report. Chestnut Hill, MA: TIMSS & PIRLS International Study Center, Boston College.
12. Mislevy, R. (1991). Randomization-based inference about latent variables from complex samples. Psychometrika, 56, 177-196.
13. Mislevy, R., Beaton, A., Kaplan, B., & Sheehan, K. (1992). Estimating population characteristics from sparse matrix samples of item responses. Journal of Educational Measurement, 29, 133-161.
14. Mislevy, R., Johnson, E., & Muraki, E. (1992). Scaling procedures in NAEP. Journal of Educational Statistics, 17, 131-154.
15. Mullis, I., Kennedy, A., Martin, M., & Sainsbury, M. (2006). PIRLS assessment framework and specifications. Chestnut Hill, MA: TIMSS & PIRLS International Study Center, Boston College.
16. Mullis, I., Martin, M., Ruddock, G., O'Sullivan, C., Arora, A., & Erberber, E. (2005). TIMSS 2007 assessment frameworks. Chestnut Hill, MA: TIMSS & PIRLS International Study Center, Boston College.
17. Muthén, L., & Muthén, B. (1998-2007). Mplus user's guide (5th ed.). Los Angeles, CA: Muthén & Muthén.
18. National Center for Education Statistics. (n.d.). NAEP Data Explorer. In National Center for Education Statistics. Retrieved January 7, 2010, from http://nces.ed.gov/nationsreportcard/naepdata/
19. Olson, J., Martin, M., & Mullis, I. (Eds.). (2008). TIMSS 2007 technical report. Chestnut Hill, MA: TIMSS & PIRLS International Study Center, Boston College.
20. Organisation for Economic Co-operation and Development. (n.d.). Participating countries. Retrieved December 31, 2009, from http:http:// www.pisa.oecd.org/pages/0,3417,en_32252351_32236225_ 1_ 1_ 1_1_ 1,00.html
21. Organisation for Economic Co-operation and Development. (2005). PISA 2003 data analysis manual. Paris: Author.
22. Organisation for Economic Co-operation and Development. (2009). PISA 2006 technical report. Paris: Author.
23. Pfeffermann, C., Skinner, D., Holmes, H., Goldstein, H., & Rasbash, J. (1998). Weighting for unequal probabilities in multilevel models. Journal of the Royal Statistical Society, Series B (Statistical Methodology), 60, 23-40.
24. Rabe-Hesketh, S., & Skrondal, A. (2006). Multilevel modelling of complex survey data. Journal of the Royal Statistical Society. Series A (General), 169, 805-827.
25. Raudenbush, S., Bryk, A., Cheong, Y., & Congdon, R. (2004). HLM 6 [Manual]. Lincolnwood, IL: Scientific Software International.
26. Rubin, D. (1987). Multiple imputation for nonresponse in sample surveys. New York: John Wiley.
27. Rust, K. (1985). Variance estimation for complex estimators in sample surveys. Journal of Official Statistics, 1, 381-397.
28. SAS (Version 9) [Computer software and manual]. (2003). Cary, NC: SAS Institute.
29. Schafer, J. (1999). Multiple imputation: A primer. Statistical Methods in Medical Research, 8, 3-15.
30. Schafer, J. (2003). Multiple imputation in multivariate problems when the imputation and analysis models differ. Statistica Neerlandica, 57, 19-35.
31. SPSS (Version 16.0) [Computer software and manual]. (2007). Chicago: SPSS, Inc.
32. Stapleton, L. (2008). Variance estimation using replication methods in structural equation modeling with complex sample data. Structural Equation Modeling, 15, 183-210.
33. SUDAAN (Release 8.0) [Manual]. (2001). Research Triangle Park, NC: Research Triangle Institute.
34. Thomas, S., & Heck, R. (2001). Analysis of large-scale secondary data in higher education research: Potential perils associated with complex sampling designs. Research in Higher Education, 42, 517-540.
35. Thomas, S., Heck, R., & Bauer, K. (2005). Weighting and adjusting for design effects in secondary data analysis. New Directions for Institutional Research, 2005(127), 51-72.
36. TIMSS & PIRLS International Study Center. (n.d.). Countries. Retrieved December 29, 2009, from http://timss.bc.edu/pirls2001i/pirls2001_countries.html
37. TIMSS & PIRLS International Study Center. (2008a). Countries participating. Retrieved December 29, 2009, from http://timss.bc.edu/pirls2006/countries.html
38. TIMSS & PIRLS International Study Center. (2008b). Countries participating. Retrieved December 29, 2009, from http://timss.bc.edu/timss2003i/countries.html
39. TIMSS & PIRLS International Study Center. (2009). Countries participating. Retrieved December 29, 2009, from http://timss.bc.edu/TIMSS2007/countries.html
40. von Davier, M., Gonzalez, E., & Mislevy, R. (2009). Plausible values: What are they and why do we need them? IERI Monograph Series: Issues and Methodologies in Large-Scale Assessments, 2, 9-36.
41. von Davier, M., Sinharay, S., Oranje, A., & Beaton, A. (2006). The statistical procedures used in National Assessment of Educational Progress: Recent developments and future directions. In C. R. Rao & S. Sinharay (Eds.), Handbook of statistics: Vol. 26. Psychometrics (pp. 1039-1055). Amsterdam: Elsevier.
42. WesVar 4.3 [Manual]. (2007). Rockville, MD: Westat.