The Late Pretest Problem in Randomized Control Trials of Education Interventions

Journal of Educational and Behavioral Statistics

Volumn 35, Issue 4, 2010, Pages 379-406

  Schochet, Peter Z ^a  

^a MATHEMATICA POLICY RESEARCH   (United States)

Author keywords

causal inference theory; education interventions; late pretests; pretest posttest designs; randomized control trials

Indexed keywords

EID: 78649787248     PISSN: 10769986     EISSN: None     Source Type: Journal    
DOI: 10.3102/1076998609353112     Document Type: Article

References (27)

1. Allison, P. (1990). Change scores as dependent variables in regression analysis. In C. Clogg (Ed.), Sociological methodology (Vol. 20, pp. 93-114). Oxford, England: Blackwell.
2. Bloom, H., Hayes, L., & Black, A. (2005). Using covariates to improve precision. New York, NY: MDRC.
3. Cochran, W. (1963). Sampling techniques. New York, NY: Wiley.
4. Cohen, J. (1988). Statistical power analysis for behavioral sciences. Hillside, NJ: Erlbaum.
5. Cook, T.D., Wong, V.C., Taylor, J., Gandhi, A., Kendziora, K., Choi, K.,et al.(2009). Impacts of school improvement status on students with disabilities: Feasibility report. Washington, DC: American Institutes for Research.
6. Donner, A., & Klar, N. (2000). Design and analysis of cluster randomization trials in health research. London, England: Arnold.
7. Dynarski, M., & Agodini, R. (2003). The effectiveness of educational technology: issues and recommendations for the national study. Princeton, NJ: Mathematica Policy Research, Inc.
8. Decker, P., Glazerman, S., & Mayer, D. (2004). The effects of teach for America on students: Findings from a national evaluation. Princeton, NJ: Mathematica Policy Research, Inc.
9. Freedman, D. (2008). On regression adjustments to experimental data. Advances in Applied Mathematics, 40, 180-193.
10. Hedges, L., & Hedberg, E. (2007). Intraclass correlation values for planning group-randomized trials in education. Educational Evaluation and Policy Analysis, 29, 60-87.
11. Hill, C., Bloom, H., Black, A., & Lipsey, M. (2007). Empirical benchmarks for interpreting effect sizes in research. New York, NY: MDRC.
12. Holland, P. (1986). Statistics and causal inference. Journal of the American Statistical Association, 81, 945-960.
13. Imbens, G., & Rubin, D. (2007). Causal inference: Statistical methods for estimating causal effects in biomedical, social, and behavioral sciences. New York, NY: Cambridge University Press.
14. Jackson, R., McCoy, A., Pistorino, C., Wilkinson, A., Burghardt, J., & Clark, M... Schmidt, S.R. (2007). National evaluation of early reading first. Final report to Congress. Washington, DC: U.S. Department of Education, Institute of Education Sciences.
15. Kish, L. (1965). Survey sampling. New York, NY: Wiley.
16. Mayer, D., Peterson, P., Myers, D., Tuttle, C., & Howell, W. (2002). School choice in New York city: An evaluation of the school choice scholarships program. Washington, DC: Mathematica Policy Research, Inc.
17. Murray, D. (1998). Design and analysis of group-randomized trials. Oxford University Press.
18. Neyman, J. (1923/1990). On the application of probability theory to agricultural experiments: Essay on principles. Section 9. Translated by D. M. Dabrowska and T. P. Speed. Statistical Science 5, 465-472 (Original work published in 1923).
19. Oakes, J., & Feldman, H. (2001). Statistical power for nonequivalent pretest-posttest designs: The impact of change-score versus ANCOVA models. Evaluation Review, 25, 3-28.
20. Puma, M., Bell, S., Cook, R., Heidi, R.C., Lopez, M.,et al.(2005). Head start impact findings: First year findings. Final Report. Washington, DC: U.S. Department of Health and Human Services, Administration for Children and Families.
21. Rao, C. (1973). Linear statistical inference and its applications. New York, NY: Wiley.
22. Raudenbush, S. (1997). Statistical analysis and optimal design for cluster randomized trials. Psychological Methods, 2, 173-185.
23. Reichardt, C. (1979). Quasi-experimentation: Design and analysis for field settings. Boston, MA: Houghton Mifflin.
24. Rubin, D. (1974). Estimating causal effects of treatments in randomized and nonrandomized studies. Journal of Education Psychology, 66, 688-701.
25. Schochet, P. (2008). Statistical power for random assignment evaluations of education programs. Journal of Educational and Behavioral Statistics, 33, 62-87.
26. Schochet, P. (2009). Is regression adjustment supported by the Neyman model for causal inference? Journal of Statistical Planning and Inference, 140, 246-259.
27. Yang, L., & Tsiatis, A. (2001). Efficiency study of estimators for a treatment effects in a pretest-posttest trial. American Statistician, 55, 314-321.