An examination of the precision and technical accuracy of the first wave of group-randomized trials funded by the institute of education sciences

Educational Evaluation and Policy Analysis

Volumn 31, Issue 3, 2009, Pages 298-318

  Spybrook, Jessaca ^a   Raudenbush, Stephen W ^b  

^a WESTERN MICHIGAN UNIVERSITY   (United States)

^b UNIVERSITY OF CHICAGO   (United States)

Author keywords

Experimental design; Randomized trial; Statistical power

Indexed keywords

EID: 74349087924     PISSN: 01623737     EISSN: 19351062     Source Type: Journal    
DOI: 10.3102/0162373709339524     Document Type: Article

References (33)

1. Bloom, H.S. (2005). Randomizing groups to evaluate place-based programs. In H. S. Bloom (Ed.), Learning more from social experiments: Evolving analytic approaches (pp. 115-172). New York: Russell Sage Foundation.
2. Bloom, H.S., Bos, J.M., & Lee, S.W. (1999). Using cluster random assignment to measure program impacts: Statistical implications for the evaluation of education programs. Evaluation Review, 23(4), 445-469.
3. Bloom, H.S., & Riccio, J.A. (2002). Using place-based random assignment and comparative interrupted time-series analysis to evaluate the jobs-plus employment program for public housing residents. New York: MDRC.
4. Bloom, H.S., Richburg-Hayes, L., & Black, A.R. (2007). Using covariates to improve precision: Empirical guidance for studies that randomize schools to measure the impacts of educational interventions. Educational Evaluation and Policy Analysis, 29(1), 30-59.
5. Borman, G., Slavin, R.E., Cheung, A., Chamberlain, A., Madden, N.A., & Chambers, B. (2005). Success for all: First year results from the national randomized field trial. Educational Evaluation and Policy Analysis, 27(1), 1-22.
6. Boruch, R.F. (2005). Better evaluation for evidence-based policy: Place randomized trials in education, criminology, welfare, and health. The Annals of American Academy of Political and Social Science, 599.
7. Boruch, R.F., & Foley, E. (2000). The honestly experimental society. In L. Bickman (Ed.), Validity and social experiments: Donald Campbell's legacy (pp. 193-239). Thousand Oaks, CA: Sage.
8. Boruch, R.F., May, H., Turner, H., Lavenberg, J., Petrosino, A., de Moya, D.,et al.(2004). Estimating the effects of interventions that are deployed in many places: Place-randomized trials. American Behavioral Scientist, 47, 608-633.
9. Cohen, J. (1988). Statistical power analyses for the behavioral sciences. Hillsdale, NJ: Lawrence Erlbaum.
10. Cook, T.D. (2005). Emergent principles for the design, implementation, and analysis of cluster-based experiments in social science. The Annals of American Academy of Political and Social Science, 599.
11. Cook, T.D., & Payne, M. (2002). Objecting to the objections to using random assignment. In F. Mosteller & R. Boruch (Eds.), Evidence matters: Randomized field trials in education research (pp. 150-178). Washington, DC: Brookings Institution.
12. Donner, A., & Klar, N. (2000). Design and analysis of cluster randomization trials in health research. London: Arnold.
13. Flay, B.R., & Collins, L. (2005). Historical review of school-based randomized trials for evaluation problem behavior prevention programs. The Annals of the American Academy of Political and Social Science, 599, 115-146.
14. Grimshaw, J., Eccles, M., Campbell, M., & Elbourne, D. (2005). Cluster randomized trials of professional and organizational behavior change interventions in health care research. The Annals of the American Academy of Political and Social Science, 599, 71-93.
15. Hedges, L., & Hedberg, E.C. (2007). Intraclass correlation values for planning group-randomized trials in education. Educational Evaluation and Policy Analysis, 29(1), 60-87.
16. Institute of Education Sciences. Retrieved October 9, 2006, from http://www.Ed.gov/about/offices/list/ies/index.html.
17. Kirk, R.E. (1982). Experimental design: Procedures for the behavioral sciences. Belmont, CA: Brooks/ Cole.
18. Leviton, L.C., & Horbar, J.D. (2005). Cluster randomized trials for the evaluation of strategies designed to promote evidence-based practice in perinatal and neonatal medicine. The Annals of the American Academy of Political and Social Science, 599, 94-114.
19. Liu, X., Spybrook, J., Congdon, R., Martinez, A., & Raudenbush, S.W. (2006). Optimal design for longitudinal and multilevel research [V1.83]. Lincolnwood, IL: HLM Software.
20. Mathematica Policy Research, Inc. (2006). The national evaluation of reading comprehension interventions: Design report. Princeton, NJ: Author.
21. Murray, D.M. (1998). Design and analysis of group-randomized trials. New York: Oxford University Press.
22. Murray, D.M., & Blitstein, J.L. (2003). Methods to reduce the impact of intraclass correlation in group-randomized trials. Evaluation Review, 27(1), 79-103.
23. Murray, D.M., & Short, B. (1995). Intra-class correlation among measures related to alcohol use by young adults: Estimates, correlates, and applications in intervention studies. Journal of Studies on Alcohol, 56(6), 681-694.
24. Raudenbush, S.W. (1997). Statistical analysis and optimal design for cluster randomized trials. Psychological Methods, 2(2), 173-185.
25. Raudenbush, S.W., & Bryk, A.S. (2002). Hierarchical linear models: Applications and data analysis methods. Thousand Oaks, CA: Sage.
26. Raudenbush, S.W., Martinez, A., & Spybrook, J. (2007). Strategies for improving precision in group-randomized experiments. Educational Evaluation and Policy Analysis, 29(1), 5-29.
27. Schochet, P. (2008). Statistical power for random assignment evaluations of education programs. Journal of Educational and Behavioral Statistics, 33(1), 62-87.
28. Shadish, W.R., Cook, T.D., & Campbell, T.D. (2002). Experimental and quasi-experimental designs for generalized causal inference. Boston: Houghton Mifflin.
29. Siddiqui, O., Hedeker, D., Flay, B.R., & Hu, F.B. (1996). Intraclass correlation estimates in school-based smoking prevention study: Outcome and mediating variables, by sex and ethnicity. American Journal of Epidemiology, 144(4), 425-433.
30. Sikkema, K.J. (2005). HIV prevention among women in low-income housing developments: issues and intervention outcomes in a place-based randomized controlled trial. The Annals of the American Academy of Political and Social Science, 599, 52-7.
31. Teruel, G.M., & Davis, B. (2000). Final report: An evaluation of the impact of PROGRESA cash payments on private inter-household transfers. Washington, DC: International Food Policy Research Institute.
32. Ukoumunne, O.C., Gulliford, M.C., Chinn, S., Sterne, A.C., & Burney, P.F. (1999). Methods for evaluating area-wide and organisation-based interventions in health and health care: A systematic review. Health Technology Assessment, 3(5), 1-99.
33. Weisburd, D. (2005). Hot spots policing experiments and criminal justice research: Lessons from the field. The Annals of the American Academy of Political and Social Science, 599, 220-245.