New empirical evidence for the design of group randomized trials in education

Journal of Research on Educational Effectiveness

Volumn 3, Issue 2, 2010, Pages 157-198

  Jacob, Robin ^a   Zhu, Pei ^b   Bloom, Howard ^b  

^a UNIVERSITY OF MICHIGAN   (United States)

^b MDRC   (United States)

Author keywords

Group randomized trials; Intraclass correlations; R2; Study design

Indexed keywords

EID: 79960912051     PISSN: 19345747     EISSN: 19345739     Source Type: Journal    
DOI: 10.1080/19345741003592428     Document Type: Article

References (33)

1. Bloom, H. S. (2005). Randomizing groups to evaluate place-based programs. In H. S. Bloom (Ed.), Learning more from social experiments: Evolving analytic approaches (pp. 115-172). New York: Russell Sage Foundation.
2. Bloom, H. S., Richburg-Hayes, L., & Black, A. (2007). Using covariates to improve precision for studies that randomize schools to evaluate educational interventions. Educational Evaluation and Policy Analysis, 29, 30-59.
3. Carrodus, M. L., & Giles, D. E. A. (1992). The exact distribution of R2 when the regression disturbances are autocorrelated. Economics Letters, 38, 375-380.
4. Conners, C. K. (2000). Conners' Rating Scales-Revised Technical Manual. North Tonawanda, New York: Multi Health Systems.
5. Detterman, D. (1988). Cognitive Abilities Tests. Cleveland, OH: Department of Psychology, Case Western Reserve University.
6. Fisher, R. A. (1925). Statistical methods for research workers. Edinburgh, Scotland: Oliver and Boyd.
7. Garet, M. S., Cronen, S., Eaton, M., Kurki, A., Ludwig, M., Jones, W., et al. (2008). The impact of two professional development interventions on early reading instruction and achievement (NCEE 2008-4030). Washington, DC: National Center for Education Evaluation and Regional Assistance, Institute of Education Sciences, U.S. Department of Education.
8. Hedberg, E. C., Santana, R., & Hedges, L. (2004, April). The variance structure of academic achievement in America. Paper presented at the 2004 annual meeting of the American Educational Research Association.
9. Hedges, L. (2004). Correcting significance tests for clustering. Working paper, University of Chicago.
10. Hedges, L. V., & Hedberg, E. C. (2007). Intra-class correlation values for planning group-randomized trials in education. Educational Evaluation and Policy Analysis, 29, 60-87.
11. Helland, I. S. (1987). On the interpretation and use of R2 in regression analysis. Biometrics, 43, 61-69.
12. Hill, C. J., Bloom, H. S., Black, A. R., & Lipsey, M. W. (2008). Empirical benchmarks for interpreting effect sizes in research. Child Development Perspectives, 2(3), 172-177.
13. Lonigan, C. J., Wagner, R. K., Torgesen, J., & Rashotle, C. (2002). Preschool Comprehensive Test of Phonological and Print Processing. Florida, State University.
14. Moerbeek, M. (2004). The consequence of ignoring a level of nesting in multilevel analysis. Multivariate Behavioral Research, 39, 129-149.
15. Murphy, J.M., Wehler, C., Pagano, M., Little, M., Kleinman, R., & Jellinek, M. (1998). Relationship between hunger and psychosocial functioning in low-income American children. Journal of the American Academy of Child and Adolescent Psychiatry, 37(2), 163-170.
16. Murray, D. M., & Blitstein, J. L. (2004). Methods to reduce the impact of intra-class correlation in group-randomized trials. Evaluation Review, 27(1), 79-103.
17. Murray, D. M., & Short, B. (1995). Intra-class correlation among measures related to alcohol use by young adults: Estimates, correlates and applications in intervention studies. Journal of Studies on Alcohol, 56(6), 681-694.
18. Ohtani, K. (2000). Bootstrapping R2 and adjusted R2 in regression analysis. Economic Modelling, 17, 473-483.
19. Press, S. J., & Zellner, A. (1978). Posterior distribution for the multiple correlation coefficient with fixed regressors. Journal of Econometrics, 8, 307-321.
20. Raudenbush, S.W. (1997). Statistical analysis and optimal design for group randomized trials. Psychological Methods, 2(2), 173-185.
21. Raudenbush, S.W., & Bryk, A. S. (2002). Hierarchical linear models: Applications and data analysis methods. Newbury Park, CA: Sage.
22. Raudenbush, S. W., Martinez, A., & Spybrook, J. (2007). Strategies for improving precision in group-randomized experiments. Educational Evaluation and Policy Analysis, 29, 5-29.
23. Reardon, S. F. (2007). Thirteen ways of looking at the black-white achievement gap. Unpublished manuscript, Graduate School of Education, Stanford University, Standford, CA.
24. Rothbart, M. K., Ahadi, S. A., Hershey, K. L., & Fisher, P. (2001). Investigations of temperament at 3-7 years: The Children's Behavior Questionnaire. Child Development, 72, 1394-1408.
25. Schochet, P. (2008). Statistical Power for Random Assignment Evaluations of Education Programs. Journal of Educational and Behavioral Statistics, 33(1), 62-87.
26. Schochet, P. A. (2005). Statistical power for random assignment evaluations of education programs. Princeton, NJ: Mathematica Policy Research.
27. Siddiqui, O., Hedeker, D., Flay, B. R., & Hu, F. B. (1996). Intra-class correlation estimates in a school-based smoking prevention study: Outcome and mediating variables by gender and ethnicity. American Journal of Epidemiology, 144, 425-433.
28. Simeon, D. T., & Grantham-McGregor, S. (1989). Effects of missing breakfast on the cognitive functions of school children with differing nutritional status. American Journal of Clinical Nutrition, 49, 646-653.
29. Spybrook, J. H. (2007). The statistical power of group randomized trials funded by the Institute of Education Sciences. Unpublished doctoral dissertation, University of Michigan School of Education, Ann Arbor, MI.
30. Ukoumunne, O. C., Gulliford, M. C., Chinn, S., Sterne, J. A. C., & Burney, P. F. J. (1999). Methods for evaluating area-wide and organisation-based interventions in health and health care: A systematic review. Health Technology Assessment, 3(5), 1-99.
31. U.S. Department of Agriculture, Food and Nutrition Service. (2004). Evaluation of the School Breakfast Program Pilot Project: Final report. (Nutrition Assistance Programs Report Series CN-04-SBP). Washington, DC: Author.
32. Visscher, P. M. (1998). On the sampling variance of intra-class correlations and genetic correlations. Genetics, 149, 1605-1614.
33. Wechsler, D. (1991). WISC-III manual. San Antonio, TX: Psychological Corporation, Harcourt Brace.