Reference Values of Within-District Intraclass Correlations of Academic Achievement by District Characteristics: Results From a Meta-Analysis of District-Specific Values

Evaluation Review

Volumn 38, Issue 6, 2014, Pages 546-582

  Hedberg, E C ^a,b   Hedges, Larry V ^c  

^a ARIZONA STATE UNIVERSITY   (United States)

^b University of Chicago   (United States)

^c NORTHWESTERN UNIVERSITY   (United States)

Author keywords

education; methodological development

Indexed keywords

EDUCATIONAL STATUS; HUMAN; META ANALYSIS; ORGANIZATION AND MANAGEMENT; RANDOMIZED CONTROLLED TRIAL (TOPIC); REFERENCE VALUE; SCHOOL; STANDARDS; STATISTICS AND NUMERICAL DATA; UNITED STATES;

EDUCATIONAL STATUS; HUMANS; RANDOMIZED CONTROLLED TRIALS AS TOPIC; REFERENCE VALUES; SCHOOLS; UNITED STATES;

EID: 84919679646     PISSN: 0193841X     EISSN: 15523926     Source Type: Journal    
DOI: 10.1177/0193841X14554212     Document Type: Article

References (51)

1. Agodini R.,Harris B.,Thomas M.,Murphy R.,Gallagher L.Achievement effects of four early elementary school math curricula: Findings for first and second graders (NCEE 2011–4001). Washington, DC: U.S. Department of Education, Institute of Education Sciences, National Center for Education Evaluation and Regional Assistance; 2010:
2. Bloom H. S.Learning more from social experiments: Evolving analytic approaches. New York, NY: Russell Sage Foundation; 2005:
3. Bloom H. S.,Bos J. M.,Lee S.-W.Using cluster random assignment to measure program impacts statistical implications for the evaluation of education programs.Evaluation Review. 1999;23:445-469
4. Bloom H. S.,Richburg-Hayes L.,Black A. R.Using covariates to improve precision for studies that randomize schools to evaluate educational interventions.Educational Evaluation and Policy Analysis. 2007;29:30-59
5. Bloom H. S.,Zhu P.,Jacob R.,Raudenbush S.,Martinez A.,Lin F.Empirical issues in the design of group-randomized studies to measure the effects of interventions for children. New York, NY: MDRC; 2008:
6. Borenstein M.,Hedges L.,Rothstein H.CRT-Power. Teaneck, NJ: Biostat; 2012:
7. BorensteinM.HedgesL. V.HigginsJ. P.RothsteinH. R. (2011). Introduction to meta-analysis. Wiley.com.
8. Bottge B. A.,Grant T. S.,Stephens A. C.,Rueda E.Advancing the math skills of middle school students in technology education classrooms.NASSP Bulletin. 2010;94:81-106
9. Bradshaw C. P.,Mitchell M. M.,Leaf P. J.Examining the effects of schoolwide positive behavioral interventions and supports on student outcomes results from a randomized controlled effectiveness trial in elementary schools.Journal of Positive Behavior Interventions. 2010;12:133-148
10. Brandon P. R.,Harrison G. M.,Lawton B. E.SAS code for calculating intraclass correlation coefficients and effect size benchmarks for site-randomized education experiments.American Journal of Evaluation. 2013;34:85-90
11. Calderón M.,Slavin R.,Sánchez M.Effective instruction for English learners.The Future of Children. 2011;21:103-127
12. Fantuzzo J. W.,Gadsden V. L.,McDermott P. A.An integrated curriculum to improve mathematics, language, and literacy for Head Start children.American Educational Research Journal. 2011;48:763-793
13. Fisher R. A.Statistical methods for research workers. New Delhi, India: Genesis; 1925:
14. Fulmer S. M.,Frijters J. C.Motivation during an excessively challenging reading task: The buffering role of relative topic interest.The Journal of Experimental Education. 2011;79:185-208
15. Gersten R.,Dimino J.,Jayanthi M.,Kim J. S.,Santoro L. E.Teacher study group impact of the professional development model on reading instruction and student outcomes in first grade classrooms.American Educational Research Journal. 2010;47:694-739
16. Goodson B.,Wolf A.,Bell S.,Turner H.,Finney P. B.,Garcia S.The Effectiveness of a Program to Accelerate Vocabulary Development in Kindergarten (VOCAB).: First grade follow-up impact report and exploratory analyses of kindergarten impacts. Washington, DC: U.S. Department of Education, Institute of Education Sciences, National Center for Education Evaluation and Regional Assistance; 2011:
17. Hamre B. K.,Pianta R. C.,Burchinal M.,Field S.,LoCasale-Crouch J.,Downer J. T.A course on effective teacher-child interactions effects on teacher beliefs, knowledge, and observed practice.American Educational Research Journal. 2012;49:88-123
18. Hayes R. J.,Moulton L. H.,Press C.Cluster randomised trials. London, England: CRC press; 2009:
19. Hedberg E. C.RDPOWER: Stata module to perform power calculations for random designs. Statistical Software Components. 2012:
20. Hedges L. V.,Hedberg E. C.Intraclass correlation values for planning group-randomized trials in education.Educational Evaluation and Policy Analysis. 2007a;29:60-87
21. Hedges L. V.,Hedberg E. C.Intraclass correlations for planning group randomized experiments in rural education.Journal of Research in Rural Education. 2007b;22:22-10
22. HedgesL. V.HedbergE. C. (2011). Variance Almanac of Academic Achievement 2013. Retrieved fromhttps://arc.uchicago.edu/reese/variance-almanac-academic-achievement
23. Hedges L. V.,Hedberg E. C.Intraclass Correlations and Covariate Outcome Correlations for Planning Two and Three-Level Cluster-Randomized Experiments in Education.Evaluation review. 2014;37 (6): 445-489
24. Hedges L. V.,Rhoads C. H.Correcting an analysis of variance for clustering.British Journal of Mathematical and Statistical Psychology. 2011;64:20-37
25. Hedges L. V.,Vevea J. L.Fixed-and random-effects models in meta-analysis.Psychological Methods. 1998;3:486
26. Isenberg E.,Glazerman S.,Bleeker M.,Johnson A.,Lugo-Gil J.,Grider M.Impacts of comprehensive teacher induction: Results from the Second Year of a Randomized Controlled Study. NCEE 2009-4072. Washington, DC: U.S. Department of Education, National Center for Educational Evaluation and Regional Assistance, Institute of Education Sciences; 2009 2008:
27. Keaton P.Documentation to the NCES Common Core of Data Public Elementary/Secondary School Universe Survey: School Year 2010–11 (NCES 2012-338rev) U.S. Department of Education. Washington, DC: National center for Education Statistics; 2012:
28. Kim J. S.,Capotosto L.,Hartry A.,Fitzgerald R.Can a mixed-method literacy intervention improve the reading achievement of low-performing elementary school students in an after-school program? Results from a randomized controlled trial of READ 180 enterprise.Educational Evaluation and Policy Analysis. 2011;33:183-201
29. Lane K. L.,Harris K.,Graham S.,Driscoll S.,Sandmel K.,Morphy P.Self-regulated strategy development at tier 2 for second-grade students with writing and behavioral difficulties: A randomized controlled trial.Journal of Research on Educational Effectiveness. 2011;4:322-353
30. Laura B.,McMeeking S.,Orsi R.,Cobb R. B.Effects of a teacher professional development program on the mathematics achievement of middle school students.Journal for Research in Mathematics Education. 2012;43:159-181
31. Marley S. C.,Levin J. R.,Glenberg A. M.What cognitive benefits does an activity-based reading strategy afford young native American readers?.The Journal of Experimental Education. 2010;78:395-417
32. Marley S. C.,Szabo Z.,Levin J. R.,Glenberg A. M.Investigation of an activity-based text-processing strategy in mixed-age child dyads.The Journal of Experimental Education. 2011;79:340-360
33. McQuillin S.,Smith B.,Strait G.Randomized evaluation of a single semester transitional mentoring program for first year middle school students: A cautionary result for brief, school-based mentoring programs.Journal of Community Psychology. 2011;39:844-859
34. Olson C. B.,Kim J. S.,Scarcella R.,Kramer J.,Pearson M.,van Dyk D. A.Enhancing the interpretive reading and analytical writing of mainstreamed English learners in secondary school results from a randomized field trial using a cognitive strategies approach.American Educational Research Journal. 2012;49:323-355
35. Phelan J.,Choi K.,Vendlinski T.,Baker E.,Herman J.Differential improvement in student understanding of mathematical principles following formative assessment intervention.The Journal of Educational Research. 2011;104:330-339
36. Raudenbush S. W.Statistical analysis and optimal design for cluster randomized trials.Psychological Methods. 1997;2:173-185
37. Reis S. M.,McCoach D. B.,Little C. A.,Muller L. M.,Kaniskan R. B.The effects of differentiated instruction and enrichment pedagogy on reading achievement in five elementary schools.American Educational Research Journal. 2011;48:462-501
38. Rose R. A.,Woolley M. E.,Orthner D. K.,Akos P. T.,Jones-Sanpei H. A.Increasing teacher use of career-relevant instruction a randomized control trial of careerstart.Educational Evaluation and Policy Analysis. 2012;34:295-312
39. Sarama J.,Clements D. H.,Wolfe C. B.,Spitler M. E.Longitudinal evaluation of a scale-up model for teaching mathematics with trajectories and technologies.Journal of Research on Educational Effectiveness. 2012;5:105-135
40. Schochet P. Z.Statistical power for random assignment evaluations of education programs.Journal of Educational and Behavioral Statistics. 2008;33:62-87
41. Slavin R. E.,Cheung A.,Holmes G.,Madden N. A.,Chamberlain A.Effects of a data-driven district reform model on state assessment outcomes.American Educational Research Journal. 2012;:
42. Springer M. G.,Pane J. F.,Le V.-N.,McCaffrey D. F.,Burns S. F.,Hamilton L. S.,Stecher B.Team pay for performance experimental evidence from the round rock pilot project on team incentives.Educational Evaluation and Policy Analysis. 2012;34:367-390
43. Spybrook J.,Puente A. C.,Lininger M.An examination of the impact of changes in federal policies on the landscape of educational research in the USA.Effective Education. 2011;3:83-88
44. Spybrook J.,Raudenbush S. W.An examination of the precision and technical accuracy of the first wave of group-randomized trials funded by the institute of education sciences.Educational Evaluation and Policy Analysis. 2009;31:298-318
45. Spybrook J.,Raudenbush S. W.,Liu X.-F.,Congdon R.,Martínez A.Optimal design for longitudinal and multilevel research: Documentation for the “Optimal Design” software. Ann Arbor, MI: Survey Research Center of the Institute of Social Research at University of Michigan; 2006:
46. VanDerHeyden A.,McLaughlin T.,Algina J.,Snyder P.Randomized evaluation of a supplemental grade-wide mathematics intervention.American Educational Research Journal. 2012;49:1251-1284
47. Vaughn S.,Klingner J. K.,Swanson E. A.,Boardman A. G.,Roberts G.,Mohammed S. S.,Stillman-Spisak S. J.Efficacy of collaborative strategic reading with middle school students*.American Educational Research Journal. 2011;48:938-964
48. Vaughn S.,Wexler J.,Roberts G.,Barth A. A.,Cirino P. T.,Romain M. A.Effects of individualized and standardized interventions on middle school students with reading disabilities.Exceptional children. 2011;77:391-407
49. Westine C. D.,Spybrook J.,Taylor J. A.An empirical investigation of variance design parameters for planning cluster-randomized trials of science achievement.Evaluation Review. 2014;:
50. Wirkala C.,Kuhn D.Problem-based learning in K–12 education is it effective and how does it achieve its effects?.American Educational Research Journal. 2011;48:1157-1186
51. Wolf P.,Gutmann B.,Puma M.,Kisida B.,Rizzo L.,Eissa N.,Carr M.Evaluation of the DC opportunity scholarship program: Final report. NCEE 2010-4018. Washington, DC: National Center for Education Evaluation and Regional Assistance; 2010: