Meta-Analysis With Complex Research Designs: Dealing With Dependence From Multiple Measures and Multiple Group Comparisons

Review of Educational Research

Volumn 84, Issue 3, 2014, Pages 328-364

  Scammacca, Nancy ^a   Roberts, Greg ^a   Stuebing, Karla K ^b  

^a UNIVERSITY OF TEXAS AT AUSTIN   (United States)

^b UNIVERSITY OF HOUSTON   (United States)

Author keywords

heterogeneity analysis; meta analysis; statistical dependence

Indexed keywords

EID: 84905098291     PISSN: 00346543     EISSN: None     Source Type: Journal    
DOI: 10.3102/0034654313500826     Document Type: Article

References (40)

1. Ahn S.,Ames A. J.,Myers N. D.A review of meta-analyses in education: Methodological strengths and weaknesses.Review of Educational Research. 2012;82:436-476.
2. Reporting standards for research in psychology: Why do we need them? What might they be?.American Psychologist. 2008;63:839-851.
3. Becker B. J.Handbook of applied multivariate statistics and mathematical modeling. Tinsley H. A.Brown S. D., ed. San Diego, CA: Academic Press; 2000:499-525.
4. Berkeley S.,Scruggs T. E.,Mastropieri M. A.Reading comprehension instruction for students with learning disabilities, 1995-2006: A meta-analysis.Remedial and Special Education. 2010;31:423-436.
5. Borenstein M.,Hedges L. V.,Higgins J. P. T.,Rothstein H. R.Introduction to meta-analysis. Borenstein M.Hedges L. V.Higgins J. P. T.Rothstein H. R., ed. Chichester, England: John Wiley; 2009a:215-216.
6. Borenstein M.,Hedges L. V.,Higgins J. P. T.,Rothstein H. R.Introduction to meta-analysis. Borenstein M.Hedges L. V.Higgins J. P. T.Rothstein H. R., ed. Chichester, England: John Wiley; 2009b:225-238.
7. Borenstein M.,Hedges L. V.,Higgins J. P. T.,Rothstein H. R.Introduction to meta-analysis. Borenstein M.Hedges L. V.Higgins J. P. T.Rothstein H. R., ed. Chichester, England: John Wiley; 2009c:239-242.
8. Borenstein M.,Hedges L. V.,Higgins J. P. T.,Rothstein H. R.Comprehensive meta analysis. Englewood, NJ: Biostat; 2011:.
9. Card N.Applied meta-analysis for social science research. New York, NY: Guilford; 2012:.
10. Chambers E. A.An introduction to meta-analysis with articles from the Journal of Educational Research (1992-2002).Journal of Educational Research. 2004;98:35-44.
11. Cheung M. W. L.Fixed-effects meta-analyses as multiple-group structural equation models.Structural Equation Modeling. 2010;17:481-509.
12. CheungM. W. L. (2013). Modeling dependent effect sizes with three-level meta-analyses: A structural equation modeling approach. Psychological Methods. Advance online publication. doi:10.1037/a0032968.
13. Cooper H.Synthesizing research: A guide for literature reviews. 1998). Synthesizing research: A guide for literature reviews (3rd ed.). Thousand Oaks, CA: Sage.Thousand Oaks, CA: Sage; 1998:.
14. D'Agostino J.,Murphy J.A meta-analysis of Reading Recovery in United States schools.Educational Evaluation and Policy Analysis. 2004;26:23-38.
15. de Vibe M.,Bjørndal A.,Tipton E.,Hammerstrøm K.,Kowalski K.Mindfulness based stress reduction (MBSR) for improving health, quality of life, and social functioning in results.Campbell Systematic Reviews. 2012;3:.
16. Dieckmann N. F.,Malle B. F.,Bodner T. E.An empirical assessment of meta-analytic practice.Review of General Psychology. 2009;13:101-115.
17. Edmonds M. S.,Vaughn S.,Wexler J.,Reutebuch C. K.,Cable A.,Tackett K. K.A synthesis of reading interventions and effects on reading outcomes for older struggling readers.Review of Educational Research. 2009;79:262-300.
18. Flynn L. J.,Zheng X.,Swanson H. L.Instructing struggling older readers: A selective meta-analysis of intervention research.Learning Disabilities Research & Practice. 2012;27:21-32.
19. Gersten R.,Chard D. J.,Jayanthi M.,Baker S. K.,Morphy P.,Flojo J.Mathematics instruction for students with learning disabilities: A meta-analysis of instructional components.Review of Educational Research. 2009;79:1202-1242.
20. Gleser L. J.,Olkin I.The handbook of research synthesis. Cooper H.Hedges L. V., ed. New York, NY: Russell Sage Foundation; 1994:339-355.
21. Graham S.,Hebert M.Writing to read: A meta-analysis of the impact of writing and writing instruction on reading.Harvard Educational Review. 2011;81:710-744.
22. Hedges L. V.Distribution theory for Glass's estimator of effect size and related estimators.Journal of Education Statistics. 1981;6:107-128.
23. Hedges L. V.,Tipton E.,Johnson M. C.Robust variance estimation in meta-regression with dependent effect size estimates.Research Synthesis Methods. 2010;1:39-65.
24. Institute of Education Sciences. (2008). What works clearinghouse study review standards. Retrieved from http://ies.ed.gov/ncee/wwc/pdf/reference_resources/wwc_version1_standards.pdf.
25. Kalaian H. A.,Raudenbush S. W.A multivariate mixed linear model for meta-analysis.Psychological Methods. 1996;1:227-235.
26. Kim R.,Becker B.The degree of dependence between multiple-treatment effect sizes.Multivariate Behavioral Research. 2010;45:213-238.
27. Konstantopoulos S.Fixed effects and variance components estimation in three-level meta-analysis.Research Synthesis Methods. 2011;2:61-76.
28. Marín-Martínez F.,Sánchez-Meca J.Averaging dependent effect sizes in meta-analysis: A cautionary note about procedures.Spanish Journal of Psychology. 1999;2:32-38.
29. Peabody Research Institute. (n.d.). Methods resources. Retrieved from http://peabody.vanderbilt.edu/research/pri/methods_resources.php.
30. Raudenbush S. W.,Becker B. J.,Kalaian H.Modeling multivariate effect sizes.Psychological Bulletin. 1988;103:111-120.
31. Rosenthal R.,Rubin D. B.Meta-analytic procedures for combining studies with multiple effect sizes.Psychological Bulletin. 1986;99:400-406.
32. Samson J. E.,Ojanen T.,Hollo A.Social goals and youth aggression: Meta-analysis of prosocial and antisocial goals.Social Development. 2012;21:645-666.
33. Scammacca N.,Roberts G.,Vaughn S.,Edmonds M.,Wexler J.,Reutebuch C. K., Torgesen J. K.Reading interventions for adolescent struggling readers: A meta-analysis with implications for practice. Portsmouth, NH: RMC Research Corporation, Center on Instruction; 2007:.
34. Scammacca N.,Roberts G.,Vaughn S.,Stuebing K.A meta-analysis of interventions for struggling readers in Grades 4-12: 1980-2011. Journal of Learning Disabilities.:.
35. Tanner-Smith E. E.,Tipton E.Robust variance estimation with dependent effect sizes: Practical considerations including a software tutorial in Stata and SPSS. 2013;:.
36. Tanner-Smith E. E.,Wilson S. J.,Lipsey M. W.The comparative effectiveness of outpatient treatment for adolescent substance abuse: A meta-analysis.Journal of Substance Abuse Treatment. 2013;44:145-158.
37. Tran L.,Sanchez T.,Arellano B.,Swanson H. L.A meta-analysis of the RTI literature for children at risk for reading disabilities.Journal of Learning Disabilities. 2011;44:283-295.
38. Uttal D. H.,Meadow N. G.,Tipton E.,Hand L. L.,Alden A. R.,Warren C.,Newcombe N. S.The malleability of spatial skills: A meta-analysis of training studies.Psychological Bulletin. 2013;139:352-402.
39. Van den Noortgate W.,López-López J. A.,Marín-Martínez F.,Sánchez-Meca J.Three-level meta-analysis of dependent effect sizes.Behavior Research Methods. 2013;45:576-594.
40. Wilson S. J.,Tanner-Smith E. E.,Lipsey M. W.,Steinka-Fry K.,Morrison J.Dropout prevention and intervention programs: Effects on school completion and dropout among school-aged children and youth.Campbell Systematic Reviews. 2011;8:.