Meta-analysis of multiple outcomes: a multilevel approach

Behavior Research Methods

Volumn 47, Issue 4, 2014, Pages 1274-1294

  Van den Noortgate, Wim ^a   López López, José Antonio ^b   Marín Martínez, Fulgencio ^c   Sánchez Meca, Julio ^c  

^a UNIVERSITY OF LEUVEN   (Belgium)

^b UNIVERSITY OF BRISTOL   (United Kingdom)

^c UNIVERSITY OF MURCIA   (Spain)

Author keywords

Dependence; Meta analysis; Multilevel; Multiple outcomes

Indexed keywords

COMPUTER SIMULATION; EPIDEMIOLOGY; HUMAN; META ANALYSIS (TOPIC); SAMPLE SIZE; STANDARD; STATISTICAL ANALYSIS; TREATMENT OUTCOME;

COMPUTER SIMULATION; DATA INTERPRETATION, STATISTICAL; HUMANS; META-ANALYSIS AS TOPIC; REFERENCE STANDARDS; SAMPLE SIZE; SAMPLING STUDIES; TREATMENT OUTCOME;

EID: 84947033036     PISSN: 1554351X     EISSN: 15543528     Source Type: Journal    
DOI: 10.3758/s13428-014-0527-2     Document Type: Article

References (32)

1. Ahn, S., Ames, A. J., & Myers, N. D. (2012). A review of meta-analyses in education: Methodological strengths and weaknesses. Review of Educational Research, 82, 436–476.
2. Arends, L. R., Voko, Z., & Stijnen, T. (2003). Combining multiple outcome measures in a meta-analysis: An application. Statistics in Medicine, 22, 1335–1353.
3. Becker, B. J. (2000). Multivariate meta-analysis. In H. E. A. Tinsley & E. D. Brown (Eds.), Handbook of applied multivariate statistics and mathematical modeling (pp. 499–525). Orlando, FL: Academic Press.
4. Borenstein, M., Hedges, L. V., Higgins, J. P. T., & Rothstein, H. R. (2010). A basic introduction to fixed-effect and random-effects models for meta-analysis. Research Synthesis Methods, 1, 97–111.
5. Cheung, M.W.-L. (2013). Modelling dependent effect sizes with three-level meta-analyses: A structural equation modelling approach. Psychological Methods. Advance online publication.
6. Cheung, S.F., & Chan, D.K.-S. (2014). Meta-analyzing dependent correlations: AnSPSS macro and an R script. Behavior Research, 46, 331–345.
7. Cohen, J. (1988). Statistical power analysis for the behavioral sciences (2nd ed.). Hillsdale, NJ: Lawrence Erlbaum.
8. Geeraert, L., Van den Noortgate, W., Grietens, H., & Onghena, P. (2004). The effects of early prevention programs for families with young children at risk for physical child abuse and neglect. A meta-analysis. Child Maltreatment, 9, 277–291.
9. Gleser, L. J., & Olkin, I. (1994). Stochastically dependent effect sizes. In H. Cooper & L. V. Hedges (Eds.), The handbook of research synthesis (pp. 339–355). New York: Russell Sage Foundation.
10. Hedges, L. V. (1981). Distribution theory for Glass's estimator of effect size and related estimators. Journal of Educational Statistics, 6, 107–128.
11. Hedges, L. V., Tipton, E., & Johnson, M. C. (2010). Robust variance estimation of meta-regression with dependent effect size estimates. Research Synthesis Methods, 1, 39–65.
12. Higgins, J. P. T., Thompson, S. G., & Spiegelhalter, D. J. (2009). A re-evaluation of random-effects meta-analysis. Journal of the Royal Statistical Society, Series A, 172, 137–159.
13. Hox, J. (2002). Multilevel analysis. Techniques and applications. Mahwah, NJ: Erlbaum.
14. Ishak, K. J., Platt, R. W., Joseph, L., & Hanley, J. A. (2008). Impact of approximating or ignoring within-study covariances in multivariate meta-analyses. Statistics in Medicine, 27, 670–686.
15. Jackson, D., Riley, R., & White, I. R. (2011). Multivariate meta-analysis: Potential and promise. Statistics in Medicine, 30, 2481–2498.
16. Kalaian, H. A., & Raudenbush, S. W. (1996). A multivariate mixed linear model for meta-analysis. Psychological Methods, 1, 227–235.
17. Konstantopoulos, S. (2011). Fixed effects and variance components estimation in three-level meta-analysis. Research Synthesis Methods, 2, 61–76.
18. Littell, R. C., Milliken, G. A., Stroup, W. W., Wolfinger, R. D., & Schabenberger, O. (2006). SAS® system for mixed models (2nd ed.). Cary, NC: SAS Institute Inc.
19. Maas, C. J. M., & Hox, J. J. (2005). Sufficient sample sizes for multilevel modeling. Methodology, 1, 85–91.
20. Mood, A. M., Graybill, F. A., & Boes, D. C. (1974). Introduction to the theory of statistics. New York: McGraw-Hill.
21. Osburn, H. G., & Callender, J. C. (1992). A note on the sampling variance of the mean uncorrected correlation in meta-analysis and validity generalization. Journal of Applied Psychology, 77, 115–122.
22. Raudenbush, S. W. (2009). Analyzing effect sizes: Random effects models. In H. M. Cooper, L. V. Hedges, & J. C. Valentine (Eds.), The handbook of research synthesis and meta-analysis (2nd ed., pp. 295–315). New York: Russell Sage Foundation.
23. Raudenbush, S. W., Becker, B. J., & Kalaian, H. A. (1988). Modeling multivariate effect sizes. Psychological Bulletin, 103, 111–120.
24. Raudenbush, S. W., & Bryk, A. S. (2002). Hierarchical linear models: Applications and data analysis methods (2nd ed.). London: Sage Publications.
25. Riley, R. D. (2009). Multivariate meta-analysis: The effect of ignoring within-study correlation. Journal of the Royal Statistical Society, Series A, 172, 789–811.
26. Rosa-Alcázar, A. I., Sánchez-Meca, J., Gómez-Conesa, A., & Marín-Martínez, F. (2008). Psychological treatment of obsessive-compulsive disorder: A meta-analysis. Clinical Psychology Review, 28, 1310–1325.
27. Scammacca, N., Roberts, G., & Stuebing, K.K. (2013). Meta-analysis with complex research designs: Dealing with dependence from multiple measures and multiple group comparisons. Review of Educational Research. Advance online publication.
28. Stevens, J. R., & Taylor, A. M. (2009). Hierarchical dependence in meta-analysis. Journal of Educational and Behavioral Statistics, 34, 46–73.
29. Van den Noortgate, W., López-López, J. A., Marín-Martínez, F., & Sánchez-Meca, J. (2013). Three level meta-analyses of dependent effect sizes. Behavior Research Methods, 45, 576–594.
30. Van den Noortgate, W., & Onghena, P. (2003). Multilevel meta-analysis: A comparison with traditional meta-analytical procedures. Educational and Psychological Measurement, 63, 765–790.
31. Van den Noortgate, W., & Onghena, P. (2005). Parametric and nonparametric bootstrap methods for meta-analysis. Behavior Research Methods, 37, 11–22.
32. van Houwelingen, H. C., Arends, L. R., & Stijnen, T. (2002). Advanced methods in meta-analysis: Multivariate approach and meta regression. Statistics in Medicine, 21, 589–624.