Small sample adjustments for robust variance estimation with meta-regression

Psychological Methods

Volumn 20, Issue 3, 2015, Pages 375-393

  Tipton, Elizabeth ^a  

^a Och Spine at New York Presbyterian Hospitals   (United States)

Author keywords

Dependent; Effect size; Meta analysis; Robust statistics

Indexed keywords

HUMAN; META ANALYSIS (TOPIC); REGRESSION ANALYSIS; STATISTICAL ANALYSIS;

DATA INTERPRETATION, STATISTICAL; HUMANS; META-ANALYSIS AS TOPIC; REGRESSION ANALYSIS;

EID: 84941261747     PISSN: 1082989X     EISSN: None     Source Type: Journal    
DOI: 10.1037/met0000011     Document Type: Article

References (33)

1. Ahn, S., Ames, A. J., & Myers, N. D. (2012). A review of meta-analyses in education: Methodological strengths and weaknesses. Review of Educational Research, 82, 436-476. doi:10.3102/0034654312458162
2. Bell, R. M.,&McCaffrey, D. F. (2002). Bias reduction in standard errors for linear regression with multi-stage samples. Survey Methodology, 28, 169-181.
3. Cafri, G., Kromrey, J. D., & Brannick, M. T. (2010). A meta-meta-analysis: Empirical review of statistical power, Type I error rates, effect sizes, and model selection of meta-analyses published in psychology. Multivariate Behavioral Research, 45, 239-270. doi:10.1080/00273171003680187
4. Chesher, A., & Austin, G. (1991). The finite-sample distributions of heteroskedasticity robust Wald statistics. Journal of Econometrics, 47, 153-173. doi:10.1016/0304-4076(91)90082-O
5. Cohen, J. (1988). Statistical power analysis for the behavioral sciences (2nd ed). Hillsdale, NJ: Erlbaum.
6. De Vibe, M., Bjørndal, A., Tipton, E., Hammerstrøm, K. T., Kowalski, K., & Bjørndal, A. (2012). Mindfulness based stress reduction (MBSR) for improving health, quality of life and social functioning in adults. Campbell Systematic Reviews, 8(3). Retrieved from http://campbellcollaboration.org/lib/project/117/
7. Falkenström, F., Markowitz, J. C., Jonker, H., Philips, B., & Holmqvist, R. (2013). Can psychotherapists function as their own controls? Metaanalysis of the crossed therapist design in comparative psychotherapy trials. Journal of Clinical Psychiatry, 74, 482-491. doi:10.4088/JCP.12r07848
8. Gleser, L. J., & Olkin, I. (2009). Stochastically dependent effect sizes. In H. Cooper, L. V. Hedges, & J. C. Valentine (Eds.), The handbook of research synthesis and meta-analysis (2nd ed., pp. 357-376). New York, NY: Russell Sage Foundation.
9. Hedges, L. V., Tipton, E., & Johnson, M. (2010). Robust variance estimation in meta-regression with dependent effect size estimates. Research Synthesis Methods, 1, 39-65. doi:10.1002/jrsm.5
10. Higgins, J. P. T., & Thompson, S. G. (2002). Quantifying heterogeneity in a meta-analysis. Statistics in Medicine, 21, 1539-1558. doi:10.1002/sim.1186
11. Huber, P. (1967). The behavior of maximum-likelihood estimates under nonstandard conditions. In L. M. LeCam & J. Neyman (Eds.), Proceedings of the Fifth Berkeley Symposium on Mathematical Statistics and Probability (pp. 221-233). Berkeley: University of California Press.
12. Imbens, G. W., & Kolesar, M. (2013). Robust standard errors in small samples: Some practical advice (NBER Working Paper 18478). Retrieved from the National Bureau of Economic Research website: http://www.nber.org/papers/w18478
13. Jackson, D., Riley, R., & White, I. R. (2011). Multivariate meta-analysis: Potential and promise. Statistics in Medicine, 30, 2481-2498. doi: 10.1002/sim.4172
14. Konstantopoulos, S. (2011). Fixed effects and variance components estimation in three-level meta-analysis. Research Synthesis Methods, 2, 61-76. doi:10.1002/jrsm.35
15. Liang, K. L., & Zeger, S. L. (1986). Longitudinal data analysis using generalized linear models. Biometrika, 73, 13-22. doi:10.1093/biomet/73.1.13
16. Long, J. S., & Ervin, L. H. (2000). Using heteroscedasticity consistent standard errors in the linear regression model. The American Statistician, 54, 217-224.
17. MacKinnon, J. G. (2013). Thirty years of heteroskedasticity-robust inference. In X. Chen & N. R. Swanson (Eds.), Recent advances and future directions in causality, prediction, and specification analysis: Essays in honor of Halbert L. White Jr. (pp. 437-461). New York, NY: Springer. doi:10.1007/978-1-4614-1653-1_17
18. MacKinnon, J. G., & White, H. (1985). Some heteroskedastity-consistent covariance matrix estimators with improved finite sample properties. Journal of Econometrics, 29, 305-325. doi:10.1016/0304-4076(85)90158-7
19. McCaffrey, D. F., Bell, R. M., & Botts, C. H. (2001). Generalizations of biased reduced linearization. In Proceedings of the Annual Meeting of the American Statistical Association, August 5-9, 2001. Retrieved from http://www.amstat.org/sections/SRMS/Proceedings/y2001/Proceed/00264.pdf
20. Polanin, J. R. (2013). Addressing the issue of meta-analysis multiplicity in education and psychology (Doctoral dissertation). Available from Pro-Quest Dissertations and Theses database. (Accession Order No.3566462)
21. R Development Core Team. (2012). R: A language and environment for statistical computing [Computer software]. Retrieved from http://www .R-project.org/
22. Raudenbush, S. W., & Bryk, A. (2002). Hierarchical linear models: Applications and data analysis methods (Vol. 1). Thousand Oaks, CA: Sage.
23. Samson, J. E., Ojanen, T., & Hollo, A. (2012). Social goals and youth aggression: Meta-analysis of prosocial and antisocial goals. Social Development, 21, 645-666. doi:10.1111/j.1467-9507.2012.00658.x
24. Satterthwaite, F. E. (1946). An approximate distribution of estimates of variance components. Biometrics, 2, 110-114. doi:10.2307/3002019
25. Skinner, C. J. (1989). Domain means, regression and multivariate analyses. In C. J. Skinner, D. Holt, & T. M. F. Smith (Eds.), Analysis of complex surveys (pp. 59-88). New York, NY: Wiley.
26. Tan, J. Y. R., Huedo-Medina, T. B., Lennon, C. A., White, A. C., & Johnson, B. T. (2010). Us versus them in context: Meta-analysis as a tool for geotemporal trends in intergroup relations. International Journal of Conflict and Violence, 4, 288-297.
27. Tanner-Smith, E. E., & Lipsey, M. W. (2013). Brief intervention effects on adolescent and young adults' alcohol consumption: A systematic review and meta-analysis. Manuscript submitted for publication.
28. Tanner-Smith, E. E., & Tipton, E. (2013). Robust variance estimation with dependent effect sizes: Practical considerations including a software tutorial in Stata and SPSS. Research Synthesis Methods. Advance online publication. doi:10.1002/jrsm.1091
29. Tipton, E. (2013). Robust variance estimation in meta-regression with binary dependent effects. Research Synthesis Methods, 4, 169-187. doi:10.1002/jrsm.1070
30. Uttal, D. H., Meadow, N. G., Tipton, E., Hand, L. L., Alden, A. R., Warren, C., & Newcombe, N. S. (2013). The malleability of spatial skills: A meta-analysis of training studies. Psychological Bulletin, 139, 352-402. doi:10.1037/a0028446
31. White, H. (1980). A heteroscedastity-consistent covariance matrix and a direct test for heteroscedasticity. Econometrica, 48, 817-838. doi: 10.2307/1912934
32. Williams, R. (2012). Using robust standard errors to combine multiple regression estimates with meta-analysis (Doctoral dissertation). Available from ProQuest Dissertations and Theses database. (Accession Order No. 3526372)
33. Wilson, S. J., Tanner-Smith, E. E., Lipsey, M. W., Steinka-Fry, K., & Morrison, J. (2011). Dropout prevention and intervention programs: Effects on school completion and dropout among school-aged children and youth. Campbell Systematic Reviews, 8. doi:10.4073/csr.2011.8