The use of meta-analytic statistical significance testing

Research Synthesis Methods

Volumn 6, Issue 1, 2015, Pages 63-73

  Polanin, Joshua R ^a   Pigottb, Terri D ^b  

^a VANDERBILT UNIVERSITY   (United States)

^b LOYOLA UNIVERSITY CHICAGO   (United States)

Author keywords

Meta analysis; Meta review; Moderator analyses; Multiplicity corrections; Power analysis; Statistical significance testing; Type I errors

Indexed keywords

BIOSTATISTICS; EDUCATION; HUMAN; META ANALYSIS (TOPIC); PROCEDURES; PSYCHOLOGY; STATISTICAL ANALYSIS; STATISTICAL MODEL; STATISTICS AND NUMERICAL DATA;

BIOSTATISTICS; DATA INTERPRETATION, STATISTICAL; EDUCATION; HUMANS; META-ANALYSIS AS TOPIC; MODELS, STATISTICAL; PSYCHOLOGY;

EID: 84938693245     PISSN: None     EISSN: 17592887     Source Type: Journal    
DOI: 10.1002/jrsm.1124     Document Type: Article

References (37)

1. Archer J. 2000. Sex differences in aggression between heterosexual partners: a meta-analytic review. Psychological Bulletin 126(5): 651.
2. Bender R, Bunce C, Clarke M, Gates S, Lange S, Pace NL, Thorlund K. 2008. Attention should be given to multiplicity issues in systematic reviews. Journal of Clinical Epidemiology 61: 857-865. DOI: 10.1016/j.jclinepi.2008.03.004.
3. Borenstein M, Hedges LV, Higgins JPT, Rothstein HR. 2009. Introduction toMeta-analysis. New York, NY: John Wiley & Sons.
4. Cafri G, Kromrey JD, Brannick MT. 2010. A meta-meta-analysis: empirical review of statistical power, type I error rates, effect sizes, and model selection of meta-analyses published in psychology. Multivariate Behavioral Research 45: 239-270. DOI: 10.1080/00273171003680187.
5. Cohen J. 1994. The earth is round (p<0.05). American Psychologist 49: 997-1001. DOI: 10.1037//0003-066X.49.12.997.
6. Cook TD, Campbell DT. 1979. Quasi-experimentation: Design and Analysis Issues for Field Settings. New York, NY: Rand McNally.
7. Cooper H. 2010. Research synthesis and meta-analysis (4th ed.). Thousand Oaks, CA: Sage Publications.
8. Cooper H, Koenka AC. 2012. The overview of reviews: unique challenges and opportunities when research syntheses are the principal elements of new integrative scholarship. American Psychologist 67: 446-462. DOI: 10.1037/a0027119.
9. Cumming G. 2014. The new statistics: why and how. Psychological Science 25: 7-29.
10. Dudoit S, Popper S, Boldrick JC. 2009. Multiple hypothesis testing in microarray experiments. Statistical Science 18(1): 71-103.
11. Fisher RA. 1935. The Design of Experiments. Oxford, England: Oliver & Boyd.
12. Hedges LV, Olkin I. 1985. Statistical Methods for Meta-analysis. London, England: Academic Press Inc.
13. Hedges LV, Pigott TD. 2001. The power of statistical tests in meta-analysis. Psychological Methods 6: 203-217.
14. Hedges LV, Pigott TD. 2004. The power of statistical tests for moderators in meta-analysis. Psychological Methods 9(4): 426-445.
15. Higgins JPT, Lane PW, Anagnostelis B, Anzures-Cabrera J, Baker NF, Cappelleri JC, Haughie S, Hollis S, Lewis SC, Moneuse P, Whitehead A. 2013. A tool to assess the quality of a meta-analysis. Research Synthesis Methods. DOI: 10.1002/jrsm.1092.
16. Hochberg Y, Tamhane AC. 1987. Multiple Comparison Procedures. New York, NY: John Wiley & Sons.
17. Hochberg Y. 1988. A sharper Bonferroni procedure for multiple tests of significance. Biometrika 75(4): 800-802.
18. Howell DC. 2006. Statistical Methods for Psychology. Independence, KY: Cengage Learning Tools.
19. IBM Corp. 2012. IBM SPSS Statistics for Windows. Version 21.0 [Software]. Armonk, NY: IBM Corp.
20. Keselman HJ, Cribbie R, Holland B. 1999. The pairwise multiple comparison multiplicity problem: an alternative approach to familywise and comparison wise type I error control. Psychological Methods 4: 58-69. DOI: 10.1037//1082-989X.4.1.58.
21. Keselman HJ, Miller CW, Holland B. 2011. Many test of significance: new methods for controlling type I errors. Psychological Methods 16: 420-431. DOI: 10.1037/a0025810.
22. Lipsey MW, Wilson DB. 1993. The efficacy of psychological, educational, and behavioral treatment: confirmation from meta-analysis. American Psychologist 48: 1181.
23. Lipsey MW, Wilson DB. 2001. Practical meta-analysis (2nd ed.). Thousand Oaks, CA: Sage Publications.
24. Lohr S. 1999. Sampling: Design and Analysis. Independence, KY: Cengage Learning.
25. Matt GE, Cook TD. 2009. Threats to the validity of generalized inferences (pp. 537-660). In H Cooper, LV Hedges, JC Valentine (Eds.), The Handbook of Research Synthesis and Meta-analysis. New York, NY: The Russell Sage Foundation.
26. Neyman J, Pearson ES. 1928. On the use and interpretation of certain test criteria for purposes of statistical inference: Part I. Biometrika 20A: 175-240. DOI: 10.2307/2331945.
27. Pigott TD. 2012. Advances in meta-analysis. New York, NY: Springer.
28. Scheffe H. 1959. The Analysis of Variance. New York, NY: Wiley & Sons, Ltd.
29. Schmidt FL. 1996. Statistical significance testing and cumulative knowledge in psychology: implications for training of researchers. Psychological Methods 1: 115-129. DOI: 10.1037//1082-989X.1.2.115.
30. Schmidt FL, Hunter JE. 2014. Methods of Meta-analysis: Correcting Error and Bias in Research Findings (3rd Ed.). New York, NY: The Russell Sage Foundation.
31. Sutton AJ, Higgins J. 2008. Recent developments in meta-analysis. Statistics in Medicine 27: 625-650.
32. Sutton AJ, Abrams KR. 2001. Bayesian methods in meta-analysis and evidence synthesis. Statistical Methods in Medical Research 10(4): 277-303.
33. Tukey JW. 1949. Comparing Individual Means in the Analysis of Variance. Unpublished Doctoral Dissertation. Princeton University. New Jersey.
34. Tendal B, Nüesch E, Higgins JPT, Jüni P, Gøtzsche PC. 2011. Multiplicity of data in trial reports and the reliability of meta-analyses: empirical study. British Medical Journal 343: 1-13. DOI: 10.1136/bmj.d4829.
35. Valentine JC, Pigott TD, Rothstein HR. 2010. How many studies do you need? A primer on statistical power for meta-analysis. Journal of Educational and Behavioral Statistics 35: 215-247.
36. Wickham H. 2009. ggplot2: Elegant Graphics for Data Analysis. New York, NY: Springer.
37. Williams RT. 2012. Using Robust Standard Errors to Combine Multiple Estimates with Meta-Analysis. Unpublished Doctoral Dissertation. Loyola University Chicago. Chicago, IL.