Limitations of Significance Testing in Clinical Research: A Review of Multiple Comparison Corrections and Effect Size Calculations with Correlated Measures

Anesthesia and Analgesia

Volumn 122, Issue 3, 2016, Pages 825-830

  Vasilopoulos, Terrie ^a   Morey, Timothy E ^b   Dhatariya, Ketan ^b   Rice, Mark J ^c  

^a UNIVERSITY OF FLORIDA COLLEGE OF MEDICINE   (United States)

^b NORFOLK AND NORWICH UNIVERSITY HOSPITAL   (United Kingdom)

^c VANDERBILT UNIVERSITY MEDICAL CENTER   (United States)

Author keywords

[No Author keywords available]

Indexed keywords

ARTICLE; BONFERRONI CORRECTION; CLINICAL RESEARCH; CORRELATIONAL STUDY; DIAGNOSTIC ERROR; EFFECT SIZE; FAMILY WISE ERROR RATE; METHODOLOGY; MULTIPLE COMPARISON CORRECTION; NULL HYPOTHESIS; PRIORITY JOURNAL; PROBABILITY; SIMULATION; ALGORITHM; COMPARATIVE STUDY; COMPUTER SIMULATION; HUMAN; MEDICAL RESEARCH; SAMPLE SIZE; STATISTICS AND NUMERICAL DATA;

ALGORITHMS; BIOMEDICAL RESEARCH; COMPUTER SIMULATION; HUMANS; RESEARCH DESIGN; SAMPLE SIZE;

EID: 84959064433     PISSN: 00032999     EISSN: 15267598     Source Type: Journal    
DOI: 10.1213/ANE.0000000000001107     Document Type: Article

References (36)

1. Cao J, Zhang S, Multiple comparison procedures. JAMA 2014 312 543 4
2. Sterne JA, Smith GD, Sifting the evidence-what's wrong with significance tests? Phys Ther 2001 81 8 1464 9
3. Bender R, Lange S, Adjusting for multiple testing-when and how? J Clin Epidemiol 2001 54 343 9
4. Feise RJ, Do multiple outcome measures require p-value adjustment? BMC Med Res Methodol 2002 2 8
5. Wason JM, Stecher L, Mander AP, Correcting for multiple-testing in multi-Arm trials: is it necessary and is it done? Trials 2014 15 364
6. Gelman A, Hill J, Yajima M, Why we (usually) don't have to worry about multiple comparisons. J Res Educ Effect 2012 5 189 211
7. Stacey AW, Pouly S, Czyz CN, An analysis of the use of multiple comparison corrections in ophthalmology research. Invest Ophthalmol Vis Sci 2012 53 1830 4
8. Holland BS, Copenhaver MD, Improved Bonferroni-type multiple testing procedures. Psychol Bull 1988 104 145
9. Eichstaedt KE, Kovatch K, Maroof DA, A less conservative method to adjust for familywise error rate in neuropsychological research: the Holm's sequential Bonferroni procedure. Neuro Rehabilitation 2013 32 693 6
10. Glickman ME, Rao SR, Schultz MR, False discovery rate control is a recommended alternative to Bonferroni-type adjustments in health studies. J Clin Epidemiol 2014 67 850 7
11. Holm S, A simple sequentially rejective multiple test procedure. Scan J Stat 1979 6 65 70
12. Perneger TV, Adjusting for multiple testing in studies is less important than other concerns. BMJ 1999 318 1288
13. Benjamini Y, Hochberg Y, Controlling the false discovery rate: a practical and powerful approach to multiple testing. J R Stat Soc Ser B 1995 57 289 300
14. Verhoeven KJ, Simonsen KL, McIntyre LM, Implementing false discovery rate control: increasing your power. Oikos 2005 108 643 7
15. Hutson AD, A semiparametric bootstrap approach to correlated data analysis problems. Comput Methods Programs Biomed 2004 73 129 34
16. Hung HM, Wang SJ, Multiple comparisons in complex clinical trial designs. Biom J 2013 55 420 9
17. Neuhäuser M, How to deal with multiple endpoints in clinical trials. Fundam Clin Pharmacol 2006 20 515 23
18. Romano JP, Shaikh AM, Wolf M, Control of the false discovery rate under dependence using the bootstrap and subsampling. Test 2008 17 417 42
19. Westfall PH, Tobias R, Wolfringer R, Multiple Comparisons and Multiple Tests Using SAS 2011 2nd ed Cary, NC SAS® Press
20. Westfall PH, Young SS, Resampling-Based Multiple Testing: Examples and Methods for P-Value Adjustment 1993 Vol 279 New York, NY John Wiley & Sons
21. Yekutieli D, Benjamini Y, Resampling-based false discovery rate controlling multiple test procedures for correlated test statistics. J Stat Planning Inference 1999 82 171 96
22. Bland JM, Altman DG, Statistics Notes: Bootstrap resampling methods. BMJ 2015 350 h2622
23. Carpenter J, Bithell J, Bootstrap confidence intervals: when, which, what? A practical guide for medical statisticians. Stat Med 2000 19 1141 64
24. Chernick MR, Bootstrap Methods: A Guide for Practitioners and Researchers 2011 Vol 619 New York, NY John Wiley & Sons
25. Efird JT, Nielsen SS, A method to compute multiplicity corrected confidence intervals for odds ratios and other relative effect estimates. Int J Environ Res Public Health 2008 5 394 8
26. Ludbrook J, Multiple inferences using confidence intervals. Clin Exp Pharmacol Physiol 2000 27 212 5
27. Murphy GS, Szokol JW, Avram MJ, Greenberg SB, Shear T, Vender JS, Gray J, Landry E, The effect of single low-dose dexamethasone on blood glucose concentrations in the perioperative period: a randomized, placebo-controlled investigation in gynecologic surgical patients. Anesth Analg 2014 118 1204 12
28. Cumming G, Replication and p intervals: p values predict the future only vaguely, but confidence intervals do much better. Perspect Psychol Sci 2008 3 286 300
29. Goodman SN, A comment on replication, p-values and evidence. Stat Med 1992 11 875 9
30. Shafer SL, Dexter F, Publication bias, retrospective bias, and reproducibility of significant results in observational studies. Anesth Analg 2012 114 931 2
31. Cumming G, The new statistics: why and how. Psychol Sci 2014 25 7 29
32. Durlak JA, How to select, calculate, and interpret effect sizes. J Pediatr Psychol 2009 34 917 28
33. Cohen J, Statistical Power for the Social Sciences 1988 Hillsdale, NJ Laurence Erlbaum and Associates
34. Dunlap WP, Cortina JM, Vaslow JB, Burke MJ, Meta-Analysis of experiments with matched groups or repeated measures designs. Psychol Methods 1996 1 170
35. Morris SB, DeShon RP, Combining effect size estimates in meta-Analysis with repeated measures and independent-groups designs. Psychol Methods 2002 7 105 25
36. Olejnik S, Algina J, Generalized eta and omega squared statistics: measures of effect size for some common research designs. Psychol Methods 2003 8 434 47