Robustness of multiple testing procedures against dependence

Annals of Statistics

Volumn 37, Issue 1, 2009, Pages 332-358

  Clarke, Sandy ^a   Hall, Peter ^a  

^a UNIVERSITY OF MELBOURNE   (Australia)

Author keywords

False discovery rate; Family wise error rate; Linear process; Moving average; Multiplicity; Significance level; Simultaneous hypothesis testing; Time series

Indexed keywords

EID: 65349145181     PISSN: 00905364     EISSN: None     Source Type: Journal    
DOI: 10.1214/07-AOS557     Document Type: Article

References (46)

1. BENJAMINI, Y. and HOCHBERG, Y. (1995). Controlling the false discovery rate: A practical and powerful approach to multiple testing. J. Roy. Statist. Soc. Ser. B 57 289-300. MR1325392
2. BENJAMINI, Y. and HOCHBERG, Y. (2000). On the adaptive control of the false discovery fate in multiple testing with independent statistics. J. Educ. Behav. Statist. 25 60-83.
3. BENJAMINI, Y. andYEKUTIELI, D. (2001). The control of the false discovery rate in multiple testing under dependency. Ann. Statist. 29 1165-1188. MR1869245
4. BERNHARD, G., KLEIN, M. and HOMMEL, G. (2004). Global and multiple test procedures using ordered p-values-a review. Statist. Papers 45 1-14. MR2019782
5. BLAIR, R. C., TROENDLE, J. F. and BECK, R.W. (1996). Control of familywise errors in multiple endpoint assessments via stepwise permutation tests. Statist. Med. 15 1107-1121.
6. BROWN, B. W. and RUSSELL, K. (1997). Methods correcting for multiple testing: Operating characteristics. Statist. Med. 16 2511-2528.
7. DUDOIT, S., SHAFFER, J. P. andBOLDRICK, J. C. (2003).Multiple hypothesis testing in microarray experiments. Statist. Sci. 18 73-103. MR1997066
8. DUNNETT, C. W. and TAMHANE, A. C. (1995). Step-up testing of parameters with unequally correlated estimates. Biometrics 51 217-227.
9. EFRON, B. (2007). Correlation and large-scale simultaneous significance testing. J. Amer. Statist. Assoc. 102 93-103. MR2293302
10. FINNER, H. and ROTERS, M. (1998). Asymptotic comparison of step-down and step-up multiple test procedures based on exchangeable test statistics. Ann. Statist. 26 505-524. MR1626043
11. FINNER, H. and ROTERS, M. (1999). Asymptotic comparison of the critical values of step-down and step-up multiple comparison procedures. J. Statist. Plann. Inference 79 11-30. MR1704211
12. FINNER, H. and ROTERS, M. (2000). On the critical value behavior of multiple decision procedures. Scand. J. Statist. 27 563-573. MR1795781
13. FINNER, H. and ROTERS, M. (2002). Multiple hypotheses testing and expected number of type I errors. Ann. Statist. 30 220-238. MR1892662
14. GENOVESE, C. andWASSERMAN, L. (2004). A stochastic process approach to false discovery control. Ann. Statist. 32 1035-1061. MR2065197
15. GODFREY, G. K. (1985). Comparing the means of several groups.NewEng. J. Med. 311 1450-1456.
16. GOTZSCHE, P. C. (1989). Methodology and overt and hidden bias in reports of 196 double-blind trials of nonsteroidal antiinflammatory drugs in rheumatoid arthritis. Control Clin. Trials 10 31-56.
17. HOCHBERG, Y. (1988). A sharper Bonferroni procedure for multiple tests of significance. Biometrika 75 800-802. MR0995126
18. HOCHBERG, Y. and BENJAMINI, Y. (1990). More powerful procedures for multiple testing. Statist. Med. 9 811-818.
19. HOCHBERG, Y. and TAMHANE, A. C. (1987). Multiple Comparison Procedures. Wiley, New York. MR0914493
20. HOLLAND, B. and CHEUNG, S. H. (2002). Familywise robustness criteria for multiple-comparison procedures. J. Roy. Statist. Soc. Ser. B 64 63-77. MR1881845
21. HOMMELL, G. (1988). A comparison of two modified Bonferroni procedures. Biometrika 76 624-625. MR1040659
22. KESSELMAN, H. J., CRIBBIE, R. and HOLLAND, B. (2002). Controlling the rate of Type I error over a large set of statistical tests. Brit. J. Math. Statist. Psych. 55 27-39.
23. LEHMANN, E. L. and ROMANO, J. P. (2005). Testing Statistical Hypotheses, 3rd ed. Springer, New York. MR2135927
24. LEHMANN, E. L., ROMANO, J. P. and SHAFFER, J. P. (2005). On optimality of stepdown and stepup multiple test procedures. Ann. Statist. 33 1084-1108. MR2195629
25. LUDBROOK, J. (1991). On making multiple comparisons in clinical and experimental pharmacology and physiology. Clin. Exper. Pharm. Physiol. 18 379-392.
26. OLEJNIK, S., LI, J. M., SUPATTATHUM, S. and HUBERTY, C. J. (1997). Multiple testing and statistical power with modified Bonferroni procedures. J. Educ. Behav. Statist. 22 389-406.
27. OTTENBACHER, K. J. (1991a). Statistical conclusion validity: An empirical analysis of multiplicity in mental retardation research. Amer. J. Ment. Retard. 95 421-427.
28. OTTENBACHER, K. J. (1991b). Statistical conclusion validity-multiple inferences in rehabilitation research. Amer. J. Phys. Med. Rehab. 70 317-322.
29. OTTENBACHER, K. J. (1998). Quantitative evaluation of multiplicity in epidemiology and public health research. Amer. J. Epidemiology 147 615-619.
30. OTTENBACHER, K. J. and BARRETT, K. A. (1991). Measures of effect size in the reporting of rehabilitation research. Amer. J. Phys. Med. Rehab. 70 S131-S137.
31. PIGEOT, I. (2000). Basic concepts of multiple tests-A survey. Statist. Papers 41 3-36. MR1746085
32. POCOCK, S. J., HUGHES, M. D. and LEE, R. J. (1987). Statistical problems in reporting of clinical trials. J. Amer. Statist. Assoc. 84 381-392.
33. ROM, D. M. (1990). A sequentially rejective test procedure based on a modified Bonferroni inequality. Biometrika 77 663-665. MR1087860
34. ROSENBERG, P. S., CHE, A. and CHEN, B. E. (2006). Multiple hypothesis testing strategies for genetic case-control association studies. Statist. Med. 25 3134-3149. MR2247232
35. SARKAR, S. K. (1998). Some probability inequalities for ordered MTP2 random variables: A proof of the Simes conjecture. Ann. Statist. 26 494-504. MR1626047
36. SARKAR, S. K. (2006). False discovery and false nondiscovery rates in single-step multiple testing procedures. Ann. Statist. 34 394-415. MR2275247
37. SARKAR, S. K. and CHANG, C. K. (1997). The Simes method for multiple hypothesis testing with positively dependent test statistics. J. Amer. Statist. Assoc. 92 1601-1608. MR1615269
38. SCHMIDT, R. and STADTMÜLLER, U. (2006). Nonparametric estimation of tail dependence. Scand. J. Statist. 33 307-335. MR2279645
39. SCHMIDT, T. (2007). Coping with copulas. In Copulas-From Theory to Applications in Finance (J. Rank, ed.) 3-34. Risk Books, London.
40. SEN, P. K. (1999). Some remarks on Simes-type multiple tests of significance. J. Statist. Plann. Inference 82 139-145. MR1736438
41. SHAO, Q.-M. (1999). A Cramér type large deviation result for Student's t -statistic. J. Theoret. Probab. 12 385-398. MR1684750
42. SIMES, R. J. (1986). An improved Bonferroni procedure for multiple tests of significance. Biometrika 73 751-754. MR0897872
43. SMITH, D. E., CLEMENS, J., CREDE, W., HARVEY, M. and GRACELY, E. J. (1987). Impact of multiple comparisons in randomized clinical trials. Amer. J. Med. 83 545-550.
44. YEKUTIELI, D., REINER-BENAIM, A., BENJAMINI, Y., ELMER, G. I., KAFKAFI, N., LETWIN, N. E. and LEE, N. H. (2006). Approaches to multiplicity issues in complex research in microarray analysis. Statist. Neerlandica 60 414-437. MR2291384
45. WANG, Q. (2005). Limit theorems for self-normalized large deviations. Electron. J. Probab. 10 1260-1285. MR2176384
46. WRIGHT, S. P. (1992). Adjusted p-values for simultaneous inference. Biometrics 48 1005-1013.