Multiple testing for exploratory research

Statistical Science

Volumn 26, Issue 4, 2011, Pages 584-597

  Goeman, Jelle ^a   Solari, Aldo ^b  

^a LEIDEN UNIVERSITY MEDICAL CENTER   (Netherlands)

^b UNIVERSITY OF MILANO BICOCCA   (Italy)

Author keywords

Closed testing; Confidence set; False discovery pro portion

Indexed keywords

EID: 84864528337     PISSN: 08834237     EISSN: None     Source Type: Journal    
DOI: 10.1214/11-STS356     Document Type: Article

References (35)

1. BENJAMINI, Y. and HELLER, R. (2008). Screening for partial conjunction hypotheses. Biometrics 64 1215-1222. MR2522270
2. BENJAMINI, Y. and HOCHBERG, Y. (1995). Controlling the false discovery rate: A practical and powerful approach to multiple testing. J. Roy. Statist. Soc. Ser. B 57 289-300. MR1325392
3. BENJAMINI, Y. and HOCHBERG, Y. (2000). On the adaptive control of the false discovery rate in multiple testing with independent statistics. Journal of Educational and Behavioral Statistics 25 60-83.
4. BENJAMINI, Y. and YEKUTIELI, D. (2001). The control of the false discovery rate in multiple testing under dependency. Ann. Statist. 29 1165-1188. MR1869245
5. BENJAMINI, Y. and YEKUTIELI, D. (2005). False discovery rate adjusted multiple confidence intervals for selected parameters. J. Amer. Statist. Assoc. 100 71-81. MR2156820
6. BITTMAN, R. M., ROMANO, J. P., VALLARINO, C. and WOLF, M. (2009). Optimal testing of multiple hypotheses with common effect direction. Biometrika 96 399-410. MR2507151
7. BRANNATH, W. and BRETZ, F. (2010). Shortcuts for locally consonant closed test procedures. J. Amer. Statist. Assoc. 105 660-669. MR2724850
8. EDWARDS, D. and MADSEN, J. (2007). Constructing multiple test procedures for partially ordered hypothesis sets. Stat. Med. 26 5116-5124. MR2412460
9. EFRON, B., TIBSHIRANI, R., STOREY, J. D. and TUSHER, V. (2001). Empirical Bayes analysis of a microarray experiment. J. Amer. Statist. Assoc. 96 1151-1160. MR1946571
10. FINNER, H. and ROTERS, M. (2001). On the false discovery rate and expected type I errors. Biom. J. 43 985-1005. MR1878272
11. GOEMAN, J. J. and MANSMANN, U. (2008). Multiple testing on the directed acyclic graph of gene ontology. Bioinformatics 24 537-544.
12. GOEMAN, J. J. and SOLARI, A. (2010). The sequential rejection principle of familywise error control. Ann. Statist. 38 3782-3810. MR2766868
13. GRECHANOVSKY, E. and HOCHBERG, Y. (1999). Closed procedures are better and often admit a shortcut. J. Statist. Plann. Inference 76 79-91. MR1673341
14. HERSON, J. (2009). Data and Safety Monitoring Committees in Clinical Trials. CRC Press, Boca Raton, FL.
15. HOMMEL, G. (1983). Tests of the overall hypothesis for arbitrary dependence structures. Biometrical J. 25 423-430. MR0735888
16. HOMMEL, G., BRETZ, F. and MAURER, W. (2007). Powerful short-cuts for multiple testing procedures with special reference to gatekeeping strategies. Stat. Med. 26 4063-4073. MR2405792
17. HUANG, Y. and HSU, J. C. (2007). Hochberg's step-up method: Cutting corners off Holm's step-down method. Biometrika 94 965-975. MR2416802
18. JIN, J. and CAI, T. T. (2007). Estimating the null and the proportional of nonnull effects in large-scale multiple comparisons. J. Amer. Statist. Assoc. 102 495-506. MR2325113
19. LANGAAS, M., LINDQVIST, B. H. and FERKINGSTAD, E. (2005). Estimating the proportion of true null hypotheses, with application to DNA microarray data. J. R. Stat. Soc. Ser. B Stat. Methodol. 67 555-572. MR2168204
20. LARNER, M. (1996). Mass and its relationship to physical measurements. MS305 Data Project, Dept. Mathematics, Univ. Queensland, Australia.
21. MARCUS, R., PERITZ, E. and GABRIEL, K. R. (1976). On closed testing procedures with special reference to ordered analysis of variance. Biometrika 63 655-660. MR0468056
22. MARENNE, G., DALMASSO, C., PERDRY, H., GÉNIN, E. and BROËT, P. (2009). Impaired performance of FDR-based strategies in whole-genome association studies when SNPs are excluded prior to the analysis. Genetic Epidemiology 33 45-53.
23. MEINSHAUSEN, N. (2006). False discovery control for multiple tests of association under general dependence. Scand. J. Statist. 33 227-237. MR2279639
24. MEINSHAUSEN, N. (2008). Hierarchical testing of variable importance. Biometrika 95 265-278. MR2521583
25. MEINSHAUSEN, N. andBÜHLMANN, P. (2005). Lower bounds for the number of false null hypotheses for multiple testing of associations under general dependence structures. Biometrika 92 893-907. MR2234193
26. ROMANO, J. P. and WOLF, M. (2007). Control of generalized error rates in multiple testing. Ann. Statist. 35 1378-1408. MR2351090
27. SARKAR, S. K. (1998). Some probability inequalities for ordered MTP2 random variables: A proof of the Simes conjecture. Ann. Statist. 26 494-504. MR1626047
28. SCHWEDER, T. and SPJØTVOLL, E. (1982). Plots of p-values to evaluate many tests simultaneously. Biometrika 69 493-502.
29. ŠIDÁK, Z. (1967). Rectangular confidence regions for the means of multivariate normal distributions. J. Amer. Statist. Assoc. 62 626-633. MR0216666
30. SIMES, R. J. (1986). An improved Bonferroni procedure for multiple tests of significance. Biometrika 73 751-754. MR0897872
31. STOREY, J. D. (2002). A direct approach to false discovery rates. J. R. Stat. Soc. Ser. B Stat. Methodol. 64 479-498. MR1924302
32. TUKEY, J. (1980). We need both exploratory and confirmatory. Amer. Statist. 34 23-25.
33. VAN DE WIEL, M., BERKHOF, J. and VAN WIERINGEN, W. (2009). Testing the prediction error difference between 2 predictors. Biostatistics 10 550-560.
34. VAN DER LAAN, M. J., DUDOIT, S. and POLLARD, K. S. (2004). Augmentation procedures for control of the generalized familywise error rate and tail probabilities for the proportion of false positives. Stat. Appl. Genet. Mol. Biol. 3 Art. 15, 27 pp. (electronic). MR2101464
35. ZAYKIN, D., ZHIVOTOVSKY, L. A.,WESTFALL, P. andWEIR, B. (2002). Truncated product method for combining p-values. Genetic Epidemiology 22 170-185.