Balanced control of generalized error rates

Annals of Statistics

Volumn 38, Issue 1, 2010, Pages 598-633

  Romano, Joseph ^a   Wolf, Michael ^b  

^a STANFORD UNIVERSITY   (United States)

^b UNIVERSITY OF ZURICH   (Switzerland)

Author keywords

Bootstrap; False discovery proportion; Generalized family wise error rate; Multiple testing; Step down procedure

Indexed keywords

EID: 73949135000     PISSN: 00905364     EISSN: None     Source Type: Journal    
DOI: 10.1214/09-AOS734     Document Type: Article

References (42)

1. BENJAMINI, Y. and HOCHBERG, Y. (1995). Controlling the false discovery rate: A practical and powerful approach to multiple testing. J. Roy. Statist. Soc. Ser. B 57 289-300. MR1325392
2. BENJAMINI, Y., KRIEGER, A.M. and YEKUTIELI, D. (2006). Adaptive linear step-up procedures that control the false discovery rate. Biometrika 93 491-507. MR2261438
3. BENJAMINI, Y. and YEKUTIELI, D. (2001). The control of the false discovery rate in multiple testing under dependency. Ann. Statist. 29 1165-1188. MR1869245
4. BERAN, R. (1986). Simulated power functions. Ann. Statist. 14 151-173. MR0829560
5. BERAN, R. (1988). Balanced simultaneous confidence sets. J. Amer. Statist. Assoc. 83 679-686. MR0963795
6. BERAN, R. (1988). Prepivoting test statistics: A bootstrap view of asymptotic refinements. J. Amer. Statist. Assoc. 83 687-697. MR0963796
7. DAVISON, A. C. andHINKLEY, D. V. (1997). Bootstrap Methods and Their Application. Cambridge Univ. Press, Cambridge. MR1478673
8. DUDOIT, S., GILBERT, H. and VAN DER LAAN, M. J. (2008). Resampling-based empirical Bayesmultiple testing procedures for controlling generalized tail probability and expected value error rates: Focus on the false discovery rate and simulation study. Biom. J. 50 716-744.
9. DUDOIT, S., SHAFFER, J. P. and BOLDRICK, J. C. (2003). Multiple hypothesis testing in microarray experiments. Statist. Sci. 18 71-103. MR1997066 (Pubitemid 41461823)
10. DUDOIT, S., VAN DER LAAN, M. J. and POLLARD, K. S. (2004). Multiple testing. I. Singlestep procedures for control of general type I error rates. Stat. Appl. Genet. Mol. Biol. 3 71. Available at http://www.bepress.com/sagmb/ vol3/iss1/art13. MR2101462
11. EFRON, B. and TIBSHIRANI, R. J. (1993). An Introduction to the Bootstrap. Chapman & Hall, New York. MR1270903
12. GENOVESE, C. R. and WASSERMAN, L. (2004). A stochastic process approach to false discovery control. Ann. Statist. 32 1035-1061. MR2065197
13. HALL, P. (1992). The Bootstrap and Edgeworth Expansion. Springer, New York. MR1145237
14. HALL, P. andWILSON, S. (1991). Two guidelines for bootstrap hypothesis testing. Biometrics 47 757-762. MR1132543
15. HOLM, S. (1979). A simple sequentially rejective multiple test procedure. Scand. J. Statist. 6 65-70. MR0538597
16. HOMMEL, G. andHOFFMAN, T. (1988). Controlled uncertainty. In Multiple Hyptheses Testing (P. Bauer, G. Hommel and E. Sonnemann, eds.) 154-161. Springer, Heidelberg.
17. KORN, E. L., TROENDLE, J. F., MCSHANE, L. M. and SIMON, R. (2004). Controlling the number of false discoveries: Application to high-dimensional genomic data. J. Statist. Plann. Inference 124 379-398. MR2080371
18. LAHIRI, S. N. (2003). Resampling Methods for Dependent Data. Springer, New York. MR2001447
19. LEHMANN, E. L. and ROMANO, J. P. (2005). Generalizations of the family-wise error rate. Ann. Statist. 33 1138-1154. MR2195631
20. LEHMANN, E. L. and ROMANO, J. P. (2005). Testing Statistical Hypotheses, 3rd ed. Springer, New York. MR2135927
21. PERONE PACIFICO, M., GENOVESE, C. R., VERDINELLI, I. and WASSERMAN, L. (2004). False discovery control for random fields. J. Amer. Statist. Assoc. 99 1002-1014. MR2109490
22. POLITIS, D. N., ROMANO, J. P. and WOLF, M. (1999). Subsampling. Springer, New York. MR1707286
23. POLLARD, K. S. and VAN DER LAAN, M. J. (2003). Multiple testing for gene expression data: An investigation of null distributions with consequences for the permutation test. In Proceedings of the 2003 International MultiConference in Computer Science and Engineering, METMBS'03 Conference 3-9.
24. ROGERS, J. and HSU, J. (2001). Multiple comparisons of biodiversity. Biom. J. 43 617-625. MR1863493 (Pubitemid 33588685)
25. ROMANO, J. P. (1988). A bootstrap revival of some nonparametric distance tests. J. Amer. Statist. Assoc. 83 698-708. MR0963797
26. ROMANO, J. P. and SHAIKH, A. M. (2006). On step-down control of the false discovery proportion. In 2nd Lehmann Symposium-Optimality (J. Rojo, ed.). Institute of Mathematical Statistics Lecture Notes-Monograph Series. 49 33-50. Inst. Math. Statist., Beachwood, OH. MR2337829
27. ROMANO, J. P. and SHAIKH, A. M. (2006). Stepup procedures for control of generalizations of the family-wise error rate. Ann. Statist. 34 1850-1873. MR2283720
28. ROMANO, J. P., SHAIKH, A. M. and WOLF, M. (2008). Control of the false discovery rate under dependence using the bootstrap and subsampling (with discussion). Test 17 417-442. MR2470085
29. ROMANO, J. P., SHAIKH, A. M. andWOLF, M. (2008). Formalized data snooping based on generalized error rates. Econometric Theory 24 404-447. MR2422863
30. ROMANO, J. P. andWOLF, M. (2005). Exact and approximate step-down methods for multiple hypothesis testing. J. Amer. Statist. Assoc. 100 94-108. MR2156821
31. ROMANO, J. P. andWOLF, M. (2007). Control of generalized error rates in multiple testing. Ann. Statist. 35 1378-1408. MR2351090
32. SARKAR, S. K. (2002). Some results on false discovery rate in stepwise multiple testing procedures. Ann. Statist. 30 239-257. MR1892663
33. SHAO, J. and TU, D. (1995). The Jackknife and the Bootstrap. Springer, New York. MR1351010
34. SPJØTVOLL, E. (1972). On the optimality of some multiple comparison procedures. Ann. Math. Statist. 43 398-411. MR0301871
35. STOREY, J. D., TAYLOR, J. E. and SIEGMUND, D. (2004). Strong control, conservative point estimation and simultaneous conservative consistency of false discovery rates: A unified approach. J. R. Stat. Soc. Ser. B Stat. Methodol. 66 187-205. MR2035766
36. TROENDLE, J. F. (2000). Stepwise normal theory test procedures controlling the false discovery rate. J. Statist. Plann. Inference 84 139-158. MR1747501
37. TROENDLE, J. F. (2008). Comment on "Control of the false discovery rate under dependence using the bootstrap and subsampling," by J. Romano, A. Shaikh and M. Wolf. Test 17 456-457.
38. TU, W. and ZHOU, X. (2000). Pairwise comparison of the means of skewed data. J. Statist. Plann. Inference 88 59-74. MR1767559
39. VAN DER LAAN, M. J., BIRKNER, M. D. and HUBBARD, A. E. (2005). Empirical Bayes and resampling based multiple testing procedure controlling tail probability of the proportion of false positives. Stat. Appl. Genet.Mol. Biol. 4 32. Available at http://www.bepress.com/sagmb/vol4/iss1/art29/. MR2170445
40. VAN DER LAAN, M. J., DUDOIT, S. and POLLARD, K. S. (2004). Augmentation procedures for control of the generalized family-wise error rate and tail probabilities for the proportion of false positives. Stat. Appl. Genet. Mol. Biol. 3 27. Available at http://www.bepress.com/sagmb/vol3/iss1/art15/. MR2101464
41. WESTFALL, P. H. and YOUNG, S. S. (1993). Resampling-Based Multiple Testing: Examples and Methods for P-Value Adjustment. Wiley, New York.
42. YEKUTIELI, D. and BENJAMINI, Y. (1999). Resampling-based false discovery rate controlling multiple test procedures for correlated test statistics. J. Statist. Plann. Inference 82 171-196. MR1736442