Multiple hypotheses testing based on ordered p ValuesA historical survey with applications to medical research

Journal of Biopharmaceutical Statistics

Volumn 21, Issue 4, 2011, Pages 595-609

  Hommel, Gerhard ^a   Bretz, Frank ^b   Maurer, Willi ^b  

^a UNIVERSITY MEDICAL CENTER MAINZ   (Germany)

^b NOVARTIS PHARMA AG   (Switzerland)

Author keywords

Cutoff tests; False discovery rate; Familywise error rate; Global test; Multiple test procedure; Weighted test procedure

Indexed keywords

ANALYTICAL ERROR; ARTICLE; CLINICAL TRIAL (TOPIC); CONTROLLED STUDY; EXPLORATORY RESEARCH; MEDICAL RESEARCH; MOTIVATION; NOMENCLATURE; NULL HYPOTHESIS; POWER ANALYSIS; PRIORITY JOURNAL; REFERENCE VALUE; STATISTICAL ANALYSIS; STATISTICAL SIGNIFICANCE;

BIOMEDICAL RESEARCH; CLINICAL TRIALS AS TOPIC; DATA INTERPRETATION, STATISTICAL; ENDPOINT DETERMINATION; MODELS, STATISTICAL; PROBABILITY;

EID: 79957500606     PISSN: 10543406     EISSN: 15205711     Source Type: Journal    
DOI: 10.1080/10543406.2011.552879     Document Type: Article

References (51)

1. Bauer, P. (1991). Multiple testing in clinical trials. Statistics in Medicine 10:871-890.
2. Bauer, P., Röhmen, J., Maurer, W., Hothorn, L. (1998). Testing strategies in multi-dose experiments including active control. Statistics in Medicine 17:2133-2146. (Pubitemid 28442020)
3. Benjamini, Y. (2010). Simultaneous and selective interence: Current successes and future challenges. Biometrical Journal 52:708-721.
4. Benjamini, Y., Hochberg, Y. (1995). Controlling the false discovery rate: A practical and powerful approach to multiple testing. Journal of the Royal Statistical Society, Series B 57:289-300.
5. Benjamini, Y., Hochberg, Y. (1997). Multiple hypothesis testing with weights. Scandinavian Journal of Statistics 24:407-418.
6. Benjamini, Y., Yekutieli, D. (2001). The control of the false discovery rate in multiple testing under dependency. Annals of Statistics 29:1165-1188. (Pubitemid 33575263)
7. Bergmann, B., Hommel, G. (1988). Improvements of general multiple test procedures for redundant systems of hypotheses. In: Bauer, P., Hommel, G., Sonnemann, E., eds. Multiple Hypothesenprüfung-Multiple Hypotheses Testing. New York: Springer, pp. 100-115.
8. Bernhard, G., Klein, M., Hommel, G. (2004). Global and multiple test procedures using ordered p values-A review. Statistical Papers 45:1-14. (Pubitemid 38157095)
9. Brannath, W., Bretz, F., Maurer, W., Sarkar, S. (2009). Weighted and trimmed Simes' tests for two hypotheses. Biometrical Journal 51:885-898.
10. Bretz, F., Maurer, W., Brannath, W., Posch, M. (2009). A graphical approach to sequentially rejective multiple test procedures. Statistics in Medicine 28:586-604.
11. Califf, R. M., Boolell, M., Haffner, S. M., Bethel, M. A., McMurray, J., Duggal, A., Holman, R. R. (2008). Prevention of diabetes and cardiovascular disease in patients with impaired glucose tolerance: Rationale and design of the Nateglinide And Valsartan in Impaired Glucose Tolerance Outcomes Research (NAVIGATOR) Trial. American Heart Journal 15:623-632.
12. Dmitrienko, A., Offen, W. W., Westfall, P. H. (2003). Gatekeeping strategies for clinical trials that do not require all primary effects to be significant. Statistics in Medicine 22:2387-2400. (Pubitemid 36949661)
13. European Medicines Agency. (2002). CPMP Points to Consider on Multiplicity Issues in Clinical Trials. London, UK.
14. Food and Drug Administration. (1999). Guidance for Industry-Clinical Development Programs for Drugs, Devices, and Biological Products for the Treatment of Rheumatoid Arthritis (RA). Washington, DC: U.S. Department of Health and Human Services.
15. Gabriel, K. R. (1969). Simultaneous test procedures-Some theory of multiple comparisons. Annals of Mathematical Statistics 40:224-250.
16. Genovese, C. R., Roeder, K., Wasserman, L. (2006). False discovery control with p value weighting. Biometrika 93:509-524. (Pubitemid 44599847)
17. Hochberg, Y. (1988). A sharper Bonferroni procedure for multiple significance testing. Biometrika 75:800-802.
18. Hochberg, Y., Liberman, U. (1994). An extended Simes test. Statistics and Probability Letters 21:101-105.
19. Hochberg, Y., Tamhane, A. C. (1987). Multiple Comparison Procedures. New York: Wiley.
20. Holm, S. (1979). A simple sequentially rejective multiple test procedure. Scandinavian Journal of Statistics 6:65-70.
21. Hommel, G. (1979). Entwicklung Einer Statistischen Teststrategie für Komplexe Medizinische Fragestellungen. Habilitationsschrift: Erlangen, Germany. Reprint Erlangen: Verlag Palm und Enke, 1982.
22. Hommel, G. (1983). Tests of the overall hypothesis for arbitrary dependence structures. Biometrical Journal 25:423-430.
23. Hommel, G. (1986). Multiple test procedures for arbitrary dependence structures. Metrika 33:321-336.
24. Hommel, G. (1988). A stagewise rejective multiple test procedure based on a modified Bonferroni test. Biometrika 75:383-386.
25. Hommel, G. (1989). A comparison of two modified Bonferroni procedures. Biometrika 76:624-625.
26. Hommel, G., Bernhard, G. (1999). Bonferroni procedures for logically related hypotheses. Journal of Statistical Planning and Inference 82:119-128.
27. Hommel, G., Bretz, F. (2008). Aesthetics and power considerations in multiple testing-A contradiction? Biometrical Journal 50:657-666.
28. Hommel, G., Kropf, S. (2005). Tests for differentiation in gene expression using a datadriven order or weights for hypotheses. Biometrical Journal 47:554-562. (Pubitemid 41229386)
29. Hommel, G., Bretz, F., Maurer, W. (2007). Powerful short-cuts for multiple testing procedures with special reference to gatekeeping strategies. Statistics in Medicine 26:4063-4073. (Pubitemid 47399340)
30. International Conference on Harmonization. (1998). Statistical Principles for Clinical Trials (ICH E9). Geneva, Switzerland.
31. Kieser, M., Reitmeir, P., Wassmer, G. (1995). Test procedures for clinical trials with multiple endpoints. In: Vollmar, J., ed. Biometrie in der Chemisch-Pharmazeutischen Industrie. Vol. 6. Stuttgart: Fischer Verlag, pp. 41-60.
32. Kling, Y. (2005). Issues of Multiple Hypothesis Testing in Statistical Process Control. Thesis, The Neiman Library of Exact Sciences & Engineering, Tel Aviv University, Tel Aviv, Israel.
33. Marcus, R., Peritz, E., Gabriel, K. R. (1976). On closed testing procedures with special reference to ordered analysis of variance. Biometrika 63:655-660.
34. Maurer, W., Hothorn, L. A., Lehmacher, W. (1995). Multiple comparisons in drug clinical trials and preclinical assays: a-priori ordered hypotheses. In: Vollmar, J., ed. Biometrie in der Chemisch-Pharmazeutischen Industrie. Vol. 6. Stuttgart: Fischer Verlag, pp. 3-18.
35. Maurer, W., Mellein, B. (1988). On new multiple tests based on independent p values and assessment of their power. In: Bauer, P., Hommel, G., Sonnemann, E., eds. Multiple Hypothesenprüfung-Multiple Hypotheses Testing. New York: Springer, pp. 48-66.
36. Röhmel, G., Streitberg, B. (1987). Zur Konstruktion globaler Tests. EDV in Medizin und Biologie 18:7-11.
37. Rüger, B. (1978). Das maximale signifikanzniveau des tests: "Lehne H0 ab, wenn k unter n gegebenen Tests zur Ablehnung führen." Metrika 25:171-178.
38. Samuel-Cahn, E. (1996). Is the Simes improved Bonferroni procedure conservative? Biometrika 83:928-933.
39. Sarkar, S. K. (1998). Some probability inequalities for ordered MTP2 random variables: A proof of the Simes conjecture. Annals of Statistics 26:494-504.
40. Sarkar, S. K., Chang, C.-K. (1997). The Simes method for multiple hypotheses testing with positively dependent test statistics. Journal of the American Statistical Association 92:1601-1608.
41. Savage, I. R. (1961). Probability inequalities of the Tchebycheff type. Journal of Research of the National Bureau of Standards 65B:211-222.
42. Shaffer, J. P. (1986). Modified sequentially rejective multiple test procedures. Journal of the American Statistical Association 81:826-831.
43. Simes, R. J. (1986). An improved Bonferroni procedure for multiple tests of significance. Biometrika 73:751-754.
44. Sonnemann, E., Finner, H. (1988). Vollständigkeitssätze für multiple Testprobleme. In: Bauer, P., Hommel, G., Sonnemann, E., eds. Multiple Hypothesenprüfung-Multiple Hypotheses Testing. New York: Springer, pp. 121-135.
45. Tamhane, A. J., Liu, L. (2008). On weighted Hochberg procedures. Biometrika 95:279-294. (Pubitemid 351797209)
46. Westfall, P. H., Krishen, A. (2001). Optimally weighted, fixed sequence and gatekeeper multiple testing procedures. Journal of Statistical Planning and Inference 99:25-40. (Pubitemid 33619467)
47. Westfall, P. H., Tobias, R. D. (2007). Multiple testing of general contrasts: Truncated closure and the extended Shaffer-Royen method. Journal of the American Statistical Association 102:487-494.
48. Westfall, P. H., Young, S. S. (1993). Resampling-Based Multiple Testing: Examples and Methods for P-Value Adjustment. New York: Wiley.
49. Westfall, P. H., Krishen, A., Young, S. S. (1998). Using prior information to allocate significance levels for multiple endpoints. Statistics in Medicine 17:2107-2119. (Pubitemid 28442018)
50. Wiens, B. L. (2003). A fixed sequence Bonferroni procedure for testing multiple endpoints. Pharmaceutical Statistics 2:211-215.
51. Wiens, B. L., Dmitrienko, A. (2005). The fallback procedure for evaluating a single family of hypotheses. Biopharmaceutical Statistics 15:929-942. (Pubitemid 41540979)