Many Tests of Significance: New Methods for Controlling Type I Errors

Psychological Methods

Volumn 16, Issue 4, 2011, Pages 420-431

  Keselman, H J ^a   Miller, Charles W ^b   Holland, Burt ^c  

^a UNIVERSITY OF MANITOBA   (Canada)

^b MERCK RESEARCH LABORATORIES   (United States)

^c TEMPLE UNIVERSITY   (United States)

Author keywords

FDR control; FWER; K FWER; Stepwise methods; Type I error control

Indexed keywords

ARTICLE; METHODOLOGY; PSYCHOLOGY; STATISTICAL MODEL; STATISTICS;

MODELS, STATISTICAL; PSYCHOLOGY; RESEARCH DESIGN; STATISTICS AS TOPIC;

EID: 83755224807     PISSN: 1082989X     EISSN: None     Source Type: Journal    
DOI: 10.1037/a0025810     Document Type: Article

References (28)

1. Benjamini Y., Hochberg Y. Controlling the false discovery rate: A practical and powerful approach to multiple testing. Journal of the Royal Statistical Society B 1995, 57:289-300.
2. Bryk A.S., Raudenbush S.W. Hierarchical linear models: Applications and data analysis methods 1992, Sage, Newbury Park, CA.
3. Dunn O.J. Multiple comparisons among means. Journal of the American Statistical Association 1961, 56:52-64. doi: 10.2307/2282330.
4. Dunnett C.W. A multiple comparison procedure for comparing several treatments with a control. Journal of the American Statistical Association 1955, 50:1096-1121. doi: 10.2307/2281208.
5. Efron B., Tibshirani R., Storey J.D., Tusher V. Empirical Bayes analysis of a microarray experiment. Journal of the American Statistical Association 2001, 96:1151-1160.
6. Graham J.M. Self-expansion and flow in couples' momentary experiences: An experience sampling study. Journal of Personality and Social Psychology 2008, 95:679-694. doi: 10.1037/0022-3514.95.3.679.
7. Hochberg Y. A sharper Bonferroni procedure for multiple tests of significance. Biometrika 1988, 75:800-802. doi: 10.1093/biomet/75.4.800.
8. Hochberg Y., Tamhane A.C. Multiple comparison procedures 1987, Wiley, New York, NY.
9. Holm S. A simple sequentially rejective multiple test procedure. Scandinavian Journal of Statistics 1979, 6:65-70.
10. Joe H. Multivariate models and dependency concepts. CRC Monographs on Statistics & Applied Probability 1997, 19-56. Chapman & Hall, New York, NY.
11. Karlin S., Rinott Y. Classes of orderings of measures and related correlation inequalities: I. Multivariate totally positive distributions. Journal of Multivariate Analysis 1980, 10:467-498. doi: 10.1016/0047-259X(80)90065-2.
12. Keselman H.J., Cribbie R., Holland B. The pairwise multiple comparison multiplicity problem: An alternative approach to familywise/comparisonwise Type I error control. Psychological Methods 1999, 4:58-69. doi: 10.1037/1082-989X.4.1.58.
13. Lehmann E.L., Romano J.P. Generalizations of the familywise error rate. Annals of Statistics 2005, 33:1138-1154. doi: 10.1214/009053605000000084.
14. Maxwell S.E., Delaney H.D. Designing experiments and analyzing data: A model comparison perspective 2004, Erlbaum, Mahwah, NJ. 2nd ed.
15. Miller R.G. Simultaneous statistical inference 1981, McGraw-Hill, New York, NY. 2nd ed.
16. Romano J.P., Shaikh A.M. Step-up procedures for control of generalizations of the familywise error rate. Annals of Statistics 2006, 34:1850-1873. doi: 10.1214/009053606000000461.
17. Rothman K.J. No adjustments are needed for multiple comparisons. Epidemiology 1990, 1:43-46. doi: 10.1097/00001648-199001000-00010.
18. Ryan T.A. The experiment as the unit for computing rates of error. Psychological Bulletin 1962, 59:301-305. doi: 10.1037/h0040562.
19. Sarkar S.K. Some probability inequalities for ordered MTP2 random variables: A proof of the Simes conjecture. Annals of Statistics 1998, 26:494-504. doi: 10.1214/aos/1028144846.
20. Sarkar S.K. A note on the monotonicity of the critical values of a step-up test. Journal of Statistical Planning and Inference 2000, 87:241-249. doi: 10.1016/S0378-3758(99)00200-1.
21. Sarkar S.K. Some results on false discovery rate in stepwise multiple testing procedures. Annals of Statistics 2002, 30:239-257. doi: 10.1214/aos/1015362192.
22. Sarkar S.K. Generalizing Simes' test and Hochberg's stepup procedure. Annals of Statistics 2008, 36:337-363. doi: 10.1214/009053607000000550.
23. Sarkar S.K., Chang C.K. The Simes method for multiple hypothesis testing with positively dependent test statistics. Journal of the American Statistical Association 1997, 92:1601-1608. doi: 10.2307/2965431.
24. Spanier G.B. Measuring dyadic adjustment: New scales for assessing the quality of marriage and similar dyads. Journal of Marriage & the Family 1976, 38:15-28. doi: 10.2307/350547.
25. Tenorio-Martínez R., Lara-Munoz M.D., Medina-Mora M.E. Measurement of problems in activities and participation in patients with anxiety, depression and schizophrenia using the ICF check-list. Social Psychiatry and Psychiatric Epidemiology 2009, 44:377-384. doi: 10.1007/s00127-008-0449-3.
26. Tukey, J. W. (1953). The problem of multiple comparisons. Unpublished manuscript, Department of Statistics, Princeton University.
27. Westfall P.H., Tobias R.D., Rom D., Wolfinger R.D., Hochberg Y. Multiple comparisons and multiple tests 1999, SAS Institute, Cary, NC.
28. Westfall P.H., Young S.S. Resampling-based multiple testing 1993, Wiley, New York, NY.