The pairwise multiple comparison multiplicity problem: An alternative approach to familywise and comparisonwise type I error control

Psychological Methods

Volumn 4, Issue 1, 1999, Pages 58-69

  Keselman, H J ^a   Cribbie, Robert ^a   Holland, Burt ^b  

^a UNIVERSITY OF MANITOBA   (Canada)

^b NONE

Author keywords

[No Author keywords available]

Indexed keywords

EID: 0033441052     PISSN: 1082989X     EISSN: None     Source Type: Journal    
DOI: 10.1037/1082-989X.4.1.58     Document Type: Review

References (41)

1. Bakan, D. (1966). The test of significance in psychological research. Psychological Bulletin, 66, 423-437.
2. Benjamini, Y., & Hochberg, Y. (1995). Controlling the false discovery rate: A practical and powerful approach to multiple testing. Journal of the Royal Statistical Society, Series B, 57, 289-300.
3. Benjamini, Y., Hochberg, Y., & Kling, Y. (1994). False discovery rate controlling procedures for pairwise comparisons. Unpublished manuscript.
4. Brown, M. B., & Forsythe, A. B. (1974). The small sample behavior of some statistics which test the equality of several means. Technometrics, 16, 129-132.
5. Clark, H. H. (1976). Reply to Wike and Church. Journal of Verbal Learning and Verbal Behavior, 15, 257-261.
6. Dijkstra, J. B., & Werter, P. S. P. J. (1981). Testing the equality of several means when the population variances are unequal. Communications in Statistics, Simulation and Computation, B10, 557-569.
7. Fisher, R. A. (1935). The design of experiments. Edinburgh, Scotland: Oliver & Boyd.
8. Games, P. A. (1971). Multiple comparisons of means. American Educational Research Journal, 8, 531-565.
9. Greenwald, A. G. (1975). Consequences of prejudice against the null hypothesis. Psychological Bulletin, 82, 1-20.
10. Hayter, A. J. (1986). The maximum familywise error rate of Fisher's least significant difference test. Journal of the American Statistical Association, 81, 1000-1004.
11. Hochberg, Y. (1988). A sharper Bonferroni procedure for multiple tests of significance. Biometrika, 75, 800-802.
12. Keselman, H. J. (1994). Stepwise and simultaneous multiple comparison procedures of repeated measures' means. Journal of Educational Statistics, 19, 127-162.
13. Keselman, H. J., Huberty, C. J., Lix, L. M., Olejnik, S., Cribbie, R. A., Donahue, B., Kowalchuk, R. K., Lowman, L. L., Petoskey, M. D., Keselman, J. C., & Levin, J. R. (in press). Statistical practices of educational researchers: An analysis of their ANOVA, MANOVA, and ANCOVA analyses. Review of Educational Research.
14. Levin, J. R., Serlin, R. C., & Seaman, M. A. (1994). A controlled, powerful multiple-comparison strategy for several situations. Psychological Bulletin, 115, 153-159.
15. Lix, L. M., & Keselman, H. J. (1995). Approximate degrees of freedom tests: A unified perspective on testing for mean equality. Psychological Bulletin, 117, 547-560.
16. Miller, R. G., Jr. (1981). Simultaneous statistical inference (2nd ed.). New York: Springer-Verlag.
17. Ramsey, P. H. (1978). Power differences between pairwise multiple comparisons. Journal of the American Statistical Association, 73, 479-487.
18. Ramsey, P. H. (1981). Power of univariate pairwise multiple comparison procedures. Psychological Bulletin, 90, 352-366.
19. Ramsey, P. H. (1993). Multiple comparisons of independent means. In L. Edwards (Ed.), Applied analysis of variance in the behavioral sciences (pp. 25-61). New York: Marcel Dekker.
20. Rothman, K. J. (1990). No adjustments are needed for multiple comparisons. Epidemiology, 1, 43-46.
21. Ryan, T. A. (1959). Multiple comparisons in psychological research. Psychological Bulletin, 56, 26-47.
22. Ryan, T. A. (1960). Significance tests for multiple comparison of proportions, variances and other statistics. Psychological Bulletin, 57, 318-328.
23. Ryan, T. A. (1962). The experiment as the unit for computing rates of error. Psychological Bulletin, 59, 301-305.
24. Sail, J., & Lehman, A. (1996). JMP start statistics: A guide to statistical and data analysis using JMP and JMP IN software. Cary, NC: SAS Institute.
25. SAS Institute, Inc. (1985). SAS User's Guide: Basics Version 5 Edition. Cary, NC: Author.
26. SAS Institute, Inc. (1989). SAS/IML software: Usage and reference, Version 6. Cary, NC: Author.
27. Saville, D. J. (1990). Multiple comparison procedures: The practical solution. The American Statistician, 44, 174-180.
28. Seaman, M. A., Levin, J. R., & Serlin, R. C. (1991). New developments in pairwise multiple comparisons: Some powerful and practicable procedures. Psychological Bulletin, 110, 577-586.
29. Shaffer, J. P. (1986). Modified sequentially rejective multiple test procedures. Journal of the American Statistical Association, 81, 826-831.
30. Shaffer, J. P. (1995). Multiple hypothesis testing: A review. Annual Review of Psychology, 46, 561-584.
31. Tomarken, A. J., & Serlin, R. C. (1986). Comparison of ANOVA alternatives under variance heterogeneity and specific noncentrality structures. Psychological Bulletin, 99, 90-99.
32. Toothaker, L. E. (1991). Multiple comparisons for researchers. Newbury Park, CA: Sage.
33. Tukey, J. W. (1953). The problem of multiple comparisons. In H. I. Braun (Ed.), The collected works of John W. Tukey: Vol. VIII. Multiple comparisons: 1948-1983 (pp. 1-300). New York: Chapman & Hall.
34. Welch, B. L. (1938). The significance of the difference between two means when the population variances are unequal. Biometrika, 29, 350-362.
35. Welch, B. L. (1951). On the comparison of several mean values: An alternative approach. Biometrika, 38, 330-336.
36. Welsch, R. E. (1977a). Stepwise multiple comparison procedures. Journal of the American Statistical Association, 72, 566-575.
37. Welsch, R. E. (1977b, August). Tables for stepwise multiple comparison procedures (Working Paper No. 949-77). Cambridge, MA: Massachusetts Institute of Technology.
38. Westfall, P. H., & Young, S. S. (1993). Resampling-based multiple testing: Examples and methods for p-value adjustments. New York: Wiley.
39. Wilcox, R. R. (1994). A one-way random effects model for trimmed means. Psychometrika, 59, 289-306.
40. Wilcox, R. R., Charlin, V. L., & Thompson, K. L. (1986). New Monte Carlo results on the robustness of the ANOVA F, W, and F* statistics. Communications in Statistics, Simulation and Computation, 15, 933-943.
41. Wilson, W. (1962). A note on the inconsistency inherent in the necessity to perform multiple comparisons. Psychological Bulletin, 59, 296-300.