Pairwise multiple comparison test procedures: An update for clinical child and adolescent psychologists

Journal of Clinical Child and Adolescent Psychology

Volumn 33, Issue 3, 2004, Pages 623-645

  Keselman, H J ^a   Cribbie, Robert A ^b   Holland, Burt ^c  

^a UNIVERSITY OF MANITOBA   (Canada)

^b YORK UNIVERSITY   (Canada)

^c TEMPLE UNIVERSITY   (United States)

Author keywords

[No Author keywords available]

Indexed keywords

ADOLESCENT; ANALYSIS OF VARIANCE; ARTICLE; CHILD; CHILD PSYCHIATRY; COMPARATIVE STUDY; HUMAN; METHODOLOGY; SENSITIVITY AND SPECIFICITY; STATISTICS;

ADOLESCENT; ADOLESCENT PSYCHIATRY; ANALYSIS OF VARIANCE; CHILD; CHILD PSYCHIATRY; HUMANS; RESEARCH DESIGN; SENSITIVITY AND SPECIFICITY;

EID: 4444326101     PISSN: 15374416     EISSN: None     Source Type: Journal    
DOI: 10.1207/s15374424jccp3303_19     Document Type: Article

References (86)

1. Akaike, H. (1974). A new look at the statistical model identification. IEEE Transactions on Automatic Control, AC-19, 716-723.
2. Bellanti, C. J., & Bierman, K. L. (2000). Disentangling the impact of low cognitive ability and inattention on social behavior and peer relationships. Journal of Clinical Child Psychology, 29, 66-75.
3. Benjamini, Y., & Hochberg, Y. (1995). Controlling the false discovery rate: A practical and powerful approach to multiple testing. Journal of the Royal Statistical Society, Series B, 57, 289-300.
4. Benjamini, Y., & Hochberg, Y. (2000). On the adaptive control of the false discovery rate in multiple testing with independent statistics. Journal of Educational and Behavioral Statistics, 25, 60-83.
5. Benjamini, Y., Hochberg, Y., & Kling, Y. (1994). False discovery rate controlling procedures for pairwise comparisons. Unpublished manuscript.
6. Bofinger, E., Hayter, A. J., & Liu, W. (1993). The construction of upper confidence bounds on the range of several location parameters. Journal of the American Statistical Association, 88, 906-911.
7. Callan, D. E., Jones, J. A., Munhall, K., Callan, A. M., Kroos, C., & Vatikiotis-Bateson, E. (2003). Neural processes underlying perceptual enhancement of visual speech gestures. Neuroreport, 14, 2213-2218.
8. Carlson, C. L., & Mann, M. (2002). Sluggish cognitive tempo predicts a different pattern of impairment in the attention deficit hyperactivity disorder, predominantly inattentive type. Journal of Clinical Child and Adolescent Psychology, 31, 123-129.
9. Cheung, S. H., & Chan, W. S. (1996). Simultaneous confidence intervals for pairwise multiple comparisons in a two-way unbalanced design. Biometrics, 52, 463-472.
10. Copenhaver, M. D., & Holland, B. (1988). Computation of the distribution of the maximum Studentized range statistic with application to multiple significance testing of simple effects. Journal of Statistical Computation and Simulation, 30,1-15.
11. Cribbie, R. A., & Keselman, H. J. (2003a). Pairwise multiple comparisons: A model comparison approach versus stepwise procedures. British Journal of Mathematical and Statistical Psychology, 56, 167-182.
12. Cribbie, R. A., & Keselman, H. J. (2003b). The effects of non-normality on parametric, nonparametric and model comparison approaches to pairwise comparisons. Educational and Psychological Measurement, 63, 615-635.
13. Daly, S. L., & Glenwick, D. S. (2000). Personal adjustment and perceptions of grandchild behavior in custodial grandmothers. Journal of Clinical Child Psychology, 29, 108-118.
14. Dayton, C. M. (1998). Information criteria for the paired-comparisons problem. The American Statistician, 52, 144-151.
15. Dudoit, S., Shaffer, J. P., & Boldrick, J. C. (2003). Multiple hypothesis testing in microarray experiments. Statistical Science, 18, 71-103.
16. Duncan, D. B. (1955). Multiple range and multiple F tests. Biometrics, 11, 1-42.
17. Dunnett, C. W. (1980). Pairwise multiple comparisons in the unequal variance case. Journal of the American Statistical Association, 75, 796-800.
18. Einot, I., & Gabriel, K. R. (1975). A study of the powers of several methods of multiple comparisons. Journal of the American Statistical Association, 70, 574-583.
19. Epkins, C. C. (2000). Cognitive specificity in internalizing and externalizing problems in community and clinic-referred children. Journal of Clinical Child Psychology, 29, 199-208.
20. Fisher, R. A. (1935). The design of experiments. Edinburgh, England: Oliver & Boyd.
21. Games, P. A. (1971). Multiple comparisons of means. American Educational Research Journal, 8, 531-565.
22. Games, P. A., & Howell, J. F. (1976). Pairwise multiple comparison procedures with unequal n's and/or variances. Journal of Educational Statistics, 1, 113-125.
23. Genovese, C., & Wasserman, L. (2002). Operating characteristics and extensions of the False Discovery Rate procedure. Journal of the Royal Statistical Society, Series B, 64, 1151-1160.
24. Gross, A. M. (1976). Confidence interval robustness with long-tailed symmetric distributions. Journal of the American Statistical Association, 71, 409-416.
25. Hampel, F. R., Ronchetti, E. M., Rousseeuw, P. J., & Stahel, W. A. (1986). Robust statistics: The approach based on influence functions. New York: Wiley.
26. Hancock, G. R., & Klockars, A. J. (1996). The quest for α: Developments in multiple comparison procedures in the quarter century since Games (1971). Review of Educational Research, 66, 269-306.
27. Hayter, A. J. (1984). A proof of the conjecture that the Tukey-Kramer multiple comparisons procedure is conservative. Annals of Statistics, 12, 61-75.
28. Hayter, A. J. (1986). The maximum familywise error rate of Fisher's least significant difference test. Journal of the American Statistical Association, 81, 1000-1004.
29. Hochberg, Y. (1988). A sharper Bonferroni procedure for multiple tests of significance. Biometrika, 75, 800-802.
30. Hochberg, Y., & Tamhane, A. C. (1987). Multiple comparison procedures. New York: Wiley.
31. Holland, B., & Cheung, S. H. (2002). Family size robustness criteria for multiple comparison procedures. Journal of the Royal Statistical Society, Series B, 64, 63-77.
32. Hsuing, T., & Olejnik, S. (1994). Power of pairwise multiple comparisons in the unequal variance case. Communications in Statistics: Simulation and Computation, 23, 691-710.
33. Huang, C. J., & Dayton C. M. (1995). Detecting patterns of bivariate mean vectors using model-selection criteria. British Journal of Mathematical and Statistical Psychology, 48, 129-147.
34. Huber, P. J. (1981). Robust statistics. New York: Wiley.
35. Jaccard, J., & Guilamo-Ramos, V. (2002a). Analysis of variance frameworks in clinical child and adolescent psychology: Advanced issue and recommendations. Journal of Clinical Child Psychology, 31, 278-294.
36. Jaccard, J., & Guilamo-Ramos, V. (2002b). Analysis of variance frameworks in clinical child and adolescent psychology: Issues and recommendations. Journal of Clinical Child and Adolescent Psychology, 31, 130-146.
37. Keselman, H. J., Cribbie, R. A., & Holland, B. (1999). The pairwise multiple comparison multiplicity problem: An alternative approach to familywise and comparisonwise Type I error control. Psychological Methods, 4, 58-69.
38. Keselman, H. J., Games, P. A., & Rogan, J. C. (1979). Protecting the overall rate of Type 1 errors for pairwise comparisons with an omnibus test statistic. Psychological Bulletin, 86, 884-888.
39. Keselman, H. J., Huberty, C. J., Lix, L. M., Olejnik, S., Cribbie, R. A., Donahue, B., et al. (1998). Statistical practices of educational researchers: An analysis of their ANOVA, MANOVA, and ANCOVA analyses. Review of Educational Research, 68, 350-386.
40. Keselman, H. J., Keselman, J. C., & Games, P. A. (1991). Maximum familywise Type I error rate: The least significant difference, Newman-Keuls, and other multiple comparison procedures. Psychological Bulletin, 110, 155-161.
41. Keselman, H. J., Lix, L. M., & Kowalchuk, R. K. (1998). Multiple comparison procedures for trimmed means. Psychological Methods, 3, 123-141.
42. Keselman, H. J., Othman, A. R., Wilcox, R. R., & Fradette, K. (2004). The new and improved two-sample t test. Psychological Science, 15, 47-51.
43. Keselman, H. J., Wilcox, R. R., & Lix, L. M. (2003). A generally robust approach to hypothesis testing in independent and correlated groups designs. Psychophysiology, 40, 586-596.
44. Keuls, M. (1952). The use of the "Studentized range" in connection with an analysis of variance. Euphytica, 1, 112-122.
45. Kirk, R. E. (1995). Experimental design: Procedures for the behavioral sciences. Toronto: Brooks/Cole.
46. Kramer, C. Y. (1956). Extension of the multiple range test to group means with unequal numbers of replications. Biometrics, 12, 307-310.
47. Leary, M. R. (1983). A brief version of the Fear of Negative Evaluation Scale. Personality and Social Psychology Bulletin, 9, 371-375.
48. Levin, J. R., Serlin, R. C., & Seaman, M. A. (1994). A controlled, powerful multiple-comparison strategy for several situations. Psychological Bulletin, 115, 153-159.
49. Lix, L. M., & Keselman, H. J. (1995). Approximate degrees of freedom tests: A unified perspective on testing for mean equality. Psychological Bulletin, 117, 547-560.
50. Marcus, R., Peritz, E., & Gabriel, K. R. (1976). On closed testing procedures with special reference to ordered analysis of variance. Biometrika, 63, 655-660.
51. Micceri, T. (1989). The unicorn, the normal curve, and other improbable creatures. Psychological Bulletin, 105, 156-166.
52. Miller, R. G., Jr. (1981). Simultaneous statistical inferencen(2nd ed.). New York: Springer-Verlag.
53. Newman, D. (1939). The distribution of the range in samples from a normal population, expressed in terms of an independent estimate of standard deviation. Biometrika, 31, 20-30.
54. Norusis, M. J. (2002). SPSS 11.0 Guide to data analysis. Upper Saddle River, NJ: Prentice Hall.
55. Olejnik, S., & Lee, J. (1990, April). Multiple comparison procedures when population variances differ. Paper presented at the annual meeting of the American Educational Research Association, Boston.
56. Pearson, E. S. (1931). The analysis of variance in cases of non-normal variation. Biometrika, 23, 114-133.
57. Podolski, C.-L., & Nigg, J. T. (2001). Parent stress and coping in relation to child ADHD severity and associated child disruptive behavior problems. Journal of Clinical Child Psychology, 30, 503-513.
58. Rothman, K. (1990). No adjustments are needed for multiple comparisons. Epidemiology, 1, 43-46.
59. Ryan, T. A. (1960). Significance tests for multiple comparison of proportions, variances, and other statistics. Psychological Bulletin, 57, 318-328.
60. Ryan, T. A. (1962). The experiment as the unit for computing rates of error. Psychological Bulletin, 59, 305.
61. Sarkar, S. K. (1998). Probability inequalities for ordered MTP2 random variables: A proof of the Simes conjecture. Annals of Statistics, 26, 494-504.
62. Sarkar, S. K., & Chang, C.-K. (1997). The Simes method for multiple hypothesis testing with positively dependent test statistics. Journal of the American Statistical Association, 92, 1601-1608.
63. SAS Institute Inc. (1989). SAS/IML software: Usage and reference, version 6 (1st ed.). Cary, NC: Author.
64. SAS Institute Inc. (1999). SAS/STAT user's guide, Version 7. Cary, NC: Author.
65. Satterthwaite, F. E. (1946). An approximate distribution of estimates of variance components. Biometrics Bulletin, 2, 110-114.
66. Saville, D. J. (1990). Multiple comparison procedures: The practical solution. The American Statistician, 44, 174-180.
67. Scheffé, H. (1959). The analysis of variance. New York: Wiley.
68. Seaman, M. A., Levin, J. R., & Serlin, R. C. (1991). New developments in pairwise multiple comparisons: Some powerful and practicable procedures. Psychological Bulletin, 110, 577-586.
69. Staudte, R. G., & Sheather, S. J. (1990). Robust estimation and testing. New York: Wiley.
70. Toothaker, L. E. (1991). Multiple comparisons for researchers. Newbury Park, CA: Sage.
71. Tukey, J. W. (1953). The problem of multiple comparisons. Unpublished manuscript, Princeton University.
72. Tukey, J. W. (1960). A survey of sampling from contaminated normal distributions. In I. Olkin et al. (Eds.), Contributions to probability and statistics. Stanford University Press, Stanford.
73. Watson, D., & Friend, R. (1969). Measurement of social-evaluative anxiety. Journal of Consulting and Clinical Psychology, 33, 448-457.
74. Welch, B. L. (1938). The significance of the difference between two means when population variances are unequal. Biometrika, 38, 330-336.
75. Welch, B. L. (1951). On the comparison of several mean values: An alternative approach. Biometrika, 38, 330-336.
76. Welsch, R. E. (1977). Stepwise multiple comparison procedures. Journal of the American Statistical Association, 72, 566-575.
77. Westfall, P. H., Tobias, R. D., Rom, D., Wolfinger, R. D., & Hochberg, Y. (1999). Multiple comparisons and multiple tests. Cary, NC: SAS Institute.
78. Westfall, P. H., & Young, S. S. (1993). Resampling-based multiple testing: Examples and methods for p-value adjustment. New York: Wiley.
79. Wilcox, R. R. (1990). Comparing the means of two independent groups. Biometrics Journal, 32, 771-780.
80. Wilcox, R. R. (1995). ANOVA: The practical importance of heteroscedastic methods, using trimmed means versus means, and designing simulation studies. British Journal of Mathematical and Statistical Psychology, 48, 99-114.
81. Wilcox, R. R. (1997). Introduction to robust estimation and hypothesis testing. San Diego: Academic Press.
82. Wilcox, R. R. (2002). Understanding the practical advantages of modern ANOVA methods. Journal of Clinical Child and Adolescent Psychology, 31, 399-412.
83. Wilcox, R. R. (2003). Applying contemporary statistical techniques. New York: Academic.
84. Wilcox, R. R., Charlin, V. L., & Thompson, K. L. (1986). New Monte Carlo results on the robustness of the ANOVA F, W and F* statistics. Communications in Statistics: Simulation and Computation, 15, 933-943.
85. Wilcox, R. R., & Keselman, H. J. (2003). Modern robust data analysis methods: Measures of central tendency. Psychological Methods, 8, 254-274.
86. Yuen, K. K., & Dixon, W. J. (1973). The approximate behavior of the two-sample trimmed t. Biometrika, 60, 369-374.