Consequences of assumption violations revisited: A quantitative review of alternatives to the one-way analysis of variance F test

Review of Educational Research

Volumn 66, Issue 4, 1996, Pages 579-619

  Lix, Lisa M ^a   Keselman, Joanne C ^a   Keselman, H J ^a  

^a UNIVERSITY OF MANITOBA   (Canada)

Author keywords

[No Author keywords available]

Indexed keywords

EID: 0040517457     PISSN: 00346543     EISSN: None     Source Type: Journal    
DOI: 10.3102/00346543066004579     Document Type: Article

References (82)

1. Aiken, L. S., & West, S. G. (1991). Multiple regression: Testing and interpreting interactions. Newbury Park, CA: Sage.
2. * Alexander, R. A., & Govern, D. M. (1994). A new and simpler approximation for ANOVA under variance heterogeneity. Journal of Educational Statistics, 19, 91-101.
3. Algina, J. (1994). Some alternative approximate tests for a split plot design. Multivariate Behavioral Research, 29, 365-384.
4. Algina, J., & Olejnik, S. F. (1984). Implementing the Welch-James procedure with factorial designs. Educational and Psychological Measurement, 44, 39-48.
5. * Algina, J., Oshima, T. C., & Lin, W. Y. (1994). Type I error rates for Welch's test and James's second-order test under nonnormality and inequality of variance when there are two groups. Journal of Educational and Behavioral Statistics, 19, 275-291.
6. Algina, J., Oshima, T. C., & Tang, K. L. (1991). Robustness of Yao's, James', and Johansen's tests under variance-covariance heteroscedasticity and nonnormality. Journal of Educational Statistics, 16, 125-139.
7. Allison, D. B., Gorman, B. S., & Primavera, L. H. (1993). Some of the most common questions asked of statistical consultants: Our favorite responses and recommended readings. Genetic, Social, and General Psychology Monographs, 119, 153-185.
8. * Alyasin, M. M. (1991). An empirical investigation of the behavior of some parametric and nonparametric tests for frequently encountered data in educational research. Dissertation Abstracts International, 52, 1721A. (University Microfilms No. D A9129714)
9. * Bishop, T. A. (1976). Heteroscedastic ANOVA, MANOVA, and multiple comparisons. Dissertation Abstracts International, 37, 5736B. (University Microfilms No. 77-10, 494)
10. Bishop, T. A., & Dudewicz, E. J. (1978). Exact analysis of variance with unequal variances: Test procedures and tables. Technometrics, 20, 419-430.
11. Blair, R. C., & Higgins, J. J. (1980). A comparison of the power of the Wilcoxon's ranksum statistic to that of student's t statistic under various non-normal distributions. Journal of Educational Statistics, 5, 309-335.
12. * Boehnke, K. (1984). F-and H-test assumptions revisited. Educational and Psychological Measurement, 44, 609-617.
13. * Boneau, C. A. (1960). The effects of violations of assumptions underlying the t test. Psychological Bulletin, 57, 49-64.
14. Box, G. E. P. (1954). Some theorems on quadratic forms applied in the study of analysis of variance problems: I. Effect of inequality of variance in the one-way classification. Annals of Mathematical Statistics, 25, 290-302.
15. Bradley, J. V. (1978). Robustness? British Journal of Mathematical and Statistical Psychology, 31, 144-152.
16. * Bradley, J. V. (1980). Nonrobustness in classical tests on means and variances: A large-scale sampling study. Bulletin of the Psychonomic Society, 15, 275-278.
17. * Brown, M. B., & Forsythe, A. B. (1974). The small sample behavior of some statistics which test the equality of several means. Technometrics, 16, 129-132.
18. * Budescu, D. V., & Appelbaum, M. I. (1981). Variance stabilizing transformations and the power of the F test. Journal of Educational Statistics, 6, 55-74.
19. Chatterjee, S., & Yilmaz, M. (1992). A review of regression diagnostics for behavioral research. Applied Psychological Measurement, 16, 209-227.
20. * Clinch, J. J., & Keselman, H. J. (1982). Parametric alternatives to the analysis of variance. Journal of Educational Statistics, 7, 207-214.
21. Cochran, W. G. (1947). Some consequences when the assumptions for the analysis of variance are not satisfied. Biometrics, 3, 22-38.
22. * Dijkstra, J. B., & Werter, P. S. P. J. (1981). Testing the equality of several means when the population variances are unequal. Communications in Statistics - Simulation and Computation, B10, 557-569.
23. Dixon, W. J. (1992). BMDP statistical software manual (Vol. 1). Los Angeles: University of California Press.
24. Eisenhart, C. (1947). The assumptions underlying the analysis of variance. Biometrics, 3, 1-21.
25. * Feir-Walsh, B. J., & Toothaker, L. E. (1974). An empirical comparison of the ANOVA F test, normal scores test and Kruskal-Wallis test under violations of assumptions. Educational and Psychological Measurement, 34, 789-799.
26. * Fenstad, G. U. (1983). A comparison between the U and V tests in the Behrens-Fisher problem. Biometrika, 70, 300-302.
27. * Gabriel, K. R., & Lachenbruch, P. A. (1967). Non-parametric ANOVA in small samples: A Monte Carlo study of the adequacy of the asymptotic approximation (Mimeo Series No. 569). Chapel Hill: University of North Carolina.
28. Glass, G. V, Peckham, P. D., & Sanders, J. R. (1972). Consequences of failure to meet assumptions underlying the fixed effects analyses of variance and covariance. Review of Educational Research, 42, 237-288.
29. Harwell, M. R. (1992). Summarizing Monte Carlo results in methodological research. Journal of Educational Statistics, 17, 297-313.
30. Harwell, M. R., Rubinstein, E. N., Hayes, W. S., & Olds, C. C. (1992). Summarizing Monte Carlo results in methodological research: The one-and two-factor fixed effects ANOVA cases. Journal of Educational Statistics, 17, 315-339.
31. * Havelick, L. L., & Peterson, N. L. (1974). Robustness of the t test: A guide for researchers on the effect of violations of assumptions. Psychological Reports, 34, 1095-1114.
32. Hedges, L. V. (1994). Fixed effects models. In H. Cooper & L. V. Hedges (Eds.), The handbook of research synthesis (pp. 285-299). New York: Russell Sage Foundation.
33. Hedges, L. V., & Olkin, I. (1985). Statistical methods for meta-analysis. New York: Harcourt, Brace, Jovanovich.
34. Hsiung, T. H., Olejnik, S., & Huberty, C. J. (1994). Comment on a Wilcox test statistic for comparing means when variances are unequal. Journal of Educational Statistics, 19, 111-118.
35. Hsiung, T. H., Olejnik, S., & Oshima, T. C. (1994). A SAS/IML program for applying the James second-order test in two-factor fixed-effect ANOVA models. Educational and Psychological Measurement, 54, 696-698.
36. James, G. S. (1951). The comparison of several groups of observations when the ratios of the population variances are unknown. Biometrika, 38, 324-329.
37. Keselman, H. J., Kowalchuk, R. K., & Lix, L. M. (1996). Robust nonorthogonal analyses revisited: An update based on trimmed means and Winsorized variances. Manuscript submitted for publication.
38. Keselman, H. J., Lix, L. M., & Kowalchuk, R. K. (1996). Multiple comparison procedures for trimmed means. Manuscript submitted for publication.
39. * Kohr, R. L., & Games, P. A. (1974). Robustness of the analysis of variance, the Welch procedure and a Box procedure to heterogeneous variances. Journal of Experimental Education, 43, 61-69.
40. Kruskal, W. H., & Wallis, W. A. (1952). Use of ranks in one-criterion variance analysis. Journal of the American Statistical Association, 47, 583-621.
41. Krutchkoff, R. G. (1988). One-way fixed effects analysis of variance when the error variances may be unequal. Journal of Statistical Computation and Simulation, 30, 259-271.
42. * Levine, D. W., & Dunlap, W. P. (1982). Power of the F test with skewed data: Should one transform or not? Psychological Bulletin, 92, 272-280.
43. * Levy, K. J. (1978). An empirical comparison of the ANOVA F-test with alternatives which are more robust against heterogeneity of variance. Journal of Statistical Computation and Simulation, 8, 49-57.
44. * Lin, L. I. K., & Sanford, R. L. (1983). The robustness of the likelihood ratio test, the nonparametric sum rank test, and F-ratio tests when the populations are from the negative binomial family. Communications in Statistics - Simulation and Computation, 12, 523-529.
45. Lix, L. M., & Keselman, H. J. (1995). Approximate degrees of freedom tests: A unified perspective on testing for mean equality. Psychological Bulletin, 117, 547-560.
46. Lix, L. M., Keselman, H. J., & Carriere, K. C. (1995). To trim or not to trim: Tests of location equality under heteroscedasticity and nonnormality. Manuscript submitted for publication.
47. * McSweeney, M., & Penfield, D. (1969). The normal scores test for the c-sample problem. British Journal of Mathematical and Statistical Psychology, 22, 177-192.
48. Micceri, T. (1989). The unicorn, the normal curve, and other improbable creatures. Psychological Bulletin, 105, 156-164.
49. * Nath, R., & Duran, B. S. (1981). The rank transform in the two-sample location problem. Communications in Statistics - Simulation and Computation, B10, 383-394.
50. * Neave, H. R., & Granger, C. W. J. (1968). A Monte Carlo study comparing various two-sample tests for differences in mean. Technometrics, 10, 509-522.
51. * Norton, D. W. (1952). An empirical investigation of the effects of nonnormality and heterogeneity upon the F-test of analysis of variance. Unpublished doctoral dissertation, State University of Iowa.
52. Oshima, T. C., & Algina, J. (1992a). A SAS program for testing the hypothesis of the equal means under heteroscedasticity: James's second-order test. Educational and Psychological Measurement, 52, 117-118.
53. m test under heteroscedasticity and non-normality. British Journal of Mathematical and Statistical Psychology, 42, 255-263.
54. Pearson, E. S. (1931). The analysis of variance in cases of non-normal variation. Biometrika, 23, 114-133.
55. * Ramsey, P. P. (1977). An empirical investigation of Type I error under conditions of heterogeneity of variance and sample size: The one-way fixed effects model. Unpublished doctoral dissertation, Hofstra University.
56. * Randolph, E. A. (1990). Type I error of the one-way fixed effects analysis of variance when the assumption of homogeneous variances is violated and sample sizes are equal. Dissertation Abstracts International, 53, 923B. (University Microfilms No. DA9220055)
57. * Randolph, E. A., Barcikowski, R. S., & Robey, R. R. (1994, April). Choosing a viable alternative to ANOVA when variances are unequal: Power and Type I error of ANOVA and ANOVA alternatives. Paper presented at the Annual Meeting of the American Educational Research Association, New Orleans, LA.
58. * Rasmussen, J. L. (1985). The power of Student's t and Wilcoxon w statistics. Evaluation Review, 9, 505-510.
59. * Rasmussen, J. L. (1986). An evaluation of parametric and non-parametric tests on modified and non-modified data. British Journal of Mathematical and Statistical Psychology, 39, 213-220.
60. Rubin, A. S. (1983). The use of weighted contrasts in analysis of models with heterogeneity of variance. Proceedings of the Business and Economic Statistics Section, American Statistical Association, 347-352.
61. SAS Institute, Inc. (1989). SAS/STAT user's guide, Version 5 (Vol. 2, 4th ed.). Cary, NC: Author.
62. SAS Institute, Inc. (1990). SAS/IML software: Usage and reference, Version 6. Cary, NC: Author.
63. Sawilowsky, S. S. (1990). Nonparametric tests of interaction in experimental design. Review of Educational Research, 60, 91-126.
64. Scheffe, H. (1959). The analysis of variance. New York: Wiley.
65. SPSS, Inc. (1990). SPSS reference guide. Chicago: Author.
66. Stegman, C. E., Timm, N. H., & Hritz, R. J. (1973). The Behrens-Fisher problem: Review of univariate and multivariate solutions (Working Paper No. 20). Pittsburgh, PA: University of Pittsburgh, Department of Educational Research.
67. Stevens, J. P. (1984). Outliers and influential data points in regression analysis. Psychological Bulletin, 95, 334-344.
68. * Tan, W. Y., & Tabatabai, M. A. (1986). Some Monte Carlo studies on the comparison of several means under heteroscedasticity and robustness with respect to departure from normality. Biometrical Journal, 28, 801-814.
69. Tang, K. L., & Algina, J. (1993). Performance of four multivariate tests under variance-covariance heteroscedasticity. Multivariate Behavioral Research, 28, 391-405.
70. * Tomarken, A. J., & Serlin, R. C. (1986). Comparison of ANOVA alternatives under variance heterogeneity and specific noncentrality structures. Psychological Bulletin, 99, 90-99.
71. Welch, B. L. (1951). On the comparison of several mean values: An alternative approach. Biometrika, 38, 330-336.
72. * Wike, E. W., & Church, J. D. (1982). Nonrobustness in F tests: 2. Further extensions of Bradley's study. Bulletin of the Psychonomic Society, 20, 168-170.
73. Wilcox, R. R. (1987). New designs in analysis of variance. Annual Review of Psychology, 38, 29-60.
74. * Wilcox, R. R. (1988). A new alternative to the ANOVA F and new results on James's second-order method. British Journal of Mathematical and Statistical Psychology, 41, 109-117.
75. Wilcox, R. R. (1989). Adjusting for unequal variances when comparing means in one-way and two-way fixed effects ANOVA models. Journal of Educational Statistics, 14, 269-278.
76. Wilcox, R. R. (1993). Robustness in ANOVA. In L. K. Edwards (Ed.), Applied analysis of variance in behavioral science (pp. 345-374). New York: Marcel Dekker.
77. Wilcox, R. R. (1994). A one-way random effects model for trimmed means. Psychometrika, 59, 289-306.
78. Wilcox, R. R. (1995). ANOVA: A paradigm for low power and misleading measures of effect size? Review of Educational Research, 65, 51-77.
79. Wilcox, R. R. (1996). Statistics for the social sciences. New York: Academic Press.
80. * Wilcox, R. R., Charlin, V. L., & Thompson, K. L. (1986). New Monte Carlo results on the robustness of the ANOVA F, W, and F* statistics. Communications in Statistics - Simulation and Computation, 15, 933-943.
81. * Winter, J. M. (1976). A comparison of several analysis of variance statistics by Type I errorrates. Dissertation Abstracts International, 37, 1769B. (University Microfilms No. 7014411)
82. * Zimmerman, D. W. (1985). Power functions of the t test and Mann-Whitney U test under violation of parametric assumptions. Perceptual and Motor Skills, 61, 467-470.