Modern Robust Statistical Methods: An Easy Way to Maximize the Accuracy and Power of Your Research

American Psychologist

Volumn 63, Issue 7, 2008, Pages 591-601

  Erceg Hurn, David M ^a   Mirosevich, Vikki M ^b  

^a UNIVERSITY OF WESTERN AUSTRALIA   (Australia)

^b DEPARTMENT OF HEALTH   (Australia)

Author keywords

effect size; nonparametric statistics; robust statistics; significance testing; software

Indexed keywords

ARTICLE; COMPUTER PROGRAM; HUMAN; NONPARAMETRIC TEST; RESEARCH; STATISTICAL ANALYSIS; STATISTICS;

DATA INTERPRETATION, STATISTICAL; HUMANS; RESEARCH; SOFTWARE; STATISTICS, NONPARAMETRIC;

EID: 55349126094     PISSN: 0003066X     EISSN: None     Source Type: Journal    
DOI: 10.1037/0003-066X.63.7.591     Document Type: Article

References (63)

1. Abebe A., Crimin K., and McKean J.W. Rank-based procedures for linear models: Applications to pharmaceutical science data. Drug Information Journal 35 (2001) 947-971
2. Acion L., Peterson J.J., Temple S., and Ardnt S. Probabilistic index: An intuitive non-parametric approach to measuring the size of treatment effects. Statistics in Medicine 25 (2006) 591-602
3. Akritas M.G., Arnold S.F., and Brunner E. Nonparametric hypotheses and rank statistics for unbalanced factorial designs. Journal of the American Statistical Association 92 (1997) 258-265
4. Algina J., Keselman H.J., and Penfield R.D. An alternative to Cohen's standardized mean difference effect size: A robust parameter and confidence interval in the two independent groups case. Psychological Methods 10 (2005) 317-328
5. Algina J., Keselman H., and Penfield R. Effect sizes and their intervals: The two-level repeated measures case. Educational and Psychological Measurement 65 (2005) 241-258
6. Algina J., Keselman H.J., and Penfield R.D. Confidence interval coverage for Cohen's effect size statistic. Educational and Psychological Measurement 66 (2006) 945-960
7. Algina J., Keselman H., and Penfield R. Confidence intervals for an effect size when variances are not equal. Journal of Modern Applied Statistical Methods 5 (2006) 2-13
8. American Psychological Association. Publication manual of the American Psychological Association. 5th ed. (2001), Author, Washington, DC
9. Boneau C.A. The effects of violations of assumptions underlying the t test. Psychological Bulletin 57 (1960) 49-64
10. Box G.E.P. Non-normality and tests on variances. Biometrika 40 (1953) 318-335
11. Bradley J.V. A common situation conducive to bizarre distribution shapes. The American Statistician 31 (1977) 147-150
12. Bradley J.V. Robustness?. British Journal of Mathematical and Statistical Psychology 31 (1978) 144-152
13. Bradley J.V. Nonrobustness in Z, t, and F tests at large sample sizes. Bulletin of the Psychonomic Society 16 (1980) 333-336
14. Brunner E., Domhof S., and Langer F. Nonparametric analysis of longitudinal data in factorial experiments (2002), Wiley, New York
15. Brunner E., and Puri M.L. Nonparametric methods in factorial designs. Statistical Papers 42 (2001) 1-52
16. Conover W.J., and Iman R.L. Rank transformations as a bridge between parametric and nonparametric statistics. The American Statistician 35 (1981) 121-129
17. Crimin, K., Abebe, A., & McKean, J. W. (in press). Robust general linear models and graphics via a user interface. Journal of Modern Applied Statistical Methods. (Available from Ash Abebe at abebeas@auburn.edu or from Joe McKean at joseph.mckean@wmich.edu)
18. Cumming G., Fidler F., Leonard M., Kalinowski P., Christiansen A., Kleinig A., et al. Statistical reform in psychology: Is anything changing?. Psychological Science 18 (2007) 230-232
19. D'Agostino R. Tests for the normal distribution. In: D'Agostino R.B., and Stephens M.A. (Eds). Goodness-of-fit techniques (1986), Dekker, New York 367-420
20. Fahoome, G., & Sawilowsky, S. S. (2000, April 24-28). Review of twenty nonparametric statistics and their large sample approximations. Paper presented at the annual meeting of the American Education Research Association. Retrieved from http://www.eric.ed.gov/ERICWebPortal/contentdelivery/servlet/ERICServlet?accno=ED441031
21. Glass G.V., and Hopkins K.D. Statistical methods in education and psychology. 3rd ed. (1996), Allyn & Bacon, Boston
22. Glass G.V., Peckham P.D., and Sanders J.R. Consequences of failure to meet assumptions underlying the fixed effects analyses of variance and covariance. Review of Educational Research 42 (1972) 237-288
23. Grissom R.J. Probability of the superior outcome of one treatment over another. Journal of Applied Psychology 79 (1994) 314-316
24. Grissom R.J. Heterogeneity of variance in clinical data. Journal of Consulting and Clinical Psychology 68 (2000) 155-165
25. Grissom R.J., and Kim J.J. Review of assumptions and problems in the appropriate conceptualization of effect size. Psychological Methods 6 (2001) 135-146
26. Grissom R.J., and Kim J.J. Effect sizes for research: A broad practical approach (2005), Erlbaum, Mahwah, NJ
27. Harwell M.R. Summarizing Monte Carlo results in methodological research. Journal of Educational Statistics 17 (1992) 297-313
28. Harwell M.R., Rubinstein E.N., Hayes W.S., and Olds C.C. Summarizing Monte Carlo results in methodological research: The one- and two-factor fixed effects ANOVA cases. Journal of Educational Statistics 17 (1992) 315-339
29. Hettmansperger T.P., and McKean J.W. Robust nonparametric statistical methods (1998), Arnold, London
30. Higgins J.J. Introduction to modern nonparametric statistics (2004), Brooks/Cole, Pacific Grove, CA
31. Hultsch D.F., MacDonald S.W.S., and Dixon R.A. Variability in reaction time performance of younger and older adults. Journals of Gerontology, Series B: Psychological Sciences and Social Sciences 57 (2002) P101-P115
32. Jaccard J., and Guilamo-Ramos V. Analysis of variance frameworks in clinical child and adolescent psychology: Advanced issues and recommendations. Journal of Clinical Child Psychology 31 (2002) 278-294
33. Keppel G., and Wickens T.D. Design and analysis: A researcher's handbook. 4th ed. (2004), Pearson, Upper Saddle River, NJ
34. Keselman H.J., Algina J., Lix L.M., Wilcox R.R., and Deering K. A generally robust approach for testing hypotheses and setting confidence intervals for effect sizes. Psychological Methods 13 (2008) 110-129
35. Keselman H.J., Huberty C.J., Lix L.M., Olejnik S., Cribbie R.A., Donahue B., et al. Statistical practices of educational researchers: An analysis of their ANOVA, MANOVA, and ANCOVA analyses. Review of Educational Research 68 (1998) 350-386
36. Kraemer H.C., and Kupfer D.J. Size of treatment effects and their importance to clinical research and practice. Biological Psychiatry 59 (2006) 990-996
37. Kromrey, J. D., & Coughlin, K. B. (2007, November). ROBUST_ES: A SAS macro for computing robust estimates of effect size. Paper presented at the annual meeting of the SouthEast SAS Users Group, Hilton Head, SC. Retrieved from http://analytics.ncsu.edu/sesug/2007/PO19.pdf
38. Leech, N. L., & Onwuegbuzie, A. J. (2002, November). A call for greater use of nonparametric statistics. Paper presented at the Annual Meeting of the Mid-South Educational Research Association. Retrieved from http://www.eric.ed.gov/ERICWebPortal/contentdelivery/servlet/ERICServlet?accno=ED471346
39. Lindquist E.F. Design and analysis of experiments in psychology and education (1953), Houghton Mifflin, Boston
40. Lix L.M., Keselman J.C., and Keselman H.J. Consequences of assumption violations revisited: A quantitative review of alternatives to the one-way analysis of variance "F" test. Review of Educational Research 66 (1996) 579-619
41. McGraw K.O., and Wong S.P. A common language effect size statistic. Psychological Bulletin 111 (1992) 361-365
42. McKean J.W. Robust analysis of linear models. Statistical Science 19 (2004) 562-570
43. Micceri T. The unicorn, the normal curve, and other improbable creatures. Psychological Bulletin 105 (1989) 156-166
44. Miller J. A warning about median reaction time. Journal of Experimental Psychology: Human Perception and Performance 14 (1988) 539-543
45. Onwuegbuzie A.J., and Levin J.R. Without supporting statistical evidence, where would reported measures of substantive importance lead us? To no good effect. Journal of Modern Applied Statistical Methods 2 (2003) 131-151
46. Ramsey P.H. Exact Type I error rates for robustness of Student's t test with unequal variances. Journal of Educational Statistics 5 (1980) 337-349
47. SAS Institute. SAS/STAT 9.1 user's guide (2004). http://support.sas.com/documentation/onlinedoc/91pdf/sasdoc_91/stat_ug_7313.pdf Retrieved February 1, 2008, from http://support.sas.com/documentation/onlinedoc/91pdf/sasdoc_91/stat_ug_7313.pdf
48. Sawilowsky S.S. Nonparametric tests of interaction in experimental design. Review of Educational Research 60 (1990) 91-126
49. Sawilowsky S.S., and Blair R.C. A more realistic look at the robustness and Type II error properties of the t test to departures from population normality. Psychological Bulletin 111 (1992) 352-360
50. Serlin R.C., and Harwell M.R. More powerful tests of predictor subsets in regression analysis under nonnormality. Psychological Methods 9 (2004) 492-509
51. Shah D.A., and Madden L.V. Nonparametric analysis of ordinal data in designed factorial experiments. Phytopathology 94 (2004) 33-43
52. Taylor D.H. Latency models for reaction time distributions. Psychometrika 30 (1965) 157-163
53. Terpstra J.T., and McKean J.W. Rank-based analyses of linear models using R. Journal of Statistical Software 14 (2005). http://www.jstatsoft.org/v14/i07 Retrieved from http://www.jstatsoft.org/v14/i07
54. Toothaker L.E., and Newman D. Nonparametric competitors to the two-way ANOVA. Journal of Educational and Behavioral Statistics 19 (1994) 237-273
55. Vargha A., and Delaney H.D. A critique and improvement of the CL common language effect size statistics of McGraw and Wong. Journal of Educational and Behavioral Statistics 25 (2000) 101-132
56. Wilcox R.R. How many discoveries have been lost by ignoring modern statistical methods?. American Psychologist 53 (1998) 300-314
57. Wilcox R.R. Fundamentals of modern statistical methods (2001), Springer, New York
58. Wilcox R.R. Applying contemporary statistical techniques (2003), Academic Press, San Diego, CA
59. Wilcox R.R. Introduction to robust estimation and hypothesis testing. 2nd ed. (2005), Academic Press, San Diego, CA
60. Wilcox R.R., Charlin V.L., and Thompson K.L. New Monte Carlo results on the robustness of the ANOVA f, w and f statistics. Communications in Statistics: Simulation and Computation 15 (1986) 933-943
61. Wilcox R.R., and Keselman H.J. Modern robust data analysis methods: Measures of central tendency. Psychological Methods 8 (2003) 254-274
62. Zimmerman D.W. Invalidation of parametric and nonparametric statistical tests by concurrent violation of two assumptions. Journal of Experimental Education 67 (1998) 55-68
63. Zimmerman D.W. Statistical significance levels of nonparametric tests biased by heterogeneous variances of treatment groups. Journal of General Psychology 127 (2000) 354-364