Equivalence Tests: A Practical Primer for t Tests, Correlations, and Meta-Analyses

Social Psychological and Personality Science

Volumn 8, Issue 4, 2017, Pages 355-362

  Lakens, Daniël ^a  

^a EINDHOVEN UNIVERSITY OF TECHNOLOGY   (Netherlands)

Author keywords

equivalence testing; null hypothesis significance testing; power analysis; research methods

Indexed keywords

EID: 85021754824     PISSN: 19485506     EISSN: 19485514     Source Type: Journal    
DOI: 10.1177/1948550617697177     Document Type: Article

References (39)

1. Anderson S. F., Maxwell S. E., (2016). There’s more than one way to conduct a replication study: Beyond statistical significance. Psychological Methods, 21, 1–12. doi:https://doi.org/11037/met0000051
2. Berger R. L., Hsu J. C., (1996). Bioequivalence trials, intersection-union tests and equivalence confidence sets. Statistical Science, 11, 283–302.
3. Chow S.-C., Shao J., Wang H., (2002). A note on sample size calculation for mean comparisons based on noncentral t-statistics. Journal of Biopharmaceutical Statistics, 12, 441–456.
4. Cumming G., Fidler F., (2009). Confidence intervals: Better answers to better questions. Zeitschrift Für Psychologie/Journal of Psychology, 217, 15–26. doi:10.1027/0044-3409.217.1.15
5. Cumming G., Finch S., (2001). A primer on the understanding, use, and calculation of confidence intervals that are based on central and noncentral distributions. Educational and Psychological Measurement, 61, 532–574. doi:10.1177/0013164401614002
6. Delacre M., Lakens D., Leys C., (2017). Why psychologists should by default use Welch’s t-test instead of Student’s t-test with unequal group sizes. International Review of Social Psychology, 30, 92–101. doi:10.5334/irsp.82
7. Dienes Z., (2014). Using Bayes to get the most out of non-significant results. Quantitative Psychology and Measurement, 5, 781. doi:10.3389/fpsyg.2014.00781
8. Dienes Z., (2016). How Bayes factors change scientific practice. Journal of Mathematical Psychology. doi:10.1016/j.jmp.2015.10.003
9. Eskine K. J., (2013). Wholesome foods and wholesome morals? Organic foods reduce prosocial behavior and harshen moral judgments. Social Psychological and Personality Science, 4, 251–254. doi:10.1177/1948550612447114
10. Faul F., Erdfelder E., Lang A.-G., Buchner A., (2007). G* Power 3: A flexible statistical power analysis program for the social, behavioral, and biomedical sciences. Behavior Research Methods, 39, 175–191.
11. Finch S., Cumming G., Thomason N., (2001). Colloquium on effect sizes: The roles of editors, textbook authors, and the publication manual reporting of statistical inference in the journal of applied psychology: Little evidence of reform. Educational and Psychological Measurement, 61, 181–210. doi:10.1177/0013164401612001
12. Gigerenzer G., (1998). Surrogates for theories. Theory and Psychology, 8, 195–204.
13. Goertzen J. R., Cribbie R. A., (2010). Detecting a lack of association: An equivalence testing approach. British Journal of Mathematical and Statistical Psychology, 63, 527–537. doi:10.1348/000711009X475853
14. Hauck D. W. W., Anderson S., (1984). A new statistical procedure for testing equivalence in two-group comparative bioavailability trials. Journal of Pharmacokinetics and Biopharmaceutics, 12, 83–91. doi:10.1007/BF01063612
15. Julious S. A., (2004). Sample sizes for clinical trials with normal data. Statistics in Medicine, 23, 1921–1986. doi:10.1002/sim.1783
16. Kruschke J., (2010). Doing Bayesian data analysis: A tutorial introduction with R. Burlington, MA: Academic Press.
17. Lakens D., (2013). Calculating and reporting effect sizes to facilitate cumulative science: A practical primer for t-tests and ANOVAs. Frontiers in Psychology, 4. doi:10.3389/fpsyg.2013.00863
18. Lakens D., (2014). Performing high-powered studies efficiently with sequential analyses: Sequential analyses. European Journal of Social Psychology, 44, 701–710. doi:10.1002/ejsp.2023
19. Maxwell S. E., Kelley K., Rausch J. R., (2008). Sample size planning for statistical power and accuracy in parameter estimation. Annual Review of Psychology, 59, 537–563. doi:10.1146/annurev.psych.59.103006.093735
20. Maxwell S. E., Lau M. Y., Howard G. S., (2015). Is psychology suffering from a replication crisis? What does “failure to replicate” really mean? American Psychologist, 70, 487–498. doi:10.1037/a0039400
21. Meyners M., (2012). Equivalence tests—A review. Food Quality and Preference, 26, 231–245. doi:10.1016/j.foodqual.2012.05.003
22. Minitab 17 Statistical Software. (2010). [Computer software]. State College, PA: Minitab, Inc.
23. Moery E., Calin-Jageman R. J., (2016). Direct and conceptual replications of Eskine (2013): Organic food exposure has little to no effect on moral judgments and prosocial behavior. Social Psychological and Personality Science, 7, 312–319. doi:10.1177/1948550616639649
24. Morey R. D., Chambers C. D., Etchells P. J., Harris C. R., Hoekstra R., Lakens D., …Zwaan R. A., (2016). The peer reviewers’ openness initiative: Incentivizing open research practices through peer review. Royal Society Open Science, 3, 150547.
25. Morey R. D., Lakens D., (2017). Why most of psychology is statistically unfalsifiable. Manuscript submitted for publication.
26. Piaggio G., Elbourne D. R., Altman D. G., Pocock S. J., Evans S. J., Group C., (2006). Reporting of noninferiority and equivalence randomized trials: An extension of the CONSORT statement. Journal of the American Medical Association, 295, 1152–1160.
27. Quertemont E., (2011). How to statistically show the absence of an effect. Psychologica Belgica, 51, 109. doi:10.5334/pb-51-2-109
28. Rogers J. L., Howard K. I., Vessey J. T., (1993). Using significance tests to evaluate equivalence between two experimental groups. Psychological Bulletin, 113, 553.
29. Rouder J. N., Speckman P. L., Sun D., Morey R. D., Iverson G., (2009). Bayesian t tests for accepting and rejecting the null hypothesis. Psychonomic Bulletin & Review, 16, 225–237. doi:10.3758/PBR.16.2.225
30. Ruxton G. D., (2006). The unequal variance t-test is an underused alternative to Student’s t-test and the Mann-Whitney U test. Behavioral Ecology, 17, 688–690. doi:10.1093/beheco/ark016
31. Satterthwaite F. E., (1946). An approximate distribution of estimates of variance components. Biometrics Bulletin, 2, 110–114. doi:10.2307/3002019
32. Schuirmann D. J., (1987). A comparison of the two one-sided tests procedure and the power approach for assessing the equivalence of average bioavailability. Journal of Pharmacokinetics and Biopharmaceutics, 15, 657–680.
33. Seaman M. A., Serlin R. C., (1998). Equivalence confidence intervals for two-group comparisons of means. Psychological Methods, 3, 403–411. doi:10.1037/1082-989X.3.4.403
34. Senn S., (2007). Statistical issues in drug development (2nd ed.). Hoboken, NJ: Wiley.
35. Simonsohn U., (2015). Small telescopes detectability and the evaluation of replication results. Psychological Science, 26, 559–569. doi:10.1177/0956797614567341
36. Viechtbauer W., (2010). Conducting meta-analyses in R with the metafor package. Journal of Statistical Software, 36, 1–48.
37. Weber R., Popova L., (2012). Testing equivalence in communication research: Theory and application. Communication Methods and Measures, 6, 190–213. doi:10.1080/19312458.2012.703834
38. Welch B. L., (1947). The generalization of student’s’ problem when several different population variances are involved. Biometrika, 34, 28–35. doi:10.2307/2332510
39. Wellek S., (2010). Testing statistical hypotheses of equivalence and noninferiority (2nd ed.). Boca Raton, FL: CRC Press.