Four reasons to prefer Bayesian analyses over significance testing

Psychonomic Bulletin and Review

Volumn 25, Issue 1, 2018, Pages 207-218

  Dienes, Zoltan ^a   Mclatchie, Neil ^b  

^a UNIVERSITY OF SUSSEX   (United Kingdom)

^b LANCASTER UNIVERSITY   (United Kingdom)

Author keywords

Bayes factor; Bayesian statistics; Power; Significance testing; Statistics

Indexed keywords

BAYES THEOREM; HUMAN; RESEARCH; STATISTICS;

BAYES THEOREM; HUMANS; RESEARCH; STATISTICS AS TOPIC;

EID: 85016097948     PISSN: 10699384     EISSN: 15315320     Source Type: Journal    
DOI: 10.3758/s13423-017-1266-z     Document Type: Article

References (55)

1. Baguley, T. (2009). Standardized or simple effect size: What should be reported? British Journal of Psychology, 100, 603–617.
2. Baguley, T., & Kaye, W. S. (2010). Review of Understanding psychology as a science: An introduction to scientific and statistical inference. British Journal of Mathematical & Statistical Psychology, 63, 695–698.
3. Banerjee, P., Chatterjee, P., & Sinha, J. (2012). Is it light or dark? Recalling moral behavior changes perception of brightness. Psychological Science, 23, 407–409.
4. Berry, D. A. (1996). Statistics: A Bayesian Perspective. Belmont: Duxbury.
5. Brandt, M. J., IJzerman, H., & Blanken, I. (2014). Does recalling moral behavior change the perception of brightness? Social Psychology, 45, 246–252.
6. Button, K. S., Ioannidis, J. P. A., Mokrysz, C., Nosek, B. A., Flint, J., Robinson, E. S. J., & Munafò, M. R. (2013). Power failure: Why small sample size undermines the reliability of neuroscience. Nature Reviews Neuroscience, 14, 365–376.
7. Cohen, J. (1988). Statistical power analysis for the behavioural sciences (2nd ed.). Hillsdale, NJ: Erlbaum.
8. Dienes, Z. (2008). Understanding psychology as a science: An introduction to scientific and statistical inference. Palgrave Macmillan. Website for associated online Bayes factor calculator: http://www.lifesci.sussex.ac.uk/home/Zoltan_Dienes/inference/Bayes.htm
9. Dienes, Z. (2014). Using Bayes to get the most out of non-significant results. Frontiers in Psycholology, 5, 781. doi:10.3389/fpsyg.2014.00781
10. Dienes, Z. (2015). How Bayesian statistics are needed to determine whether mental states are unconscious. In M. Overgaard (Ed.), Behavioural Methods in Consciousness Research (pp. 199–220). Oxford: Oxford University Press.
11. Dienes, Z. (2016). How Bayes factors change scientific practice. Journal of Mathematical Psychology, 72, 78–89.
12. Etz, A., & Vandekerckhove, J. (2016). A Bayesian perspective on the reproducibility project: Psychology. PLoS ONE, 11(2), e0149794. doi:10.1371/journal.pone.0149794
13. Gibney, E. (2016). Morphing neutrinos provide clue to antimatter mystery. Nature, 536, 261–262.
14. Gibson, C. E., Losee, J., & Vitiello, C. (2014). A replication attempt of stereotype susceptibility (Shih, Pittinsky, & Ambady, 1999): Identity salience and shifts in quantitative performance. Social Psychology, 45, 194–198.
15. Good, I. J. (1983). Good thinking: The foundations of probability and its applications. University of Minnesota Press.
16. Guan, M., & Vandekerckhove, J. (2016). A Bayesian approach to mitigation of publication bias. Psychonomic Bulletin and Review, 23, 74–86.
17. Ioannides, J. P. A. (2008). Why most discovered true associations are inflated. Epidemiology, 19(5), 640–648.
18. Ioannidis, J. P. A. (2005). Why most published research findings are false. Chance, 18(4), 40–47.
19. Jeffreys, H. (1939). Theory of probability. Oxford: Clarendon.
20. Johnson, D. J., Cheung, F., & Donnellan, M. B. (2014). Does cleanliness influence moral judgments? A Direct replication of Schnall, Benton, and Harvey (2008). Social Psychology, 45, 209–215.
21. Kruschke, J. K. (2013). Bayesian estimation supersedes the t test. Journal of Experimental Psychology: General, 142(2), 573–603.
22. Kruschke, J. K., Liddell, T. M. (2017) The Bayesian new statistics: hypothesis testing, estimation, meta-analysis, and power analysis from a Bayesian perspective. Psychonomic Bulletin & Review. doi:10.3758/s13423-016-1221-4
23. Kühberger, A., Fritz, A., & Scherndl, T. (2014). Publication bias in psychology: A diagnosis based on the correlation between effect size and sample size. PLoS ONE, 9(9), e105825. doi:10.1371/journal.pone.0105825
24. Lee, M. D., & Wagenmakers, E.-J. (2013). Bayesian cognitive modeling: A practical course. Cambridge: Cambridge University Press.
25. Lindley, D. V. (1957). A statistical paradox. Biometrika, 44(1–2), 187–192.
26. Lindley, D. V. (1993). The analysis of experimental data: The appreciation of tea and wine. Teaching Statistics, 15, 22–25.
27. Lindley, D. V. (2004). That wretched prior. Significance, 1, 85–87.
28. Lynott, D., Corker, K. S., Wortman, J., Connell, L., Donnellan, M. B., Lucas, R. E., & O’Brien, K. (2014). Replication of “Experiencing Physical Warmth Promotes Interpersonal Warmth” by Williams and Bargh (2008). Social Psychology, 45, 216–222.
29. Moon, A., & Roeder, S. S. (2014). A secondary replication attempt of stereotype susceptibility (Shih, Pittinsky, & Ambady, 1999). Social Psychology, 45, 199–201.
30. Morey, R. (2015). http://bayesfactor.blogspot.co.uk/2015/01/on-verbal-categories-for-interpretation.html. Accessed 2 September 2016.
31. Morey, R. D., & Rouder, J. N. (2011). Bayes factor approaches for testing interval null hypotheses. Psychological Methods, 16(4), 406–419.
32. Morey, R. D., Romeijn, J.-W., & Rouder, J. N. (2016). The philosophy of Bayes factors and the quantification of statistical evidence. Journal of Mathematical Psychology, 72, 6–18.
33. Nosek, B. A., & Lakens, D. (2014). Registered Reports: A Method to Increase the Credibility of Published Results. Social Psychology, 45, 137–141.
34. Open Science Collaboration. (2015). Estimating the reproducibility of psychological science. Science, 349(6251), 943–951.
35. Popper, K. R. (1959). The logic of scientific discovery. London: Hutchinson.
36. Popper, K. R. (1963). Conjectures and Refutations: The growth of scientific knowledge. London: Routledge.
37. Rouder, J. N. (2014). Optional Stopping: No Problem For Bayesians. Psychonomic Bulletin & Review, 21, 301–308.
38. Rouder, J. N., Morey, R. D., Verhagen, J. A., Province, J. M., & Wagenmakers, E.-J. (2016). Is there a free lunch in inference? Topics in Cognitive Science, 8, 520–547.
39. Royall, R. M. (1997). Statistical evidence: A likelihood paradigm. London: Chapman and Hall.
40. Schnall, S., Benton, J., & Harvey, S. (2008). With a clean conscience cleanliness reduces the severity of moral judgments. Psychological science, 19(12), 1219-1222.
41. Schönbrodt, F. D., Wagenmakers, E.-J., Zehetleitner, M., & Perugini, M. (2015). Sequential hypothesis testing with Bayes factors: Efficiently testing mean differences. Psychological Methods. doi:10.1037/met0000061
42. Shih, M., Pittinsky, T. L., & Ambady, N. (1999). Stereotype susceptibility: Identity salience and shifts in quantitative performance. Psychological Science, 10, 80–83.
43. Simmons, J. P., Nelson, L. D., & Simonsohn, U. (2013). Life after P-Hacking. Unpublished manuscript: http://papers.ssrn.com/sol3/papers.cfm?abstract_id=2205186. Accessed 9 November 2015.
44. van Elk, M., Matzke, D., Gronau, Q. F., Guan, M., Vandekerckhove, J., & Wagenmakers, E.-J. (2015). Meta-analyses are no substitute for registered replications: A skeptical perspective on religious priming. Frontiers in Psychology, 6, 1365.
45. Vanpaemel, W. (2010). Prior sensitivity in theory testing: An apologia for the Bayes factor. Journal of Mathematical Psychology, 54, 491–498.
46. Vanpaemel, W. (2011). Constructing informative model priors using hierarchical methods. Journal of Mathematical Psychology, 55, 106–117.
47. Vanpaemel, W., & Lee, M. D. (2012). Using priors to formalize theory: Optimal attention and the generalized context model. Psychonomic Bulletin & Review, 19, 1047–1056.
48. Verhagen, A. J., & Wagenmakers, E.-J. (2014). Bayesian tests to quantify the result of a replication attempt. Journal of Experimental Psychology: General, 143, 1457–1475.
49. Wagenmakers, E.-J., Lee, M. D., Rouder, J. N., & Morey, R. D. (2014). Another statistical paradox. http://www.ejwagenmakers.com/submitted/AnotherStatisticalParadox.pdf. Accessed 9 November 2015
50. Wagenmakers, E.-J., Verhagen, A. J., Ly, A., Bakker, M., Lee, M. D., Matzke, D., … Morey, R. D. (2015). A power fallacy. Behavior Research Methods, 47(4), 913–917.
51. Wetzels, R., Matzke, D., Lee, M. D., Rouder, J. N., Iverson, G. J., & Wagenmakers, E. J. (2011). Statistical evidence in experimental psychology: An empirical comparison using 855 t tests. Perspectives on Psychological Science, 6, 291–298.
52. Wilcox, R. R. (2005). Introduction to robust estimation and hypothesis testing (2nd ed.). Academic: London.
53. Williams, L. E., & Bargh, J. A. (2008). Experiencing physical warmth promotes interpersonal warmth. Science, 322, 306–307.
54. Ziliak, S. T., & McCloskey, D. N. (2008). The cult of statistical significance: How the standard error cost us jobs, justice and lives. Ann Arbor: The University of Michigan Press.
55. Ziori, E., & Dienes, Z. (2015). Facial beauty affects implicit and explicit learning of men and women differently. Frontiers in Psycholology, 6, 1124. doi:10.3389/fpsyg.2015.01124