Beyond reporting statistical significance: Identifying informative effect sizes to improve scientific communication

Public Understanding of Science

Volumn 28, Issue 4, 2019, Pages 468-485

  Hanel, Paul HP ^a   Mehler, David MA ^b,c  

^a UNIVERSITY OF BATH   (United Kingdom)

^b CARDIFF UNIVERSITY   (United Kingdom)

^c UNIVERSITY OF MÜNSTER   (Germany)

Author keywords

Cohen s d; Cohen s U3; effect size; scientific communication; statistical communication; statistical significance

Indexed keywords

EID: 85062728405     PISSN: 09636625     EISSN: 13616609     Source Type: Journal    
DOI: 10.1177/0963662519834193     Document Type: Article

References (52)

1. Allen CPG, Mehler DMA, (2018) Open science challenges, benefits and tips in early career and beyond. Psyarxiv Preprints. Available at: https://doi.org/10.31234/osf.io/3czyt
2. Bain PG, Hornsey MJ, Bongiorno R, Jeffries C, (2012) Promoting pro-environmental action in climate change deniers. Nature Climate Change 2(8): 600–603.
3. Baker DA, Ware JM, Schweitzer NJ, Risko EF, (2017) Making sense of research on the neuroimage bias. Public Understanding of Science 26(2): 251–258.
4. Bonney R, Shirk JL, Phillips TB, Wiggins A, Ballard HL, Miller-Rushing AJ. (2014) Next steps for citizen science. Science 343(6178): 1436–1437.
5. Bowley SAL, (1915) An Elementary Manual of Statistics. London: P.S. King & Son.
6. Brown University Science Center (2014) Quick guide to science communication. Retrieved from https://www.brown.edu/academics/science-center/sites/brown.edu.academics.science-center/files/uploads/Quick_Guide_to_Science_Communication_0.pdf
7. Cohen J, (1988) Statistical Power Analysis for the Behavioral Sciences, 2nd ed. Hillsdale, NY: Erlbaum.
8. Cooper BEJ, Lee WE, Goldacre BM, Sanders TAB, (2012) The quality of the evidence for dietary advice given in UK national newspapers. Public Understanding of Science 21(6): 664–673.
9. Cumming G, (2014) The new statistics: Why and how. Psychological Science 25(1): 7–29.
10. Faul F, Erdfelder E, Buchner A, Lang AG, (2009) Statistical power analyses using G*Power 3.1: Tests for correlation and regression analyses. Behavior Research Methods 41(4): 1149–1160.
11. Feingold A, (1994) Gender differences in personality: A meta-analysis. Psychological Bulletin 116(3): 429–456.
12. Gardiner A, Sullivan M, Grand A, (2018) Who are you writing for? Differences in response to blog design between scientists and nonscientists. Science Communication 40(1): 109–123.
13. Haller H, Krauss S, (2002) Misinterpretations of significance: A problem students share with their teachers? Methods of Psychological Research Online 7(12). Available at: http://www.metheval.uni-jena.de/lehre/0405-ws/evaluationuebung/haller.pdf
14. Hanel PHP, Maio GR, Manstead ASR, (2018) A new way to look at the data: Similarities between groups of people are large and important. Journal of Personality and Social Psychology. Epub ahead of print 31 December. DOI: 10.1037/pspi0000154.
15. Hoekstra R, Morey RD, Rouder JN, Wagenmakers EJ, (2014) Robust misinterpretation of confidence intervals. Psychonomic Bulletin & Review 21(5): 1157–1164.
16. Hoffrage U, Gigerenzer G, (1998) Using natural frequencies to improve diagnostic inferences. Journal of the Association of American Medical Colleges 73(5): 538–540.
17. Im S, Varma K, Varma S, (2017) Extending the seductive allure of neuroscience explanations effect to popular articles about educational topics. British Journal of Educational Psychology 87: 518–534.
18. Inman HF, Bradley EL, (1989) The overlapping coefficient as a measure of agreement between probability distributions and point estimation of the overlap of two normal densities. Communications in Statistics—Theory and Methods 18(10): 3851–3874.
19. Jaffee S, Hyde JS, (2000) Gender differences in moral orientation: A meta-analysis. Psychological Bulletin 126(5): 703–726.
20. Kahan DM, Peters E, Dawson EC, Slovic P, (2013) Motivated numeracy and enlightened self-government (Cultural Cognition Project Working Paper No 116). Available at: https://papers.ssrn.com/sol3/papers.cfm?abstract_id=2319992
21. Lakens D, (2013) Calculating and reporting effect sizes to facilitate cumulative science: A practical primer for t-tests and ANOVAs. Frontiers in Psychology 4; 863.
22. Lakens D, (2017) Equivalence tests: A practical primer for t tests, correlations, and meta-analyses. Social Psychological and Personality Science 8: 355–362.
23. Lazard A, Atkinson L, (2015) Putting environmental infographics center stage: The role of visuals at the elaboration likelihood model’s critical point of persuasion. Science Communication 37(1): 6–33.
24. Lichtenstein S, Slovic P, Fischhoff B, Layman B, Combs B, (1978) Judged frequency of lethal events. Journal of Experimental Psychology: Human Learning and Memory 4(6): 551.
25. McGraw KO, Wong SP, (1992) A common language effect size statistic. Psychological Bulletin 111: 361–365.
26. Marshall IJ, Wolfe CDA, McKevitt C, (2012) Lay perspectives on hypertension and drug adherence: Systematic review of qualitative research. BMJ 345: e3953.
27. Miguel E, Camerer C, Casey K, Cohen J, Esterling KM, Gerber A. (2014) Promoting transparency in Social Science Research. Science 343(6166): 30–31.
28. Montoya RM, Horton RS, Vevea JL, Citkowicz M, Lauber EA, (2017) A re-examination of the mere exposure effect: The influence of repeated exposure on recognition, familiarity, and liking. Psychological Bulletin 143(5): 459–498.
29. Morey RD, Chambers CD, Etchells PJ, Harris CR, Hoekstra R, Lakens D. (2016) The peer reviewers’ openness initiative. Royal Society Open Science. Available at: http://www.research.ed.ac.uk/portal/en/publications/the-peer-reviewers-openness-initiative(4c5dfd3a-fa82-4fb9-994b-bfadb09f711b).html
30. Nakagawa S, Cuthill IC, (2007) Effect size, confidence interval and statistical significance: A practical guide for biologists. Biological Reviews 82(4): 591–605.
31. National Institutes of Health (2017) A checklist for communicating science and health research to the public. Available at: https://www.nih.gov/institutes-nih/nih-office-director/office-communications-public-liaison/clear-communication/science-health-public-trust/checklist-communicating-science-health-research-public
32. Nicoll A, Laukamm-josten U, Mwizarubi B, Mayala C, Mkuye M, Nyembela G. (1993) Lay health beliefs concerning HIV and AIDS—a barrier for control programmes. AIDS Care 5(2): 231–241.
33. Norman SM, Avolio BJ, Luthans F, (2010) The impact of positivity and transparency on trust in leaders and their perceived effectiveness. The Leadership Quarterly 21(3): 350–364.
34. Nurse P, (2006) US biomedical research under siege. Cell 124(1): 9–12.
35. Plavén-Sigray P, Matheson GJ, Schiffler BC, Thompson WH, (2017) Research: The readability of scientific texts is decreasing over time. Elife 6: e27725.
36. Posavac EJ, Sinacore JM, (1984) Improving the understanding of statistical significance: Reporting Effect Size. Knowledge 5(4): 503–508.
37. Research Councils UK (n.d.) Public engagement with research strategy. Available at: http://www.rcuk.ac.uk/documents/publications/rcukperstrategy-pdf/
38. Richard FD, Bond CF, Jr Stokes-Zoota JJ, (2003) One hundred years of social psychology quantitatively described. Review of General Psychology 7(4): 331–363.
39. Ruscio J, (2008) A probability-based measure of effect size: Robustness to base rates and other factors. Psychological Methods 13(1): 19–30.
40. Samuelson W, Zeckhauser R, (1988) Status quo bias in decision making. Journal of Risk and Uncertainty 1(1): 7–59.
41. Schindler PL, Thomas CC, (1993) The structure of interpersonal trust in the workplace. Psychological Reports 73(2): 563–573.
42. Science Media Center (2012) 10 best practice guidelines for reporting science & health stories. Available at: http://www.sciencemediacentre.org/wp-content/uploads/2012/09/10-best-practice-guidelines-for-science-and-health-reporting.pdf
43. Sedgwick P, (2014) Clinical significance versus statistical significance. BMJ 348: g2130.
44. Spence A, Poortinga W, Butler C. (2011) Perceptions of climate change and willingness to save energy related to flood experience. Nature Climate Change 1(1): 46–49.
45. Sullivan GM, Feinn R, (2012) Using effect size—or why the p value is not enough. Journal of Graduate Medical Education 4(3): 279–282.
46. Sumner P, Vivian-Griffiths S, Boivin J, Pidgeon NF, (2014) The association between exaggeration in health related science news and academic press releases: Retrospective observational study. BMJ 349: g7015.
47. Szucs D, Ioannidis JPA, (2017) Empirical assessment of published effect sizes and power in the recent cognitive neuroscience and psychology literature. PLOS Biology 15(3): e2000797.
48. Thompson B, (2002) What future quantitative social science research could look like: Confidence intervals for effect sizes. Educational Researcher 31(3): 25–32.
49. Thomson ME, Önkal D, Avcioğlu A, Goodwin P, (2004) Aviation risk perception: A comparison between experts and novices. Risk Analysis 24(6): 1585–1595.
50. Tromovitch P, (2015) The lay public’s misinterpretation of the meaning of ‘significant’: A call for simple yet significant changes in scientific reporting. Journal of Research Practice 11(1): P1.
51. Tversky A, Kahneman D, (1973) Availability: A heuristic for judging frequency and probability. Cognitive Psychology 5(2): 207–232.
52. Zajonc RB, (1968) Attitudinal effects of mere exposure. Journal of Personality and Social Psychology 9: 1–27.