Erratum to: Damaging Real Lives through Obstinacy: Re-Emphasising Why Significance Testing is Wrong (Sociological Research Online, (2016), 21, 1, (102-115), 10.5153/sro.3857);Damaging real lives through obstinacy: Reemphasising why significance testing is wrong

Sociological Research Online

Volumn 21, Issue 1, 2016, Pages 102-115

  Gorard, Stephen ^a  

^a DURHAM UNIVERSITY   (United Kingdom)

Author keywords

Ethics of social research; Inverse probabilities; Significance testing; Statistical errors

Indexed keywords

EID: 84959299581     PISSN: None     EISSN: 13607804     Source Type: Journal    
DOI: 10.1177/1360780417731066     Document Type: Erratum

References (89)

1. AMERICAN PSYCHOLOGICAL ASSOCIATION (2010) Publication manual of the APA (6th ed.), Washington, DC.
2. BAKAN, D. (1966) The Test of Significance in Psychological Research, Psychological Bulletin, 77, p 423-437. [doi:10.1037/h0020412]
3. BERGER, J. and Sellke, T. (1987) Testing a Point Null Hypothesis: The Irreconcilability ofP Values and Evidence (with comments), Journal of the American Statistical Association, 82, 1, p 112-39. [doi:10.2307/2289131]
4. BERK, R. (2004) Regression analysis: A constructive critique, Thousand Oaks, CA: Sage.
5. BERK, R. and Freedman, D. (2001) Statistical assumptions as empirical commitments, http://www.stat.berkeley.edu/~census/berk2.pdf, accessed 030714.
6. BERKSON, J. (1938) Some difficulties of interpretation encountered in the application of the chi-square test, Journal of the American Statistical Association, 33, p 526-536 [doi:10.1080/01621459.1938.10502329]
7. BORING, E. (1919) Mathematical vs. scientific importance, Psychological Bulletin, 16, p 335-338 [doi:10.1037/h0074554]
8. CAMILLI, G. (1996) Standard errors in educational assessment: a policy analysis perspective, Education Policy Analysis Archives, 4, 4, p 17 [doi:10.14507/epaa.v4n4.1996]
9. Carlin, J., Taylor, P. and Nolan, T. (1998) School based bicycle safety education and bicycle injuries in children: a case control study, Injury Prevention, 4, p 22-7. [doi:10.1136/ip.4.1.22]
10. CARVER, R. (1978) The case against statistical significance testing, Harvard Educational Review, 48, p 378-399. [doi:10.17763/haer.48.3.t490261645281841]
11. CHATFIELD, C. (1991) Avoiding statistical pitfalls, Statistical Science, 6, 240-268. [doi:10.1214/ss/1177011686]
12. COHEN, J. (1994) 'The Earth is Round (p<.05)', American Psychologist, 49, 12, p 997-1003. [doi:10.1037/0003-066X.49.12.997]
13. CUMMING, G. (2014) The new statistics: why and how, Psychological Science, 25, 1, p 7-29. [doi:10.1177/0956797613504966]
14. DANIEL, L. (1998) Statistical significance testing: A historical overview of misuse and misinterpretation with implications for the editorial policies of educational journals, Research in the Schools, 5, 2, p 23-32
15. DEMING, W. (1975) On probability as a basis for action, American Statistician, 29, p 146-152.
16. DONNELLY C., Woodroffe R., Cox D.et al. Impact of localized badger culling on tuberculosis incidence in British cattle, NATURE, 426, p 834-837 [doi:10.1038/nature02192]
17. DONG, N. and Lipsey, M. (2011) Biases in estimating treatment effects due to attrition in randomised controlled trials, SREE Conference, http://eric.ed.gov/?id=ED517992.
18. DYREGROV, A. (1998) Psychological Debriefing-An Effective Method?, Traumatology, 4, 2, p. 6-15 [doi:10.1177/153476569800400203]
19. ENGMAN, A. (2013) Is there life afterP<0.05? Statistical significance and quantitative sociology, Quality Quantity, 47, 1, p 257-270. [doi:10.1007/s11135-011-9516-z]
20. FALK, R. and Greenbaum C. (1995) Significance tests die hard: the amazing persistence of a probabilistic misconception, Theory and Psychology, 5, p 75-98 [doi:10.1177/0959354395051004]
21. FIDLER, F., Thomson, N., Cumming, G., Finch, S. and Leeman, J. (2004) Editors Can Lead Researchers to Confidence Intervals, but Can't Make Them Think: Statistical Reform Lessons From Medicine, Psychological Science, 15, 2, p 119-126. [doi:10.1111/j.0963-7214.2004.01502008.x]
22. FILHO, D., Paranhos, R., da Rocha, E., Batista, M., da Silva, J., Santos, M. and Marino, J. (2013) W hen is statistical significance not significant?, http://www.scielo.br/pdf/bpsr/v7n1/02.pdf.
23. FREEDMAN, D. (2004) Sampling, in M. Lewis-Beck, A. Bryman and T. Liao (Eds) Sage Encyclopaedia of Social Science Research Methods (Thousand Oaks, CA: Sage), p 987-991 [doi:10.4135/9781412950589.n879]
24. GIBBS, B., Shafer, K. and Miles, A. (2015) Inferential statistics and the use of administrative data in US educational research, International Journal of Research and Method in Educatio, 10.1080/1743727X.2015.1113249 [doi:10.1080/1743727X.2015.1113249]
25. GLASS, G. (2014) Random selection, random assignment and Sir Ronald Fisher, Psychology of Education Review, 38, 1, p 12-13
26. GORARD, S. (2002) Ethics and equity: pursuing the perspective of non-participants, Social Research Update, 39, p 4
27. GORARD, S. (2003) Understanding probabilities and re-considering traditional research methods training, Sociological Research Online, 8, 1, 12 pages.
28. GORARD, S. (2006) Using everyday numbers effectively in research: Not a book about statistics, London: Continuum.
29. GORARD, S. (2013) Research Design: Robust approaches for the social sciences, London: SAGE.
30. GORARD, S. (2015b) Context and Implications for: Rethinking "quantitative" methods and the development of new researchers, Review of Education, 3, 1, p 97-99, doi: 10.1002/rev3.3042. [doi:10.1002/rev3.3042]
31. GORARD, S. (2015c) Rethinking "quantitative" methods and the development of new researchers, Review of Education, 3, 1, p 72-96, doi: 10.1002/rev3.3041. [doi:10.1002/rev3.3041]
32. GORARD, S. and Gorard, J. (2015a) What to do instead of significance testing? Calculating the 'number of counterfactual cases needed to disturb a finding', International Journal of Social Research Methodology, http://dx.doi.org/10.1080/13645579.2015.1091235. [doi:10.1080/13645579.2015.1091235]
33. GORARD, S. and Gorard, J. (2015b) Explaining the number of counterfactual cases needed to disturb a finding: a reply to Kuha and Sturgis, International Journal of Social Research Methodology, http://dx.doi.org/10.1080/13645579.2015.1126494. [doi:10.1080/13645579.2015.1126494]
34. GORARD, S., Roberts, K. and Taylor, C. (2004) What kind of creature is a design experiment?, British Educational Research Journal, 30, 4, p 575-590. [doi:10.1080/0141192042000237248]
35. GREENWALD, A. (1975) Consequences of Prejudice Against the Null Hypothesis, Psychological Bulletin, 82, p 20. [doi:10.1037/h0076157]
36. GUTTMAN, L. (1985) The Illogic of Statistical Inference for Cumulative Science, Applied Stochastic Models and Data Analysis, 1, p 3-10. [doi:10.1002/asm.3150010103]
37. HALSEY, L., Curran-Everett, D., Vowler, S. and Drummond, G. (2015) The fickle p value generates irreproducible results, Nature Methods, 12, 3, p 179-185. [doi:10.1038/nmeth.3288]
38. HUBBARD, R. and Lindsay, R. (2008) Why P Values Are Not a Useful Measure of Evidence in Statistical Significance Testing, Theory & Psychology, 18, 1, p 69-88. [doi:10.1177/0959354307086923]
39. HUBBARD, R. and Meyer, CK. (2013) The rise of statistical significance testing in public administration research and why this is a mistake, Journal of Business and Behavioral Sciences, 25, p 1.
40. HUBBARD, R. and Ryan, P. (2000) The Historical Growth of Statistical Significance Testing in Psychology—and Its Future Prospects (with discussion), Educational and Psychological Measurement, 60, p 661-696.
41. HUNTER, J. (1997) Needed: A Ban on the Significance Test, Psychological Science, 8, 1, p 3-7. [doi:10.1111/j.1467-9280.1997.tb00534.x]
42. JEFFREYS, H. (1937) Theory of probability, Oxford: Oxford University Press.
43. JONES, D. and Matloff, N. (1986) Statistical hypothesis testing in biology: a contradiction in terms, Journal of Economic Entomology, 79, p 1156-1160. [doi:10.1093/jee/79.5.1156]
44. KLINE, R. (2004) Beyond Significance Testing: Reforming Data Analysis Methods in Behavioral Research, Washington, DC: American Psychological Association. [doi:10.1037/10693-000]
45. KRUEGER, J. (2001) Null hypothesis significance testing: On the survival of a flawed method, American Psychologist, 56, p 16-26. [doi:10.1037/0003-066X.56.1.16]
46. KUHBERGER, A., Fritz, A. and Schemdl, T. (2014) Publication bias in psychology, PLOSone, doi: 10.137/journal.pome.0105825.
47. LEAHY, E. (2005) Alphas and Asterisks: The Development of Statistical Significance Testing Standards in Sociology, http://sociology.arizona.edu/sites/sociology.arizona.edu/files/files-user/Leahey_SF2005.pdf (accessed 7/12/15).
48. LE FANU J. (1999) The rise and fall of modern medicine, New York: Little, Brown.
49. LECOUTRE, B. and Poitevineau, J. (2011) The Significance Test Controversy and the Bayesian Alternative, http://statprob.com/encyclopedia/SignificanceTestControversyAndTheBayesianAlternative.html.
50. LIPSEY, M., Puzio, K., Yun, C., Hebert, M., Steinka-Fry, K., Cole, M., Roberts, M., Anthony, K. and Busick, M. (2012) Translating the statistical representation of the effects of education interventions into more readily interpretable forms, Washington DC: Institute of Education Sciences, p 13.
51. LOFTUS, G. (1991) On the tyranny of hypothesis testing in the social sciences, Contemporary Psychology, 36, p 102-105. [doi:10.1037/029395]
52. MATTHEWS, R. (1998a) Bayesian Critique of Statistics in Health: The great health hoa, http://www2.isye.gatech.edu/~brani/isyebayes/bank/pvalue.pdf.
53. MATTHEWS, R. (1998b) Flukes and flaws, Prospect, 20 November 1998. Available online at http:// www.prospectmagazine.co.uk/features/flukesandflaws (accessed 14 July2014).
54. MEEHL, P. (1967) Theory - testing in psychology and physics: A methodological paradox, Philosophy of Science, 34, p 103-115 [doi:10.1086/288135]
55. MORRISON, D. and Henkel, R. (1969) Significance tests reconsidered, American Sociologist, 4, p 131-140.
56. MORRISON, D E, Henkel, RE. (1970) Significance tests reconsidered. in: Morrison DE, Henkel RE (Eds.) The significance test controversy. A reader. Butterworth, London; 1970: p 182-198.
57. MURTONEN, M. and Lehtinen, E. (2003) Difficulties experienced by education and sociology students in quantitative methods courses, Studies in Higher Education, 28, 2, p 171-185. [doi:10.1080/0307507032000058064]
58. NELDER, J. (1999) Statistics for the Millennium: From Statistics to Statistical Science, The Statistician, 48 (Part 2), p 257-269. [doi:10.1111/1467-9884.00187]
59. NESTER, M. (1996) An Applied Statistician's Creed, Applied Statistics, 45, 4, p 401-410. [doi:10.2307/2986064]
60. NICKERSON, R. (2000) Null hypothesis significance testing: a review of an old and continuing controversy, Psychological Methods, 5, 2, p 241-301. [doi:10.1037/1082-989X.5.2.241]
61. NIX, T. and Barnette, J. (1998) The data analysis dilemma: Ban or abandon, A Review of null hypothesis significance testing, Research in the Schools, 5, 2, 3-14
62. OAKES, M. (1986) Statistical Inference: A Commentary for the Social and Behavioural Sciences, Chichester: Wiley.
63. ORLITZKY, M. (2011) How Can Significance Tests Be Deinstitutionalized?, Organizational Research Methods, 15, 2, p 199-228. [doi:10.1177/1094428111428356]
64. PARKHURST, D. (1985) Interpreting failure to reject a null hypothesis, Bulletin of the Ecological Society of America, 66: p 301-302.
65. PETROSINO, A., Turpin-Petrosino, C. and Finckenauer, J. (2000) Programs can have harmful effects!: Lessons from experiments of programs such as scared straight. Crime and Delinquency, 46, 1, p 354-379. [doi:10.1177/0011128700046003006]
66. PIGOTT, T., Valentince, J., Polanin, J., Williams, R. and Canada, D. (2013) Outcome-reporting bias in education research, Educational Researcher, 42, 8, p 424-432. [doi:10.3102/0013189X13507104]
67. PLATT, J. (1996) A History of Sociological Research Methods in the United States, Cambridge University Press.
68. ROSENTHAL, R. (1979) 'The "file drawer problem" and tolerance for null results', Psychological Bulletin, 86, p 638-641. [doi:10.1037/0033-2909.86.3.638]
69. ROZEBOOM, W. (1960) The fallacy of the null hypothesis significance test, Psychological Bulletin, 57, p 416-428. [doi:10.1037/h0042040]
70. ROZEBOOM, W. (1997) Good science is abductive not hypothetico-deductive, in Harlow, L., Mulaik, S. and Steiger. J (Eds.) What if there were no significance tests?, New Jersey: Erlbaum.
71. SALSBURG, D. (1985) The Religion of Statistics as Practiced in Medical Journals,The American Statistician, 39, p 220-223.
72. SALSBURG, D.S. 1986. Statistics for Toxicologists. New York, Marcel Dekker, Inc.
73. SHAVER, J. (1993). What statistical significance testing is, and what it is not. Journal of Experimental Education, 61, p 293-316. [doi:10.1080/00220973.1993.10806592]
74. SAVITZ, D. (1993) 'Is statistical significance testing useful in interpreting data?', Reproductive Toxicology, 7, 2, p 95-100. [doi:10.1016/0890-6238(93)90242-Y]
75. SCHMIDT, F. (1996) Statistical significance testing and cumulative knowledge in psychology: Implications for training of researchers, Psychological Methods, 1, p 115-129. [doi:10.1037/1082-989X.1.2.115]
76. SCHMIDT, F. and Hunter, J. (1997) Eight common but false objections to the discontinuation of significance testing in the analysis of research data, in Lisa A. Harlow, Stanley A. Mulaik, and James H. Steiger (Eds.) What if there were no significance tests? Mahwah, NJ: Lawrence Erlbaum Associates. p 37-64.
77. SCHWAB, A, Abrahamson, E., Starbuck, W. and Fidler, F. (2011) Researchers Should Make Thoughtful Assessments Instead of Null-Hypothesis Significance Tests, Organization Science, 22, 4, p 1105-1120. doi:10.1287/orsc.1100.0557
78. SHROUT, P. (1997) Should Significance Tests be Banned? Introduction to a Special Section Exploring the Pros and Cons, Psychological Science, 8, 1, p 2. [doi:10.1111/j.1467-9280.1997.tb00533.x]
79. SIMMONS J, Nelson L. and Simonsohn U. (2011) False-positive psychology: undisclosed flexibility in data collection and analysis allows presenting anything as significant, Psychological Science, 11: p 1359-66. doi: 10.1177/0956797611417632. [doi:10.1177/0956797611417632]
80. STANG, A., Poole, C. and Kuss, O. (2010) The ongoing tyranny of statistical significance testing in biomedical research, European Journal of Epidemiology, 25, 4, p 225-230 [doi:10.1007/s10654-010-9440-x]
81. SUTER, G. (1996) Abuse of hypothesis testing statistics in ecological risk assessment, Human and Ecological Risk Assessment, 2, p 331-347. [doi:10.1080/10807039609383611]
82. THOMPSON, B. (2004) The "significance" crisis in psychology and education, The Journal of Socio-Economics, 33, p. 607-613. [doi:10.1016/j.socec.2004.09.034]
83. TRAFIMOW, D. and Rice, S. (2009) A test of the null hypothesis significance testing procedure correlation argument, The Journal of General Psychology, 136, 3, p 261-269. [doi:10.3200/GENP.136.3.261-270]
84. TRYON, W. (1998) The Inscrutable Null Hypothesis, American Psychologist, 53, p 796. [doi:10.1037/0003- 066X.53.7.796.b]
85. TUKEY, J. (1991) The Philosophy of Multiple Comparisons, Statistical Science, 6, 1, p 100-16. [doi:10.1214/ss/1177011945]
86. WALSTER, G. and Cleary T. (1970) A Proposal for a New Editorial Policy in the Social Sciences, The American Statistician, 241, p. 16-19.
87. WATTS, D. (1991) Why is introductory statistics difficult to learn?, The American Statistician, 45, 4, p 290-291.
88. WHITE, JW, Rassweiler, A., Samhouri, J., Stier, A. and White, C. (2013) Ecologists should not use statistical significance tests to interpret simulation model results, Oikos, 123, 4, p 385-388. [doi:10.1111/j.1600-0706.2013.01073.x]
89. YATES, F. (1964) Sir Ronald Fisher and the design of experiments, Biometrics 20, p 307-321. [doi:10.2307/2528399]