The insignificance of statistical significance testing

Journal of Wildlife Management

Volumn 63, Issue 3, 1999, Pages 763-772

  Johnson, Douglas H ^a  

^a U S GEOLOGICAL SURVEY   (United States)

Author keywords

Bayesian approaches; Confidence interval; Interval; Null hypothesis; P value; Power analysis; Scientific hypothesis test; Statistical hypothesis test

Indexed keywords

DATA INTERPRETATION; STATISTICAL ANALYSIS;

EID: 0032808670     PISSN: 0022541X     EISSN: None     Source Type: Journal    
DOI: 10.2307/3802789     Document Type: Article

References (65)

1. ABELSON, R. P. 1997. A retrospective on the significance test ban on 1999 (If there were no significance tests, they would be invented). Pages 117-141 in L. L. Harlow, S. A. Mulaik, and J. H. Steiger, editors. What if there were no significance tests? Lawrence Erlbaum Associates, Mahwah, New Jersey, USA.
2. ANSCOMBE, F. J. 1956. Discussion on Dr. David's and Dr. Johnson's paper. Journal of the Royal Statistical Society 18:24-27.
3. BAKAN, D. 1966. The test of significance in psychological research. Psychological Bulletin 66:423-437.
4. BARNARD, G. 1998. Pooling probabilities. New Scientist 157:47.
5. BAUERNFEIND, R. H. 1968. The need for replication in educational research. Phi Delta Kappan 50: 126-128.
6. BAYES, T. 1763. An essay toward solving a problem in the doctrine of chances. Philosophical Transactions of the Royal Society, London 53:370-418.
7. BERGER, J. O. 1985. Statistical decision theory and Bayesian analysis. Springer-Verlag, Berlin, Germany.
8. _, AND D. A. BERRY. 1988. Statistical analysis and illusion of objectivity. American Scientist 76: 159-165.
9. _, AND M. DELAMPADY. 1987. Testing precise hypotheses. Statistical Science 2:317-352.
10. _, AND T. SELLKE. 1987. Testing a point null hypothesis: the irreconcilability of P values and evidence. Journal of the American Statistical Association 82:112-122.
11. BERKSON, J. 1938. Some difficulties of interpretation encountered in the application of the chi-square test. Journal of the American Statistical Association 33:526-542.
12. BOX, G. E. P. 1980. Sampling and Bayes' inference in scientific modelling and robustness. Journal of the Royal Statistical Society 143:383-430.
13. _, AND G. C. TIAO. 1973. Bayesian inference in statistical analysis. Addison-Wesley, Reading, Massachusetts, USA.
14. BUCKLAND, S. T., K. P. BURNHAM, AND N. H. AUGUSTIN. 1997. Model selection: an integrated part of inference. Biometrics 53:603-618.
15. BURNHAM, K. P., AND D. R. ANDERSON. 1998. Model selection and inference: a practical information-theoretic approach. Springer-Verlag, New York, New York, USA.
16. CAMPBELL, M. 1992. Confidence intervals. Royal Statistical Society News and Notes 18:4-5.
17. CARLIN, B. P., AND T. A. LOUIS. 1996. Bayes and empirical Bayes methods for data analysis. Chapman & Hall, London, United Kingdom.
18. CARVER, R. P. 1978. The case against statistical significance testing. Harvard Educational Review 48:378-399.
19. CLARK, C. A. 1963. Hypothesis testing in relation to statistical methodology. Review of Educational Research 33:455-473.
20. COHEN, J. 1988. Statistical power analysis for the behavioral sciences. Second edition. Lawrence Erlbaum Associates, Hillsdale, New Jersey, USA.
21. _. 1994. The earth is round ( p < .05). American Psychologist 49:997-1003.
22. DAYTON, P. K. 1998. Reversal of the burden of proof in fisheries management. Science 279:821-822.
23. DEGROOT, M. H. 1970. Optimal statistical decisions. McGraw-Hill, New York, New York, USA.
24. DEMING, W. E. 1975. On probability as a basis for action. American Statistician 29:146-152.
25. ELLISON, A. M. 1996. An introduction to Bayesian inference for ecological research and environmental decision-making. Ecological Applications 6:1036-1046.
26. GERARD, P. D., D. R. SMITH, AND G. WEERAKKODY. 1998. Limits of retrospective power analysis. Journal of Wildlife Management 62:801-807.
27. GOOD, I. J. 1982. Standardized tail-area probabilities. Journal of Statistical Computation and Simulation 16:65-66.
28. GUTTMAN, L. 1985. The illogic of statistical inference for cumulative science. Applied Stochastic Models and Data Analysis 1:3-10.
29. HEDGES, L. V., AND I. OLKIN. 1985. Statistical methods for meta-analysis. Academic Press, New York. New York, USA.
30. HOLLING, C. S., editor. 1978. Adaptive environmental assessment and management. John Wiley & Sons, Chichester, United Kingdom.
31. HOWSON, C., AND P. URBACH. 1991. Bayesian reasoning in science. Nature 350:371-374.
32. HUBERTY, C. J. 1993. Historical origins of statistical testing practices: the treatment of Fisher versus Neyman-Pearson views in textbooks. Journal of Experimental Education 61:317-333.
33. JOHNSON, D. H. 1995. Statistical sirens: the allure of nonparametrics. Ecology 76:1998-2000.
34. LOFTUS, G. R. 1991. On the tyranny of hypothesis testing in the social sciences. Contemporary Psychology 36:102-105.
35. MATTHEWS, R. 1997. Faith, hope and statistics. New-Scientist 156:36-39.
36. MCLEAN, J. E., AND J. M. ERNEST. 1998. The role of statistical significance testing in educational research. Research in the Schools 5:15-22.
37. MEEHL, P. E. 1997. The problem is epistemology, not statistics: replace significance tests by confidence intervals and quantify accuracy of risky numerical predictions. Pages 393-425 in L. L. Harlow, S. A. Mulaik, and J. H. Steiger, editors. What if there were no significance tests? Lawrence Erlbaum Associates, Mahwah, New Jersey, USA.
38. MULAIK, S. A., N. S. RAJU, AND R. A. HARSHMAN. 1997. There is a time and a place for significance testing. Pages 65-115 in L. L. Harlow, S. A. Mulaik, and J. H. Steiger, editors. What if there were no significance tests? Lawrence Erlbaum Associates, Mahwah, New Jersey, USA.
39. NESTER, M. R. 1996. An applied statistician's creed. Applied Statistics 45:401-410.
40. NUNNALLY, J. C. 1960. The place of statistics in psychology. Educational and Psychological Measurement 20:641-650.
41. PETERMAN, R. M. 1990. Statistical power analysis can improve fisheries research and management. Canadian Journal of Fisheries and Aquatic Sciences 47:2-15
42. PLATT, J. R. 1964. Strong inference. Science 146:347-353.
43. POPPER, K. R. 1959. The logic of scientific discovery. Basic Books, New York, New York, USA.
44. PRATT, J. W., H. RAIFFA, AND R. SCHLAIFER. 1995. Introduction to statistical decision theory. MIT Press, Cambridge, Massachusetts, USA.
45. PREECE, D. A. 1990. R. A. Fisher and experimental design: a review. Biometrics 46:925-935.
46. QUINN, J. F., AND A. E. DUNHAM. 1983. On hypothesis testing in ecology and evolution. American Naturalist 122:602-617.
47. REICHARDT, C. S, AND H. F. GOLLOB. 1997. When confidence intervals should be used instead of statistical tests, and vice versa. Pages 259-284 in L. L. Harlow, S. A. Mulaik, and J. H. Steiger, editors. What if there were no significance tests? Lawrence Erlbaum Associates, Mahwah, New Jersey, USA.
48. RINDSKOPF, D. M. 1997. Testing "small," not null, hypotheses: classical and Bayesian approaches. Pages 319-332 in L. L. Harlow, S. A. Mulaik, and J. H. Steiger, editors. What if there were no significance tests? Lawrence Erlbaum Associates, Mahwah, New Jersey, USA.
49. SAVAGE, I. R. 1957. Nonparametric statistics. Journal of the American Statistical Association 52:331-344.
50. SCHMIDT, F. L., AND J. E. HUNTER. 1997. Eight common but false objections to the discontinuation of significance testing in the analysis of research data. Pages 37-64 in L. L. Harlow, S. A. Mulaik, and J. H. Steiger, editors. What if there were no significance tests? Lawrence Erlbaum Associates, Mahwah, New Jersey, USA.
51. SCHMITT, S. A. 1969. Measuring uncertainty: an elementary introduction to Bayesian statistics. Addison-Weslev, Reading, Massachusetts, USA.
52. SHAVER, J. P. 1993. What statistical significance testing is, and what it is not. Journal of Experimental Education 61:293-316.
53. SIMBERLOFF, D. 1990. Hypotheses, errors, and statistical assumptions. Herpetologica 46:351-357.
54. STEIDL, R. J., J. P. HAYES, AND E. SCHAUBER. 1997. Statistical power analysis in wildlife research. Journal of Wildlife Management 61:270-279.
55. STEIGER, J. H., AND R. T. FOULADI. 1997. Noncentrality interval estimation and evaluation of statistical models. Pages 221-257 in L. L. Harlow, S. A. Mulaik, and J. H. Steiger, editors. What if there were no significance tests? Lawrence Erlbaum Associates, Mahwah, New Jersey, USA.
56. THE WILDLIFE SOCIETY. 1995. Journal news. Journal of Wildlife Management 59:196-198.
57. THOMAS, L., AND C. J. KREBS. 1997. Technological tools. Bulletin of the Ecological Society of America 78:126-139.
58. TOFT, C. A., AND P. J. SHEA. 1983. Detecting community-wide patterns: estimating power strengthens statistical inference. American Naturalist 122: 618-625.
59. TUKEY, J. W. 1969. Analyzing data: sanctification or detective work? American Psychologist 24:83-91.
60. UNDERWOOD, A. J. 1997. Experiments in ecology: their logical design and interpretation using analysis of variance. Cambridge University Press, Cambridge, United Kingdom.
61. WALTERS, C. J. 1986. Adaptive management of renewable resources. MacMillan, New York, New York, USA.
62. _, AND R. GREEN. 1997. Valuation of experimental management options for ecological systems. Journal of Wildlife Management 61:987-1006.
63. WOLFSON, L. J., J. B. KADANE, AND M. J. SMALL. 1996. Bayesian environmental policy decisions: two case studies. Ecological Applications 6:1056-1066.
64. YATES, F. 1964. Sir Ronald Fisher and the design of experiments. Biometrics 20:307-321.
65. ZELLNER, A. 1987. Comment. Statistical Science 2: 339-341.