How the Maximal Evidence of P-Values Against Point Null Hypotheses Depends on Sample Size

American Statistician

Volumn 70, Issue 4, 2016, Pages 335-341

  Held, Leonhard ^a   Ott, Manuela ^a  

^a UNIVERSITY OF ZURICH   (Switzerland)

Author keywords

Evidence; F Test; Minimum Bayes factor; p value; Sample size; t Test

Indexed keywords

EID: 84996497586     PISSN: 00031305     EISSN: 15372731     Source Type: Journal    
DOI: 10.1080/00031305.2016.1209128     Document Type: Article

References (32)

1. Bayarri, M. J., Benjamin, D. J., Berger, J. O., and Sellke, T. M., (2016), “Rejection Odds and Rejection Ratios:A Proposal for Statistical Practice in Testing Hypotheses,” Journal of Mathematical Psychology, 72, 90–103.
2. Berger, J. O., and Sellke, T., (1987), “Testing a Point Null Hypothesis:The Irreconcilability of P Values and Evidence” (with discussion), Journal of the American Statistical Association, 82, 112–139.
3. Bernardo, J. M., and Smith, A. F. M., (2000), Bayesian Theory, Wiley Series in Probability and Statistics, Chichester:Wiley.
4. Copas, J. B., (1983), “Regression, Prediction and Shrinkage,” Journal of the Royal Statistical Society, Series B, 45, 311–354.
5. Edwards, W., Lindman, H., and Savage, L. J., (1963), “Bayesian Statistical Inference in Psychological Research,” Psychological Review, 70, 193–242.
6. Freeman, P. R., (1993), “The Role of p-values in Analysing Trial Results,” Statistics in Medicine, 12, 1443–1452.
7. Goodman, S. N., (1992), “A Comment on Replication, P-values and Evidence,” Statistics in Medicine, 11, 875–879.
8. ——— (1999a), “Towards Evidence-Based Medical Statistics. 1.:The P value Fallacy,” Annals of Internal Medicine, 130, 995–1004.
9. ——— (1999b), “Towards Evidence-based Medical Statistics. 2.:The Bayes Factor,” Annals of Internal Medicine, 130, 1005–1013.
10. ——— (2001), “Of P-values and Bayes:A Modest Proposal,” Epidemiology, 12, 295–297.
11. Greenland, S., (2006), “Bayesian Perspectives for Epidemiological Research:I. Foundations and Basic Methods,” International Journal of Epidemiology, 35, 765–775.
12. ——— (2011), “Null Misinterpretation in Statistical Testing and its Impact on Health Risk Assessment,” Preventive Medicine, 53, 225–228.
13. Greenland, S., and Poole, C., (2013), “Living with p Values:Resurrecting a Bayesian Perspective on Frequentist Statistics,” Epidemiology, 24, 62–68.
14. Held, L., (2010), “A Nomogram for P Values,” BMC Medical Research Methodology, 10, 21.
15. ——— (2013), “Reverse-Bayes Analysis of Two Common Misinterpretations of Significance Tests,” Clinical Trials, 10, 236–242.
16. Held, L., Sabanés Bové, D., and Gravestock, I., (2015), “Approximate Bayesian Model Selection with the Deviance Statistic,” Statistical Science, 30, 242–257.
17. Hu, J., and Johnson, V. E., (2009), “Bayesian Model Selection Using Test Statistics,” Journal of the Royal Statistical Society, Series B, 71, 143–158.
18. Johnson, V. E., (2005), “Bayes Factors Based on Test Statistics,” Journal of the Royal Statistical Society, Series B, 67, 689–701.
19. ——— (2008), “Properties of Bayes Factors Based on Test Statistics,” Scandinavian Journal of Statistics, 35, 354–368.
20. ——— (2013), “Revised Standards for Statistical Evidence,” Proceedings of the National Academy of Sciences, 110, 19313–19317.
21. Kass, R., and Raftery, A., (1995), “Bayes Factors,” Journal of the American Statistical Association, 90, 773–795.
22. Kass, R. E., and Wasserman, L., (1995), “A Reference Bayesian Test for Nested Hypotheses and its Relationship to the Schwarz Criterion,” Journal of the American Statistical Association, 90, 928–934.
23. Liang, F., Paulo, R., Molina, G., Clyde, M. A., and Berger, J. O., (2008), “Mixtures of g Priors for Bayesian Variable Selection,” Journal of the American Statistical Association, 103, 410–423.
24. Matthews, R., (2001), “Methods for Assessing the Credibility of Clinical Trial Outcomes,” Drug Information Journal, 35, 1469–1478.
25. Royall, R. M., (1986), “The Effect of Sample Size on the Meaning of Significance Tests,” The American Statistician, 40, 313–315.
26. Sabanés Bové, D., and Held, L., (2011), “Hyper-g Priors for Generalized Linear Models,” Bayesian Analysis, 6, 387–410.
27. Sellke, T., Bayarri, M. J., and Berger, J. O., (2001), “Calibration of p Values for Testing Precise Null Hypotheses,” The American Statistician, 55, 62–71.
28. Spiegelhalter, D. J., Abrams, K. R., and Myles, J. P., (2004), Bayesian Approaches to Clinical Trials and Health-Care Evaluation. New York:Wiley.
29. Vovk, V. G., (1993), “A Logic of Probability, with Application to the Foundations of Statistics” (with Discussion and a Reply by the Author), Journal Royal Statistical Society, Series B, 55, 317–351.
30. Wagenmakers, E.-J., (2007), “A Practical Solution to the Pervasive Problems of p Values,” Psychonomic Bulletin & Review, 14, 779–804.
31. Wasserstein, R. L., and Lazar, N. A., (2016), “The ASA’s Statement on p-values:Context, Process, and Purpose,” The American Statistician, 70, 129–133.
32. Zellner, A., (1986), “On Assessing Prior Distributions and Bayesian Regression Analysis with g-prior Distributions,” in Bayesian Inference and Decision Techniques:Essays in Honor of Bruno de Finetti, Vol. 6 of Studies in Bayesian Econometrics and Statistics, eds. P. K., Goel and A., Zellner, chap. 5, pp. 233–243. Amsterdam:North-Holland.