Confusion Over Measures of Evidence (p's) Versus Errors (α's) in Classical Statistical Testing

American Statistician

Volumn 57, Issue 3, 2003, Pages 171-178

  Hubbard, Raymond ^a   Bayarri, M J ^b  

^a DRAKE UNIVERSITY   (United States)

^b UNIVERSITY OF VALENCIA   (Spain)

Author keywords

Conditional error probabilities; Fisher; Hypothesis test; Neyman Pearson; Significance test; Values

Indexed keywords

EID: 0042074083     PISSN: 00031305     EISSN: 15372731     Source Type: Journal    
DOI: 10.1198/0003130031856     Document Type: Article

References (47)

1. Bayarri, M. J., and Berger, J. O. (2000), “P Values for Composite Null Models,” Journalof the American StatisticalAssociation, 95, 1127-1142.
2. Berger, J. O. (1985), Statistical Decision Theory and Bayesian Analysis (2nd ed.), New York: Springer-Verlag.
3. Berger, J. O. (2000), “Bayesian Analysis: A Look at Today and Thoughts of Tomorrow,” Journal of the American StatisticalAssociation, 95, 1269-1276.
4. Berger, J. O. (2002), “Could Fisher, Jeffreys, and Neyman Have Agreed on Testing?” ISDS (Duke University) discussion paper 02-01.
5. Berger, J. O., and Delampady, M. (1987), “Testing Precise Hypotheses” (with comments), StatisticalScience, 2, 317-352.
6. Berger, J. O., and Pericchi, L. (2001), “Objective Bayesian Methods for Model Selection: Introduction and Comparison” (with comments), in Model Selection, ed. P. Lahiri, Institute of Mathematical Statistics Lecture Notes— Monograph Series, Volume 38, 135-207.
7. Berger, J. O., and Sellke, T. (1987), “Testing a Point Null Hypothesis: The Irreconcilability of P Values and Evidence” (with comments), Journal ofthe American StatisticalAssociation, 82, 112-139.
8. Casella, G., and Berger, R. L. (1987), “Reconciling Bayesian and Frequentist Evidence in the One-Sided Testing Problem” (with comments), Journal of the American StatisticalAssociation, 82, 106-139.
9. Donahue, R. M. J. (1999), “A Note on Information Seldom Reported Via the P Value,” The American Statistician, 53, 303-306.
10. Fisher, R. A. (1925), Statistical Methods for Research Workers, Edinburgh: Oliver and Boyd.
11. Fisher, R. A. (1926), “The Arrangement of Field Experiments,” Journal of the Ministry of AgricultureforGreat Britain, 33, 503-513.
12. Fisher, R. A. (1929), “The Statistical Method in Psychical Research,” in Proceedings ofthe Society forPsychical Research, London, 39, pp. 189-192.
13. Fisher, R. A. (1935a), The Design ofExperiments, Edinburgh: Oliver and Boyd.
14. Fisher, R. A. (1935b), “The Logic of Inductive Inference,” Journal ofthe Royal Statistical Society, 98, 39-54.
15. Fisher, R. A. (1945), “The Logical Inversion of the Notion of the Random Variable,” Sankhya, 7, 129-132.
16. Fisher, R. A. (1959), StatisticalMethodsand Scientific Inference, (2nd ed., revised), Edinburgh: Oliver and Boyd.
17. Fisher, R. A. (1960), “Scientific Thought and the Refinement of Human Reasoning,” Journal ofthe OperationsResearch Society of Japan, 3, 1-10.
18. Fisher, R. A. (1966), The Design of Experiments (8th ed.), Edinburgh: Oliver and Boyd.
19. Gibbons, J. D. (1986), “P-Values,” in Encyclopedia ofStatisticalSciences, eds. S. Kotz and N. L. Johnson, New York: Wiley, pp. 366-368.
20. Gibbons, J. D., and Pratt, J. W. (1975), “P-values: Interpretation and Methodology,” The American Statistician, 29, 20-25.
21. Gigerenzer, G. (1993), “The Superego, the Ego, and the Id in Statistical Reasoning,” in A Handbook for Data Analysis in the Behavioral Sciences— MethodologicalIssues, eds. G. Keren and C. A. Lewis, Hillsdale, NJ: Erlbaum, pp. 311-339.
22. Goodman, S. N. (1992), “A Comment on Replication, P-Values and Evidence,” Statistics in Medicine, 11, 875-879.
23. Goodman, S. N. (1993), “p Values, Hypothesis Tests, and Likelihood: Implications for Epidemiology of a Neglected Historical Debate,” American Journal of Epidemiology, 137,485-496.
24. Goodman, S. N. (1999), “Toward Evidence-Based Medical Statistics. 1: The P Value Fallacy,” Annals of Internal Medicine, 130, 995-1004.
25. Hinkley, D. V. (1987), “Comment,” Journalofthe American StatisticalAssoci-ation, 82, 128-129.
26. Hung, H. M. J., O’Neill, R. T., Bauer, P., and Kohne, K. (1997), “The Behavior of the P-Value When the Alternative Hypothesis is True,” Biometrics, 53, 11-22.
27. Johnstone, D. J. (1986), “Tests of Significance in Theory and Practice” (with comments), The Statistician, 35,491-504.
28. Johnstone, D. J. (1987a), “On the Interpretation of Hypothesis Tests Following Neyman and Pearson,” in ProbabilityandBayesian Statistics, ed. R. Viertl, New York: Plenum Press, pp. 267-277.
29. Johnstone, D. J. (1987b), “Tests of Significance Following R.A. Fisher,” British Journal for the PhilosophyofScience, 38, 481-499.
30. Kyburg, H. E. (1974), The Logical Foundations of Statistical Inference, Dordrecht: Reidel.
31. Lehmann, E. L. (1993), “The Fisher, Neyman-Pearson Theories of Testing Hypotheses: One Theory or Two?” Journal oftheAmerican StatisticalAssocia-tion, 88, 1242-1249.
32. Neyman, J. (1950), First Course in Probabilityand Statistics, New York: Holt.
33. Neyman, J. (1952), Lectures and Conferences on Mathematical Statistics and Probability (2nd ed., revised and enlarged), Washington, DC: Graduate School, U.S. Department of Agriculture.
34. Neyman, J. (1955), “TheProblemoflnductivelnference,” CommunicationsonPure andAppliedMathematics, 8, 13-45.
35. Neyman, J. (1967), “R.A. Fisher (1890-1962), An Appreciation,” Science, 156, 1456-1460.
36. Neyman, J. (1976), “TheEmergence ofMathematical Statistics:AHistorical Sketch with Particular Reference to the United States,” in On the History ofStatistics and Probability, ed. D.B. Owen, New York: Marcel Dekker, pp. 149-193.
37. Neyman, J. (1977), “Frequentist Probability and Frequentist Statistics,” Synthese, 36, 97-131.
38. Neyman, J., and Pearson, E. S. (1928 a), “OntheUse and InterpretationofCertain Test Criteria for Purposes of Statistical Inference. Part I,” Biometrika, 20A, 175-240.
39. Neyman, J., and Pearson, E. S. (1928b), “On the Use and Interpretation of Certain Test Criteria for Purposes of Statistical Inference. Part II,” Biometrika, 20A, 263-294.
40. Neyman, J., and Pearson, E. S. (1933), “On the Problem of the Most Efficient Tests of Statistical Hypotheses,” Philosophical Transactions of the Royal Society of London, Ser. A, 231,289-337.
41. Pearson, E. S. (1962), “Some Thoughts on Statistical Inference,” Annals of Mathematical Statistics, 33, 394-403.
42. Rao, C. R. (1992), “R.A. Fisher: TheFounderof ModernStatistics,” Statistical Science, 7, 34-48.
43. Royall, R. M. (1997), Statistical Evidence: A Likelihood Paradigm, New York: Chapman and Hall.
44. Schervish, M. J. (1996), “P Values: What They Are and What They Are Not,” The American Statistician, 50, 203-206.
45. Seidenfeld, T. (1979), PhilosophicalProblemsofStatisticalInference: Learning from R. A. Fisher, Dordrecht: Reidel.
46. Sellke, T., Bayarri, M. J., and Berger, J. O. (2001), “Calibration of p Values for Testing Precise Null Hypotheses,” The American Statistician, 55, 62-71.
47. Spielman, S. (1974),“TheLogicofTests ofSignificance,” PhilosophyofScience, 41,211-226.