Evaluating statistical difference, equivalence, and indeterminacy using inferential confidence intervals: An integrated alternative method of conducting null hypothesis statistical tests

Psychological Methods

Volumn 6, Issue 3, 2001, Pages 371-386

  Tryon, Warren W ^a  

^a FORDHAM UNIVERSITY   (United States)

Author keywords

[No Author keywords available]

Indexed keywords

ARTICLE; CONFIDENCE INTERVAL; HUMAN; PSYCHOLOGICAL MODEL; PSYCHOLOGY; PSYCHOMETRY; STATISTICAL ANALYSIS; STATISTICS;

CONFIDENCE INTERVALS; DATA INTERPRETATION, STATISTICAL; HUMANS; MODELS, PSYCHOLOGICAL; PSYCHOLOGY; PSYCHOMETRICS;

EID: 0035751681     PISSN: 1082989X     EISSN: None     Source Type: Journal    
DOI: 10.1037/1082-989x.6.4.371     Document Type: Review

References (86)

1. Abelson, R. P. (1997a). On the surprising longevity of flogged horses: Why there is a case for the significance test. Psychological Science, 8, 12-15.
2. Abelson, R. P. (1997b). A retrospective on the significance test ban of 1999 (If there were no significance tests, they would be invented). In L. L. Harlow, S. A. Mulaik, & J. H. Steiger (Eds.), What if there were no significance tests? (pp. 117-141). Mahwah, NJ: Erlbaum.
3. Anastasi, A. (1988). Psychological testing (6th ed.). New York: Macmillan.
4. Anderson, S., & Hauck, W. W. (1983). A new procedure for testing equivalence in comparative bioavailability and other clinical trials. Communications in Statistics -Theory and Methods, 12, 2663-2692.
5. Andrews, H. P., Snee, R. D., & Sarner, M. H. (1980). Graphical display of means. The American Statistician, 34, 195-199.
6. Arbuthnot, J. (1710). An argument for divine providence, taken from the constant regularity observ'd in the births of both sexes. Philosophical Transactions of the Royal Society of London, 27, 186-190.
7. Bakan, D. (1966). The test of significance in psychological research. Psychological Bulletin, 66, 423-437.
8. Benjamini, Y., & Hochberg, Y. (1995). Controlling the false discovery rate: A practical and powerful approach to multiple testing. Journal of the Royal Statistical Society, Series B, 57, 289-300.
9. Berkson, J. (1942). Tests of significance considered as evidence. Journal of the American Statistical Association, 37, 325-335.
10. Blackwelder, W. C. (1982). "Proving the null hypothesis" in clinical trials. Controlled Clinical Trials, 3, 345-353.
11. Carver, R. P. (1978). The case against statistical significance testing. Harvard Educational Review, 48, 378-399.
12. Carver, R. P. (1993). The case against statistical significance testing, revisited. Journal of Experimental Education, 61, 287-292.
13. Chow, S. L. (1998). Statistical significance: Rationale, validity, and utility. Behavioral and Brain Sciences, 21, 169-239.
14. Cohen, J. (1994). The world is round (p < .05). American Psychologist, 49, 997-1003.
15. Dar, R., Serlin, R. C., & Omer, H. (1994). Misuse of statistical tests in three decades of psychotherapy research. Journal of Consulting and Clinical Psychology, 62, 75-82.
16. Detsky, A. S., & Sackett, D. L. (1985). When was a 'negative' clinical trial big enough? How many patients you needed depends on what you found. Archives of Internal Medicine, 145, 709-712.
17. Dunnett, C. W., & Gent, M. (1977). Significance testing to establish equivalence between treatments with special reference to data in the form of 2 × 2 tables. Biometrics, 33, 593-602.
18. Estes, W. K. (1997a). On the communication of information by displays of standard errors and confidence intervals. Psychonomic Bulletin & Review, 4, 330-341.
19. Estes, W. K. (1997b). Significance testing in psychological research: Some persisting issues. Psychological Science, 8, 18-20.
20. Fisher, R. A. (1925). Statistical methods for research workers. London: Oliver & Boyd.
21. Frick, R. W. (1995). Accepting the null hypothesis. Memory & Cognition, 23, 132-138.
22. Gabriel, K. R. (1978). A simple method of multiple comparisons of means. Journal of the American Statistical Association, 73, 724-729.
23. Glass, G. V., & Hopkins, K. D. (1970). Statistical methods in education and psychology (3rd ed.). Boston: Allyn and Bacon.
24. Goldstein, H., & Healy, M. J. R. (1995). The graphical presentation of a collection of means. Journal of the Royal Statistical Society, 158A, Part 1, 175-177.
25. Guilford, J. P., & Fruchter, B. (1973). Fundamental statistics in psychology and education (5th ed.). New York: McGraw-Hill.
26. Hagen, R. L. (1997). In praise of the null hypothesis statistical test. American Psychologist, 52, 15-24.
27. Harlow, L. L. (1997). Significance testing introduction and overview. In L. L. Harlow, S. A. Mulaik, & J. H. Steiger (Eds.), What if there were no significance tests? (pp. 1-17). Mahwah, NJ: Erlbaum.
28. Harlow, L. L., Mulaik, S. A., & Steiger, J. H. (Eds.). (1997). What if there were no significance tests? Mahwah, NJ: Erlbaum.
29. Harris, R. J. (1997a). Reforming significance testing via three-valued logic. In L. L. Harlow, S. A. Mulaik, & J. H. Steiger (Eds.), What if there were no significance tests? (pp. 145-174). Mahwah, NJ: Erlbaum.
30. Harris, R. J. (1997b). Significance tests have their place. Psychological Science, 8, 8-11.
31. Hochberg, Y. (1988). A sharper Bonferroni procedure for multiple tests of significance. Biometrika, 75, 800-802.
32. Hochberg, Y., Weiss, G., & Hart, S. (1982). On graphical procedures for multiple comparisons. Journal of the American Statistical Association, 77, 767-772.
33. Hodges, J., & Lehmann, E. (1954). Testing the approximate validity of statistical hypotheses. Journal of the Royal Statistical Society, Series B, 16, 261-268.
34. Hogben, L. (1957). Statistical theory. London: Allen & Unwin.
35. Howell, D. C. (1997). Statistical methods for psychology (4th ed.). Belmont, CA: Duxbury Press.
36. Hunter, J. E. (1997). Needed: A ban on the significance test. Psychological Science, 8, 3-7.
37. Hunter, J. E., & Schmidt, F. L. (1990). Methods of meta-analysis: Correcting error and bias in research findings (pp. 29-33). Newbury Park, CA: Sage.
38. Kaiser, H. F. (1960). Directional statistical decisions. Psychological Review, 67, 160-167.
39. Keselman, H. J., Cribbie, R., & Holland, B. (1999). The pairwise multiple comparison multiplicity problem: An alternative approach to familywise and comparisonwise Type I error control. Psychological Methods, 4, 58-69.
40. Kirk, R. E. (1982). Experimental design: Procedures for the behavioral sciences (2nd ed.). Belmont, CA: Brooks/Cole.
41. Kish, L. (1959). Some statistical problems in research design. American Sociological Reveiw, 24, 328-338.
42. Krantz, D. H. (1999). The null hypothesis testing controversy in psychology. Journal of the American Statistical Association, 44, 1372-1381.
43. Loftus, G. R. (1991). On the tyranny of hypothesis testing in the social sciences. Contemporary Psychology, 36, 102-105.
44. Loftus, G. R. (1993). A picture is worth a thousand p values: On the irrelevance of hypothesis testing in the microcomputer age. Behavior Research Methods, Instruments & Computers, 25, 250-256.
45. Loftus, G. R. (1995). Data analysis as insight: Reply to Morrison and Weaver. Behavior Research Methods, Instruments & Computers, 27, 57-59.
46. Loftus, G. R. (1996). Psychology will be much better science when we change the way we analyse data. Current Directions in Psychological Science, 5, 161-171.
47. Loftus, G. R., & Mason, M. E. J. (1994). Using confidence intervals in within-subject designs. Psychonomic Bulletin & Review, 1, 476-490.
48. Lykken, D. E. (1968). Statistical significance in psychological research. Psychological Bulletin, 70, 151-159.
49. Makuch, R., & Simon, R. (1978). Sample size requirements for evaluating a conservative therapy. Cancer Treatment Reports, 62, 1037-1040.
50. Markus, K. A. (2001). The converse inequality argument against tests of statistical significance. Psychological Methods, 6, 147-160.
51. McGill, R., Tukey, J. W., & Larsen, W. A. (1978). Variation of box plots. The American Statistician, 32, 12-16.
52. McMan, J. C. (1995, August). Statistical significance testing fantasies in introductory psychology textbooks. Paper presented at the 103rd Annual Convention of the American Psychological Association, New York.
53. Meehl, P. E. (1967). Theory testing in psychology and physics: A methodological paradox. Philosophy of Science, 34, 103-115.
54. Meehl, P. E. (1978). Theoretical risks and tabular asterisks: Sir Karl, Sir Ronald, and the slow progress of soft psychology. Journal of Consulting and Clinical Psychology, 46, 806-834.
55. Meehl, P. E. (1990a). Appraising and amending theories: The strategy of Lakatosian defense and two principles that warrant it. Psychological Inquiry, 1, 108-141.
56. Meehl, P. E. (1990b). Why summaries of research on psychological theories are often uninterpretable. Psychological Reports, 66, 195-244.
57. Metzler, C. M. (1979). Bioavailability - a problem in equivalence. Biometrics, 30, 309-317.
58. Morrison, D. E., & Henkel, R. E. (1970). The significance test controversy. Chicago: Aldine.
59. Mulaik, S. A., Raju, N. S., & Harshman, R. A. (1997). There is a time and place for significance testing. In L. L. Harlow, S. A. Mulaik, & J. H. Steiger (Eds.), What if there were no significance tests? (pp. 65-115). Mahwah, NJ: Erlbaum.
60. Nickerson, R. S. (2000). Null hypothesis significance testing: A review of an old and continuing controversy. Psychological Methods, 5, 241-301.
61. Nunnally, J. C. (1967). Psychometric theory. New York: McGraw-Hill.
62. Pearce, S. C. (1992). Introduction to Fisher (1925) statistical methods for research workers. In S. Kotz & N. L. Johnson (Eds.), Breakthroughs in statistics, Volume II: Methodology and distributions (pp. 59-65). New York: Springer-Verlag.
63. Rindskopf, D. M. (1997). Testing "small," but not null, hypotheses: Classical and Bayesian approaches. In L. L. Harlow, S. A. Mulaik, & J. H. Steiger (Eds.), What if there were no significance tests? (pp. 319-332). Mahwah, NJ: Erlbaum.
64. Rogers, J. L., Howard, K. I., & Vessey, J. T. (1993). Using significant tests to evaluate equivalence between two experimental groups. Psychological Bulletin, 113, 553-565.
65. Rosenthal, R. (1995). Progress in clinical psychology: Is there any? Clinical Psychology: Science and Practice, 2, 133-150.
66. Rozeboom, W. W. (1960). The fallacy of the null hypothesis signficance test. Psychological Bulletin, 57, 416-428.
67. Scarr, S. (1997). Rules of evidence: A larger context for the statistical debate. Psychological Science, 8, 16-17.
68. Schmidt, F. (1996). APA Board of Scientific Affairs to study issue of significance testing, make recommendations. The Score Newsletter, 19, 1, 6.
69. Schmidt, F., & Hunter, J. E. (1997). Eight common but false objections to the discontinuation of significance testing in the analysis of research data. In L. L. Harlow, S. A. Mulaik, & J. H. Steiger (Eds.), What if there were no significance tests? (pp. 37-64). Mahwah, NJ: Erlbaum.
70. Selwyn, M. R., Dempster, A. P., & Hall, N. R. (1981). A Bayesian approach to bioequivalence for the 2 × 2 changeover design. Biometrics, 37, 11-21.
71. Selwyn, M. R., & Hall, N. R. (1984). On Bayesian methods for bioequivalence. Biometrics, 40, 1103-1108.
72. Serlin, R. C. (1993). Confidence intervals and the scientific method: A case for Holm on the range. Journal of Experimental Education, 61, 350-360.
73. Serlin, R. C., & Lapsley, D. K. (1985). Rationality in psychological research: The good-enough principle. American Psychologist, 40, 73-83.
74. Shaffer, J. P. (1986). Modified sequentially rejected multiple test procedures. Journal of the American Statistical Association, 81, 826-831.
75. Shaffer, J. P. (1995). Multiple hypothesis testing: A review. Annual Review of Psychology, 46, 561-584.
76. Shrout, P. E. (1997). Should significance tests be banned? Introduction to special section exploring the pros and cons. Psychological Sciences, 8, 1-2.
77. Steiger, J. H., & Fouladi, R. T. (1997). Noncentrality interval estimation and the evaluation of statistical models. In L. L. Harlow, S. A. Mulaik, & J. H. Steiger (Eds.), What if there were no significance tests? (pp. 221-257). Hillsdale, NJ: Erlbaum.
78. Sterling, T. D. (1959). Publication decisions and their possible effects on inferences drawn from tests of significance - or vice versa. Journal of the American Statistical Association, 54, 30-34.
79. Tukey, J. W. (1991). The philosophy of multiple comparisons. Statistical Science, 6, 100-116.
80. Westlake, W. J. (1972). Use of confidence intervals in analysis of comparative bioavailability trials. Journal of Pharmaceutical Science, 61, 1340-1341.
81. Westlake, W. J. (1976). Symmetrical confidence intervals for bioequivalence trials. Biometrics, 32, 741-744.
82. Westlake, W. J. (1979). Statistical aspects of comparative bioavailability trials. Biometrics, 35, 273-280.
83. Westlake, W.J. (1981). Response to T. B. L. Kirkwood: Bioequivalence testing - A need to rethink. Biometrics, 37, 589-594.
84. Wilkinson, L., & the Task Force on Statistical Inference. (1999). Statistical methods in psychology journals. American Psychologist, 54, 594-604.
85. Williams, V. S. L., Jones, L. V., & Tukey, J. W. (1999). Controlling error in multiple comparisons with examples from state-to-state differences in educational achievement. Journal of Education and Behavioral Statistics, 24, 42-69.
86. Winer, B. J., Brown, D. R., & Michels, K. M. (1991). Statistical principles in experimental design (3rd ed.) New York: McGraw-Hill.