Biases of success rate differences shown in binomial effect size displays

Psychological Methods

Volumn 9, Issue 2, 2004, Pages 183-197

  Hsu, Louis M ^a  

^a FAIRLEIGH DICKINSON UNIVERSITY   (United States)

Author keywords

[No Author keywords available]

Indexed keywords

ARTICLE; BINOMIAL DISTRIBUTION; HUMAN; PSYCHOLOGICAL MODEL;

BINOMIAL DISTRIBUTION; HUMANS; MODELS, PSYCHOLOGICAL;

EID: 2542599307     PISSN: 1082989X     EISSN: None     Source Type: Journal    
DOI: 10.1037/1082-989X.9.2.183     Document Type: Article

References (58)

1. Agresti, A. (1989). Tutorial on modeling ordered categorical response data. Psychological Bulletin, 105, 290-301.
2. Brunner, E., & Munzel, U. (2000). The nonparametric Behrens-Fisher problem: Asymptotic theory and a small sample approximation. Biometric Journal, 42, 17-25.
3. Carroll, J. B. (1961). The nature of data, or how to choose a correlation coefficient. Psychometrika, 26, 347-382.
4. Cliff, N. (1993). Dominance statistics: Ordinal analyses to answer ordinal questions. Psychological Bulletin, 114, 494-509.
5. Cliff, N. (1996). Ordinal methods for behavioral data analysis. Mahwah, NJ: Erlbaum.
6. Cohen, J. (1988). Statistical power analysis for the behavioral sciences. Hillsdale, NJ: Erlbaum.
7. Crow, E. L. (1991). Response to Rosenthal's comment "How are we doing in soft psychology?" American Psychologist, 46, 1083.
8. Delaney, H. D., & Vargha, A. (2002). Comparing several robust tests of stochastic equality with ordinally scaled variables and small to moderate sized samples. Psychological Methods, 7, 485-503.
9. Fleiss, J. L., Levin, B., & Paik, M. C. (2003). Statistical methods for rates and proportions (3rd ed.). New York: Wiley.
10. Ghiselli, E. E., Campbell, J. P., & Zedeck, S. (1981). Measurement theory for the behavioral sciences. San Francisco: Freeman.
11. Grissom, R. J. (1994a). Probability of the superior outcome of one treatment over another. Journal of Applied Psychology, 79, 314-316.
12. Grissom, R. J. (1994b). Statistical analysis of ordinal categorical status after therapies. Journal of Consulting and Clinical Psychology, 62, 281-284.
13. Grissom, R. J. (1996). The magical number .7 +/- .2: Meta-meta-analysis of the probability of superior outcome in comparisons involving therapy, placebo, and control. Journal of Consulting and Clinical Psychology, 64, 973-982.
14. Grissom, R. J. (2000). Heterogeneity of variance in clinical data. Journal of Consulting and Clinical Psychology, 68, 155-165.
15. Grissom, R. J., & Kim, J. J. (2001). Review of assumptions and problems in the appropriate conceptualization of effect size. Psychological Methods, 6, 135-146.
16. Hiller, J. B., Rosenthal, R., Bornstein, R. F., Berry, D. T. R., & Brunell-Neuleib, S. (1999). A comparative meta-analysis of Rorschach and MMPI validity. Psychological Assessment, 11, 278-296.
17. Hsu, L. M. (1993). Using Cohen's tables to determine the maximum power attainable in two-sample tests when one sample is limited in size. Journal of Applied Psychology, 78, 303-305.
18. Hsu, L. M. (1996). On the identification of clinically significant client changes: Re-interpretation of Jacobson's cut scores. Journal of Psychopathology and Behavior Assessment, 18, 371-386.
19. Hsu, L. M. (2000). Effects of directionality of significance tests on the bias of accessible effect sizes. Psychological Methods, 5, 333-342.
20. Hsu, L. M. (2002a). Diagnostic validity statistics and the MCMI-III. Psychological Assessment, 14, 410-422.
21. Hsu, L. M. (2002b). Fail-safe Ns for one- vs. two-tailed tests lead to different conclusions about publication bias. Understanding Statistics, 1, 85-100.
22. n, Cohen's kappa, Scott's ∏, and Aickin's α. Understanding Statistics, 2, 205-219.
23. Huberty, C. J. (2002). A history of effect size indices. Educational and Psychological Measurement, 62, 227-240.
24. Hunter, J. E., Schmidt, F. L., & Jackson, G. B. (1982). Meta-analysis: Cumulating research findings across studies. Beverly Hills, CA: Sage.
25. Jacobson, N. S., & Truax, P. (1991). Clinical significance: A statistical approach to defining meaningful change in psychotherapy research. Journal of Consulting and Clinical Psychology, 59, 12-19.
26. Kazdin, A. E. (2003). Research design in clinical psychology (4th ed.). Boston: Allyn & Bacon.
27. Lipsey, M. W. (1990). Design sensitivity: Statistical power for experimental research. Newbury Park, CA: Sage.
28. Lipsey, M. W., & Wilson, D. B. (1993). The efficacy of psychological, educational, and behavioral treatment. American Psychologist, 48, 1181-1209.
29. McGraw, K. O. (1991). Problems with the BESD: A comment on Rosenthal's "How are we doing in soft psychology?" American Psychologist, 46, 1084-1086.
30. McGraw, K. O., & Wong, S. P. (1992). A common-language effect size statistic. Psychological Bulletin, 111, 361-365.
31. Meyer, G. J., Finn, S. E., Eyde, L. D., Kay, G. G., Moreland, K. L., Dies, R. R., et al. (2001). Psychological testing and psychological assessment. American Psychologist, 56, 128-165.
32. Nunnally, J. C. (1978). Psychometric theory. New York: McGraw-Hill.
33. Preece, P. F. (1983). A measure of experimental effect size based on success rates. Educational and Psychological Measurement, 43, 763-766.
34. Rosenthal, R. (1987). Judgment studies: Design, analysis, and meta-analysis. New York: Cambridge University Press.
35. Rosenthal, R. (1990). How are we doing in soft psychology? American Psychologist, 45, 775-777.
36. Rosenthal, R. (1991). Meta-analytic procedures for social research (Rev. ed.). Newbury Park, CA: Sage.
37. Rosenthal, R. (1994). Parametric measures of effect size. In H. Cooper & L. V. Hedges (Eds.), The handbook of research synthesis (pp. 231-244). New York: Russell Sage Foundation.
38. Rosenthal, R. (1995a). Progress in clinical psychology: Is there any? Clinical Psychology: Science and Practice, 2, 133-149.
39. Rosenthal, R. (1995b). Writing meta-analytic reviews. Psychological Bulletin, 118, 183-192.
40. Rosenthal, R. (2002). Covert communication in classrooms, clinics, courtrooms, and cubicles. American Psychologist, 57, 839-849.
41. Rosenthal, R., & Rosnow, R. L. (1991). Essentials of behavioral research: Methods and data analysis (2nd ed.). New York: McGraw-Hill.
42. Rosenthal, R., & Rubin, D. B. (1982). A simple, general purpose display of magnitude of experimental effect. Journal of Educational Psychology, 74, 166-169.
43. Rosnow, R. L., & Rosenthal, R. (1988). Focused tests of significance and effect size estimation in counseling psychology. Journal of Counseling Psychology, 35, 203-208.
44. Rosnow, R. L., & Rosenthal, R. (1989). Statistical procedures and the justification of knowledge in psychological science. American Psychologist, 44, 1276-1284.
45. Rosnow, R. L., & Rosenthal, R. (1996). Computing contrasts, effect sizes, and counter-nulls on other people's published data: General procedures for research consumers. Psychological Methods, 1, 331-340.
46. Rosnow, R. L., Rosenthal, R., & Rubin, D. B. (2000). Contrasts and correlations in effect-size estimation. Psychological Science, 11, 446-453.
47. Sheppard, W. F. (1898). On the application of the theory of error to cases of normal distributions and normal correlations. Proceedings of the Royal Society, 62, 170.
48. Sheskin, D. J. (2000). Handbook of parametric and nonparametric statistical procedures. New York: Chapman & Hall/CRC.
49. Strahan, R. F. (1991). Remarks on the binomial effect size display. American Psychologist, 46, 1083-1084.
50. Stuart, A., & Ord, J. K. (1987). Kendall's advanced theory of statistics (Vol. 1). New York: Oxford University Press.
51. Svartberg, M., & Stiles, T. C. (1991). Comparative effects of short-term psychodynamic psychotherapy: A meta-analysis. Journal of Consulting and Clinical Psychology, 59, 704-714.
52. Taylor, H. C., & Russell, J. T. (1939). The relationship of validity coefficients to the practical effectiveness of tests in selection: Discussion and tables. Journal of Applied Psychology, 23, 565-578.
53. Thompson, K. N., & Schumacker, R. E. (1997). An evaluation of Rosenthal and Rubin's binomial effect size display. Journal of Educational and Behavioral Statistics, 22, 109-117.
54. Vargha, A., & Delaney, H. D. (2000). A critique and improvement of the CL common language effect size of McGraw and Wong. Journal of Educational and Behavioral Statistics, 25, 101-132.
55. Vargha, A., Rudas, T., Delaney, H. D., & Maxwell, S. E. (1996). Dichotomization, partial correlation, and conditional independence. Journal of Educational and Behavioral Statistics, 21, 264-282.
56. Walker, H. M., & Lev, J. (1953). Statistical inference. New York: Holt.
57. Wilcox, R. R. (1997). Introduction to robust estimation and hypothesis testing. New York: Academic Press.
58. Wilcox, R. R. (2003). Applying contemporary statistical techniques. New York: Academic Press.