The value of RCT evidence depends on the quality of statistical analysis

Behaviour Research and Therapy

Volumn 46, Issue 2, 2008, Pages 270-281

  Faulkner, Cathy ^a   Fidler, Fiona ^a   Cumming, Geoff ^a  

^a LA TROBE UNIVERSITY   (Australia)

Author keywords

Clinical significance; Confidence intervals; Effect size; Randomized controlled trial; Statistical significance

Indexed keywords

ANXIETY DISORDER; ARTICLE; CONFIDENCE INTERVAL; E-MAIL; EFFECT SIZE; HEALTH SURVEY; MEDICAL LITERATURE; MOOD DISORDER; PSYCHIATRY; PSYCHOLOGY; PSYCHOTHERAPY; QUALITY CONTROL; RANDOMIZED CONTROLLED TRIAL; STATISTICAL ANALYSIS; STATISTICAL SIGNIFICANCE; SUBSTANCE ABUSE; TREATMENT OUTCOME;

CONFIDENCE INTERVALS; DATA INTERPRETATION, STATISTICAL; HUMANS; PSYCHOMETRICS; QUALITY CONTROL; RANDOMIZED CONTROLLED TRIALS AS TOPIC; RESEARCH DESIGN; TREATMENT OUTCOME;

EID: 38649096607     PISSN: 00057967     EISSN: None     Source Type: Journal    
DOI: 10.1016/j.brat.2007.12.001     Document Type: Article

References (33)

1. American Psychological Association. Publication manual of the American Psychological Association. 5th ed. (2001), Author, Washington, DC
2. Bezeau S., and Graves R. Statistical power and effect sizes of clinical neuropsychology research. Journal of Clinical and Experimental Neuropsychology 23 (2001) 399-406
3. Bloin D.C., and Riopelle A.J. On confidence intervals for within-subjects designs. Psychological Methods 10 4 (2005) 397-412
4. CONSORT. (n.d.). CONSORT statement. Retrieved 28 December 2007, from 〈http://www.consort-statement.org/〉.
5. Cumming G., and Finch S. A primer on the understanding, use, and calculation of confidence intervals that are based on central and noncentral distributions. Educational and Psychological Measurement 61 (2001) 532-574
6. Cumming G., and Finch S. Inference by eye: Confidence intervals and how to read pictures of data. American Psychologist 60 2 (2005) 170-180
7. Crosby R.D., Wonderlich S.A., Mitchell J.E., de Zwaan M., Engel S.G., Connolly K., et al. An empirical analysis of eating disorders and anxiety disorders publications (1980-2000)-Part II: Statistical hypothesis testing. International Journal of Eating Disorders 39 (2006) 49-54
8. Faulkner, C., Fidler, F., & Cumming, G. (2007). Psychologists find confidence intervals easier and more informative than significance tests, for results of randomized controlled trials. Manuscript in preparation.
9. Fidler F., Cumming G., Burgman M., and Thomason N. Statistical reform in medicine, psychology and ecology. The Journal of Socio-Economics 33 5 (2004) 615-631
10. Fidler F., Cumming G., Thomason N., Pannuzzo D., Smith J., Fyffe P., et al. Toward improved statistical reporting in the Journal of Consulting and Clinical Psychology. Journal of Consulting & Clinical Psychology 73 1 (2005) 136-143
11. Fidler F., Faulkner C., and Cumming G. Analyzing and presenting outcomes. In: Nezu A.M., and Nezu C.M. (Eds). Evidence-based outcome research: A practical guide to conducting randomized controlled trials for psychosocial interventions (2008), Oxford University Press, New York 315-334
12. Finch S., Cumming G., and Thomason N. Reporting of statistical inference in the Journal of Applied Psychology: Little evidence of reform. Educational & Psychological Measurement 61 (2001) 181-210
13. Finch S., Cumming G., Williams J., Palmer L., Griffith E., Alders C., et al. Reform of statistical inference in psychology: The case of Memory & Cognition. Behavior Research Methods, Instruments & Computers 36 2 (2004) 312-324
14. Gigerenzer G., and Hoffrage U. How to improve Bayesian reasoning without instruction: Frequency formats. Psychological Review 102 (1995) 684-704
15. Harlow L.L. Significance testing in introduction and overview. In: Harlow L.L., Muliak S.A., and Steiger J.H. (Eds). What if there were no significance tests? (1997), Lawrence Erlbaum, Mahwah, NJ, USA 1-17
16. Hunter J.E., and Schmidt F.L. Methods of meta-analysis: Correcting error and bias in research findings. 2nd ed. (2004), Sage, Thousand Oaks, CA
17. Kazdin A.E. The meanings and measurement of clinical significance. Journal of Consulting & Clinical Psychology 67 3 (1999) 332-339
18. Kirk R.E. Practical significance: A concept whose time has come. Educational & Psychological Measurement 56 (1996) 746-759
19. Kirk R.E. Promoting good statistical practices: Some suggestions. Educational and Psychological Measurement 61 (2001) 213-218
20. Luborsky L., Rosenthal R., Diguer L., Andrusyna T.P., Berman J.S., Levitt J.T., et al. The dodo bird verdict is alive and well-Mostly. Clinical Psychology: Science and Practice 9 1 (2002) 2-12
21. Masson M., and Loftus G.R. Using confidence intervals for graphically based interpretation. Canadian Journal of Experimental Psychology 57 (2003) 203-220
22. Newcombe R.G., and Altman D.G. Proportions and their differences. In: Altman D.G., Machin D., Bryant T.N., and Gardner M.J. (Eds). Statistics with confidence: Confidence intervals and statistical guidelines. 2nd ed. (2000), BMJ Books, Bristol, UK
23. Robbins C.J. Attributions and depression: Why is the literature so inconsistent?. Journal of Personality and Social Psychology 54 (1988) 880-889
24. Rossi J. A case study in the failure of psychology as a cumulative science: The spontaneous recovery of verbal learning. In: Harlow L.L., Mulaik S.A., and Steiger J.H. (Eds). What if there were no significance tests? (1997), Lawrence Erlbaum, Mahwah, NJ 175-198
25. Sedlmeier P., and Gigerenzer G. Do studies of statistical power have an effect on the power of studies?. Psychological Bulletin 105 (1989) 309-316
26. Shapiro D.A., and Shapiro D. Meta-analysis of comparative therapy outcome studies: A replication and refinement. Psychological Bulletin 92 3 (1982) 581-604
27. Smith L.D., Best L.A., Stubbs A., Archibald A.B., and Robertson-Nay R. Constructing knowledge: The role of graphs and tables in hard and soft psychology. American Psychologist 57 10 (2002) 749-761
28. Smith M.L., and Glass G.V. Meta-analysis of psychotherapy outcome studies. American Psychologist 32 (1977) 752-760
29. Thompson B., and Snyder P.A. Statistical significance and reliability analyses in recent Journal of Counseling & Development research articles. Journal of Counseling & Development 76 (1998) 436-441
30. Vacha-Haase T., and Ness C.N. Statistical significance testing as it relates to practice: Use within Professional Psychology: Research and Practice. Professional Psychology: Research and Practice 30 (1999) 104-105
31. Vacha-Haase T., and Nilsson J.E. Statistical significance reporting: Current trends and uses in MECD. Measurement and Evaluation in Counseling and Development 31 (1998) 46-57
32. Wampold B.E., Mondin G.W., Moody M., Stich F., Benson K., and Ahn H. A meta-analysis of outcome studies comparing bona fide psychotherapies: Empirically, "All must have prizes". Psychological Bulletin 122 3 (1997) 203-215
33. Wilkinson L., and Task Force on Statistical Inference. Statistical methods in psychology journals: Guidelines and explanations. American Psychologist 54 (1999) 594-604