Retrospective Statistical Power: Fallacies and Recommendations

Newborn and Infant Nursing Reviews

Volumn 10, Issue 1, 2010, Pages 55-59

  Wang, Lihshing Leigh ^a  

^a UNIVERSITY OF CINCINNATI   (United States)

Author keywords

Confidence interval; Effect size; Observed power; Post hoc power; Retrospective power; Statistical testing

Indexed keywords

EID: 76549111331     PISSN: 15273369     EISSN: None     Source Type: Journal    
DOI: 10.1053/j.nainr.2009.12.012     Document Type: Article

References (42)

1. Cohen J. Statistical Power Analysis for the Behavioral Sciences (1988), Academic, New York
2. Cohen J. A power primer. Psychol Bull 112 (1992) 155-159
3. Cohen J. The earth is round (p < .05). Am Psychol 49 (1994) 997-1003
4. Levin J.R., and Robinson D.H. Rejoinder: statistical hypothesis testing, effect size estimation, and the conclusion coherence of primary research studies. Educ Res 29 (2000) 34-36
5. Onwuegbuzie A.J., and Leech N.L. Post hoc power: a concept whose time has come. Underst Stat 3 (2004) 201-230
6. Hubbard R., Lindsay R.M., and Why P. values are not a useful measure of evidence in statistical significance testing. Theory Psychol 18 (2008) 69-88
7. Kline R.B. Beyond Significance Testing: Reforming Data Analysis Methods in Behavioral Research (2004), American Psychological Association, Washington, DC
8. Goodman S.N., and Berlin J.A. The use of predicted confidence intervals when planning experiments and the misuse of power when interpreting results. Ann Intern Med 121 (1994) 200-206
9. Robinson D.H., and Levin J.R. Reflections on statistical and substantive significance, with a slice of replication. Educ Res 26 (1997) 21-26
10. In: Rothstein H.R., Sutton A.J., and Borenstein M. (Eds). Publication Bias in Meta-analysis: Prevention, Assessment and Adjustments (2005), Wiley, Chichester (UK)
11. Rotton J., Foos P.W., Vanmeek L., and Levitt M. Publication practices and the file drawer problem - A survey of published authors. J Soc Behav Pers 10 (1995) 1-13
12. Erturk S.M. Retrospective power analysis: when?. Radiology 237 (2005) 743
13. Hoenig J.M., and Heisey D.M. The abuse of power: the pervasive fallacy of power calculations for data analysis. Am Stat 55 (2001) 19-24
14. O'Keefe K.J. Post hoc power, observed power, a priori power, retrospective power, prospective power, achieved power: Sorting out appropriate uses of statistical power analyses. Commun Methods Meas 1 (2007) 291-299
15. Smith S.D. Statistical tools in the quest for truth: hypothesis testing, confidence intervals, and the power of clinical studies. Ophthalmology 115 (2008) 423-424
16. Fidler F., and Cumming G. The new stats: attitudes for the 21st century. In: Osborne J.W. (Ed). Best Practices in Quantitative Methods (2008), Sage, Thousand Oaks (Calif) 1-12
17. Schmidt F.L., and Hunter J.E. Eight common but false objections to the discontinuation of significance testing in the analysis of research data. In: Harlow L.A., Mulaik S.A., and Steiger J.H. (Eds). What If There Were No Significance Tests? (1997), Lawrence Erlbaum, Mahwah (NJ) 37-64
18. American Educational Research Association. Standards for reporting on empirical social science research in AERA publications. Educ Res 35 (2006) 33-40
19. American Psychological Association. Publication Manual of the American Psychological Association. 6th ed. (2009), American Psychological Association, Washington, DC
20. Mulaik S.A., Raju N.S., and Harshman R.A. There is a time and a place for significance testing. In: Harlow L.L., Mulaik S.A., and Steiger J.H. (Eds). What If There Were No Significance Tests? (1997), Lawrence Erlbaum, Mahwah (NJ) 65-115
21. Wainer H., and Robinson D.H. Shaping up the practice of Null Hypothesis Significance Testing. Educ Res 32 (2003) 22-30
22. Yuan K.-H., and Maxwell S. On the post hoc power in testing mean differences. J Educ Behav Stat 30 (2005) 141-167
23. Thomas L. Retrospective power analysis. Conserv Biol 11 (1997) 276-280
24. Zumbo B.D., and Hubley A.M. A note on misconceptions concerning prospective and retrospective power. The Statistician 47 Part 2 (1998) 385-388
25. Gillett R. Post hoc power analysis. J Appl Psychol 79 (1994) 783-785
26. Thomas L., and Krebs C.J. A review of statistical power analysis software. Bull Ecol Soc Am 78 (1997) 126-139
27. Sedlmeier P., and Gigerenzer G. Do studies of statistical power have an effect on the power of studies?. Psychol Bull 105 (1989) 309-316
28. Sun S, Pan W, Wang L. Rethinking Observed Power: Concept, Practice, and Implications. Manuscript submitted for publication in methodology: European Journal of Research Methods for the Behavioral and Social Sciences; 2009.
29. Thompson B. Foundations of Behavioral Statistics: An Insight-Based Approach (2006), Guilford, New York
30. Finch S., Cumming G., and Thomason N. Reporting of statistical inference in the Journal of Applied Psychology: little evidence of reform. Educ Psychol Meas 61 (2001) 181-210
31. Gerard P.D., Smith D.R., and Weerakkody G. Limits of retrospective power analysis. J Wildl Manag 62 (1998) 801-807
32. Oakes J.M., and Feldman H.A. Statistical power for nonequivalent pretest-posttest designs. Eval Rev 25 (2001) 3-28
33. Levine M., and Ensom M.H.H. Post hoc power analysis: an idea whose time has passed?. Pharmacotherapy 21 (2001) 405-409
34. Goodman S.N. A comment on replication, P-values and evidence. Stat Med 11 (1992) 857-859
35. Matcham J., McDermott M.P., and Lang A.E. GDNF in Pakinson's disease: the perils of post-hoc power. J Neurosci Methods 163 (2007) 193-196
36. Hogarty K.Y., and Kromrey J.D. RETR_PWR: an SAS macro for retrospective statistical power analysis. Behav Res Methods Instrum Comput 35 (2003) 585-589
37. Froman T., and Shneyderman A. Replicability reconsidered: an excessive range of possibilities. Underst Stat 3 (2004) 365-373
38. Steiger J.H., and Fouladi R.T. Noncentrality interval estimation and the evaluation of statistical models. In: Harlow L.L., Mulaik S.A., and Steiger J.H. (Eds). What If There Were No Significance Tests? (1997), Erlbaum, Mahwah (NJ) 221-257
39. Taylor D.J., and Muller K.E. Computing confidence bounds for power and sample size of the general linear univariate model. Am Stat 49 (1995) 43-47
40. Wilkinson L., and Task Force on Statistical Inference. APA Board of Scientific Affairs. Statistical methods in psychology journals: guidelines and explanations. Am Psychol. 54 (1999) 594-604
41. Colegrave N., and Ruxton G.D. Confidence intervals are a more useful complement to nonsignificant tests than are power calculations. Behav Ecol 14 (2003) 446-447
42. Overall J.E. Classical statistical hypotheses within the context of Bayesian theory. Psychol Bull 71 (1969) 285-292