Accuracy of effect size estimates from published psychological experiments involving multiple trials

Journal of General Psychology

Volumn 138, Issue 4, 2011, Pages 281-291

  Brand, Andrew ^a   Bradley, M T ^b   Best, Lisa A ^b   George, Stoica ^b  

^a KING'S COLLEGE LONDON   (United Kingdom)

^b UNIVERSITY OF NEW BRUNSWICK   (Canada)

Author keywords

Alpha levels; Correlation; Effect size; Monte Carlo; Multiple assessments; Power

Indexed keywords

HUMAN; META ANALYSIS (TOPIC); METHODOLOGY; MONTE CARLO METHOD; PROCEDURES; PSYCHOLOGY; SAMPLE SIZE; STATISTICAL ANALYSIS;

DATA INTERPRETATION, STATISTICAL; HUMANS; META-ANALYSIS AS TOPIC; MONTE CARLO METHOD; PSYCHOLOGY, EXPERIMENTAL; RESEARCH DESIGN; SAMPLE SIZE;

EID: 84855809930     PISSN: 00221309     EISSN: 19400888     Source Type: Journal    
DOI: 10.1080/00221309.2011.604365     Document Type: Article

References (25)

1. Anastasi, A. (1988). Psychological testing (6th ed.). New York, NY: Macmillan.
2. Baguley, T. (2009). Standardized or simple effect size: What should be reported? British Journal of Psychology, 100, 603-617.
3. Begg, C. B. (1994) Publication bias. In H. Copper & L. V. Hedges (Eds.), The handbook of research synthesis (pp. 400-409.). New York, NY: Russell Sage Found.
4. Bradley, D. R. & Russell, R. L. (1998). Some cautions regarding statistical power in splitplot designs. Behavior Research Methods, Instruments, & Computers, 30, 462-477. (Pubitemid 128685502)
5. Bradley, M. T. & Stoica G. (2004) Diagnosing estimate distortion due to significance testing in literature on detection of deception. Perceptual and Motor Skills, 98, 827-839. (Pubitemid 39109436)
6. Brand, A., Bradley,M. T., Best L. A., & Stoica, G. (2008). Accuracy of effect size estimates from published psychological research. Perceptual and Motor Skills, 106, 645-649.
7. Brand, A., Bradley, M. T., Best L. A., & Stoica, G. (2011). Multiple trials may yield exaggerated effect size estimates. Journal of General Psychology, 138, 1-11.
8. Cohen, J. (1962). The statistical power of abnormal-social psychological research:Areview. Journal of Abnormal and Social Psychology, 69, 145-153.
9. Cohen, J. (1988). Statistical Power Analysis for the Behaviors Sciences, 2nd edition. Hillsdale, NJ: Erlbaum.
10. Dienes, Z. (2011). Bayesian versus orthodox statistics: Which side are you on? Perspectives on Psychological Science, 6(3), 274-290.
11. Gliner, J. A., Leech, N. L., and Morgan, G. A. (2002). Problems with null hypothesis significance testing (NHST): What do the textbooks say? The Journal of Experimental Education, 71(1), 83-92.
12. Goodman, S.N. (1999). Toward evidence-based medical statistics. 2: The Bayes Factor. Annals of Internal Medicine, 130(12):1005.
13. Hubbard, R., & Lindsay, R. M. (2008).Why P values are not a useful measure of evidence in statistical significance testing. Theory & Psychology, 18, 69-88.
14. Kraemer,H. C., Gardner, C., Brooks, J.O., & Yesavage, J. A. (1998).Advantages of excluding underpowered studies in meta-analysis: Inclusionist versus exclusionist viewpoints. Psychological Methods, 3, 23-31.
15. Lane, D. M, & Dunlap, W. P. (1978). Estimating effect size: Bias resulting from the significance criterion in editorial decisions. British Journal of Mathematical and Statistical Psychology, 31, 107-112.
16. Overall, J. E. (1996). How many repeated measurements are useful? Journal of Clinical Psychology, 52, 243-252. (Pubitemid 26250449)
17. Rouder, J. N., Speckman, P. L., Sun, D., Morey, R. D., & Iverson, G. (2009). Bayesian t tests for accepting and rejecting the null hypothesis. Psychonomic Bulletin Review, 16(2), 225-237.
18. Rossi, J. S. (1990). Statistical power of psychological research:What have we gained in 20 years? Journal of Consulting and Clinical Psychology, 58, 646-656. (Pubitemid 20356009)
19. Sedlmeier, P., & Gigerenzer, G. (1989). Do studies of statistical power have an effect on the power of studies? Psychological Bulletin, 105, 309-316.
20. Sterling, T. D. (1959). Publication power of studies? Psychological Bulletin, 105, 309-316.
21. decisions and their possible effects on inferences drawn from tests of significance or vice versa. Journal of the American Statistical Association, 54, 30-34.
22. Sterling, T. D., RosenbaumW. L., & Weinkam J. J. (1995). Publication decisions revisited: the effect of the outcome of statistical tests on the decision to publish and vice versa. American Statistician, 49, 108-112.
23. Trafimow, D. (2003).Hypothesis testing and theory evaluation at the boundaries: Surprising insights from Bayes's theorem. Psychological Review, 110, 526-535. (Pubitemid 37189436)
24. Trafimow, D., & Rice, S. (2009). A test of the Null Hypothesis Significance Testing Procedure correlation argument. Journal of General Psychology, 136, 261-269.
25. Wilkinson, L., & APA Task Force on Statistical Inference. (1999). Statistical methods in psychology journals: Guidelines and explanations. American Psychologist, 54, 594-604.