Survey of editors and reviewers of high-impact psychology journals: Statistical and research design problems in submitted manuscripts

Journal of Psychology: Interdisciplinary and Applied

Volumn 145, Issue 3, 2011, Pages 195-209

  Harris, Alex ^a   Reeder, Rachelle ^a   Hyun, Jenny ^a  

^a CENTER FOR HEALTH CARE EVALUATION   (United States)

Author keywords

journal editors; scientific reporting; statistics; survey

Indexed keywords

HUMAN; INFORMATION PROCESSING; JOURNAL IMPACT FACTOR; METHODOLOGY; PEER REVIEW; PSYCHOLOGY; PUBLICATION; PUBLISHING; QUALITY CONTROL; REVIEW; STANDARD; STATISTICAL ANALYSIS; STATISTICS;

DATA COLLECTION; DATA INTERPRETATION, STATISTICAL; EDITORIAL POLICIES; HUMANS; JOURNAL IMPACT FACTOR; MANUSCRIPTS AS TOPIC; PEER REVIEW, RESEARCH; PERIODICALS AS TOPIC; PSYCHOLOGY; QUALITY CONTROL; RESEARCH DESIGN;

EID: 79953721465     PISSN: 00223980     EISSN: 19401019     Source Type: Journal    
DOI: 10.1080/00223980.2011.555431     Document Type: Review

References (50)

1. Annals of Internal Medicine (2011). Information for authors. Retrieved from http://www.annals.org/site/misc/ifora.xhtml#missing-data
2. Blankers,M.,Koeter,M.W.,& Schippers,G.M. (2010).Missing data approaches in ehealth research: Simulation study and a tutorial for nonmathematically inclined researchers. Journal of Medical Internet Research, 12(5), e54.
3. Cunningham, A. J., & Edmonds, C. V. I. (2005). Possible effects of psychological therapy on survival duration in cancer patients. Journal of Clinical Oncology, 23(22), 5263. (Pubitemid 46224041)
4. Cunningham, J. B., & McCrum-Gardner, E. (2007). Power, effect and sample size using gpower: Practical issues for researchers and members of research ethics committees. Evidence Based Midwifery, 5(4), 132-136.
5. Daniels, M., & Hogan, J. (2008). Missing data in longitudinal studies: Strategies for Bayesian modeling and sensitivity analysis. New York, NY: Chapman & Hall.
6. Dar, R., Serlin, R. C., & Omer, H. (1994). Misuse of statistical test in three decades of psychotherapy research. Journal of Consulting and Clinical Psychology, 62(1), 75-82. (Pubitemid 24058699)
7. Efron, B., & Tibshirani, R. (1993). An introduction to the bootstrap. Boca Raton, FL: Chapman & Hall.
8. Erdfelder, E., Faul, F., & Buchner, A. (1996). Gpower: A general power analysis program. Behavior Research Methods, Instruments, and Computers, 28(1), 1-11. (Pubitemid 126416697)
9. Faul, F., Erdfelder, E., Lang, A.-G., & Buchner, A. (2007). G*Power 3: A flexible statistical power analysis for the social, behavioral, and biomedical sciences. Behavior Research Methods, 39, 175-191.
10. Fitzmaurice, G. M., Laird, N. M., & Ware, J. H. (2004). Applied longitudinal analysis. Hoboken, NJ: Wiley.
11. Gueorguieva, R., & Krystal, J. H. (2004). Move over ANOVA: Progress in analyzing repeated-measures data and its reflection in papers published in the Archives of General Psychiatry. Archives of General Psychiatry, 61(3), 310-317. (Pubitemid 38295833)
12. Hagen, R. L. (1997). In praise of the null hypothesis statistical test. American Psychologist, 52, 15-24. (Pubitemid 127458595)
13. Harris, A. H. S., Reeder, R.,& Hyun, J. K. (2009a). Common statistical and research design problems in manuscripts submitted to high-impact psychiatry journals:What editors and reviewers want authors to know. Journal of Psychiatric Research, 43, 1231-1234.
14. Harris, A. H. S., Reeder, R.,& Hyun, J. K. (2009b). Common statistical and research design problems in manuscripts submitted to high-impact public health journals. Open Public Health Journal, 2, 44-48.
15. Hintze, J. (2008). PASS 2008. Kaysville, UT: NCSS.
16. Huberty, C., & Morris, J. (1989).Multivariate analysis versus multiple univariate analyses. Psychological Bulletin, 105, 302-308.
17. Humphreys, K., Harris, A. H. S, & Weingardt, K. R. (2008). Subject eligibility criteria can substantially influence the results of alcohol-treatment outcome research. Journal of Studies on Alcohol and Drugs, 69(5), 757-764.
18. Humphreys, K., Weingardt, K. R., & Harris, A. H. S (2007). Influence of subject eligibility criteria on compliance with National Institutes of Health guidelines for inclusion of women, minorities, and children in treatment research. Alcoholism: Clinical and Experimental Research, 31(6), 988-995. (Pubitemid 46776177)
19. Institute for Scientific Information. (2007). Journal citation report. Philadelphia, PA: Thompson Scientific.
20. Irwin, J. R., &McClelland, G. H. (2003, August). Negative consequences of dichotomizing continuous predictor variables. Journal of Marketing Research, XL, 366-371. (Pubitemid 37056582)
21. Jaeger, R. G., & Halliday, T. R. (1998). On confirmatory versus exploratory research. Herpetologica, 54(Supp.), S64-S66. (Pubitemid 28278856)
22. Jelicic, H., Phelps, E., & Lerner, R. M. (2009). Use of missing data methods in longitudinal studies: The persistence of bad practices in developmental psychology. Developmental Psychology, 45(4), 1195-1199.
23. Kim, M. H., & Do, Y. K. (2007). Strengthening causal inference in studies using nonexperimental data: An application of propensity score and instrumental variable methods. Journal of Preventive Medicine and Public Health, 40(6), 495-504.
24. Kraemer, H. C., Loew, K., & Kupfer, D. (2005). To your health: How to understand what research tells us about risk. New York, NY: Oxford University Press.
25. Kraemer, H. C., Mintz, J., Noda, A., Tinklenberg, J., & Yesavage, J. A. (2006). Caution regarding the use of pilot studies to guide power calculations for study proposals. Archives of General Psychiatry, 63(5), 484-489.
26. Kraemer, H. C., Stice, E., Kazdin, A., Offord, D., & Kupfer, D. (2001). How do risk factors work together? Mediators, moderators, and independent, overlapping, and proxy risk factors. American Journal of Psychiatry, 158(6), 848-856. (Pubitemid 32547610)
27. Lieberman, M. D., & Cunningham, W. A. (2009). Type I and Type II error concerns in fMRI research: Re-balancing the scale. Social Cognitive Affective Neuroscience, 4(4), 423-428.
28. MacCallum, R. C., Zhang, S., Preacher, K. J., & Rucker, D. D. (2002). On the practice of dichotomization of quantitative variables. Psychological Methods, 7, 19-40.
29. MacKinnon, D. P., & Lockwood, C. M. (2003). Advances in statistical methods for substance abuse prevention research. Prevention Science, 4(3), 155-171. (Pubitemid 39023969)
30. Maxwell, S. E. (2004). The persistence of underpowered studies in psychological research: Causes, consequences, and remedies. Psychological Methods, 9(2), 147-163. (Pubitemid 38706222)
31. McCaffrey, D. F., Ridgeway, G., & Morral, A. R. (2004). Propensity score estimation with boosted regression for evaluating causal effects in observational studies. Psychological Methods, 9(4), 403-425. (Pubitemid 40005040)
32. Miller, G. A., & Chapman, J. P. (2001). Misunderstanding analysis of covariance. Journal of Abnormal Psychology, 110(1), 40-48. (Pubitemid 32205156)
33. Moher, D., Cook, D. J., Eastwood, S., Olkin, I., Rennie, D., & Stroup, D. F. (1999). Improving the quality of reports of meta-analysis of randomised controlled trials: TheQUOROM statement-quality of reporting of meta-analyses. Lancet, 354(9193), 1896-1900.
34. Nickerson, R. S. (2000). Null hypothesis significance testing: A review of an old and continuing controversy. Psychological Methods, 5(2), 241-301.
35. Pocock, S. J., Hughes, M. D., & Lee, R. J. (1987). Statistical problems in the reporting of clinical trials. New England Journal of Medicine, 317(7), 426-432. (Pubitemid 17131180)
36. Psychosomatic Medicine (2011). Psychosomatic medicine statistical guidelines. Retrieved from http://www.psychosomaticmedicine.org/misc/stat.shtml
37. Raudenbush, S. W., Spybrook, J., Congdon, R., Liu, X., & Martinez, A. (2009). Optimal design software for multi-level and longitudinal research (Version 2.0) [Software]. Retrieved from http://sitemaker.umich.edu/group-based
38. Rennie, D. (1996). How to report randomized controlled trials: The CONSORT Statement. Journal of the American Medical Association, 276(8), 649.
39. Rennie, D. (2001). CONSORT revised: Improving the reporting of randomized trials. Journal of the American Medical Association, 285(15), 2006-2007. (Pubitemid 32366956)
40. Royston, P., Altman, D. G., & Sauerbrei, W. (2006). Dichotomizing continuous predictors in multiple regression: A bad idea. Statistics in Medicine, 25(1), 127-141. (Pubitemid 43136953)
41. Rubin, D. B. (1976). Inference and missing data. Biometrika, 63, 581-592.
42. Schafer, J. L. (1997). Analysis of incomplete multivariate data. New York, NY: Chapman & Hall.
43. Schafer, J. L., & Graham, J. W. (2002). Missing data: Our view of the state of the art. Psychological Methods, 7(2), 147-177.
44. Schlomer, G. L., Bauman, S., & Card, N. A. (2010). Best practices for missing data management in counseling psychology. Journal of Counseling Psychology, 57(1), 1-10.
45. Shaffii, M., Lavely, R., & L'Jaffe, R. (1975). Meditation and the prevention of alcohol abuse. American Journal of Psychiatry, 132(9), 942-945.
46. Snijders, T. A. B. (2005). Power and sample size in multilevel linear models. In B. S. Everitt & D. C. Howell (Eds.), Encyclopedia of statistics in behavioral science (Vol. 3, pp. 1570-1573). Chichester, England: Wiley.
47. Steyerberg, E.W., Eijkemans, M. J., & Habbema, J. D. (1999). Stepwise selection in small data sets: A simulation study of bias in logistic regression analysis. Journal of Clinical Epidemiology, 52(10), 935-942. (Pubitemid 29486559)
48. Stroup, D. F., Berlin, J. A., Morton, S. C., Olkin, I., Williamson, G. D., Rennie, D. et al. (2000).Meta-analysis of observational studies in epidemiology:Aproposal for reporting, Meta-analysis Of Observational Studies in Epidemiology (MOOSE) group. Journal of the American Medical Association, 283(15), 2008-2012. (Pubitemid 30214957)
49. Wang, D., Zhang, W., & Bakhai, A. (2004). Comparison of Bayesian model averaging and stepwise methods for model selection in logistic regression. Statistics in Medicine, 23(22), 3451-3467. (Pubitemid 39467863)
50. Wilkinson, L., & Task Force on Statistical Inference, A. B. o. S. A. (1999). Statistical methods in psychology journals: Guidelines and explanations. American Psychologist, 54, 594-604.