Substantial Agreement of Referee Recommendations at a General Medical Journal - A Peer Review Evaluation at Deutsches Ärzteblatt International

PLoS ONE

Volumn 8, Issue 5, 2013, Pages

  Baethge, Christopher ^a,b   Franklin, Jeremy ^b   Mertens, Stephan ^a  

^a Deutsches Ärzteblatt International Editorial Offices ^*  (Germany)

^b UNIVERSITY OF COLOGNE   (Germany)

Author keywords

[No Author keywords available]

Indexed keywords

ARTICLE; COHORT ANALYSIS; MEDICAL LITERATURE; PEER REVIEW; PREDICTION; PUBLICATION; PUBLISHING; RETROSPECTIVE STUDY;

CONSENSUS; EDITORIAL POLICIES; JOURNALISM, MEDICAL; PEER REVIEW; STATISTICS AS TOPIC;

EID: 84877084962     PISSN: None     EISSN: 19326203     Source Type: Journal    
DOI: 10.1371/journal.pone.0061401     Document Type: Article

References (22)

1. Kronick DA, (1990) Peer review in 18th-century scientific journalism. JAMA 263: 1321-2.
2. Burnham JC, (1990) The evolution of editorial peer review. JAMA 263: 1323-29.
3. Horobin DF, (1990) The philosophical basis of peer review and the suppression of innovation. JAMA 263: 1438-41.
4. US National Library of Medicine: Number of titles currently indexed for Index Medicus and MEDLINE on PubMed. Available: http://www.nlm.nih.gov/bsd/num_titles.html. Accessed 2012 Sep 6.
5. Bornmann L, Mutz R, Daniel HD, (2010) A reliability-generalization study of journal peer reviews: a multilevel meta-analysis of inter-rater reliability and its determinants. PLoS ONE 5: e14331.
6. Rothwell PM, Martyn CN, (2000) Reproducibility of peer review in clinical neuroscience - is agreement between reviewers any greater than would be expected by chance alone? Brain 123: 1964-69.
7. Marusić A, Lukić IK, Marusić M, McNamee D, Sharp D, et al. (2002) Peer review in a small and a big medical journal: case study of the Croatian Medical Journal and the Lancet. Croat Med J 43: 286-9.
8. Kravitz RL, Franks P, Feldman MD, Gerrity M, Byrne C, et al. (2010) Editorial peer reviewers' recommendations at a general medical journal: are they reliable and do editors care? PLoS ONE 5: e10072.
9. Yadollahie M, Roshanipoor M, Habibzadeh F, (2004) The agreement in reports of peer reviews in the Iranian Journal of Medical Sciences. Saudi Med J 25 (Suppl. 1) (): S36.
10. Feinstein AR, Cicchetti DV, (1990) High agreement but low kappa: I. The problems of two paradoxes. J Clin Epidemiol 43: 543-549.
11. Gwet K (2002) Kappa statistic is not satisfactory for assessing the extent of agreement between raters. Series: statistical methods for inter-rater reliability: 1 Available: http://www.agreestat.com/research_papers/kappa_statistic_is_not_satisfactory.pdf. Accessed 2012 Aug 5.
12. Gwet K, (2008) Computing inter-rater reliability and its variance in the presence of high agreement. Br J Mathemat Statist Psychol 61: 29-48.
13. Opthof T, Coronel R, Janse MJ, (2002) The significance of the peer review process against the background of bias: priority ratings of reviewers and editors and the prediction of citation, the role of geographic bias. Cardiovasc Res 56: 339-46.
14. Cicchetti DV, Feinstein AR, (1990) High agreement but low kappa: II. Resolving the paradoxes. J Clin Epidemiol 43: 551-558.
15. John Uebersax's Home Page. Available: www.john-uebersax.com/stat/raw.htm. Accessed 2013 Mar 21.
16. Fleiss JL (1981) Statistical methods for rates and proportions. New York, Wiley (2nded.).
17. Landis JR, Koch GG, (1977) The measurement of observer agreement for categorical data. Biometrics 33: 159-174.
18. Bornmann L, Daniel HD, (2009) Reviewer and editor bias in journal peer review: an investigation of manuscript refereeing at Angewandte Chemie International Edition. Res Evaluat 18: 262-272.
19. Baethge C, (2012) Impact factor-a useful tool, but not for all purposes. Dtsch Arztebl Int 109: 267-9.
20. Adler R, Ewing J, Taylor P (2008) Citation statistics. A report from the International Mathematical Union (IMU) in cooperation with the International Council of Industrial and Applied Mathematics (ICIAM) and the Institute of Mathematical Statistics (IMS). Available: www.mathunion.org/fileadmin/IMU/Report/CitationStatistics.pdf. Accessed 2013 Mar 21.
21. Kulkarni AV, Aziz B, Shams I, Busse JW, (2009) Comparison of citations in Web of Science, Scopus, and Google Scholar for articles published in general medical journals. JAMA 302: 1092-1096.
22. Mertens S, Baethge C, (2012) Standards in the face of uncertainty-peer review is flawed and under-researched, but the best we have. Dtsch Arztebl Int 109: 900-2.