The most important tasks for peer reviewers evaluating a randomized controlled trial are not congruent with the tasks most often requested by journal editors

BMC Medicine

Volumn 13, Issue 1, 2015, Pages

  Chauvin, Anthony ^b,c   Ravaud, Philippe ^a,b,c   Baron, Gabriel ^a,c   Barnes, Caroline ^b,c   Boutron, Isabelle ^a,b,c  

^a HÔTEL DIEU   (France)

^b UNIVERSITÉ PARIS DESCARTES   (France)

^c INSERM   (France)

Author keywords

Peer review; Q sort; Randomized controlled trials; Recommendations to reviewers

Indexed keywords

ARTICLE; AUTHOR; EDITOR; EVALUATION STUDY; HUMAN; JOB ANALYSIS; JOB CHARACTERISTICS; JOURNAL EDITOR; NONMEDICAL OCCUPATIONS; PEER REVIEW; PEER REVIEWER; PRACTICE GUIDELINE; PUBLISHING; SCIENTIFIC LITERATURE; WORK; WORK EXPERIENCE; JOURNAL IMPACT FACTOR; MEDICAL RESEARCH; PUBLICATION; RANDOMIZED CONTROLLED TRIAL (TOPIC);

BIOMEDICAL RESEARCH; EDITORIAL POLICIES; HUMANS; JOURNAL IMPACT FACTOR; PEER REVIEW, RESEARCH; PERIODICALS AS TOPIC; RANDOMIZED CONTROLLED TRIALS AS TOPIC;

EID: 84936880467     PISSN: None     EISSN: 17417015     Source Type: Journal    
DOI: 10.1186/s12916-015-0395-3     Document Type: Article

References (32)

1. Kassirer JP, Campion EW. Peer review: crude and understudied, but indispensable. JAMA. 1994;272:96-7.
2. Jefferson T, Rudin M, Folse S, Davidoff F. Editorial peer review for improving the quality of reports of biomedical studies. Cochrane Database Syst Rev. 2007;18:MR000016.
3. Rennie R. Editorial peer review: its development and rationale. In: Godlee F, Jefferson T, editors. Peer review in health sciences. 2nd ed. London: BMJ Books; 2003. p. 1-13.
4. Public Library of Science. Peer review-optimizing practices for online scholarly communication. In: House of Commons Science and Technology Committee, editor. Peer Review in Scientific Publications, Eighth Report of Session 2010-2012, Vol. I: Report, Together with Formal, Minutes, Oral and Written Evidence. London: The Stationery Office Limited; 2011. p. 21-2.
5. Public Library of Science. Peer review-optimizing practices for online scholarly communication. In: House of Commons Science and Technology Committee, editor. Peer Review in Scientific Publications, Eighth Report of Session 2010-2012, Vol. I: Report, Together with Formal, Minutes, Oral and Written Evidence. London: The Stationery Office Limited; 2011. p. 174-8.
6. Rubriq. How we found 15 million hours of lost time. In: Rubriq: independent peer review system. 2013. http://blog.rubriq.com/2013/06/03/how-we-found-15-million-hours-of-lost-time/. Accessed 27 Mar 2015.
7. Ghimire S, Kyung E, Kang W, Kin E. Assessment of adherence to the CONSORT statement for quality of reports on randomized controlled trial abstracts from four high-impact general medical journals. Trials. 2012;13:77.
8. Boutron I, Dutton S, Ravaud P, Altman DG. Reporting and interpretation of randomized controlled trials with statistically nonsignificant results for primary outcomes. JAMA. 2010;303:2058-64.
9. Hopewell S, Collins GS, Boutron I, Yu LM, Cook J, Shanyinde M, et al. Impact of peer review on reports of randomised trials published in open peer review journals: retrospective before and after study. BMJ. 2014;349:g4145. doi: 10.1136/bmj.g4145.
10. Turner EH, Matthew AM, Linardatos E, Tell RA, Rosenthal R. Selective publication of antidepressant trials and its confluence on apparent efficacy. N Engl J Med. 2008;358:252-60.
11. Melander H, Ahlqvist-Rastad J, Meijer G, Beermann B. Evidence b(i)ased medicine-selective reporting from studies sponsored by pharmaceutical industry: review of studies in new drug applications. BMJ. 2003;326:1171-3.
12. Rowe BH, Strome TL, Spooner C, Bilitz S, Grafstein E, Worster A. Reviewer agreement trends from four years of electronic submissions of conference abstract. BMC Med Res Methodol. 2006;6:14.
13. Jackson JL, Srinivasan M, Rea J, Fletcher KE, Kravitz RL. The validity of peer review in a general medicine journal. PLoS ONE. 2011;6:e22475. doi: 10.1371/journal.pone.0022475.
14. Weiner BK, Weiner JP, Smith HE. Spine journals: is reviewer agreement on publication recommendations greater than would be expected by chance? Spine J. 2010;10:209-11.
15. Bohannon J. Who's afraid by peer review? Science. 2013;342:60-5.
16. Schroter S, Black N, Evans S, Godlee F, Osorio L, Smith R. What errors do peer reviewers detect, and does training improve their ability to detect them? J R Soc Med. 2008;101:507-14.
17. Hirst A, Altman DG. Are peer reviewers encouraged to use reporting guidelines? A survey of 116 health research journals. PlosOne. 2012;7:e35621.
18. Van Exel J, de Graaf G. Q methodology: A sneak preview. http://qmethod.org/articles/vanExel.pdf. 2005. Accessed 28 Mar 2014.
19. Watts S, Stenner P. Doing Q methodological research: theory, method and interpretation. London: Sage Publications; 2012.
20. Brown S. Q-methodology and qualitative research. Qual Life Res. 1996;6:561-7.
21. Dennis KE. Q methodology: relevance and application to nursing research. Adv Nurs Sci. 1986;8:6-17.
22. Mathieu S, Chan AW, Ravaud P. Use of trial register information during the peer review process. PLoS One. 2013;10:8(4).
23. Stephenson W. Correlating persons instead of tests. J Personality. 1935;4:17-24. doi: 10.1111/j.1467-6494.1935.tb02022.x.
24. Hackert C, Braehler G. FlashQ. 2007. http://www.hackert.biz/flashq/. Accessed 30 Mar 2014.
25. Baxt WG, Waeckerle JF, Berlin JA, Callaham ML. Who reviews the reviewers? Feasibility of using a fictitious manuscript to evaluate peer reviewer performance. Ann Emerg Med. 1998;32:310-7.
26. Kravitz RL, Franks P, Feldman MD, Gerrity M, Byrne C, Tierney WM. Editorial peer reviewers' recommendations at a general medical journal: are they reliable and do editors care? PLoS One. 2010;5:e10072.
27. Henderson M. Problems with peer review. BMJ. 2010;340:c1409. doi: http://dx.doi.org/10.1136/bmj.c1409.
28. Yaffe MB. Re-reviewing peer review. Sci Signal. 2009;2:eg11.
29. Stahel PF, Moore EE. Peer review for biomedical publications: we can improve the system. BMC Med. 2014;12:179.
30. Matthieu S, Boutron I, Moher D, Altman DG, Ravaud P. Comparison of registered and published primary outcomes in randomized controlled trials. JAMA. 2009;302:977-84.
31. Cobo E, Cortés J, Ribera JM, Cardellach F, Selva-O'Callaghan A, Kostov B, et al. Effect of using reporting guidelines during peer review on quality of final manuscripts submitted to a biomedical journal: masked randomised trial. BMJ. 2011;22:343.
32. Mills EJ, Wu P, Gagnier J, Devereaux PJ. The quality of randomized trial reporting in leading medical journals since the revised CONSORT statement. Contemp Clin Trials. 2005;26:480-7.