What is open peer review? A systematic review

F1000Research

Volumn 6, Issue , 2017, Pages

  Ross Hellauer, Tony ^a  

^a UNIVERSITY OF GÖTTINGEN   (Germany)

Author keywords

Open peer review; Open Science; Publishing; Research evaluation; Scholarly communication

Indexed keywords

CONSENSUS; HUMAN; IDENTITY; PEER REVIEW; PUBLISHING; SYSTEMATIC REVIEW;

EID: 85028970471     PISSN: 20461402     EISSN: 1759796X     Source Type: Journal    
DOI: 10.12688/f1000research.11369.2     Document Type: Review

References (79)

1. Armstrong JS: Barriers to Scientific Contributions: The Authors Formula. Behav Brain Sci. Cambridge University Press (CUP). 1982;5(02):197-199. 10.1017/s0140525x00011201
2. Armstrong JS: Peer Review for Journals: Evidence on Quality Control Fairness, and Innovation. Sci Eng Ethics. Springer Nature. 1997;3(1):63-84. 10.1007/s11948-997-0017-3
3. Bardy AH: Bias in reporting clinical trials. Br J Clin Pharmacol. Wiley-Blackwell. 1998;46(2):147-50. 9723823 10.1046/j.1365-2125.1998.00759.x 1873669
4. Bloom T: Referee Report For: What is open peer review? A systematic review. F1000Res. 2017;6:588. 10.5256/f1000research.12273.r22301
5. Boldt A: Extending ArXiv.Org to Achieve Open Peer Review and Publishing. J Scholarly Publ. University of Toronto Press Inc. (UTPress), 2011;42(2):238-42. 10.3138/jsp.42.2.238
6. Bornmann L Herich H Joos H: In Public Peer Review of Submitted Manuscripts How Do Reviewer Comments Differ from Comments Written by Interested Members of the Scientific Community? A Content Analysis of Comments Written for Atmospheric Chemistry and Physics. Scientometrics. Springer Nature. 2012;93(3):915-29. 10.1007/s11192-012-0731-8
7. Budden AE Tregenza T Aarsen LW: Double-blind review favours increased representation of female authors. Trends Ecol Evol. 2008;23(1):4-6. 17963996 10.1016/j.tree.2007.07.008
8. Campanario JM: Peer Review for Journals as It Stands Today-Part 1. Sci Commun. SAGE Publications. 1998;19(3):181-211. 10.1177/1075547098019003002
9. Casadevall A Fang FC: Specialized science. Infect Immun. 2014;82(4):1355-1360. 24421049 10.1128/IAI.01530-13 3993417
10. Cassella M Calvi L: New Journal Models and Publishing Perspectives in the Evolving Digital Environment. IFLA Journal. SAGE Publications. 2010;36(1):7-15. 10.1177/0340035209359559
11. Chubin DE Hackett EJ: Peerless Science: Peer Review and US Science Policy.Suny Press,1990. Reference Source
12. Cronin B: Vernacular and Vehicular Language. J Am Soc Inf Sci Technol. Wiley-Blackwell. 2009;60(3):433. 10.1002/asi.21010
13. Dall'Aglio P: Peer Review and Journal Models.ArXiv:Physics/0608307,2006. Reference Source
14. Daniel HD: Guardians of Science.Wiley-Blackwell,1993. 10.1002/3527602208
15. Day C: Meet the Overlay Journal. Phys Today. AIP Publishing,2015. 10.1063/pt.5.010330
16. Dickersin K Min YI Meinert CL: Factors influencing publication of research results. Follow-up of applications submitted to two institutional review boards. JAMA. American Medical Association (AMA). 1992;267(3):374-8. 1727960 10.1001/jama.1992.03480030052036
17. EMBO Journal: About | The EMBO Journal [WWW Document].2016; (accessed 8.24.16). Reference Source
18. Ernst E Kienbacher T: Chauvinism. Nature. Springer Nature. 1991;352(6336):560. 10.1038/352560b0
19. Fanelli D: Do Pressures to Publish Increase Scientists' Bias? An Empirical Support from US States Data.Edited by Enrico Scalas. PLoS One. Public Library of Science (PLoS). 2010;5(4):e10271. 20422014 10.1371/journal.pone.0010271 2858206
20. Fang FC Casadevall A: Retracted Science and the Retraction Index. Infect Immun. American Society for Microbiology. 2011;79(10):3855-59. 21825063 10.1128/IAI.05661-11 3187237
21. Fang FC Steen RG Casadevall A: Misconduct Accounts for the Majority of Retracted Scientific Publications. Proc Natl Acad Sci U S A. Proceedings of the National Academy of Sciences. 2012;109(42):17028-33. 23027971 10.1073/pnas.1212247109 3479492
22. Fecher B Friesike S: Open Science: One Term, Five Schools of Thought.In: Bartling, S. and Friesike (Eds.), Opening Science.New York, NY: Springer,2013;17-47. 10.2139/ssrn.2272036
23. Fisher M Friedman SB Strauss B: The Effects of Blinding on Acceptance of Research Papers by Peer Review. JAMA. American Medical Association (AMA). 1994;272(2):143-46. 8015127 10.1001/jama.1994.03520020069019
24. Fitzpatrick K: Planned Obsolescence.New York, NY: NYU Press,2011. Reference Source
25. Fitzpatrick K Santo A: Open Review, A Study of Contexts and Practices. Report.2012. Reference Source
26. Ford E: Defining and Characterizing Open Peer Review: A Review of the Literature. J Scholarly Publ. University of Toronto Press Inc. (UTPress),2013;44(4):311-26. 10.3138/jsp.44-4-001
27. Ford E: Open peer review at four STEM journals: an observational overview [version 2; referees: 2 approved, 2 approved with reservations]. F1000Res. F1000 Research Ltd.2015;4:6. 25767695 10.12688/f1000research.6005.2 4350441
28. Frontiers: About Frontiers Academic Journals and Research Community.2016. Reference Source
29. Garcia JA Rodriguez-Sanchez R Fdez-Valdivia J: Authors and Reviewers Who Suffer from Confirmatory Bias. Scientometrics. Springer Nature.2016;109(2):1377-95. 10.1007/s11192-016-2079-y
30. Gillespie GW Chubin DE Kurzon GM: Experience with NIH Peer Review: Researchers Cynicism and Desire for Change. Sci Technol Hum Val. 1985;10(3):44-54. 10.1177/016224398501000306
31. Godlee F Gale CR Martyn CN: Effect on the quality of peer review of blinding reviewers and asking them to sign their reports: a randomized controlled trial. JAMA. American Medical Association (AMA).1998;280(3):237-40. 9676667 10.1001/jama.280.3.237
32. Hames I: The Changing Face of Peer Review. Sci Ed. Korean Council of Science Editors.2014;1(1):9-12. 10.6087/kcse.2014.1.9
33. Hanson B Lawrence R Meadows A: Early Adopters of ORCID Functionality Enabling Recognition of Peer Review: Two Brief Case Studies. Learn Publ. Wiley-Blackwell.2016;29(1):60-63. 10.1002/leap.1004
34. Harnad S: The Invisible Hand of Peer Review.Journal (On-line/Unpaginated). Exploit Interactive.2000. Reference Source
35. Herron DM: Is expert peer review obsolete? A model suggests that post-publication reader review may exceed the accuracy of traditional peer review. Surg Endosc. Springer Nature.2012;26(8):2275-80. 22350231 10.1007/s00464-012-2171-1
36. Ioannidis JP: Effect of the statistical significance of results on the time to completion and publication of randomized efficacy trials. JAMA. American Medical Association (AMA).1998;279(4):281-6. 9450711 10.1001/jama.279.4.281
37. Janowicz K Hitzler P: Open and Transparent: the Review Process of the Semantic Web Journal. Learn Publ. Wiley-Blackwell.2012;25(1):48-55. 10.1087/20120107
38. Jubb M: Peer Review: The Current Landscape and Future Trends. Learn Publ. Wiley-Blackwell.2016;29(1):13-21. 10.1002/leap.1008
39. Justice AC Cho MK Winker MA: Does masking author identity improve peer review quality? A randomized controlled trial. PEER Investigators. JAMA. American Medical Association (AMA).1998;280(3):240-2. 9676668 10.1001/jama.280.3.240
40. Kaplan S: Major Publisher Retracts 64 Scientific Papers in Fake Peer Review Outbreak.Washington Post,2015. Reference Source
41. Kerr S Tolliver J Petree D: Manuscript Characteristics Which Influence Acceptance for Management and Social Science Journals. Acad Manage J. The Academy of Management.1977;20(1):132-41. 10.2307/255467
42. Kravitz RL Franks P Feldman MD: Editorial peer reviewers' recommendations at a general medical journal: are they reliable and do editors care? PLoS One. Public Library of Science (PLoS).2010;5(4):e10072. 20386704 10.1371/journal.pone.0010072 2851650
43. Kriegeskorte N: Open evaluation: a vision for entirely transparent post-publication peer review and rating for science. Front Comput Neurosci. Frontiers Media SA.2012;6:79. 23087639 10.3389/fncom.2012.00079 3473231
44. Leek JT Taub MA Pineda FJ: Cooperation between referees and authors increases peer review accuracy. PLoS One. Public Library of Science (PLoS).2011;6(11):e26895. 22096506 10.1371/journal.pone.0026895 3212530
45. Link AM: US and non-US submissions: an analysis of reviewer bias. JAMA. 1998;280(3):246-7. 9676670 10.1001/jama.280.3.246
46. Lloyd ME: Gender factors in reviewer recommendations for manuscript publication. J Appl Behav Anal. Society for the Experimental Analysis of Behavior.1990;23(4):539-43. 16795738 1286270
47. Mahoney MJ: Publication Prejudices: An Experimental Study of Confirmatory Bias in the Peer Review System. Cognit Ther Res. 1977;1(2): Springer Nature:161-75. 10.1007/BF01173636
48. McNutt RA Evans AT Fletcher RH: The effects of blinding on the quality of peer review. A randomized trial. JAMA. American Medical Association (AMA).1990;263(10):1371-6. 2304216 10.1001/jama.1990.03440100079012
49. Monsen ER Van Horn L: Research: Successful Approaches. American Dietetic Association;2007. Reference Source
50. Moore S Neylon C Eve MP: Excellence R Us: University Research and the Fetishisation of Excellence. Palgrave Commun. Springer Nature.2017;3: 16105. 10.1057/palcomms.2016.105
51. Mulligan A Hall L Raphael E: Peer Review in a Changing World: An International Study Measuring the Attitudes of Researchers. J Am Soc Inf Sci Technol. Wiley-Blackwell.2013;64(1):132-61. 10.1002/asi.22798
52. Nicholson J Alperin JP: A Brief Survey on Peer Review in Scholarly Communication. The Winnower,2016. Reference Source
53. Nickerson RC Varshney U Muntermann J: A Method for Taxonomy Development and Its Application in Information Systems. Eur J Inf Syst. Springer Nature.2013;22(3):336-59. 10.1057/ejis.2012.26
54. Nobarany S Booth KS: Use of Politeness Strategies in Signed Open Peer Review. J Assoc Inf Sci Technol. Wiley-Blackwell.2015;66(5):1048-64. 10.1002/asi.23229
55. OpenAIRE: OpenAIRE's Experiments in Open Peer Review / Report. Zenodo. 2016. 10.5281/zenodo.154647
56. Perakakis P Taylor M Mazza M: Natural Selection of Academic Papers. Scientometrics. Springer Nature.2010;85(2):553-59. 10.1007/s11192-010-0253-1
57. Peters DP Ceci SJ: Peer-Review Practices of Psychological Journals: The Fate of Published Articles Submitted Again. Behav Brain Sci. Cambridge University Press (CUP).1982;5(02):187-195. 10.1017/S0140525X00011183
58. Pontika N Knoth P Cancellieri M: Fostering Open Science to Research Using a Taxonomy and an ELearning Portal. In Proceedings of the 15th International Conference on Knowledge Technologies and Data-Driven Business - i-KNOW 15. Association for Computing Machinery (ACM).2015. 10.1145/2809563.2809571
59. Pöschl U: Multi-stage open peer review: scientific evaluation integrating the strengths of traditional peer review with the virtues of transparency and self-regulation. Front Comput Neurosci. Frontiers Media SA.2012;6:33. 22783183 10.3389/fncom.2012.00033 3389610
60. PubPeer: Science Self-Corrects - Instantly. PubPeer: The Online Journal Club.2014. Reference Source
61. Research Information Network: Activities, Costs and Funding Flows in the Scholarly Communications System in the UK: Report Commissioned by the Research Information Network (RIN).2008. Reference Source
62. Ross JS Gross CP Desai MM: Effect of blinded peer review on abstract acceptance. JAMA. American Medical Association (AMA).2006;295(14):1675-80. 16609089 10.1001/jama.295.14.1675
63. Ross-Hellauer T: Review of Definitions of Open Peer Review in the Scholarly Literature 2016.2017. Data Source
64. Sandewall E: Maintaining Live Discussion in Two-Stage Open Peer Review. Front Comput Neurosci. Frontiers Media SA.2012;6:9. 22363282 10.3389/fncom.2012.00009 3282940
65. Schekman R Watt F Weigel D: The eLife approach to peer review. eLife. eLife Sciences Organisation Ltd.2013;2:e00799. 23638304 10.7554/eLife.00799 3639505
66. Schroter S Black N Evans S: Effects of Training on Quality of Peer Review: Randomised Controlled Trial. BMJ. BMJ.2004;328(7441):673-70. 14996698 10.1136/bmj.38023.700775.AE 381220
67. Smith R: Peer review: a flawed process at the heart of science and journals. J R Soc Med. SAGE Publications.2006;99(4):178-82. 16574968 1420798
68. Spier R: The History of the Peer-Review Process. Trends Biotechnol. Elsevier BV.2002;20(8):357-58. 12127284 10.1016/S0167-7799(02)01985-6
69. Steen RG Casadevall A Fang FC: Why has the number of scientific retractions increased?Edited by Gemma Elizabeth Derrick. PLoS One. Public Library of Science (PLoS).2013;8(7):e68397. 23861902 10.1371/journal.pone.0068397 3704583
70. Suber P: Open Access.Cambridge, MA: MIT Press,2012. Reference Source
71. Tennant JP Dugan JM Graziotin D: A multi-disciplinary perspective on emergent and future innovations in peer review [version 1; referees: 2 approved with reservations]. F1000Res. 2017;6:1151. 10.12688/f1000research.12037.1
72. Travis GD Collins HM: New Light on Old Boys: Cognitive and Institutional Particularism in the Peer Review System. Sci Technol Hum Val. 1991;16(3). 10.1177/016224399101600303
73. Tregenza T: Gender Bias in the Refereeing Process? Trends Ecol. 2002;17(8):349-350. 10.1016/S0169-5347(02)02545-4
74. van Rooyen S Delamothe T Evans SJ: Effect on peer review of telling reviewers that their signed reviews might be posted on the web: randomised controlled trial. BMJ. BMJ.2010;341:c5729. 21081600 10.1136/bmj.c5729 2982798
75. van Rooyen S Godlee F Evans S: Effect of open peer review on quality of reviews and on reviewers' recommendations: a randomised trial. BMJ. BMJ.1999;318(7175):23-27. 9872878 10.1136/bmj.318.7175.23 27670
76. Walker R Rocha da Silva P: Emerging trends in peer review-a survey. Front Neurosci. Frontiers Media SA.2015;9:169. 26074753 10.3389/fnins.2015.00169 4444765
77. Ware M: Peer Review: Benefits, Perceptions and Alternatives. Publishing Research Consortium 4,2008. Reference Source
78. Ware M: Peer Review: Recent Experience and Future Directions. New Review of Information Networking. Informa UK Limited.2011;16(1):23-53. 10.1080/13614576.2011.566812
79. Ware M: Peer Review Survey 2015. Publishing Research Consortium.2016. Reference Source