Emerging trends in peer review - a survey

Frontiers in Neuroscience

Volumn 9, Issue APR, 2015, Pages

  Walker, Richard ^a   da Silva, Pascal Rocha ^a  

^a EPFL   (Switzerland)

Author keywords

Anonymity; Impact metrics; Interactive review; Non selective review; Open access; Open peer review; Peer review; Preprint servers

Indexed keywords

ARTICLE; COGNITION; COGNITIVE BIAS; COMMUNITY; HUMAN; INTERPERSONAL COMMUNICATION; PEER REVIEW; PRACTICE GUIDELINE; PUBLICATION; QUANTITATIVE STUDY;

EID: 84928667465     PISSN: 16624548     EISSN: 1662453X     Source Type: Journal    
DOI: 10.3389/fnins.2015.00169     Document Type: Article

References (126)

1. Armstrong, J. S. (1980). Unintelligible management research and academic prestige. Interfaces 10 (April): 80-86. doi: 10.1287/inte.10.2.80.
2. Armstrong, J. S. (1997). Peer review for journals: evidence on quality control, fairness, and innovation. Science and Engineering Ethics 3: 63-84. doi: 10.1007/s11948-997-0017-3.
3. Armstrong, J. S. (1998). Management science: what does it have to do with management or science? Marketing Bulletin 9 (1): 1-13.
4. ArXiv. (2015a). arXiv monthly submission rates [CSV]. Retrieved February 18, 2015, from http://arxiv.org/stats/get_monthly_submissions.
5. ArXiv. (2015b). Quantitative Biology (since Sep 2003). Retrieved February 18, 2015, from http://arxiv.org/archive/q-bio.
6. Auburn University. (2014). PrePrint Servers. Retrieved July 6, 2014, from http://www.lib.auburn.edu/preprint/.
7. Benos, D. J., E. Bashari, J. M. Chaves, A. Gaggar, N. Kapoor, M. LaFrance, R. Mans, D. Mayhew, S. McGowan, A. Polter, Y. Qadri, S. Sarfare, K. Schultz, R. Splittgerber, J. Stephenson, C. Tower, R. G. Walton and A. Zotov (2007). The ups and downs of peer review. Advances in Physiology Education 31 (2): 145-152. doi: 10.1152/advan.00104.2006.
8. bioRxiv. (2015). bioRxiv beta - the preprint server for biology. Retrieved February 18, 2015, from http://biorxiv.org/search/.
9. Bornmann, L. and H.-D. Daniel (2009). Reviewer and editor biases in journal peer review: an investigation of manuscript refereeing at Angewandte Chemie International Edition. Research Evaluation 18 (4): 262-272. doi: 10.3152/095820209X477520.
10. Bradley, J. V. (1981). Pernicious publication practices. Bulletin of the Psychonomic Society 18: 31-34. doi: 10.3758/BF03333562.
11. Brown, J. A. (2010). Overlay journals, repositories and the evolution of scholarly communication. Retrieved February 13, 2015, from http://or2010.fecyt.es/Publico/Posters/index.aspx.
12. Brownlee, C. (2006). Peer review under the microscope: One journal's experiment aims to change science vetting. Science News 170 (25): 392-393. doi: 10.2307/4017427.
13. Budden, A. E., T. Tregenza, L. W. Aarssen, J. Koricheva, R. Leimu and C. J. Lortie. (2008). Double-blind review favours increased representation of female authors. Trends in ecology & evolution 23 (1): 4-6. doi: 10.1016/j.tree.2007.07.008.
14. Burnham, J. C. (1990). The evolution of editorial peer review. JAMA 263 (10): 1323-1329. doi: 10.1001/jama.1990.03440100023003.
15. Campanario, J. M. (1997). The journal scout. The Scientist 11 (9).
16. Campanario, J. M. (1998). Peer review for journals as it stands today-Part 1. Science Communication 19: 181-211. doi: 10.1177/1075547098019003002.
17. Campanario, J. M. (2009). Rejecting and resisting Nobel class discoveries: accounts by Nobel Laureates. Scientometrics 81 (2): 549-565. doi: 10.1007/s11192-008-2141-5.
18. Campbell, P. (2006). Nature peer review. Trial and debate. Retrieved February 13, 2015, from http://www.nature.com/nature/peerreview/.
19. Ccanz (2013). "Open Access megajournals - have they changed everything?" Creative Commons Aotearoa New Zealand http://creativecommons.org.nz/2013/10/open-accessmegajournals-have-they-changed-everything/Accessed February 13 2015.
20. Clarke, M. (2008). "Working double-blind." peertopeer http://blogs.nature.com/peer-topeer/2008/02/working_doubleblind.html Accessed February 13 2015.
21. Cogprints. (2015). Browse by year. Retrieved February 13, 2015, from http://cogprints.org/view/year/.
22. Cressey, D. (2014). Journals weigh up double-blind peer review. Nature. doi: 10.1038/nature.2014.15564.
23. Darling, E. S. (2014). Use of double-blind peer review to increase author diversity: diversity and double-blind review. Conservation Biology: n/a-n/a. doi: 10.1111/cobi.12333.
24. DeCoursey, T. (1999) Pros and cons of open peer review."Nature Neuroscience 2, 197-198 doi: 10.1038/nature04991.
25. Ellison, G. (2011). Is peer review in decline? Economic Inquiry 49 (3): 635-657. doi: DOI: 10.1111/j.1465-7295.2010.00261.x.
26. EMBOJ. (2015). About the EMBO Journal. Retrieved February 13, 2015, from http://emboj.embopress.org/about.
27. Emerson, G. B., W. J. Warme, F. M. Wolf, J. D. Heckman, R. A. Brand and S. S. Leopold (2010). Testing for the presence of positive-outcome bias in peer review: a randomized controlled trial. Archives of internal medicine 170 (21): 1934. doi: 10.1001/archinternmed.2010.406.
28. Fitzpatrick, K. (2009). The future of peer review. Retrieved February 13, 2015, from http://mcpress.media-commons.org/plannedobsolescence/one/the-future-of-peer-review/.
29. Fletcher, R. H. and S. W. Fletcher (1997). Evidence for the effectiveness of peer review. Sci Eng Ethics 3: 35-43. doi: 10.1007/s11948-997-0015-5.
30. Fox, J. (2014). "Post-publication review is here to stay-for the scientific 1%." Dynamic Ecology http://dynamicecology.wordpress.com/2014/03/20/post-publication-review-comesof-age-for-the-scientific-one-percent/Accessed February 13 2014.
31. Frontiers in Neuroscience. (2015). Review Guidelines. Retrieved February 13, 2015, from http://www.frontiersin.org/Neuroscience/reviewguidelines.
32. Gilbert, J. R., E. S. Williams and G. D. Lundberg (1994). Is there gender bias in JAMA's peer review process? JAMA 272 (2): 139-142.
33. Ginsparg, P. (1994). First steps Towards electronic research communication. Computers in Physics 8 (4): 390-396. doi: 10.1063/1.4823313.
34. Ginsparg, P. (2011). ArXiv at 20: The price for success - CornellCast. Retrieved February 13, 2015, from http://www.cornell.edu/video/arxiv-at-20-the-price-for-success.
35. Haldane's sieve. (2015). Welcome to Haldane's Sieve. Retrieved February 13, 2015, from http://haldanessieve.org/about/.
36. Hames, I. (2014). The changing face of peer review. Science Editing 1 (1): 9-12.
37. Harnad, S. (1979). Creative disagreement. The Sciences 19: 18-20.
38. Harnad, S. (1990). Scholarly skywriting and the prepublication continuum of scientific inquiry. Psychological science: 342-344.
39. Harnad, S. (1991). Post-Gutenberg galaxy: The fourth revolution in the means of production of knowledge. Public-access computer systems review 2 (1): 39-53.
40. Herrera, A. J. (1999). Language bias discredits the peer-review system. Nature 397 (6719): 467-467.
41. Herron, D. M. (2012). Is expert peer review obsolete? A model suggests that post-publication reader review may exceed the accuracy of traditional peer review. Surgical endoscopy 26 (8): 2275-2280. doi: 10.1007/s00464-012-2171-1.
42. Hill, S. and F. Provost (2003). The myth of the double-blind review?: author identification using only citations. ACM SIGKDD Explorations Newsletter 5 (2): 179-184.
43. Ioannidis, J. P. (2005). Why most published research findings are false. PLoS medicine 2 (8): e124. doi: 10.1371/journal.pmed.0020124.
44. Jaschik, S. (2011). "Rejecting double blind." Inside higher ed https://http://www.insidehighered.com/news/2011/05/31/american_economic_association_abandons_double_blind_journal_reviewing, Accessed February 13 2015.
45. Jaschik, S. (2012). Humanities scholars consider the role of peer review. Retrieved July 20, 2014, from http://www.insidehighered.com/news/2012/01/06/humanities-scholars-considerrole-peer-review - sthash.EMt6oWQ4.dpbs.
46. John, L. K., G. Loewenstein and D. Prelec (2012). Measuring the prevalence of questionable research practices With incentives for truth telling. Psychological Science 23 (5): 524-532. doi: 10.1177/0956797611430953.
47. Journal of Interactive Media in Education. (1996). Aims of the Journal. Retrieved February 13, 2015, from http://web.archive.org/web/20041126030702/http://www-jime.open.ac.uk/.
48. Journal of Interactive Media in Education. (2014). Journal of Interactive Media in Education. Retrieved February 13, 2015, from http://www-jime.open.ac.uk/.
49. K-theory Preprint Archives. (2015). K-theory Preprint Archives. Retrieved February 18, 2015, from http://www.math.uiuc.edu/K-theory/.
50. Kairos. (2015). Kairos: A journal of rhetoric, technology, and pedagogy. Retrieved February 13, 2015, from http://kairos.technorhetoric.net/about.html.
51. Kaiser, J. (2013). New preprint server aims to be biologists' answer to physicists' arXiv. Retrieved February 13, 2015, from http://news.sciencemag.org/biology/2013/11/newpreprint-server-aims-be-biologists-answer-physicists-arxiv.
52. Kerr, S., J. Tolliver and D. Petree (1977). Manuscript characteristics which influence acceptance for management and social science journals. The Academy of Management Journal 20 (1): 132-141. doi: 10.2307/255467.
53. Knoepfler, P. (2014). Review of Obokata stress reprogramming Nature papers. Retrieved February 13, 2015, from http://www.ipscell.com/2014/01/review-of-obokata-stressreprogramming-nature-papers/.
54. Koonin, E. (2006). Reviving a culture of scientific debate. Nature. doi: 10.1038/nature05005.
55. Koop, T. and U. Pöschl (2006). Systems - an open two-stage peer-review journal. Nature. doi: doi:10.1038/nature04988.
56. Kravitz, R. L., P. Franks, M. D. Feldman, M. Gerrity, C. Byrne and W. M. Tierney (2010). Editorial peer reviewers' recommendations at a general medical journal: are they reliable and do editors care? PLoS One 5 (4): e10072. doi: 10.2307/255467.
57. Kriegeskorte, N. (2012). Open evaluation: a vision for entirely transparent post-publication peer review and rating for science. Frontiers in computational neuroscience 6.
58. Kriegeskorte, N., A. Walther and D. Deca (2012). An emerging consensus for open evaluation: 18 visions for the future of scientific publishing. Frontiers in Computational Neuroscience. doi: 10.3389/fncom.2012.00094.
59. Laakso, M. and B.-C. Björk (2012). Anatomy of open access publishing: a study of longitudinal development and internal structure. BMC Medicine 10 (1). doi: 10.1186/1741-7015-10-124.
60. Lee, K. K.-H. (2014). Review of article: stimulus-triggered fate conversion of somatic cells into pluripotency. Retrieved February 13, 2015, from https://http://www.researchgate.net/publication/259984904_Stimulustriggered_fate_conversion_of_somatic_cells_into_pluripotency/reviews/103?_sg=Is5Ua%2B%2FlGWU%2FsSweZQiVCTQ6r53ZUE%2FBl4ouv635Z8smwN3mpLuzAEWfXOMYxYypEbkfeBeUwECVj0ahslcwfYG85lp8fD%2BOunsvbYFSMrk%3D.
61. Linear Algebraic Groups and Related Structures Preprint Server. (2015). Linear Algebraic Groups and Related Structures Preprint Server,. Retrieved February 18, 2015, from http://www.math.uni-bielefeld.de/lag/.
62. Link, A. M. (1998). US and Non-US Submissions: An Analysis of Reviewer Bias. JAMA July 15, 1998 280 (3): 246-247.
63. Lipton, R. J. (2011). "Deolalikar's claim: one year later." Gödel's Lost Letter and P=NP http://rjlipton.wordpress.com/2011/08/11/deolalikars-claim-one-year-later/Accessed February 13 2015.
64. Lloyd, M. E. (1990). Gender factors in reviewer recommendations for manuscript publication. Journal of Applied Behavior Analysis 23: 539-543. doi: 10.1901/jaba.1990.23-539.
65. Loy, M. (2011). Hindawi Publishing Corporation: Growing an Open-Access contributor-pays business model: case study update 2011. Retrieved February 13, 2015, from http://www.sr.ithaka.org/download/file/fid/532.
66. MacNealy, M. S., B. W. Speck and N. Clements (1994). Publishing in technical communication journals from the successful author's point of view. Technical communication 41 (2): 240-259.
67. Madden, S. and D. DeWitt (2006). Impact of double-blind reviewing on SIGMOD publication rates. ACM SIGMOD Record 35 (2): 29-32. doi: 10.1145/1147376.1147381.
68. Mahoney, M. J. (1977). Publication prejudices: An experimental study of confirmatory bias in the peer review system. Cognitive Therapy and Research 1 (2): 161-175. doi: 10.1007/bf01173636.
69. Mathematics on the Web. (2015). Mathematics Preprint Servers. Retrieved February 18, 2015, from http://www.mathontheweb.org/mathweb/mi-preprints.html.
70. McNutt, R. A., A. T. Evans, R. H. Fletcher and S. W. Fletcher (1990). The effects of blinding on the quality of peer review: A randomized trial. JAMA 263 (10): 1371-1376. doi: 10.1001/jama.1990.03440100079012.
71. Mole, B. (2012). The future of peer review in the humanities Is wide open. The Chronicle of Higher Education.
72. Moyle, M. and P. Polydoratou (2007). Investigating overlay journals: introducing the RIOJA Project. D-Lib Magazine 13 (September/October).
73. Multiple Authors. (2014a). "Bidirectional developmental potential in reprogrammed cells with acquired pluripotency". Retrieved February 13, 2015, from https://pubpeer.com/publications/1F3D9CBBB6A8F1953284B66EEA7887.
74. Multiple Authors (2014b). "STAP new data." Knoepfler Lab Stem Cell Blog http://www.ipscell.com/stap-new-data/Accessed February 13 2015.
75. Nature. (2006a). Overview: Nature's peer review trial. Retrieved February 13, 2015, from http://www.nature.com/nature/peerreview/debate/nature05535.html. doi: 10.1038/nature05535.
76. Nature (2006b). Peer review and fraud. Nature 444 (7122): 971-972. doi: 10.1038/444971b.
77. Nature Climate Change (2014). Blind stock-taking. Nature Climate Change 4 (1): 1-1. doi: 10.1038/nclimate2096.
78. Nature Precedings. (2015). Nature precedings - pre-publication research and preliminary findings. Retrieved February 13, 2015, from http://precedings.nature.com/.
79. Nature Publishing Group. (2015). Notice on web site - no title. Retrieved February 18, 2015, from http://www.nature.com/content/npg/23909.html.
80. Neylon, C. (2010). "Science in the Open » Blog Archive » P ? NP and the future of peer review." http://cameronneylon.net/blog/p-%e2%89%a0-np-and-the-future-of-peer-review/Accessed February 13 2015.
81. Obokata, H., T. Wakayama, Y. Sasai, K. Kojima, M. P. Vacanti, H. Niwa, M. Yamato and C. A. Vacanti (2014). Stimulus-triggered fate conversion of somatic cells into pluripotency. Nature 505 (7485): 641-647. doi: 10.1038/nature12968.
82. Ozonoff, D. (2006). Statistics in peer review. Nature. doi: 10.1038/nature04989.
83. Peters, D. P. and S. J. Ceci (1982). Peer-review practices of psychology journals: The fate of published articles, submitted again. The Behavioral and brain Sciences 5: 187-195. doi: 10.1017/S0140525X00011183.
84. PLOS ONE. (2015a). PLOS ONE guidelines for reviewers. Retrieved February 13, 2015, from http://www.plosone.org/static/reviewerGuidelines.
85. PLOS ONE. (2015b). PLOS ONE Journal information. Retrieved February 13, 2015, from http://www.plosone.org/static/information.action-1.
86. Pöschl, U. (2012). Multi-stage open peer review: scientific evaluation integrating the strengths of traditional peer review with the virtues of transparency and self-regulation. Frontiers in Computational Neuroscience 6. doi: 10.3389/fncom.2012.00033.
87. Preprints on Conservation Laws. (2015). Preprints on Conservation Laws. Retrieved February 18, 2015, from http://www.math.ntnu.no/conservation/.
88. Price, R. (2013). Peer review needs to expand so that more scientists are reviewing papers. The Guardian.
89. Prinz, F., T. Schlange and K. Asadullah (2011). Believe it or not: how much can we rely on published data on potential drug targets? Nature Reviews Drug Discovery 10 (9): 712-712. doi: 10.1038/nrd3439-c1.
90. Proceedings of the National Academiy of Sciences of the United States of America. (2015). Editorial Policies. Retrieved February 13, 2015, from http://www.pnas.org/site/authors/guidelines.xhtml.
91. PubPeer (2015). "Science self-corrects - instantly | PubPeer." http://blog.pubpeer.com/?p=117 Accessed February 13 2015.
92. Redfield, R. (2010). "Arsenic-associated bacteria (NASA's claims)." RRResearch http://rrresearch.fieldofscience.com/2010/12/arsenic-associated-bacteria-nasas.html Accessed February 13 2015.
93. Rowe, K. (2010). "From the editor: gentle numbers | Shakespeare Quarterly | MediaCommons Press." http://mcpress.media-commons.org/ShakespeareQuarterly_NewMedia/from-theeditor-gentle-numbers/Accessed February 13 2015.
94. Ryter, S. W. and A. M. Choi (2013). Testing the rebound peer review concept. Antioxidants & redox signaling 19 (7): 639-643. doi: 10.1089/ars.2013.5431.
95. Sandewall, E. (1997). A neo-classical structure for scientific publication and reviewing. Linköping Electronic Articles on Academic Policies and Trends 2.
96. Schekman, R., F. Watt and D. Weigel (2013). The eLife approach to peer review. eLife 2. doi: 10.7554/eLife.00799.
97. Scholarly Open Access. (2014). New OA publisher aims to compete with preprint servers. Retrieved February 13, 2015, from http://scholarlyoa.com/2014/05/27/new-oa-publisheraims-to-compete-with-preprint-servers/.
98. Schroter, S., N. Black, S. Evans, J. Carpenter, F. Godlee and R. Smith (2004). Effects of training on quality of peer review: randomised controlled trial. BMJ 328 (7441): 673. doi: 10.1136/bmj.38023.700775.AE.
99. Sciencepaper Online. (2015). Sciencepaper Online. Retrieved February 18, 2015, from http://www.paper.edu.cn/en.
100. Scientific Reports. (2015). Scientific Reports. Retrieved February 18, 2015, from http://www.ncbi.nlm.nih.gov/pmc/journals/1579/.
101. Scimago Lab. (2014). Scimago journal and country rank. Retrieved June 29, 2014, from http://www.scimagojr.com/journalrank.php?area=1200&category=0&country=all&year=2012&order=sjr&min=0&min_type=cd.
102. Semantic Web Journal. (2015). For reviewers. Retrieved February 13, 2015, from http://www.semantic-web-journal.net/reviewers.
103. Sen, C. K. (2012). Rebound peer review: a viable recourse for aggrieved authors? Antioxidants & redox signaling 16 (4): 293-296. doi: 10.1089/ars.2011.4424.
104. Sieber, J. E. (2012). Improving the peer-review process relies on understanding its context and culture. Nature. doi: 10.1038/nature05006.
105. Smith, A. P. (2000). The journal as an overlay on preprint databases. Learned Publishing 13 (1): 43-48. doi: 10.1087/09531510050145542.
106. Smith, R. (2006). Peer review: a flawed process at the heart of science and journals. Journal of the Royal Society of Medicine 99 (4): 178-182.
107. Smith, R. (2011). "Richard Smith: What is post publication peer review?" The BMJ http://blogs.bmj.com/bmj/2011/04/06/richard-smith-what-is-post-publication-peer-review/2014.
108. Snodgrass, R. (2006). Single-versus double-blind reviewing: an analysis of the literature. ACM Sigmod Record 35 (3): 8-21. doi: 10.1145/1168092.1168094.
109. Spier, R. (2002). The history of the peer-review process. Trends in Biotechnology 20 (8): 357-358. doi: 10.1016/S0167-7799(02)01985-6.
110. Srivastava, S. (2014). "Reflections on a foray into post-publication peer review." The hardest science http://hardsci.wordpress.com/2013/03/25/reflections-on-a-foray-into-postpublication-peer-review/Accessed February 13 2015.
111. The Economist. (2014). Let the light shine in. Retrieved February 18, 2015, from http://www.economist.com/news/science-and-technology/21604089-two-big-recentscientific-results-are-looking-shakyand-it-open-peer-review.
112. The University of Chicago Press Journals. (2014). Current Anthropology - instructions for authors. Retrieved February 13, 2015, from http://www.press.uchicago.edu/journals/ca/instruct.html?journal=ca.
113. Tomaiuolo, N. G. and J. G. Packer (2000). Pre-print servers: pushing the envelope of electronic scholarly publishing. Searcher 8 (9).
114. Travis, G. D. L. and H. M. Collins (1991). New light on old boys: cognitive and institutional particularism in the peer review system. Science, Technology, & Human Values 16 (3): 322-341. doi: 10.1177/016224399101600303.
115. Tregenza, T. (2002). Gender bias in the refereeing process? Trends in Ecology & Evolution 17 (8): 349-350.
116. Tung, A. K. H. (2006). Impact of double blind reviewing on SIGMOD publication: a more detailed analysis. SIGMOD Rec. 35 (3): 6-7. doi: 10.1145/1168092.1168093.
117. University of Wollongong. (2014). Preprint servers. Retrieved July 7, 2014, from http://www.library.uow.edu.au/eresources/UOW026409.html.
118. Van Noorden, R. (2013). Company offers portable peer review. Nature 494 (7436): 161-161. doi: 10.1038/494161a.
119. Van Rooyen, S., N. Black and F. Godlee (1999). Development of the Review Quality Instrument (RQI) for Assessing Peer Reviews of Manuscripts. Journal of Clinical Epidemiology 52 (7): 625-629. doi: 10.1016/S0895-4356(99)00047-5.
120. Van Rooyen, S., F. Godlee, S. Evans, N. Black and R. Smith (1999). Effect of open peer review on quality of reviews and on reviewers' recommendations: a randomised trial. Bmj 318 (7175): 23-27. doi: 10.1136/bmj.318.7175.23.
121. Van Rooyen, S., T. Delamothe and S. J. Evans (2010). Effect on peer review of telling reviewers that their signed reviews might be posted on the web: randomised controlled trial. BMJ: British Medical Journal 341. doi: 10.1136/bmj.c5729.
122. Veitch, E. (2010). "Does post-publication peer review work?" Speaking of Medicine http://blogs.plos.org/speakingofmedicine/2010/08/25/does-post-publication-peer-reviewwork/Accessed February 13 2015.
123. Walker, R., B. Barros, R. Conejo, K. Neumann and M. Telefont (2015). Bias in peer review: a case study. F1000Research. doi: 10.12688/f1000research.6012.1.
124. Ware, M. (2008). Peer review: benefits, perceptions and alternatives. Publishing Research Consortium.
125. Webb, T. J., B. O'Hara and R. P. Freckleton (2008). Does double-blind review benefit female authors? Trends in ecology & evolution 23 (7): 351-353.
126. Wikipedia (2014). Preprint. Wikipedia, the free encyclopedia.