Open evaluation (OE): A vision for entirely transparent post-publication peer review and rating for science

Frontiers in Computational Neuroscience

Volumn , Issue SEPTEMBER, 2012, Pages

  Kriegeskorte, Nikolaus ^a  

^a MEDICAL RESEARCH COUNCIL   (United Kingdom)

Author keywords

[No Author keywords available]

Indexed keywords

CORE FEATURES; DECISION PROCESS; EVALUATION FUNCTION; H-FACTORS; INFLUENCE EVALUATION; LOW QUALITIES; OPEN ACCESS; PEER REVIEW; PEER-REVIEW PROCESS; PUBLISHING SYSTEMS; RATING SCALE; SCIENTIFIC COMMUNITY; SCIENTIFIC FINDINGS; SCIENTIFIC LITERATURE; SCIENTIFIC PAPERS; SCIENTIFIC SOCIETY; STATISTICAL TECHNIQUES;

FUNCTION EVALUATION; PAPER; RATING; REVIEWS;

QUALITY CONTROL;

ACCESS TO INFORMATION; ANALYTICAL RESEARCH; EVALUATION RESEARCH; HUMAN; JOURNAL IMPACT FACTOR; LIBRARY SCIENCE; PEER REVIEW; PRIORITY JOURNAL; PROFIT; PUBLIC OPINION; PUBLICATION; QUALITY CONTROL; RATING SCALE; REVIEW; SCIENTIFIC LITERATURE; SYSTEM ANALYSIS; WEB BROWSER;

EID: 84866753331     PISSN: 16625188     EISSN: None     Source Type: Journal    
DOI: 10.3389/fncom.2012.00079     Document Type: Review

References (39)

1. Bachmann, T. (2011) Fair and Open Evaluation May Call for Temporarily Hidden Authorship, Caution When Counting the Votes, and Transparency of the Full Pre-publication Procedure. Front. Comput. Neurosci. 5:61. doi: 10.3389/fncom.2011.00061
2. Birukou, A., Wakeling, J. R., Bartolini, C., Casati, F., Marchese, M., Mirylenka, K., Osman, N., Ragone, A., Sierra, C., and Wassef, A. (2011) Alternatives to peer review: novel approaches for research evaluation. Front. Comput. Neurosci. 5:56. doi: 10.3389/fncom.2011.00056
3. Boldt, A. (2010) Extending ArXiv. org to Achieve Open Peer Review and Publishing, arXiv: 1011.6590v1 http://arxiv.org/abs/1011.6590
4. Breese, J. S., Heckerman, D., & Kadie, C. (1998). Empirical analysis of predictive algorithms for collaborative filtering. (G. Cooper & S. Moral, Eds.)Proceedings of the 14th conference on Uncertainty in Artificial Intelligence, 461(8), 43-52. San Francisco, CA. doi:10.1111/j.1553-2712.2011.01172.x
5. Florian, R. V. (2012) Aggregating post-publication peer reviews and ratings. Front. Comput. Neurosci. 6:31. doi: 10.3389/fncom.2012.00031
6. Frishauf P (2009) Reputation Systems: A New Vision for Publishing and Peer Review, J Participat Med. 2009(Oct);1(1):e13a. http://www.jopm.org/opinion/commentary/2009/10/21/reputation-systems-anew-vision-for-publishing-and-peer-review/print/
7. Godlee, F. (2002). Making Reviewers VisibleOpenness, Accountability, and Credit. JAMA. 2002;287(21):2762-2765
8. Goldberg, D. (1992). Using collaborative filtering to weave an information tapestry. Commun. ACM 35(12), 61-70.
9. Greaves, S., Scott, J., Clarke, M., Miller, L., Hannay, T., Thomas, A., Campbell, P. (2006) Nature's trial of open peer review. Nature Online Peer Review Debate http://www.nature.com/nature/peerreview/debate/nature05535.html
10. Groves, T (2010) Is open peer review the fairest system? Yes, BMJ 2010; 341:c6424 http://dx.doi.org/10.1136/bmj.c6424
11. Harnad, S. (1990) Scholarly Skywriting and the Prepublication Continuum of Scientific Inquiry. Psychological Science, 1, 342-343.
12. Harnad, S. (1997) Learned Inquiry and the Net: The Role of Peer Review, Peer Commentary and Copyright. Learned Publishing, 11, (4), 283-292.
13. Harnad, Stevan (2010) The Open Challenge: A Brief History. Public Service Review: European Science & Technology, 9, 13-15.
14. Hirsch, J.E. (2005) An index to quantify an individual's scientific research output. Proc Natl Acad Sci U S A. 2005 Nov 15;102(46):16569-72.
15. Hodgkinson, M (2007) Open peer review & community peer review. Journalology (blog) http://journalology.blogspot.com/2007/06/open-peer-review-community-peer-review.html
16. Khan, K (2010) Is open peer review the fairest system? No, BMJ 2010;341:c6425 http://dx.doi.org/10.1136/bmj.c6425
17. Kravitz, D. J., and Baker, C. I. (2011) Toward a New Model of Scientific Publishing: Discussion and a Proposal. Front. Comput. Neurosci. 5:55. doi: 10.3389/fncom.2011.00055
18. Kreiman, G. and Maunsell, J. (2011) Nine criteria for a measure of scientific output. Front. Comput. Neurosci. 5:48. doi: 10.3389/fncom.2011.00048
19. Lee, C. (2012) Open Peer Review by a Selected-Papers Network. Front. Comput. Neurosci. 6:1. doi: 10.3389/fncom.2012.00001
20. Neylon, C., Wu, S. (2009). Article-level metrics and the evolution of scientific impact. PLoS biology, 7(11), e1000242. Public Library of Science. doi:10.1371/journal.pbio.1000242
21. Nielsen, M. (2009). Doing science in the open. Physics World, 22(5), 30-35.
22. Nielsen, M. (2011). Reinventing Discovery: The New Era of Networked Science (p. 280). Princeton University Press.
23. Pöschl, U (2010) Interactive Open Access Publishing and Peer Review: The Effectiveness and Perspectives of Transparency and Self-Regulation in Scientific Communication and Evaluation, Liber Quarterly 19, 3/4, 293-314
24. Pöschl, U. (2012) Multi-Stage Open Peer Review: Scientific Evaluation Integrating the Strengths of Traditional Peer Review with the Virtues of Transparency and Self-Regulation. Front. Comput. Neurosci. 6:33. doi: 10.3389/fncom.2012.00033
25. Priem J., and Hemminger, B. M. (2012) Decoupling the scholarly journal. Front. Comput. Neurosci. 6:19. doi: 10.3389/fncom.2012.00019
26. Priem, J., Hemminger, B. (2010). Scientometrics 2.0: Toward new metrics of scholarly impact on the social Web. First Monday, 15(7). http://firstmonday.org/htbin/cgiwrap/bin/ojs/index.php/fm/article/view/2874/2570
27. Pulverer, B (2010) Transparency showcases strength of peer review, Nature 468, 29-31. http://dx.doi.org/10.1038/468029a
28. Sandewall, E. (2012) Maintaining Live Discussion in Two-Stage Open Peer Review. Front. Comput. Neurosci. 6:9. doi: 10.3389/fncom.2012.00009
29. Schafer, J. B., Frankowski, D., Herlocker, J., & Sen, S. (2007). Collaborative Filtering Recommender Systems. (P. Brusilovsky, A. Kobsa, & W. Nejdl, Eds.)International Journal of Electronic Business, 2(1), 77. Springer.
30. Smith, J. (1999) The Deconstructed Journal-A New Model for Academic Publishing. Learned Publishing 12(2), 79-91.
31. Smith, R. (2006) Peer review: a flawed process at the heart of science and journals, J R Soc Med. 99(4): 178-182.
32. Smith, R. (2009) In Search of an Optimal Peer Review System, J Participat Med. 1(1):e13.
33. Van Rooyen, S., Delamothe, T., Evans, S. J. W. (2010). Effect on peer review of telling reviewers that their signed reviews might be posted on the web: randomised controlled trial. British Medical Journal 341, c5729.
34. Van Rooyen, S., Godlee, F., Evans, S., Black, N., & Smith, R. (1999). Effect of open peer review on quality of reviews and on reviewers' recommendations: a randomized trial. British Medical Journal, 318(7175), 23-27.
35. Walsh, E., Rooney, M., Appleby, L., & Wilkinson, G. (2000). Open peer review: a randomised controlled trial. The British Journal of Psychiatry 176, 47-51.
36. Walther, A., and van den Bosch, J. F. (2012) FOSE: a framework for open science evaluation. Front. Comput. Neurosci. 6:32. doi: 10.3389/fncom.2012.00032
37. Ware, M. (2011) Peer review: recent experience and future directions. New Review of Information Networking, 16:23-53. DOI: 10.1080/13614576.2011.566812
38. Wicherts, J. M., Kievit, R. A., Bakker, M., and Borsboom, D. (2012) Letting the daylight in: Reviewing the reviewers and other ways to maximize transparency in science. Front. Comput. Neurosci. 6:20. doi: 10.3389/fncom.2012.00020
39. Zimmermann, J., Roebroeck, A., Uludag, K., Sack, A. T., Formisano, E., Jansma, B., De Weerd, P., and Goebel, R. (2012) Network-based statistics for a community driven transparent publication process. Front. Comput. Neurosci. 6:11. doi: 10.3389/fncom.2012.00011