The usefulness of peer review for selecting manuscripts for publication: A utility analysis taking as an example a high-impact journal

PLoS ONE

Volumn 5, Issue 6, 2010, Pages

  Bornmann, Lutz ^a   Daniel, Hans Dieter ^a,b  

^a ETH ZURICH   (Switzerland)

^b UNIVERSITY OF ZURICH   (Switzerland)

Author keywords

[No Author keywords available]

Indexed keywords

ARTICLE; MEDICAL LITERATURE; MEDICAL RESEARCH; PEER REVIEW; PREDICTIVE VALIDITY; PUBLICATION; PUBLISHING;

JOURNAL IMPACT FACTOR; PEER REVIEW, RESEARCH; PUBLISHING;

EID: 77955330873     PISSN: None     EISSN: 19326203     Source Type: Journal    
DOI: 10.1371/journal.pone.0011344     Document Type: Article

References (37)

1. Sense About Science (2004) Peer review and the acceptance of new scientific ideas. London, UK: Sense about Science (working party on equipping the public with an understanding of peer review).
2. Stossel TP (1985) Refinement in biomedical communication - a case study. Science Technology & Human Values 10: 39-43.
3. Bornmann L, Daniel H-D (2008) Selecting manuscripts for a high impact journal through peer review: a citation analysis of Communications that were accepted by Angewandte Chemie International Edition, or rejected but published elsewhere. Journal of the American Society for Information Science and Technology 59: 1841-1852.
4. Bornmann L, Daniel H-D (2008) The effectiveness of the peer review process: inter-referee agreement and predictive validity of manuscript refereeing at Angewandte Chemie. Angewandte Chemie International Edition 47: 7173-7178.
5. Bornmann L, Daniel H-D (2009) Extent of type I and type II errors in editorial decisions: a case study on Angewandte Chemie International Edition. Journal of Informetrics 3: 348-352.
6. van Raan AFJ (2004) Measuring science. Capita selecta of current main issues. In: Moed HF, Glänzel W, Schmoch U, eds. Handbook of quantitative science and technology research The use of publication and patent statistics in studies of S&T systems. Dordrecht, The Netherlands: Kluwer Academic Publishers. pp 19-50.
7. Bornmann L, Daniel H-D (2007) Convergent validation of peer review decisions using the h index: extent of and reasons for type I and type II errors. Journal of Informetrics 1: 204-213.
8. Bornmann L, Wallon G, Ledin A (2008) Does the committee peer review select the best applicants for funding? An investigation of the selection process for two European Molecular Biology Organization programmes. PLoS One 3: e3480.
9. Bornmann L, Leydesdorff L, van den Besselaar P (in press) A meta-evaluation of scientific research proposals: different ways of comparing rejected to awarded applications. Journal of Informetrics.
10. Straub DW (2008) Type II reviewing errors and the search for exciting papers. MIS Quarterly 32: V-X.
11. Straub DW (2008) Thirty years of service to the IS profession: time for renewal at MISQ? MIS Quarterly 32: iii-viii.
12. Thorngate W, Dawes RM, Foddy M (2009) Judging merit. New York, NY, USA: Psychology Press. xi, 185 p.
13. Taylor HG, Russell JT (1939) The relationship of validity coefficients to the practical effectiveness of tests in selection: discussion and tables. Journal of Applied Psychology 23: 565-578.
14. Cascio WF (1991) Costing human resources: the financial impact of behavior in organizations. Boston, MA, USA: PWS-Kent. XIV, 322 S p.
15. Cabrera EF, Raju NS (2001) Utility analysis: current trends and future directions. International Journal of Selection and Assessment 9: 92-102.
16. Bornmann L, Daniel H-D (2009) The luck of the referee draw: the effect of exchanging reviews. Learned Publishing 22: 117-125.
17. Bornmann L, Daniel H-D (2010) The manuscript reviewing process - empirical research on review requests, review sequences and decision rules in peer review. Library & Information Science Research 32: 5-12.
18. Schultz DM (in press) Are three heads better than two? How the number of reviewers and editor behavior affect the rejection rate. Scientometrics.
19. Egghe L (in press) Study of some editor-in-chief decision schemes. Research Evaluation.
20. Craig ID, Plume AM, McVeigh ME, Pringle J, Amin M (2007) Do open access articles have greater citation impact? A critical review of the literature. Journal of Informetrics 1: 239-248.
21. Smith LC (1981) Citation analysis. Library Trends 30: 83-106.
22. Bornmann L, Daniel H-D (2008) What do citation counts measure? A review of studies on citing behavior. Journal of Documentation 64: 45-80.
23. Martin BR, Irvine J (1983) Assessing basic research - some partial indicators of scientific progress in radio astronomy. Research Policy 12: 61-90.
24. van Raan AFJ (1996) Advanced bibliometric methods as quantitative core of peer review based evaluation and foresight exercises. Scientometrics 36: 397-420.
25. Lindsey D (1989) Using citation counts as a measure of quality in science. Measuring what's measurable rather than what's valid. Scientometrics 15: 189-203.
26. Jefferson T, Wager E, Davidoff F (2002) Measuring the quality of editorial peer review. Journal of the American Medical Association 287: 2786-2790.
27. Bornmann L, Mutz R, Neuhaus C, Daniel H-D (2008) Use of citation counts for research evaluation: standards of good practice for analyzing bibliometric data and presenting and interpreting results. Ethics in Science and Environmental Politics 8: 93-102.
28. Vinkler P (1997) Relations of relative scientometric impact indicators. The relative publication strategy index. Scientometrics 40: 163-169.
29. Vinkler P (1986) Evaluation of some methods for the relative assessment of scientific publications. Scientometrics 10: 157-177.
30. van Leeuwen TN (2007) Modelling of bibliometric approaches and importance of output verification in research performance assessment. Research Evaluation 16: 93-105.
31. Neuhaus C, Daniel H-D (2009) A new reference standard for citation analysis in chemistry and related fields based on the sections of Chemical Abstracts. Scientometrics 78: 219-229.
32. Daniel H-D (1993) Guardians of science. Fairness and reliability of peer review. Weinheim, Germany: Wiley-VCH.
33. Radicchi F, Fortunato S, Castellano C (2008) Universality of citation distributions: toward an objective measure of scientific impact. Proceedings of the National Academy of Sciences 105: 17268-17272.
34. Stewart JA (1994) The poisson-lognormal model for bibliometric/scientometric distributions. Information Processing & Management 30: 239-251.
35. Pöschl U (2004) Interactive journal concept for improved scientific publishing and quality assurance. Learned Publishing 17: 105-113.
36. Straub DW (2009) Why top journals accept your paper. MIS Quarterly 33: iii-x
37. Baskerville RL, Myers MD (2009) Fashion waves in information systems research and practice. MIS Quarterly 33: 647-662.