Do author-suggested reviewers rate submissions more favorably than editor-suggested reviewers? a study on atmospheric chemistry and physics

PLoS ONE

Volumn 5, Issue 10, 2010, Pages

  Bornmann, Lutz ^a   Daniel, Hans Dieter ^b,c  

^a MAX PLANCK INSTITUTE   (Germany)

^b ETH ZURICH   (Switzerland)

^c UNIVERSITY OF ZURICH   (Switzerland)

Author keywords

[No Author keywords available]

Indexed keywords

ATMOSPHERE; AUTHOR; CITATION ANALYSIS; CLINICAL RESEARCH; CONTROLLED STUDY; EDITOR; EVALUATION; HUMAN; HUMAN RELATION; JOURNAL IMPACT FACTOR; PEER REVIEW; PREDICTION; PROCESS OPTIMIZATION; PUBLICATION; QUALITY CONTROL; REVIEW; SCIENTIFIC LITERATURE; SYSTEMATIC ERROR; ARTICLE; CHEMISTRY; DATA BASE; PHYSICS; PUBLISHING;

ATMOSPHERE; CHEMISTRY; DATABASE MANAGEMENT SYSTEMS; PEER REVIEW, RESEARCH; PHYSICS; PUBLICATION BIAS;

EID: 78149423721     PISSN: None     EISSN: 19326203     Source Type: Journal    
DOI: 10.1371/journal.pone.0013345     Document Type: Review

References (50)

1. Weller AC (2002) Editorial peer review: its strengths and weaknesses. Medford, NJ, USA: Information Today, Inc.
2. Arkes HR (2003) The nonuse of psychological research at two federal agencies. Psychological Science 14: 1-6.
3. Jayasinghe UW (2003) Peer review in the assessment and funding of research by the Australian Research Council. Greater Western Sydney, Australia: University of Western Sydney.
4. Lee K, Boyd E, Bero L (2004) A look inside the black box: a description of the editorial process at three leading biomedical journals; Ottawa, Canada.
5. Tonks A (1995) Reviewers chosen by authors. British Medical Journal 311: 210.
6. European Association for Chemical and Molecular Sciences (2006) Ethical guidelines for publication in journals and reviews. Brussels, Belgium: European Association for Chemical and Molecular Sciences (EuCheMS).
7. Perlman D, Dean E (1987) The wisdom of Salomon: avoiding bias in the publication review process. In: Jackson DN, Rushton J, eds. Scientific excellence Origins and assessment. London, UK: Sage. pp 204-221.
8. Anon (2007) Gatekeepers of science. Interview with Peter Stern, editor at Science magazine. BIF Futura 22: 14-18.
9. Schroter S, Tite L, Hutchings A, Black N (2006) Differences in review quality and recommendations for publication between peer reviewers suggested by authors or by editors. JAMA 295: 314-317.
10. Grimm D (2005) Suggesting or excluding reviewers can help get your paper published. Science 309: 1974.
11. Scharschmidt BF, Deamicis A, Bacchetti P, Held MJ (1994) Chance, concurrence, and clustering - analysis of reviewers recommendations on 1,000 submissions to the Journal of Clinical Investigation. Journal of Clinical Investigation 93: 1877-1880.
12. Earnshaw JJ, Farndon JR (2000) A comparison of reports from referees chosen by authors or journal editors in the peer review process. Annals of the Royal College of Surgeons of England 82: 133-135.
13. Goldsmith LA, Blalock E, Bobkova H, Hall RP (2005) Effect of authors' suggestions concerning reviewers on manuscript acceptance. In: Rennie D, Godlee F, Flanagin A, Smith J, eds. 5th International Congress on Peer Review and Biomedical Publication: Chicago, IL, USA.
14. Wager E, Parkin E, Tamber P (2006) Are reviewers suggested by authors as good as those chosen by editors? Results of a rater-blinded, retrospective study. BMC Medicine 4: 13.
15. Rivara FP, Cummings P, Ringold S, Bergman AB, Joffe A, et al. (2007) A comparison of reviewers selected by editors and reviewers suggested by authors. Journal of Pediatrics 151.
16. Bornmann L, Daniel H-D (2009) Reviewer and editor biases in journal peer review: an investigation of manuscript refereeing at Angewandte Chemie International Edition. Research Evaluation 18: 262-272.
17. Jayasinghe UW, Marsh HW, Bond N (2003) A multilevel cross-classified modelling approach to peer review of grant proposals: the effects of assessor and researcher attributes on assessor ratings. Journal of the Royal Statistical Society Series a-Statistics in Society 166: 279-300.
18. Bailey CW (2005) Open Access Bibliography. Washington, DC, USA: Association of Research Libraries.
19. Pöschl U (2010) Interactive open access publishing and peer review: the effectiveness and perspectives of transparency and self-regulation in scientific communication and evaluation. Liber Quarterly 19: 293-314.
20. Koop T, Pöschl U (2006) Systems: an open, two-stage peer-review journal. The editors of Atmospheric Chemistry and Physics explain their journal's approach. Retrieved 26 June 2006, from http://www.nature.com/nature/peerreview/debate/nature04988.html.
21. Bingham CM, Higgins G, Coleman R, Van Der Weyden MB (1998) The Medical Journal of Australia Internet peer-review study. Lancet 352: 441-445.
22. Pöschl U (2004) Interactive journal concept for improved scientific publishing and quality assurance. Learned Publishing 17: 105-113.
23. Bornmann L, Daniel H-D (2009) The luck of the referee draw: the effect of exchanging reviews. Learned Publishing 22: 117-125.
24. Bornmann L, Daniel H-D (2010) Reliability of reviewers' ratings at an interactive open access journal using public peer review: a case study on Atmospheric Chemistry and Physics. Learned Publishing 23: 124-131.
25. Bornmann L, Marx W, Schier H, Thor A, Daniel H-D (2010) From black box to white box at open access journals: predictive validity of manuscript reviewing and editorial decisions at Atmospheric Chemistry and Physics. Research Evaluation 19: 81-156.
26. Bornmann L, Neuhaus C, Daniel H-D (in press) The effect of a two-stage publication process on the Journal Impact Factor: a case study on the interactive open access journal Atmospheric Chemistry and Physics. Scientometrics.
27. Budden AE, Aarssen L, Koricheva J, Leimu R, Lortie CJ, et al. (2008) Response to Whittaker: challenges in testing for gender bias. TRENDS in Ecology & Evolution 23: 480-481.
28. Laband DN, Piette MJ (1994) Favoritism versus search for good papers: empirical evidence regarding the behavior of journal editors. Journal of Political Economy 102: 194-203.
29. Smart S, Waldfogel J (1996) A citation-based test for discrimination at economics and finance journals. NBER working Paper, No 5460 Cambridge, MA, USA: National Bureau of Economic Research.
30. Jayasinghe UW, Marsh HW, Bond N (2001) Peer review in the funding of research in higher education: the Australian experience. Educational Evaluation and Policy Analysis 23: 343-346.
31. Agresti A (2002) Categorical data analysis. Hoboken, NJ, USA: John Wiley & Sons, Inc.
32. Cytel Software Corporation (2010) StatXact: version 9. Cambridge, MA, USA: Cytel Software Corporation.
33. Long JS, Freese J (2006) Regression models for categorical dependent variables using Stata. College Station, TX, USA: Stata Press, Stata Corporation.
34. StataCorp (2009) Stata statistical software: release 11. College Station, TX,USA: Stata Corporation.
35. Nichols A (2007) Causal inference with observational data. Stata Journal 7: 507-541.
36. Rabe-Hesketh S, Skrondal A (2008) Multilevel and longitudinal modeling using Stata. College Station, TX, USA: Stata Press.
37. Brant R (1990) Assessing proportionality in the proportional odds model for ordinal logistic regression. Biometrics 46: 1171-1178.
38. Long JS (1997) Regression models for categorical and limited dependent variables. Thousand Oaks, California, USA: Sage.
39. van Raan AFJ (1996) Advanced bibliometric methods as quantitative core of peer review based evaluation and foresight exercises. Scientometrics 36: 397-420.
40. Lindsey D (1989) Using citation counts as a measure of quality in science. Measuring what's measurable rather than what's valid. Scientometrics 15: 189-203.
41. Craig ID, Plume AM, McVeigh ME, Pringle J, Amin M (2007) Do open access articles have greater citation impact? A critical review of the literature. Journal of Informetrics 1: 239-248.
42. Cole S, Fiorentine R (1991) Discrimination against women in science: the confusion of outcome with process. In: Zuckerman H, Cole JR, Bruer JT, eds. The outer circle Women in the scientific community. London, UK: W W Norton & Company. pp 205-226.
43. Shadish WR, Cook TD, Campbell DT (2002) Experimental and quasiexperimental designs for generalized causal inference. Boston, MA, USA: Houghton Mifflin Company.
44. Peters DP, Ceci SJ (1982) Peer-review practices of psychological journals - the fate of accepted, published articles, submitted again. Behavioral and Brain Sciences 5: 187-195.
45. Ziman J (1996) Is science losing its objectivity? Nature 382: 751-754.
46. Geisler E (2001) The mires of research evaluation. The Scientist 15: 39.
47. Bornmann L, Mutz R, Daniel H-D (2008) How to detect indications of potential sources of bias in peer review: a generalized latent variable modeling approach exemplified by a gender study. Journal of Informetrics 2: 280-287.
48. Hojat M, Gonnella JS, Caelleigh AS (2003) Impartial judgment by the "gatekeepers" of science: fallibility and accountability in the peer review process. Advances in Health Sciences Education 8: 75-96.
49. Godlee F, Dickersin K (2003) Bias, subjectivity, chance, and conflict of interest. In: Godlee F, Jefferson J, eds. Peer review in health sciences, 2nd ed London, UK: BMJ Publishing Group. pp 91-117.
50. Thorngate W, Dawes RM, Foddy M (2009) Judging merit. New York, NY, USA: Psychology Press.