Feature: Professional issues exploring the peer review process: What is it, does it work, and can it be improved?

Fisheries

Volumn 34, Issue 6, 2009, Pages 270-279

  DeVries, Dennis R ^a   Marschall, Elizabeth A ^c   Stein, Roy A ^b  

^a AUBURN UNIVERSITY   (United States)

^b OHIO STATE UNIVERSITY   (United States)

^c NONE

Author keywords

[No Author keywords available]

Indexed keywords

EID: 67650021984     PISSN: 03632415     EISSN: None     Source Type: Journal    
DOI: 10.1577/1548-8446-34.6.270     Document Type: Article

References (125)

1. Adam, D., and J. Knight. 2002. Publish and be damned. Nature 419:772- 776.
2. Alpert, J. S. 2007. Peer review: the best of the blemished. The American Journal of Medicine 120:287-288.
3. Amos, K. 2008. Transparency and the peer-review process. Fisheries 33(4):197-198.
4. Anonymous. 2007. Mentors of tomorrow. Nature 447:754.
5. Bachand, R. G., and P. P. Sawallis. 2003. Accuracy in the identification of scholarly and peer-reviewed journals and the peer-review process across disciplines. The Serials Librarian 45:39-59.
6. Baxt, W. G., J. F. Waeckerle, J. A. Berlin, and M. L. Callaham. 1998. Who reviews the reviewers? Feasibility of using a fictitious manuscript to evaluate peer reviewer performance. Annals of Emergency Medicine 32:310-317.
7. Bearinger, L. H. 2006. Beyond objective and balanced: writing constructive manuscript reviews. Research in Nursing and Health 29:71-73.
8. Benos, D. J., K. L. Kirk, and J. E. Hall. 2003. How to review a paper. Advances in Physiology Education 27:47-52.
9. Black, N., S. van Rooyen, F. Godlee, R. Smith, and S. Evans. 1998. What makes a good reviewer and a good review for a general medical journal? Journal of the American Medical Association 280:231-233.
10. Bornmann, L., R. Mutz, and H. Daniel. 2007. Gender differences in grant peer review: a meta-analysis. Journal of Informetrics 1:226-238.
11. Bourne, P. E., and A. Korngreen. 2006. Ten simple rules for reviewers. PLOS Computational Biology 2:973-974.
12. Brown, R. W. 1995. Conducting an effective manuscript review. Fisheries 20(7):40-41.
13. Budden, A. E., T. Tregenza, L. W. Aarssen, J. Koricheva, R. Leimu, and C. J. Lortie. 2008. Double-blind review favours increased representation of female authors. Trends in Ecology and Evolution 23:4-6.
14. Caelleigh, A. S., J. A. Shea, and G. Penn. 2001. Selection and qualities of reviewers. Academic Medicine 76:914-916.
15. Callaham, M. 2003. The evaluation and training of peer reviewers. Pages 164-182 in F. Godlee and T. Jefferson, eds. Peer review in health sciences. BMJ Books, London.
16. Callaham, M. L., W. G. Baxt, J. F. Waeckerle, and R. L. Wears. 1998a. Reliability of editors' subjective quality ratings of peer reviews of manuscripts. Journal of the American Medical Association 280:229-231.
17. Callaham, M. L., and D. L. Schriger. 2002. Effect of structured workshop training on subsequent performance of journal peer reviewers. Annals of Emergency Medicine 40:323-328.
18. Callaham, M. L., and J. Tercier. 2007. The relationship of previous training and experience of journal peer reviewers to subsequent review quality. PLOS Medicine 4:32-40.
19. Callaham, M. L., R. L. Wears, and J. F. Waeckerle. 1998b. Effect of attendance at a training session on peer reviewer quality and performance. Annals of Emergency Medicine 32:318-322.
20. Carmi, R., and C. Koch. 2007. Improving peer review with CAR MA. Learned Publishing 20:173-176.
21. Ceci, S. J., and D. Peters. 1984. How blind is blind review? American Psychologist 39:1491-1494.
22. Cho, M. K., A. C. Justice, M. A. Winker, J. A. Berlin, F. Joseph, M. L. Callaham, and R. Drummond. 1998. Masking author identity in peer review: what factors influence masking success? Journal of the American Medical Association 280:243-245.
23. Cicchetti, D. V. 1980. Reliability of reviews for the American Psychologist: a biostatistical assessment of the data. American Psychologist 35:300- 303.
24. Cooke, S. J. 2008. Truth, faith, and transparency in peer review - perspectives on the AFS peer review process. Fisheries 33(5):242-243.
25. Council of Science Editors. 2006. CSE's white paper on promoting integrity in scientific journal publications. CSE Editorial Policy Committee, Reston, Virginia.
26. Croll, P. 2007. Editorial: peer review. British Journal of Educational Studies 55:347-350.
27. Davidoff, F. 1998. Masking, blinding, and peer review: the blind leading the blinded. Annals of Internal Medicine 128:66-68.
28. Donovan, B. 1998. The truth about peer review. Learned Publishing 11:179-184.
29. Drotar, D. 2009. Editorial: How to write effective reviews for the Journal of Pediatric Psychology. Journal of Pediatric Psychology 34:113-117.
30. Earnshaw, J. J., J. R. Farndon, P. J. Guillou, C. D. Johnson, J. A. Murie, and G. D. Murray. 2000. A comparison of reports from referees chosen by authors or journal editors in the peer review process. Annals of the Royal College of Surgeons of England 82:133-135.
31. Epstein, W. M. 1990. Confirmational response bias among social work journals. Science, Technology, and Human Values 15:9-38.
32. Erren, T. C. 2007. Judging words rather than journals or authors: a reminder that peer review continues after publication. QJM An International Journal of Medicine 100:799-801.
33. Evans, A. T., R. A. McNutt, S. W. Fletcher, and R. H. Fletcher. 1993. The characteristics of peer reviewers who produce good-quality reviews. Journal of General Internal Medicine 8:422-428.
34. Feurer, I. D., G. J. Becker, D. Picus, E. Ramirez, M. D. Darcy, and M. E. Hicks. 1994. Evaluating peer reviews: pilot testing of a grading instrument. Journal of the American Medical Association 272:98- 100.
35. Fisher, M., S. B. Friedman, and B. Strauss. 1994. The effects of blinding on acceptance of research papers by peer review. Journal of the American Medical Association 272:143-146.
36. Fisher, R. S., and L. E. Powers. 2004. Peerreviewed publication: a view from the inside. Epilepsia 45:889894
37. Fletcher, R. H., and S. W. Fletcher. 2003. The effectiveness of journal peer review. Pages 62-75 in F. Godlee and T. Jefferson, eds. Peer review in health sciences. BMJ Books, London.
38. Friedman, D. P. 1995. Manuscript peer review at the AJR: facts, figures, and quality assessment. American Journal of Roentgenology 164:1007- 1009.
39. Garfunkel, J. M., M. H. Ulshen, H. J. Hamrick, and E. E. Lawson. 1994. Effect of institutional prestige on reviewer's recommendations and editorial decisions. Journal of the American Medical Association 272:137-138.
40. Garrow, J., M. Butterfield, J. Marshall, and A. Williamson. 1998. The reported training and experience of editors in chief of specialist clinical medical journals. Journal of the American Medical Association 280:286-287.
41. Gilbert, J. R., E.S. Williams, and G. D. Lundberg. 1994. Is there a gender bias in JAMA's peer review process. Journal of the American Medical Association 272:139-142.
42. Glick, M. 2007. Peer review: an inexact but essential part of scientific publishing. Journal of the American Dental Association 138:568-571.
43. Godlee, F. 2002. Making reviewers visible: openness, accountability, and credit. Journal of the American Medical Association 287:2762-2765.
44. Godlee, F., and K. Dickersin. 2003. Bias, subjectivity, chance, and conflict of interest in editorial decisions. Pages 91-117 in F. Godlee and T. Jefferson, eds. Peer review in health sciences, 2nd edition. BMJ Publishing Group, London.
45. Godlee, F., C. R. Gale, and C. N. Martyn. 1998. Effect on the quality of peer review of blinding reviewers and asking them to sign their reports. Journal of the American Medical Association 280:237-240.
46. Goldbeck-Wood, S. 1998. What makes a good reviewer of manuscripts. British Medical Journal 316:86.
47. Goldsmith, L. A., E. N. Blalock, H. Bobkova, and R. P. Hall II. 2006. Picking your peers. Journal of Investigative Dermatology 126:1429- 1430.
48. Goodman, S. N., J. Berlin, S. W. Fletcher, and R. H. Fletcher. 1994. Manuscript quality before and after peer review and editing at Annals of Internal Medicine. Annals of Internal Medicine 121:11-21.
49. Grant, J., S. Burden, and G. Breen. 1997. No evidence of sexism in peer review. Nature 390:438.
50. Greaves, J. S., J. Scott, M. Clarke, L. Miller, T. Hannay, A. Thomas, and P. Campbell. 2006. Overview: Nature's peer review trial. Nature doi:10.1038/nature05535. Available at: www.nature.com/nature/peerreview/ debate/nature05535.html.
51. Green, S. M., and M. L. Callaham. 2006. Current status of peer review at Annals of Emergency Medicine. Annals of Emergency Medicine 48:304-308.
52. Groves, T., and K. Abbasi. 2004. Screening research papers by reading abstracts. British Medical Journal 329:470-471.
53. Gura, T. 2002. Peer review, unmasked. Nature 416:258-260.
54. Hammerschmidt, K., K. Reinhardt, and J. Rolff. 2008. Does doubleblind review favor female authors? Frontiers in Ecology and the Environment 6:354.
55. Hansen, M. J. 2002. Reviewer rights and responsibilities. Fisheries 27(10):32-33.
56. Horrobin, D. F. 1990. The philosophical basis of peer review and the suppression of innovation. Journal of the American Medical Association 263:1438.
57. -. 2001. Something rotten at the core of science? Trends in Pharmacological Sciences 22:51-52.
58. Howard, L., and G. Wilkinson. 1998. Peer review and editorial decisionmaking. British Journal of Psychiatry 173:110-113.
59. Jawaid, S. A., M. Jawaid, and M. H. Jafary 2006. Characteristics of reviewers and quality of reviews: a retrospective study of reviewers at Pakistan Journal of Medical Sciences. Pakistan Journal of Medical Sciences 22:101-06.
60. Jefferson, T., E. Wager, and F. Davidoff. 2002. Measuring the quality of editorial peer review. Journal of the American Medical Association 287:2786-2790.
61. Johnston, S. C., D. H. Lowenstein, D. M Ferriero, R. O. Messing, J. R. Oksenberg, and S. L. Hauser. 2007. Early editorial manuscript screening versus obligate peer review: a randomized trial. Annals of Neurology 61:A10-A12.
62. Jolley, J. C., and B. D. S. Graeb. 2007. Writing, publishing, and reviewing: the students' perspective. Fisheries 32(1):40-43.
63. Justice, A. C., M. K. Cho, M. A. Winker, J. A. Berlin, and R. Drummond. 1998. Does masking author identity improve peer review quality? A randomized controlled trial. Journal of the American Medical Association 280:240-242.
64. Katz, D. S., A. V. Proto, and W. W. Olmstead. 2002. Incidence and nature of unblinding by authors: our experience at two radiology journals with double-blinded peer review policies. American Journal of Radiology 179:1415-1417.
65. Kearney, M. H., and M. C. Freda. 2005. Nurse editors' views on the peer review process. Research in Nursing and Health 28:444-452.
66. Kemp, S. 2005. Editorial comment: agreement between reviewers of Journal of Economic Psychology submissions. Journal of Economic Psychology 26:779-784.
67. Kliewer, M. A., K. S. Freed, D. M. DeLong, P. J. Pickhardt, and J. M. Provenzale. 2005. Reviewing the reviewers: comparison of review quality and reviewer characteristics at the American Journal of Roentgenology 184:1731-1735.
68. Koch, D. D. 2001. Peer review: a group activity. Journal of Cataract and Refractive Surgery 27:1707.
69. Kronick, D. A. 1990. Peer review in 18th-century scientific journalism. Journal of the American Medical Association 263:1321-1322.
70. Kuyper, B. J. 1991. Bringing up scientists in the art of critiquing research. BioScience 41:248-250.
71. Landkroon, A. P., A. M. Euser, H. Veeken, W. Hart, and A. J. P. M. Overbeke. 2006. Quality assessment of reviewers' reports using a simple instrument. Obstetrics and Gynecology 108:979-985.
72. Lee, K. P., E. A. Boyd, J. M. Holroyd-Leduc, P. Bacchetti, and L. A. Bero. 2006. Predictors of publication: characteristics of submitted manuscripts associated with acceptance at major biomedical journals. Medical Journal of Australia 184:621-626.
73. Loonen, M. P. J., J. J.Hage, and M. Kon. 2005. Who benefits from peer review? An analysis of the outcome of 100 requests for review by Plastic and Reconstructive Surgery. Plastic and Reconstructive Surgery 116:1461-1472.
74. Lortie, C. J., L. W. Aarssen, A. E Budden, J. K. Koricheva, R. Leimu, and T. Tregenza. 2007. Publication bias and merit in ecology. Oikos 116:1247-1253.
75. Mahoney, M. J. 1977. Publication prejudices: an experimental study of confirmatory bias in the peer review system. Cognitive Therapy and Research 1:161-175.
76. Marsh, H. W., and S. Ball. 1981. Interjudgemental reliability of reviews for the Journal of Educational Psychology. Journal of Educational Psychology 73:872-880.
77. McCoy, E. D. 1993. Review peering: a look at the ESA's peer review process. Bulletin of Ecological Society of America 74:232-236.
78. McCutchen, C. W. 1991. Peer review: treacherous servant, disastrous master. Technology Review 94:29-40.
79. McNutt, R. A., A. T. Evans, R. H. Fletcher, and S. W. Fletcher. 1990. The effects of blinding on the quality of peer-review- a randomized trial. Journal of the American Medical Association 263:1371-1376.
80. Michels, R. 1995. Peer review. International Journal of Psycho-Analysis 76:217-221.
81. Moher, D., and A. R. Jadad. 2003. How to peer review a manuscript. Pages 183-190 in F. Godlee and T. Jefferson, eds. Peer review in health sciences. BMJ Books, London.
82. Mulligan, A. 2005. Is peer review in crisis? Oral Oncology 41:135-141.
83. Northcraft, G. 2001. From the editors. Academy of Management Journal 44:1079-1080.
84. Peters, D. P., and S. J. Ceci. 1982. Peer-review practices of psychological journals: the fate of published articles, submitted again. The Behavioral and Brain Sciences 5:187-255.
85. Plug, C. 1993. The reliability of manuscript evaluation for the South African Journal of Psychology. South African Journal of Psychology 23:43-48.
86. Poschl, U., K. S. Carslaw, T. Koop, R. Sander, W. T. Sturges, J. P. D. Abbatt, J. T. Jayne, and D. R. Worsnop. 2004. Scientific quality assurance by interactive peer review and public discussion. Abstracts of Papers of the American Chemical Society 228: U357-U357 017- CINF Part 1.
87. Provenzale, J. M., and R. J. Stanley. 2005. A systematic guide to reviewing a manuscript. American Journal of Roentgenology 185:848-854.
88. Publishing Research Consortium. 2008. Peer review in scholarly journals. Mark Ware Consulting Ltd., Bristol, UK. Available at: www. publishingresearch.net/PeerReview.htm.
89. Raelin, J. A. 2008. Refereeing the game of peer review. Academy of Management Learning and Education 7:124-129.
90. Raveendran, R. 2006. Rejecting manuscripts without external review. Indian Journal of Pharmacology 38.5 (September-October 2006).
91. Regehr, G., and G. Bordage. 2006. To blind or not to blind? What authors and reviewers prefer. Medical Education 40:832-839.
92. Relman, A. S. 1990. Peer review in scientific journals-what good is it? Western Journal of Medicine 153:520-522.
93. Rennie, D. 1998. Freedom and responsibility in medical publication. Journal of the American Medical Association 280:300-302.
94. -. 2003a. Editorial peer review: its development and rationale. Pages 1-13 in F. Godlee and T. Jefferson, eds. Peer review in health sciences. BMJ Books, London.
95. -. 2003b. Innovation and peer review. Pages 76-90 in F. Godlee and T. Jefferson, eds. Peer review in health sciences. BMJ Books, London.
96. Rivara, F. P., P. Cummings, S. Ringold, A. B. Bergman, A. Joffe, and D. A. Christakis. 2007. A comparison of reviewers selected by editors and reviewers suggested by authors. Journal of Pediatrics 151:202-205.
97. Roberts, J. C., R. H. Fletcher, and S. W. Fletcher. 1994. Effects of peer review and editing on the readability of articles published in Annals of Internal Medicine. Journal of the American Medical Association 272:119-121.
98. Robin, E. D., and C. M. Burke. 1987. Peer review in medical journals. Chest 91:252-255.
99. Rosenzweig, M. L., J. I. Davis, and J. H. Brown. 1988. How to write an influential review. Bulletin of the Ecological Society of America 69:152-155.
100. Rothwell, P. M., and C. N. Martyn. 2000. Reproducibility of peer review in clinical neuroscience: is agreement between reviewers any greater than would be expected by chance alone? Brain 123:1964- 1969.
101. Schroter, S., N. Black, S. Evans, J. Carpenter, F. Godlee, and R. Smith. 2004. Effects of training on quality of peer review: randomised controlled trial. British Medical Journal 328:673-677.
102. Schroter, S., L. Tite, A. Hutchings, and N. Black. 2006. Differences in review quality and recommendations for publication between peer reviewers suggested by authors or by editors. Journal of the American Medical Association 295:314-317.
103. Scott, W. A. 1974. Interreferee agreement on some characteristics of manuscripts submitted to the Journal of Personality and Social Psychology. American Psychologist 29:698-702.
104. Siegelman, S. S. 1991. Assassins and zealots: variations in peer review. Radiology 178:637-642.
105. Smit, C. 2006. Peer review: time for a change? BioScience 56:712-713.
106. Smith, R. 2006. Peer review: a flawed process at the heart of science and journals. Journal of the Royal Society of Medicine 99:178-182.
107. Stensrud, D. J., and H. E. Brooks. 2005. The future of peer review? Weather and Forecasting 20:825-826
108. Stossel, T. P. 1985. Reviewers status and review quality: experience of the journal of clinical investigation. The New England Journal of Medicine 312:658-659.
109. Tite, L., and S. Schroter. 2007. Why do peer reviewers decline to review? A survey. Journal of Epidemiology and Community Health 61:9-12.
110. Tregenza, T. 2002. Gender bias in the refereeing process? Trends in Ecology and Evolution 17:349-350.
111. Tsang, E. W. K., and B. S. Frey. 2007. The as-is journal review process: let authors own their ideas. Academy of Management Learning and Education 6:128-136.
112. van Rooyen, S. 2001. The evaluation of peer-review quality. Learned Publishing 14:85-91.
113. van Rooyen, S., N. Black, and F. Godlee. 1999a. Development of the review quality instrument (RQI) for assessing peer reviews of manuscripts. Journal of Clinical Epidemiology 7:625-629.
114. van Rooyen, S., F. Godlee, S. Evans, N. Black, and R. Smith. 1999b. Effect of open peer review on quality of reviews and on reviewers' recommendations: a randomised trial. British Medical Journal 318:23- 27.
115. -. 1998. Effect of blinding and unmasking on the quality of peer review: a randomized trial. Journal of the American Medical Association 280:234-237.
116. Wager, E., E. C. Parkin, and P. S. Tamber. 2006. Are reviewers suggested by authors as good as those chosen by editors? Results of a rater-blinded retrospective study. BioMed Central Medicine 4:13 doi10.1186/1741-7015-4-13.
117. Waldrop, M. W. 2008. What is Edit This? Scientific American website at www.sciam.com/article.cfm?id=science-2-point-0-great-new-toolor- great-risk.
118. Walsh, E., M. Rooney, L. Appleby, and G. Wilkinson. 2000. Open peer review: a randomised controlled trial. British Journal of Psychiatry 176:47-51.
119. Waser, N. M., M. V., Price, and R. K. Grosberg. 1992. Writing an effective manuscript review. BioScience 42:621-623.
120. Webb, T. J., B. O'Hara, and R. P. Freckleton. 2008. Does double-blind review benefit female authors? Trends in Ecology and Evolution 23:351-353.
121. Weller, A. C. 1991. Potential bias in editorial peer review: a study of U.S. medical journals. The Serials Librarian 19:95-103.
122. -. 1996. Editorial peer review: a comparison of authors publishing in two groups of U.S. medical journals. Bulletin of the Medical Library Association 84:359-366.
123. Wenneras, C., and A. Wold. 1997. Nepotism and sexism in peer-review. Nature 387:341-343.
124. Wets, K., D. Weedon, and J. Velterop. 2003. Post-publication filtering and evaluation: faculty of 1000. Learned Publishing 16:249-258.
125. Whittaker, R. J. 2008. Journal review and gender equality: a critical comment on Budden et al. Trends in Ecology and Evolution 23:478-