Peer review and legal publishing: What law librarians need to know about open, single-blind, and double-blind reviewing

Law Library Journal

Volumn 101, Issue 1, 2009, Pages 59-70

  McCormack, Nancy ^a,b  

^a QUEEN S UNIVERSITY   (Canada)

^b NONE

Author keywords

[No Author keywords available]

Indexed keywords

EID: 62149108545     PISSN: 00239283     EISSN: None     Source Type: Journal    
DOI: None     Document Type: Review

References (69)

1. Peer review is sometimes called "refereeing."
2. See SALLY BROWN, PHIL RACE & JOANNA BULL, COMPUTER-ASSISTED ASSESSMENT IN HIGHER EDUCATION 97 (1999).
3. In 2008, for example, the South Carolina Law Review began to make use of the peer review process. See South Carolina Law Review, Peer Review Pilot Project, http://www.sclawreview.org/ peerreview/index.php (last visited Sept. 17, 2008). Others are taking smaller steps toward the inevitable-The Yale Law Journal Pocket Part, Harvard Law Review, and Stanford Law Review now use a blind submission process. The Yale Law Journal, Submissions, http://yalelawjournal.org/submissions. html (last visited Sept. 17, 2008);
4. Harvard Law Review, Guidelines for Submitting Manuscripts, http://www.harvardlawreview.org/manuscript.shtml (last visited Sept. 17, 2008); The Stanford Law Review, Articles, http://lawreview.stanford.edu/submissions/ articles/index.htm (last visited Sept. 17, 2008). PrawfsBlawg (a blog featuring discussions by law professors from an assortment of American law schools) has noted the growing list of peer-reviewed law journals. See Posting of Matt Bodie to PrawfsBlawg (Project on Peer-Review Law Journals), http://prawfsblawg.blogs. com/prawfs-blawg/2006/01/project-on-peer.html (Jan. 2, 2006, 7:36 PM).
5. See Drummond Rennie, Editorial Peer Review: Its Development and Rational, in PEER REVIEW IN HEALTH SCIENCES 7 (Fiona Godlee & Tom Jefferson eds., 2d ed. 2003).
6. See Rethinking Peer Review: How the Internet is Changing Science Journals, NEW ATLANTIS, Summer 2006, at 106,107 [hereinafter Rethinking Peer Review].
7. See Richard A. Posner, Law Reviews, 46 WASHBURN L.J. 155,155 (2006).
8. David A. Kronick, Peer Review in 18th-Century Scientific Journalism, 263 JAMA 1321, 1321 (1990) (quoting author's translation of Preface to 1 Essais et observations de medicine de la Societe d'Edinbourg (1740)).
9. Id.
10. See Rethinking Peer Review, supra note 5,106-07 (citing comments of Ohio State University science historian John C. Burnham).
11. See id. at 106.
12. Id. at 107.
13. Id.
14. PENNumbra, About Penn Law Review, http://www.pennumbra.com/about (last visited Sept. 17, 2008) (beginning as the American Law Register). In addition, the Albany Law School Journal first appeared on the scene in 1875 as the "nation's first student-edited law review ⋯" Albany Law School, The History of Albany Law School, http://www.albanylaw.edu/sub.php? navigation-id=292 (last visited Sept. 17,2008). However, the journal only lasted a year. See Michael I. Swygert & Jon W. Bruce, The Historical Origins, Founding, and Early Development of Student-Edited Law Reviews, 36 HASTINGS L.J. 739, 764 (1985) (citation omitted). Amongst other vintage law reviews are the Harvard Law Review (1887), the Yale Law Journal (1891), the Columbia Law Review (which followed the Columbia Jurist first published in 1885), and the West Virginia Law Review (1894). See id. at 763-83. Swygert and Bruce do not quite regard the West Virginia Law Review as an early law school periodical because it was "the successor to a monthly publication of the West Virginia Bar called The Bar that originated in 1894." Id. at 780 n.343 (citations omitted). But cf. West Virginia University College of Law, Welcome to the West Virginia Law Review, http://law.wvu.edu/student-life/ west-virginia-law-review/west-virginia-law-review-about-us (last visited Sept. 17,2008) (stating it is "the fourth oldest law review in the country").
15. C.J.J.M. Stolker, Legal Journals: In Pursuit of a More Scientific Approach, 2 EUR. J. LEGAL EDUC. 77, 82 (2005).
16. Lawrence M. Friedman, Law Reviews and Legal Scholarship: Some Comments, 75 DENV. U. L. REV. 661, 661 (1998).
17. Richard A. Posner, The Future of the Student-Edited Law Review, 47 STAN. L. REV. 1131, 1133-34 (1995). For a searing critique of student-edited law reviews, see James Lindgren, Exchange, An Author's Manifesto, 61 U. CHI. L. REV. 527 (1994) ("Our scholarly journals are in the hands of incompetents.").
18. Richard A. Posner, THE PROBLEMATICS OF MORAL AND LEGAL THEORY 297-98 (1999) (citations omitted).
19. See Posner, supra note 6, at 157-58.
20. Posner, supra note 16, at 1136.
21. Id. at 1137, 1138.
22. Michael Kirby, Welcome to Law Reviews, 26 MELB. U. L. REV. 1, 4 (2002).
23. See also Arthur D. Austin, The "Custom of Vetting" as a Substitute for Peer Review, 32 ARIZ. L. REV. 1 (1990) (discrediting the trend of "vetting"-informal pre-publication review by colleagues-in place of conventional peer review in order to garner credibility for legal scholarship).
24. See Kirby, supra note 21, at 4. There are numerous peer-reviewed journals and law reviews in the United States, the U.K., Canada, Australia, and New Zealand. A complete list is beyond the scope of this paper, but many can be found using Washington & Lee Law School's database, and searching for "refereed" law journals. Washington & Lee Law School, Law Journals: Submissions and Ranking, http://lawlib.wlu.edu/LJ/index. aspx (last visited Sept. 26,2008). The following is offered as a sample of titles from countries outside of the United States: U.K.: Oxford University Commonwealth Law Journal and Liverpool Law Review, Canada: Alberta Law Review, Indigenous Law Journal, and Queen's Law Journal; Australia and New Zealand: Australian and New Zealand Maritime Law Journal, University of Queensland Law Journal, Melbourne University Law Review, and Sydney Law Review. Michael Kirby notes that most Australian law reviews are now peer-reviewed. See Kirby, supra note 21, at 4. The same is true in Canada.
25. See Gaell Mainguy, Mohammad R. Motamedi & Daniel Mietchen, Peer Review-The Newcomers' Perspective, 3 PLoS BIOLOGY 1534, 1534 (2005), http://biology.plosjournals.org/ perlserv/?request=get- document&doi=10.1371%2Fjournal.pbio.0030326&ct=l.
26. Editorial: Pros and Cons of Open Peer Review, 2 NATURE NEUROSCIENCE 197,198 (1999).
27. SENSE ABOUT SCI., PEER REVIEW AND THE ACCEPTANCE OF NEW SCIENTIFIC IDEAS 2 (2004), available at http://www.senseaboutscience.org.uk/pdf/PeerReview.pdf.
28. See id. at 8.
29. Id. at 5.
30. Id. at 2.
31. See, e.g., Brian Martin, Peer Review and the Origin of AIDS-A Case Study in Rejected Ideas, 43 BIOSCIENCE 624 (1993) (detailing the rejection of a nonstandard theory on the origin of AIDS to illustrate "some of the problems of peer review in dealing with unorthodox ideas").
32. See id. at 625.
33. Thomas Gold, New Ideas in Science, 3 J. SCI. EXPLORATION 103 (1989) (arguing that "on the whole the 'herd instinct' has been a disaster in science").
34. See, e.g., Kevin M. Yamamoto, What's in a Name? The Letterhead Impact Project, 22 J. LEGAL STUD. EDUC. 65 (2004).
35. Id. In Yamamoto's study, letterhead from one school versus another made no difference in terms of the time it took for the various law review personnel to respond to the author or to accept or reject the paper, but the only offer of publication came for the article that was submitted with the letterhead from a top tier law school. Id. at 67.
36. See also Lindgren, supra note 16, at 530, in which is cited the example of a "very much liked" article that was rejected because the journal editors decided not to "take a chance" with an author from a non-elite law school.
37. Amber E. Budden et al., Double-Blind Review Favours Increased Representation of Female Authors, 23 TRENDS ECOLOGY & EVOLUTION 4, 5 (2008). The journal Behavioral Ecology introduced double-blind peer review in 2001, which provided the authors an "opportunity to examine variation in author demographics associated with the review method." Id. at 4. The authors reported an increase in "female first-authored papers" compared with a similar journal, Behavioral Ecology and Sociobiology. Id. at 6. The authors concluded:
38. A difference of 7.9% in the proportion of female first-authored papers following the implementation of double-blind review in BE is three times greater than the recorded increase in female ecology graduates in the USA across the same time period and represents a 33% increase in the rep-resentation of female authors. Furthermore, this increased representation of female authors more accurately reflects the (US) life sciences academic workforce composition, which is 37% female. Id. (citations omitted).
39. See also Julie R. Gilbert et al, Is There Gender Bias in JAMA's Peer Review Process?, 272 JAMA 139 (1994) (finding gender differences in "editor and reviewer characteristics" but no gender differences in publication acceptance); Margaret E. Lloyd, Gender Factors in Reviewer Recommendations for Manuscript Publication, 23 J. APPLIED BEHAV. ANALYSIS 539 (1990) (finding increased acceptance of female-authored manuscripts by female reviewers); Robert E. Rains, Andrea's Adventures in Law Review Land, 50 J. LEGAL EDUC. 306 (2000) (examining gender bias in manuscript selection with inconclusive results).
40. Who Is Accountable?, 450 NATURE 1, 1 (2007), available at http://www.nature.com/nature/ journal/v450/n7166/pdf/450001a.pdf.
41. But see Susan Finch, Law Dean Writes of Regret Over Errors; Author Stands By Article's Conclusions, TIMES-PICAYUNE (New Orleans), Sept., 17, 2008, at 1 (noting "numerous errors" in data underlying empirical study of the Louisiana Supreme Court published in the Tulane Law Review).
42. Woo Suk Hwang et al., Evidence of a Pluripotent Human Embryonic Stem Cell Line Derived from a Cloned Blastocyst, 303 SCIENCE 1669 (2004);
43. Woo Suk Hwang et al., Patient-Specific Embryonic Stem Cells Derived from Human SCNT Blastocysts, 308 SCIENCE 1777 (2005). The two papers involved concerned the cloning of human beings and subsequent stem-cell lines. The data in both papers has been shown to be completely false. Seoul National University, the research institution at which the data was faked, delivered a report in January 2006 condemning the fraud. See Seoul National University Investigation Committee, Summary of the Final Report on Professor Woo Suk Hwang's Research Allegations (Jan. 9, 2006), available at http://www.nytimes.com/2006/01/09/science/ text-clonereport.html?pagewanted= 1 &-r= 1.
44. See supra ¶ 1.
45. Committee Report, Hwang et al., Science, 308,1777-1783 (2005) (Sept. 15,2006), available at http://www.sciencemag.0rg/cgi/data/314/ 5804/1353/DC1/1.
46. Posner, supra note 16, at 1137.
47. See Elizabeth Walsh et al., Open Peer Review: A Randomised Controlled Trial, 176 Brit. J. Psychiatry 47,50 (2000).
48. Overview: Nature's Peer Review Trial, NATURE, Dec. 2006, http://www.nature.com/nature/ peerreview/debate/nature05535.html.
49. Id. They were also sent to subject reviewers along with the papers from all the authors who declined to participate.
50. Id.
51. Id.
52. Walsh, supra note 41, at 50.
53. See id. at 49.
54. See Christine Wenneras & Agnes Wold, Commentary, Nepotism and Sexism in Peer-Review, 387 NATURE 341, 341 (1997)
55. See Tom Abate, What's the Verdict on Peer Review?, 21stC, Spring 1995, http://www.columbia .edu/cu/21stC/issue-l.l/peer.htm.
56. See also Wenneras & Wold, supra note 48, at 343.
57. See David N. Laband & Michael J. Piette, Does the "Blindness" of Peer Review Influence Manuscript Selection Efficiency?, 60 S. ECON. J. 896, 897, 905-06 (1994).
58. Id. at 906.
59. PEER REVIEW IN SCHOLARLY JOURNALS: PERSPECTIVE OF THE SCHOLARLY COMMUNITY-AN INTERNATIONAL STUDY 1 (2008) (study commissioned and funded by the Publishing Research Consortium), available at http://www.publishingresearch.net/documents/ PeerReviewFullPRCReport -final.pdf; see also Richard Smith, Editorial, Peer Review: Reform or Revolution? Time to Open up the Black Box of Peer Review, 315 BMJ: Brit. Med. J. 759, 759 (1997).
60. Frank Davidoff, Editorial, Masking, Blinding, and Peer Review: The Blind Leading the Blinded, 128 ANNALS INTERNAL MED. 66, 66 (1998); David S. Liebeskind, Letter to the Editor, The Fallacy of Double-Blinded Peer Review, 181 Am. J. Roentgenology 1422,1422 (2003) ("until... guidelines can be established and implemented, the double-blinded review process remains a myth").
61. See Posting of Bill Hooker to Action Potential (Double-blind peer review?), http://blogs .nature.com/nn/actionpotential/2005/12/doublebUnd-peer- review.html (Dec. 28, 2005,1:08 AM).
62. Douglas S. Katz et al., Incidence and Nature of Unblinding by Authors: Our Experience at Two Radiology Journals with Double-Blinded Peer Review Policies, 179 Am. J. ROENTGENOLOGY 1415, 1416 (2002).
63. See Robert A. McNutt et al., The Effects of Blinding on the Quality of Peer Review, 263 JAMA 1371,1375 (1990).
64. David N. Laband & Michael J. Piette, A Citation Analysis of the Impact of Blinded Peer Review, 272 JAMA 147,148 (1994).
65. See Nathan H. Saunders, Student-Edited Law Reviews: Reflections and Responses of an Inmate, 49 DUKE L.J. 1663, 1681 (2000).
66. Id. at 1667-68. See also Posner, supra note 16, at 1136-37.
67. See Yamamoto, supra note 32, at 84.
68. See, e.g., Bernard J. Hibbitts, Last Writes? Reassessing the Law Review in the Age of Cyberspace, 71 N.Y.U. L. REV. 615,651-52 (1996).
69. E.g., The Executive Board of the Chicago-Kent Law Review, Symposium Format as a Solution to Problems Inherent in Student-Edited Law Journals: A View from the Inside, 70 Chi.-Kent L. Rev. 141, 142 (1994-95).