The use of legal scholarship by the federal courts of appeals: An empirical study

Cornell Law Review

Volumn 96, Issue 6, 2011, Pages 1345-1374

  Schwartz, David L ^a   Petherbridge, Lee ^b  

^a CHICAGO KENT COLLEGE OF LAW   (United States)

^b LOYOLA LAW SCHOOL   (United States)

Author keywords

[No Author keywords available]

Indexed keywords

EID: 80054084637     PISSN: 00108847     EISSN: None     Source Type: Journal    
DOI: None     Document Type: Article

References (94)

1. Alex Kozinski, Who Gives a Hoot About Legal Scholarship?, 37 HOUS. L. REV. 295, 295 (2000).
2. It has garnered criticism for other reasons as well. See, e.g., Harry T. Edwards, The Growing Disjunction Between Legal Education and the Legal Profession, 91 MICH. L. REV. 34, 35 (1992) ("I see no reason why law professors should write mediocre economics, or philosophy, or literary criticism, when arts and sciences professors could be doing a better au....");
3. Richard A. Posner, Legal Scholarship Today, 115 HARV. L. REV. 1314, 1320-21 (2002) (expressing concern over how law review articles differ from the scholarly norm: "[t]hink only of the length of law review articles, how unembarrassed the legal writer is to repeat what is well known, how seldom one finds an article that begins with a clear statement of what the author thinks the article adds to the existing literature").
4. Jess Bravin, Chief Justice Roberts on Obama, Justice Stevens, Law Reviews, More, WALL ST. J. L. BLOG (Apr. 7, 2010, 7:20 PM), http://blogs.wsj. com/law/2010/04/07/chief-justiceroberts-on-obama-justice-stevens-law-reviews- more/;
5. cf. Daniel Solove, Chief Justice Roberts and Legal Scholarship, CONCURRING OPINIONS, (Apr. 8, 2010, 12:26 PM) http://www.concurringopinions.com/ archives/2010/04/chief-justice-roberts-and-legal-scholarship. html (finding Chief Justice Roberts's view "kind of glib and dismissive").
6. Adam Liptak, When Rendering Decisions, Judges Are Finding Law Reviews Irrelevant, N. Y. TIMES, Mar. 19, 2007, at A8 (quoting Chief Judge Dennis G. Jacobs). Other judges have echoed similar concerns.
7. See, e.g., Thomas L. Ambro, Citing Legal Articles in Judicial Opinions: A Sympathetic Antipathy, 80 AM. BANKR. L. J. 547, 549 (2006) ("When we [judges] do read the occasional article, we find it often not only unpersuasive, but even at times at odds with accepted means of analysis.");
8. Judith S. Kaye, One Judge's View of Academic Law Review Writing, 39 J. LEGAL EDUC. 313, 319-20 (1989) ("Prominent law reviews are increasingly dedicated to abstract, theoretical subjects... and less and less to practice and professional issues.... I am disappointed not to find more in the law reviews that is of value and pertinence to our cases.");
9. see also United States v. Six Hundred Thirty-Nine Thousand Five Hundred & Fifty-Eight Dollars ($639, 558) in U. S. Currency, 955 F.2d 712, 722 (D. C. Cir. 1992) (Silberman, J., concurring) ("I suppose, now that many of our law reviews are dominated by rather exotic offerings of increasingly out-of-touch faculty members, the temptation for judges to write about issues that interest them-whether or not raised by the parties or constituting part of the logic of the decision-is even greater.").
10. See, e.g., David Hricik & Victoria S. Salzmann, Why There Should Be Fewer Articles Like This One: Law Professors Should Write More for Legal Decision-Makers and Less for Themselves, 38 SUFFOLK U. L. REV. 761, 778 (2005) (claiming that the "trend" toward "'not merely unhelpful', but 'useless'" legal scholarship is "already apparent" (citation omitted)).
11. SECTION OF LEGAL EDUC. & ADMISSIONS TO THE BAR, AM. BAR ASS'N, LEGAL EDUCATION AND PROFESSIONAL DEVELOPMENT-AN EDUCATIONAL CONTINUUM: REPORT OF THE TASK FORCE ON LAW SCHOOLS AND THE PROFESSION: NARROWING THE GAP (1992) [hereinafter MACCRATE REPORT].
12. Remarks of Professor Aaron D. Twerski, L'65, MARQ. LAW. (Marquette Univ. Law Sch., Milwaukee, Wis.), Spring 2009, at 55, 56 (noting that prestigious law reviews appear less interested in publishing traditional doctrinal scholarship and that "young scholars [engaged in such scholarship] are justifiably afraid that when tenure time comes around their articles will be viewed as pedestrian"). Other law professors repeat these concerns. See, e.g., Hricik & Salzmann, supra note 6 (positing that law professors write more for themselves and less for decision makers and arguing that law professors should write more for decision makers and less for themselves).
13. Erik M. Jensen, The Shortest Article in Law Review History, 50 J. LEGAL EDUC. 156, 156 & n. 1 (2000) (having as its entire text: "This is it").
14. See, e.g., Richard A. Posner, The Present Situation in Legal Scholarship, 90 YALE L. J. 1113 (1981);
15. Edward L. Rubin, The Practice and Discourse of Legal Scholarship, 86 MICH. L. REV. 1835 (1988);
16. Mark Tushnet, Legal Scholarship: Its Causes and Cure, 90 YALE L. J. 1205 (1981).
17. John Minor Wisdom, Wisdom's Idiosyncrasies, 109 YALE L. J. 1273, 1278 (2000).
18. See, e.g., Robert J. Hume, Strategic-Instrument Theory and the Use of Non-Authoritative Sources by Federal Judges: Explaining References to Law Review Articles, 31 JUST. SYS. J. 291, 299 (2010) (reporting that law review citations "have become increasingly less common over time");
19. Michael D. McClintock, The Declining Use of Legal Scholarship by Courts: An Empirical Study, 51 OKLA. L. REV. 659, 660 (1998) ("This survey reveals a 47.35% decline in the use of legal scholarship by courts over the past two decades, the most notable decline occurring in the past ten years.");
20. Louis J. Sirico, Jr., The Citing of Law Reviews by the Supreme Court: 1971-1999, 75 IND. L. J. 1009, 1010 (2000) ("We find a continuing decline in [the] number of times the Court cited legal periodicals....");
21. Louis J. Sirico, Jr. & Jeffrey B. Margulies, The Citing of Law Reviews by the Supreme Court: An Empirical Study, 34 UCLA L. REV. 131, 134 (1986) (examining two three-year periods and finding a "substantial" decline in citations to legal periodicals by the Supreme Court);
22. Carissa Alden et al., Trends in Federal Judicial Citations and Law Review Articles 2 (Mar. 8, 2007) (unpublished manuscript), http://graphics8.nytimes. com/packages/pdf/national/20070319-federal-citations.pdf ("Courts cited law reviews more frequently in the 1970s and 1980s than they do today."). Hricik and Salzmann tested the "already apparent" "trend" that law review articles are deemed "'not merely unhelpful', but 'useless' to the bench and bar" in the following way: [We] surveyed every opinion written by the United States Supreme Court during the 2003-2004 term. In those opinions, 3, 998 sources were cited. Only 744 of those citations were to secondary sources, and only 108 of those citations were to law review articles. In other words, less than three percent of the sources the Supreme Court cited were the law review articles that are supposed to analyze the cutting edge of legal issues, doctrine, and theory. These numbers indicate that judges are relying on black letter law and their own legal reasoning, rather than the theories and discussions proffered by the individuals who ostensibly should be at the forefront of legal knowledge, to effectuate their opinions.
23. see also Blake Rohrbacher, Decline: Twenty-Five Years of Student Scholarship in Judicial Opinions, 80 AM. BANKR. L. J. 553, 553 (2006) (examining judicial citation to student scholarship and finding that "judicial citation of student notes has plunged since 1980").
24. But cf. Lawrence M. Friedman et al., State Supreme Courts: A Century of Style and Citation, 33 STAN. L. REV. 773, 812 (1981) (reporting an 8% increase in citation to legal scholarship by state supreme courts when comparing 1945-55 with 1960-70). Some articles do not claim to identify a trend but still describe the frequency of judicial citation to law review articles.
25. See, e.g., Louis J. Sirico, Jr. & Beth A. Drew, The Citing of Law Reviews by the United States Courts of Appeals: An Empirical Analysis, 45 U. MIAMI L. REV. 1051, 1052 (1991) (examining 1200 opinions issued in 1989 and finding that "the federal circuit courts cite law reviews infrequently");
26. see also Gregory Scott Crespi, The Influence of a Decade of Statutory Interpretation Scholarship on Judicial Rulings: An Empirical Analysis, 53 SMU L. REV. 9, 11 (2000) [hereinafter Crespi, Influence of a Decade] (finding that "almost half of the statutory interpretation articles published between 1988 and 1995 have been cited in at least one judicial opinion" but that this statistic "is a relatively high figure" when compared with citation rates in general);
27. Gregory Scott Crespi, The Influence of Two Decades of Contract Law Scholarship on Judicial Rulings: An Empirical Analysis, 57 SMU L. REV. 105, 111 (2004) (examining a subset of contract law articles and finding that 36.4% had been judicially cited). Finally, some articles address other aspects of judicial citation to law review articles.
28. See, e.g., Gregory Scott Crespi, Judicial and Law Review Citation Frequencies for Articles Published in Different "Tiers" of Law Journals: An Empirical Analysis, 44 SANTA CLARA L. REV. 897, 897 (2004) (finding that courts and scholars cite articles published in "the three most prestigious law journals at much higher rates" than articles appearing in lower-tier journals);
29. Robert M. Lawless & Ira David, The General Role Played by Specialty Law Journals: Empirical Evidence from Bankruptcy Scholarship, 80 AM. BANKR. L. J. 523, 524 (2006) (comparing citation to bankruptcy articles published in general law reviews with those published in four specialty journals);
30. Richard A. Mann, The Use of Legal Periodicals by Courts and Journals, 26 JURIMETRICS J. 400, 401 (1986) (comparing judicial and academic citations to scholarship published from 1978 to 1979);
31. William H. Manz, Citations in Supreme Court Opinions and Briefs: A Comparative Study, 94 LAW LIBR. J. 267, 267 (2002) (comparing citation in Supreme Court briefs with those in Supreme Court opinions);
32. Deborah J. Merritt & Melanie Putnam, Judges and Scholars: Do Courts and Scholarly Journals Cite the Same Law Review Articles?, 71 CHI.-KENT L. REV. 871, 872 (1996) (comparing and contrasting the law reviews most cited by legal scholars against those most cited by the courts);
33. Chester A. Newland, Legal Periodicals and the United States Supreme Court, 7 U. KAN. L. REV. 477, 479 (1959) (outlining legal periodical citations for twenty-eight justices from 1924 to 1956);
34. Michael J. Saks et al., Is There a Growing Gap Among Law, Law Practice, and Legal Scholarship?: A Systematic Comparison of Law Review Articles One Generation Apart, 30 SUFFOLK U. L. REV. 353, 361 (1996) (comparing articles published in 1960 and 1985 for, among other things, citation by courts);
35. Russell Smyth, Citing Outside the Law Reports: Citations of Secondary Authorities on the Australian State Supreme Courts Over the Twentieth Century, 18 GRIFFITH L. REV. 692, 692 (2009) (analyzing Australian state supreme court citation to secondary sources over time).
36. (citing Colleen M. Cullen & S. Randall Kalberg, Chicago-Kent Law Review Faculty Scholarship Survey, 70 CHI.-KENT L. REV. 1445, 1446-47, 1452 tbl. I (1995)).
37. See Crespi, Influence of a Decade, supra note 13, at 10.
38. When it is not feasible to collect data on the entire population, good empirical practices urge random sampling. See Lee Epstein & Gary King, The Rules of Inference, 69 U. CHI. L. REV. 1, 108 (2002).
39. We focused on the federal appellate level because, as others have recognized, "the circuit court judiciary is probably the single most important level of the federal judiciary in light of its extensive caseload and policy making authority." Frank B. Cross & Stefanie Lindquist, Judging the Judges, 58 DUKE L. J. 1383, 1385 (2009);
40. see also FRANK B. CROSS, DECISION MAKING IN THE U. S. COURTS OF APPEALS 1-2 (2007) ("[T]he circuit courts are much more important [than the United States Supreme Court] in setting and enforcing the law of the United States.");
41. DAVID E. KLEIN, MAKING LAW IN THE UNITED STATES COURTS OF APPEALS 4 (2002) ("The truth, well known but often overlooked in the media and even in serious scholarship, is that lower court judges play a major role in the development of legal doctrine.");
42. Corey Rayburn Yung, Flexing Judicial Muscle: An Empirical Study of Judicial Activism in the Federal Courts, 105 NW. U. L. REV. 1, 3 (2011) ("Although the actions of the Supreme Court are higher profile, studying the courts of appeals for activism has been substantially more informative about judges and the judiciary." (footnote omitted)). Others have surmised that the federal appellate courts should be most receptive to law review articles. See Sirico & Drew, supra note 13, at 1051 ("[T]he federal circuit courts may be the most policy-oriented tribunals and hence the most receptive to the theory-oriented discussions of the law reviews.").
43. MERRIAM-WEBSTER'S COLLEGIATE DICTIONARY 1051 (11th ed. 2004).
44. Our focus on the decisional-law-making process requires, we think, the use of reported opinions. Other prominent studies of citation counts focus exclusively on reported opinions. See, e.g., William M. Landes et al., Judicial Influence: A Citation Analysis of Federal Courts of Appeals Judges, 27 J. LEGAL STUD. 271, 271 (1998). Formal evidence of decisional law historically does not include unreported (or nonprecedential) opinions. (Pubitemid 128426690)
45. See, e.g., FED. CIR. R. 32.1 (authorizing unreported opinions and explaining: "[a]n opinion or order which is designated as nonprecedential is one determined by the panel issuing it as not adding significantly to the body of law"); id. at 36 (authorizing judgments of affirmance without opinions and explaining: "[t]he court may enter a judgment of affirmance without opinion, citing this rule, when it determines that... an opinion would have no precedential value");
46. see also Porter v. Merit Sys. Prot. Bd., 210 F. App'x 996, 996 (Fed. Cir. 2006)
47. (applying FED. CIR. R. 32.1);
48. Waddoups v. Dep't of the Air Force, 201 F. App'x 995, 996 (Fed. Cir. 2006) (deciding an appeal without a published opinion);
49. cf. FED. R. APP. P. 32.1 (introducing permission to cite to unreported opinions in briefing as of January 1, 2007, but not changing the precedential status of the cited opinion). This observation strongly suggests two things. First, while there may be reasons to study the judicial use of legal scholarship in unreported opinions, clumping unreported opinions together with reported opinions in a study examining the use of legal scholarship in the decisional-lawmaking process will be more confounding than informative. Second, judges should only rarely use legal scholarship in unreported opinions since they are not intended to add to the law. To test the second hypothesis, we examined all unreported opinions in Westlaw for all circuits from 1950 to 2008 and found that citation to legal scholarship, while it does occasionally occur, is exceptionally rare.
50. For an article describing content analysis and its application to legal studies, see Mark A. Hall & Ronald F. Wright, Systematic Content Analysis of Judicial Opinions, 96 CALIF. L. REV. 63, 79-99 (2008). For some recent examples of legal scholarship that have used content analysis as an approach to examining a body of law
51. see John R. Allison & Mark A. Lemley, The (Unnoticed) Demise of the Doctrine of Equivalents, 59 STAN. L. REV. 955, 966-76 (2007);
52. Edward K. Cheng, The Myth of the Generalist Judge, 61 STAN. L. REV. 519, 530-40 (2008);
53. Christian A. Chu, Empirical Analysis of the Federal Circuit's Claim Construction Trends, 16 BERKELEY TECH. L. J. 1075, 1096-1143 (2001);
54. Christopher A. Cotropia, Nonobviousness and the Federal Circuit: An Empirical Analysis of Recent Case Law, 82 NOTRE DAME L. REV. 911, 924-30 (2007);
55. Lee Petherbridge & R. Polk Wagner, TheThe Federal Circuit and Patentability: An Empirical Assessment of the Law of Obviousness, 85 Tex. L. Rev. 2051, 2076-102 (2007);
56. Lee Petherbridge, On the Decline of the Doctrine of Equivalents, 31 CARDOZO L. REV. 1371, 1384-1404 (2010);
57. Lee Petherbridge, Patent Law Uniformity?, 22 HARV. J. L. & TECH. 421, 438-63 (2009);
58. David L. Schwartz, Courting Specialization: An Empirical Study of Claim Construction Comparing Patent Litigation Before Federal District Courts and the International Trade Commission, 50 WM. & MARY L. REV. 1699, 1712-33 (2009);
59. David L. Schwartz, Practice Makes Perfect? An Empirical Study of Claim Construction Reversal Rates in Patent Cases, 107 MICH. L. REV. 223, 245-58 (2008);
60. R. Polk Wagner & Lee Petherbridge, Is the Federal Circuit Succeeding? An Empirical Assessment of Judicial Performance, 152 U. PA. L. REV. 1105, 1148-79 (2004).
61. Judge Edwards, for example, has been a vociferous critic of empirical studies of appellate decision making. See Harry T. Edwards & Michael A. Livermore, Pitfalls of Empirical Studies That Attempt to Understand the Factors Affecting Appellate Decisionmaking, 58 DUKE L. J. 1895, 1907-30 (2009) (discussing several limitations on empirical research into judicial decision making).
62. See Paul L. Caron, The Long Tail of Legal Scholarship, 116 YALE L. J. POCKET PART 38, 41(2006), http://www.thepocketpart.org/ylj-online/scholarship/ 59-the-long-tail-of-legalscholarship ("Citations reflect one particular end-use of an article; they do not measure how many times an article is read but not cited by a judge or professor.").
63. See George L. Priest & Benjamin Klein, The Selection of Disputes for Litigation, 13 J. LEGAL STUD. 1, 24 (1984) (explaining that because so few cases reach an appellate judgment, such cases may not be representative of all lawsuits or disputes).
64. For example, they might settle or be disposed of without any opinion. For more on unpublished opinions, see Kirt Shuldberg, Digital Influence: Technology and Unpublished Opinions in the Federal Courts of Appeals, 85 CALIF. L. REV. 541, 545-51(1997), which notes that the use of unpublished opinions was first proposed in 1964 and first implemented in the early 1970s. Others have noted that unpublished opinions often constitute a substantial percentage of a court's opinions and are often similar to published opinions in many respects.
65. See Michael Hannon, A Closer Look at Unpublished Opinions in the United States Courts of Appeals, 3 J. APP. PRAC. & PROCESS 199, 201 (2001) (reporting that less than 21% of opinions in the federal courts of appeals are published);
66. Beth Zeitlin Shaw, Please Ignore This Case: An Empirical Study of Nonprecedential Opinions in the Federal Circuit, 12 GEO. MASON L. REV. 1013, 1014, 1023-35 (2004) (reporting that the Federal Circuit publishes only 23% of its opinions and that the publication rate varies by the subject matter of the case).
67. See generally Donald R. Songer, Criteria for Publication of Opinions in the U. S. Courts of Appeals: Formal Rules Versus Empirical Reality, 73 JUDICATURE 307, 313 (1990) (demonstrating that official criteria for publication do not accurately describe differences between published and unpublished opinions). Using only reported decisions thus has the potential to provide a biased sample. See Edwards & Livermore, supra note 39, at 1922-24 (identifying the omission of unpublished decisions as a problem with the U. S. courts of appeals database that empiricists use). However, based on our examination of large numbers of unreported opinions, reported opinions contain the overwhelming majority of citations to legal scholarship. See supra note 30. It thus appears that circuit judges are not saving the use of legal scholarship for unreported opinions.
68. "Reported opinions" means those opinions Westlaw so designates. We did not distinguish between opinions for the court and alternative opinions. The Westlaw federal circuit court databases used were CTA1R, CTA2R, CTA3R, CTA4R, CTA5R, CTA6R, CTA7R, CTA8R, CTA9R, CTA10R, CTA11R, CTADCR, and CTAFEDR. As Westlaw's descriptions of each of these databases explain, they include "[r]eported cases from the federal appellate courts authoritative" in the circuit, with coverage beginning in 1945. E.g., U. S. Court of Appeals, 1st Circuit Cases, Reported, WESTLAW DATABASE DIRECTORY, http://directory.westlaw. com/scope/default.asp?db=CTA1R&RS=WDIR2.0&VR=2.0 (last visited July 18, 2011). The Eleventh and Federal Circuits did not exist until well after that date. The Federal Circuit database appears empty before the Federal Circuit's creation in 1982. The Eleventh Circuit database appears to replicate the Fifth Circuit database before 1980, the year the Fifth Circuit split into the Fifth and Eleventh Circuits. We excluded the duplicative Fifth and Eleventh Circuit data.
69. In a previous draft of this Article, we employed a search query that relied upon the Bluebook and the fact that nearly all law reviews include "law review" or "law journal" in their title. We developed the query over the course of several months and refined it through multiple conversations with Westlaw search specialists and by trial and error. Numerous people reviewed the query after several blogs featured a draft of this Article. See e.g., Iantha Haight, Court Citation of Legal Scholarship on the Rise, THE COMPETITIVE EDGE, (Aug. 17, 2011, 3:10 PM) http://blog.law.cornell.edu/library/ 2010/08/17/court-citationof-legal-scholarship-on-the-rise (noting that the search query was "impressive" and about as close to perfect "as you can reasonably get"). Unfortunately, we recently discovered that West, for reasons we do not presently understand, removes blank spaces from citations on an apparent haphazard basis. The haphazard removal appears to occur much more frequently in opinions issued before about 1997. Compounding this problem, Westlaw search specialists have recently advised us that it is not possible to craft a search query that locates citations without spaces-e.g., to the harv. l.rev. instead of harv. l.rev.-unless the journal's exact name is included as a search term. The impact of this discovery is that our original search query does not locate some of the citations to legal scholarship in Westlaw circuit court databases and we have not discovered a way of locating them without hard coding in each journal's West-formatted title. Future researchers should therefore be cautious when utilizing Westlaw for research concerning citation in Westlaw databases. To address this data inconsistency, we constructed a new query that included hard-coded West-formatted journal titles and experimented with the query until we discovered a query the positive responses to which were not substantially increased by the addition of new journals. We then regenerated all of our data.
70. See Schools of Law: The Top 100 Schools, U. S. NEWS & WORLD REP., May 2010, at 74, 74-75. Of the top fifty journals using Washington and Lee University's combined 2009 rankings, forty-eight fall within the search, and ninety-four of the top 100 are responsive.
71. See Law Journals: Submissions and Ranking, WASH. & LEE U. SCH. L., http://lawlib.wlu.edu/lj/indexOlderYears.aspx (last visited July 18, 2011) (check the box labeled "Comb." and "2009"; then click "Submit"). Excluded are Supreme Court Review (27); Law & Contemporary Problems (42); Yale Journal on Regulation (67); the Business Lawyer (68); University of Chicago Legal Forum (80); and Supreme Court Economic Review (97). See id.
72. The exact Westlaw search for opinions in each year YYYY is: "date ([YYYY]) and court." The word "court" is included because Westlaw prohibits date-only searches. The word "court" is present in all opinions in the case caption. See Christopher A. Cotropia, Determining Uniformity Within the Federal Circuit by Measuring Dissent and En Banc Review, 43 LOY. L. A. L. REV. 801, 811 n. 59 (2010).
73. We calculated the false-positive rate based upon a 5% random sample of the dataset. For the random sample, we manually reviewed the Westlaw citation results to ascertain which results were not of legal scholarship. An example of a false positive was a case that included the phrase "Letter No. 538 (July 21, 1971) by J. L. Robertson, CCH Cons. Credit Guide." Joseph v. Norman's Health Club, Inc., 532 F.2d 86, 92 n. 10 (8th Cir. 1976).
74. As points of reference, Lexis went online in 1973 and Westlaw went online in 1975. See William G. Harrington, A Brief History of Computer-Assisted Legal Research, 77 LAW LIBR. J. 543, 553 (1985).
75. HeinOnline launched in 2000, History, HEINONLINE http://home.heinonline. org/about/history (last visited July 18, 2011), and the Internet boom began in the mid-to late-1990s. LexisNexis and Westlaw may have permitted judges and lawyers to locate case law more easily, which they then inserted into opinions and briefs, respectively. Being able to easily find primary authority reduced the need to read (and cite to) law reviews to find the underlying black letter law. Furthermore, neither LexisNexis nor Westlaw included any legal scholarship until 1982; each service expanded their legal scholarship databases slowly thereafter.
76. See Bernard J. Hibbitts, Last Writes? Reassessing the Law Review in the Age of Cyberspace, 71 N. Y. U. L. REV. 615, 657-58 (1996). In the 1990s, the Internet made it much easier to find law reviews. 64 There are many other possible explanations for the results. For example, the number of law clerks afforded to each appellate court judge has increased over time.
77. Cf. I. SCOTT MESSINGER, FED. JUDICIAL CTR., ORDER IN THE COURTS: A HISTORY OF THE FEDERAL COURT CLERK'S OFFICE 58 (2002). Increasing the number of clerks may have altered who was responsible for preparing the initial draft of the opinion. The type of legal scholarship published by law reviews may have changed as well, moving away from more doctrinal scholarship. See McClintock, supra note 13, at 659 ("Judges and practitioners increasingly feel that there is a lack of legal scholarship that they can use when they face their daily case loads."). Citation practices by judges may have changed as well.
78. See generally Casey R. Fronk, The Cost of Judicial Citation: An Empirical Investigation of Citation Practices in the Federal Appellate Courts, 2010 U. ILL. J. L. TECH. & POL'Y 51 (finding an increase in overall citations of case law in opinions, but a decrease in string citations of case law over time).
79. We also experimented with a binomial regression, where the dependent variable was the number of opinions citing to legal scholarship and the number of reported opinions was an exposure variable. The results were qualitatively similar to those from the simpler OLS regression. Specifically, the same variables were statistically significant and the coefficients all had the same sign. One other variable, JCS-Supreme Court, was significant (p = 0.049) in the negative binomial model.
80. 2 = 0.7326) examining the contribution of various explanatory variables on the proportion of reported opinions citing legal scholarship from 1990 to 2008. Rep. Opinions/Active Judge measures the number of reported opinions authored per active circuit judge. Judicial Common Space refers to scores for the regional circuits and for the Supreme Court. See Lee Epstein et al., The Judicial Common Space, 23 J. L. ECON. & ORG. 303, 306-09 (2007); infra notes 71-72 and accompanying text. For the JCS scores used in this study
81. see Lee Epstein: Research, NW. L., http://epstein. law.northwestern. edu/research/JCS.html (follow "here" hyperlinks) (last visited July 18, 2011). TotalCrim measures the total number of reported and unreported criminal appellate decisions for each data point. TotalConlaw measures the number of opinions involving a constitutional issue. All of the variables in Figure 3 are significant at α = 0.05. See infra Appendix. Moreover, TotalConlaw is significant at α = 0.01 and Rep. Opinions/Active Judge is significant at α =0.001. The brackets at the end of the variables represent the standard errors. A conventional report of three regression models can be found in the Appendix.
82. See, e.g., Michael Heise, Federal Criminal Appeals: A Brief Empirical Perspective, 93 MARQ. L. REV. 825, 828 (2009) (noting the "sheer increase in federal criminal appeals over time").
83. CROSS, supra note 25, at 19 (noting that JCS scores are the "best currently available measure for circuit court judicial ideology").
84. See, e.g., Carolyn Shapiro, The Context of Ideology: Law, Politics, and Empirical Legal Scholarship, 75 MO. L. REV. 79, 79 (2010) (arguing that coding ignores the role of law in judicial decision making and presumes that cases can be broken down into a simple liberal-conservative distinction).
85. The Supreme Court ideology variable is a transformed Martin-Quinn score, which is another measure of ideology developed from votes by justices. See generally Andrew D. Martin & Kevin M. Quinn, Dynamic Ideal Point Estimation via Markov Chain Monte Carlo for the U. S. Supreme Court, 1953-1999, 10 POL. ANALYSIS 134, 145-52 (2002) (presenting a model for estimating judicial ideology and applying it to the Supreme Court Justices for the years 1953 through 1999).
86. Judges within the cohort include Judges Posner, Frank Easterbrook, Edward Becker, Edmund Lynch, Alex Kozinski, and Guido Calabresi. We note that many of these judges were academics before joining the bench. It also appears that there has been an increase in appointments of former academics to the federal circuit courts of appeals in the last thirty years. See Tracey E. George, Court Fixing, 43 ARIZ. L. REV. 1, 44 fig.1 (2001). Interestingly, the top three citing judges (Posner, Easterbrook, and Becker) are also the top three circuit judges most cited by the Supreme Court.
87. See Jeffrey A. Berger & Tracey E. George, Judicial Entrepreneurs on the U. S. Courts of Appeals: A Citation Analysis of Judicial Influence 15-17 tbl.3 (Vanderbilt Univ. Law Sch., Law & Econ. Working Paper No. 05-24), available at http://papers.ssrn. com/sol3/papers.cfm?abstract-id=789544.
88. See, e.g., F. Allan Hanson, From Key Numbers to Keywords: How Automation Has Transformed the Law, 94 LAW LIBR. J. 563, 598-600 (2002) (arguing that computer-aided legal research has fundamentally transformed legal research);
89. cf. Lawrence B. Solum, Blogging and the Transformation of Legal Scholarship, 84 WASH. U. L. REV. 1071, 1087 (2006) (noting that blogs may be viewed as shorter forms of legal scholarship).
90. But cf. Robert C. Berring, Legal Information and the Search for Cognitive Authority, 88 CALIF. L. REV. 1673, 1696 (2000) (arguing that Westlaw and Lexis did very little to change the practice of law).
91. In a study of Supreme Court citations, one researcher noted that the two justices who cited secondary sources most frequently, Brennan and Marshall, were on the liberal wing of the Court, while Rehnquist and White, two of the most conservative justices on the Court at that time, had among the lowest citation counts. See Wes Daniels, "Far Beyond the Law Reports": Secondary Source Citations in United States Supreme Court Opinions October Terms 1900, 1940, and 1978, 76 LAW LIBR. J. 1, 10 tbl.6 (1983).
92. Antonin Scalia, The Legacy of Judge Howard T. Markey, 8 J. MARSHALL REV. INTELL. PROP. L. (SPECIAL ISSUE) 1, 8 (2009).
93. This category overlaps at least somewhat with the category What Variables Explain Judicial Use of Legal Scholarship?, supra Part II. C.1, in particular with the discussion about whether there are qualities or characteristics of the form or presentation of legal scholarship that encourage judges to use it.
94. See, e.g., Gary Edmond & David Mercer, Conjectures and Exhumations: Citations of History, Philosophy and Sociology of Science in US Federal Courts, 14 LAW & LITERATURE 309 (2002) (examining the citation of scientific articles against the backdrop of Daubert challenges).