The origins, nature, and promise of empirical legal studies and a response to concerns

University of Illinois Law Review

Volumn 2011, Issue 5, 2011, Pages 1713-1738

  Eisenberg, Theodore ^a  

^a Cornell University ^*  (United States)

Author keywords

[No Author keywords available]

Indexed keywords

EID: 83155189855     PISSN: 02769948     EISSN: None     Source Type: Journal    
DOI: None     Document Type: Conference Paper

References (110)

1. See EaMPIRICAL LEGAL STUD., http://www.elsbIog.org/(last visited July 25, 2011).
2. See, e.g., Christoph Engel & Urs Schweizer, Jurimetrics: 27th International Seminar on the New Institutional Economics June 11-13, 2009, Kloster Eberbach, Germany, 166 J. INST. & THEORETICAL ECON., Mar. 2010, at 1;
3. Jason Czaruezki, An Announcement from the UK, EMPIRICAL LEGAL STUD. (Oct. 23, 2007, 8:15 PM), http://www.elsblog.org/the-empirical-legal-studi/2007/10/ an-announcement.html;
4. Michael Heise, First International Conference on Empirical Studies of Judicial Systems, EMPIRICAL LEGAL STUD. (July 1, 2008, 7:49 PM), http://www.elsblog.org/the-empirical-legal-studi/2008/07/my-cornell-coll.html;
5. Conference Itenarary, International Conference: Empirical Legal Studies (Mar. 26, 2009), http://www.law.tau.ac.il/Eng/-Uploads/dbsAttachedFiles/ EmpiricalLegalStudiesINV.pdf.
6. See, e.g., Empirical Scholarship Workshops, NW. U. SCH. L., http://www.law.northwestern.edu/faculty/empiricalworkshop/(last visited July 25, 2011).
7. See THE OXFORD HANDBOOK OF EMPIRICAL LEGAL RESEARCH (Peter Cane & Herbert M. Kritzer eds., 2010).
8. See ROBERT M. LAWLESS ET AL., EMPIRICAL METHODS IN LAW (2010).
9. Tracey E. George, An Empirical Study of Empirical Legal Scholarship: The Top Law Schools, 81 IND. L. J. 141, 142 (2006).
10. See, e.g., Leslie A. Gordon, The Empiricists: Legal Scholars at the Forefront of Data-Based Research, 82 STAN. LAW. (May 11, 2010), http://stanfordlawyer.law.stanford.edu/2010/05/the-empiricists/;
11. Linda Brandt Myers, The Journal of Empirical Legal Studies: Finding the Facts that Challenge Our Assumptions About the World, 34 CORNELL L. F., Spring 2008, at 10, http://www.lawschool.cornell.edu/research/upload/ Spring2008LawForum2.pdf.
12. The New Legal Realism Project, U. WIS. L. SCH., http://www.law.wisc.edu/ ils/newlegal.htm (last updated Oct. 28, 2009). "The Project's goal is to develop rigorous, genuinely interdisciplinary approaches to the empirical study of law." Id. (follow the link to the NLR homepage);
13. see also Mark C. Suchman & Elizabeth Mertz, Toward a New Legal Empiricism: Empirical Legal Studies and New Legal Realism, 6 ANN. REV. L. & SOC. SCI. 555 (2010) (noting the return of interest in empirical legal research within the U. S. legal academy).
14. Shari Seidman Diamond & Pam Mueller, Empirical Scholarship in Law Reviews, 6 ANN. REV. L. & SOC. SCI. 581, 587 (2010) (showing that nearly half (45.8%) of a sample of law review articles from 1998 to 2008 included some empirical content, but only 5.7% presented original empirical research);
15. Michael Heise, An Empirical Analysis of Empirical Legal Scholarship Production, 1990-2009, 2011 U. ILL. L. REV. 1739, 1746 fig.2 (showing substantial growth in empirical legal scholarship from 2000 to 2009). For a description of ELS work and growth in the area of alternative dispute resolution, and salient observations about baseline reference points
16. see Carrie J. Menkel-Meadow, Dispute Resolution, in THE OXFORD HANDBOOK OF EMPIRICAL LEGAL RESEARCH, supra note 4, at 596, 599.
17. STEVEN M. TELES, THE RISE OF THE CONSERVATIVE LEGAL MOVEMENT: THE BATTLE FOR CONTROL OF THE LAWS 183-87 (2008) (describing the Olin Foundation financing);
18. H. Lawrence Ross, Programs in Law and Social Science, LAW & SOC'Y REV. 509, 509 (1968) (describing the law and society financing).
19. Robert B. Yegge, The Law and Society Association to Date, 1 LAW & SOC'Y REV. 3, 3 (1966).
20. Richard D. Schwartz, From the Editor..., 1 LAW & SOC'Y REV. 6, 6 (1966).
21. TELES, supra note 10, at 182.
22. GEORGE L. PRIEST, CTR. FOR LEGAL POL'Y, WHAT WE KNOW AND WHAT WE DON'T KNOW ABOUT MODERN CLASS ACTIONS: A REVIEW OF THE EISENBERG-MILLER STUDY 2-3 n. 14 (2005) (describing my transforming skewed dollar amounts to a logarithmic scale as a "numerical manipulation" and claiming that I had not answered this criticism). It is correct that I had not expressly addressed this criticism because some criticisms are unworthy of response. For a primer on the statistical basis for transforming skewed amounts
23. see Theodore Eisenberg & Martin T. Wells, The Significant Association Between Punitive and Compensatory Damages in Blockbuster Cases: A Methodological Primer, 3 J. EMPIRICAL LEGAL STUD. 175 (2006), or almost any basic statistics book. Professor Priest refers to my work on punitive damages as having generated similar criticism. PRIEST, supra, at 2-3 n. 14 (noting particular criticism of my work on punitive damages). Law and economics scholar W. Kip Viscusi has since employed both the logarithmic approach to regression models of punitive damages and other aspects of my work in evaluating punitive damages.
24. See Alison F. Del Rossi & W. Kip Viscusi, The Changing Landscape of Blockbuster Punitive Damages Awards, 12 AM. L. & ECON. REV. 116, 143-52 (2010) (using logarithms in analyzing punitive damages and employing a variable, first suggested in Eisenberg and Wells, supra, to account for tobacco cases). I should add, as did Professor Priest, that we are former colleagues and remain friends. PRIEST, supra, at 2 n. 9.
25. Nevertheless, troublesome evidence may exist that Olin terminated programs in part based on ideological grounds. Thomas O. McGarity, A Movement, A Lawsuit, and the Integrity of Sponsored Law and Economics Research, 21 STAN. L. & POL'Y REV. 51, 64 n. 114 (2010).
26. See EMMA COLEMAN JORDAN & ANGELA P. HARRIS, ECONOMIC JUSTICE: RACE, GENDER, IDENTITY AND ECONOMICS at v (2d ed. 2011) (expressing the view that law and economics "focuses on questions of transactional efficiency and tends to ignore questions of distribution or of justice" but noting that law and economics "has begun to alter its once iron-clad assumption that people always act as rational maximizers").
27. For example, Judge Posner was the first recipient of the American Law and Economics Association Coase medal. See The Coase Medal, 12 AM. L. & ECON. REV. 263 (2010).
28. R. H. Coase, The Problem of Social Cost, 3 J. L. & ECON. 1 (1960).
29. TELES, supra note 10, at 95.
30. The American Law and Economics Review, AM. L. & ECON. ASS'N, http://www.amlecon.org/review.html (last visited July 25, 2011) [hereinafter The American Law and Economics Review].
31. See Journal of Empirical Legal Studies: Journal Information, WILEY, http://www.wiley.com/bw/aims.asp?ref=1740-1453&site=1 (last visited July 25, 2011).
32. For information about the first conference, see Conference on Empirical Legal Studies, U. TEX. L. SCH., http://www.utexas.edu/law/conferences/cels2006/ (last visited July 25, 2011).
33. For an example of a recent CELS program, see Conference on Empirical Legal Studies, YALE L. SCH., http://www.law.yale.edu/news/CELSprogram.htm (last visited July 25, 2011).
34. Lorenz Krüger, Preface to Volumes 1 and 2, in 1 THE PROBABILISTIC REVOLUTION: IDEAS IN HISTORY, at XV (Lorenz Krtlger et al. eds., 1987).
35. For a broad perspective on statistics in modern times, see generally the articles in STATISTICS IN THE 21ST CENTURY (Adrian E. Raftery et al. eds., 2002) (containing articles addressing statistical issues across many disciplines).
36. For a perspective on empirical methods and the law, see Theodore Eisenberg, Empirical Methods and the Law, 95 J. AM. STAT. ASS'N 665 (2000) (reprinted in STATISTICS IN THE 21ST CENTURY, supra, at 179).
37. See Victoria Nourse & Gregory Shaffer, Varieties of New Legal Realism: Can a New World Order Prompt a New Legal Theory?, 95 CORNELL L. REV. 61, 64 (2009) (suggesting that new legal realism offers an alternative to a new formalism represented by the theory of neoclassical law and economics).
38. See, e.g., Howard Erlanger et al., Is It Time for a New Legal Realism?, 2005 WIS. L. REV. 335, 337.
39. See Brian Z. Tamanaha, Understanding Legal Realism, 87 TEX. L. REV. 731, 734 (2009) ("We are all realists now."
40. (quoting George Packer, Comment, Unrealistic, NEW YORKER, Nov. 27, 2006, at 83, 83)).
41. Herbert M. Kritzer, Empirical Legal Studies Before 1940: A Bibliographic Essay, 6 J. EMPIRICAL LEGAL STUD. 925, 926 (2009) [hereinafter Kritzer, Empirical Legal Studies]. A milestone in the foundation of modern ELS was the Wisconsin Civil Litigation Research Project (CLRP).
42. DAVID M. TRUBEK ET AL., CIVIL LITIGATION RESEARCH PROJECT: FINAL REPORT (1983) (studying litigation processes, outcomes, and costs based on data from federal courts, state courts, and surveys of lawyers and households). For a sample of the work growing out of the CLRP
43. see Herbert M. Kritzer, Bibliography of Publications and Papers of the Civil Litigation Research Project, http://users.polisci.wisc.edu/kritzer/ research/clrpbib.htm (last updated Dec. 18, 2004).
44. For a recent discussion of the history of empirical research in the area of civil procedure, see Carrie J. Menkel-Meadow & Bryant Garth, Civil Procedure and Courts, in THE OXFORD HANDBOOK OF EMPIRICAL LEGAL RESEARCH, supra note 4, at 679, 683-90.
45. Kritzer, Empirical Legal Studies, supra note 38, at 925 (noting that the origins of ELS predate both the law and society and law and economics movements).
46. Brian Leiter, On So-Called "Empirical Legal Studies" and Its Problems, BRIAN LETTER'S L. SCH. REP. (July 6, 2010, 6:41 AM), http://leiterlawschool.typepad.corn/leiter/2010/07/on-socalled-empirical-legal- studies.html.
47. See, for example, Benjamin Barton's prize-winning article on the relation between teaching and scholarship. Benjamin Barton, Is There a Correlation Between Law Professor Publication Counts, Law Review Citation Counts, and Teaching Evaluations? An Empirical Study, 5 J. EMPIRICAL LEGAL STUD. 619 (2008). The article won the 2010 Law School Admission Council Philip D. Shelton Award for outstanding research in legal education.
48. See Law Journals: Submission and Ranking, WASH. & LEE U. SCH. L., http://lawUb.wlu.edu/lj/(last visited July 25, 2011) [hereinafter W&L].
49. The journal ranking system often used for nonlaw disciplines is based on a journal's impact factors in the system operated by what was originally the Institute for Scientific Information (ISI) and is now part of Thomson Reuters. Daniel B. Klein & Eric Chiang, The Social Science Citation Index: A Black Box-with an Ideological Bias?, 1 ECON. J. WATCH 134, 135 (2004). But key ISI products, such as the Social Science Citation Index (SSCI) and Journal Citation Reports (JCR), are virtually unknown to U. S. law professors, have not, to my knowledge, been used by law schools to evaluate legal scholarship and contain a much narrower range of lawrelated journals than other sources.
50. Online probing of JCR suggests that it includes 128 law journals. ISI Web of Knowledge, THOMSON REUTERS (on file with author) (list of journals from subject category "Law"). Databases such as Westlaw and Lexis include several times that number.
51. TELES, supra note 10, at 96-101.
52. The American Law and Economics Review, supra note 28.
53. Journal of Law & Economics, U. CHI. PRESS, http://ideas.repec.org/s/ uq3/jlawec.html (last updated July 7, 2011).
54. Robert A. Hillman, Questioning the "New Consensus" on Promissory Estoppel: An Empirical and Theoretical Study, 98 COLUM. L. REV. 580, 580-85 (1998). (Pubitemid 128426570)
55. James A. Henderson, Jr. & Theodore Eisenberg, The Quiet Revolution in Products Liability: An Empirical Study of Legal Change, 37 UCLA L. REV. 479, 481 (1990);
56. see also Theodore Eisenberg & James A. Henderson, Jr., Inside the Quiet Revolution in Products Liability, 39 UCLA L. REV. 731, 734-35 (1992).
57. Kevin M. Clermont & Stewart J. Schwab, How Employment Discrimination Plaintiffs Fare in Federal Court, 1 J. EMPIRICAL LEGAL STUD. 429, 429 (2004).
58. See, e.g., John H. Blume et al., An Empirical Look at Atkins v. Virginia and Its Application in Capital Cases, 76 TENN. L. REV. 625 (2009);
59. Stephen P. Garvey et al., Juror First Votes in Criminal Trials, 1 J. EMPIRICAL LEGAL STUD. 371 (2004).
60. Theodore Eisenberg & Charlotte Lanvers, What Is the Settlement Rate and Why Should We Care?, 6 J. EMPIRICAL LEGAL STUD. 111, 112 n. 1 (2009) (citing studies that misstate the settlement rate).
61. See, e.g., Theodore Eisenberg, Measuring the Deterrent Effect of Punitive Damages, 87 GEO. L. J. 347, 347 (1998) (describing incorrect characterization of punitive damages laws). (Pubitemid 128429262)
62. STANTON A. GLANTZ, PRIMER OF BIOSTATISTICS 7 (6th ed. 2005).
63. Gregory D. Curfman et al., Editorial, Expression of Concern: Bombardier et al., "Comparison of Upper Gastrointestinal Toxicity of Rofecoxib and Naproxen in Patients with Rheumatoid Arthritis," 353 NEW ENG. J. MED. 2813, 2813-14 (2005);
64. Gregory D. Curfman et al., Expression of Concern Reaffirmed, 354 NEW ENG. J. MED. 1193, 1193 (2006). Dr. Thomas A. Marciniak of the U. S. Federal Drug Administration reportedly stated that: [D]octors in the [Avandia] study correctly took note of problems that patients in the study experienced only to have those initial judgments later overruled or questioned by the company. He said sick patients were dropped from the study instead being allowed to tarnish Avandia. "This is a great way to improve your mortality statistics, which everybody tells me can't be biased," he said.
65. Gardiner Harris, F. D. A Panel Votes to Restrict Avandia, N. Y. TIMES, July 14, 2010, http://prescriptiom.blogs.nytimes.com/2010/07/14/blogging-the-f- d-a-panel-on-avandia/.
66. David H. Freedman, Lies, Damned Lies, and Medical Science, ATLANTIC, Nov. 2010, http//www.theatlantic.com/magazme/archive/2010/ll/lies-damned-lies-and- medical-science/8269/;
67. see also John P. A. Ioannidis, Why Most Published Research Findings Are False, 2 PLOS MED. 0696, 0696 (2005), http://www.plosmedicme.org/article/info: doi/10.1371/journal.pmed.0020124 (follow "Download PDF" hyperlink) ("[F]alse findings may be the majority or even the vast majority of published research claims.").
68. Joshua D. Angrist & Jörn-Steffen Pischke, The Credibility Revolution in Empirical Economics: How Better Research Design Is Taking the Con out of Econometrics, 24 J. ECON. PERSP., Spring 2010, at 3, 6 (2010).
69. William Anderson & Martin T. Wells, Numerical Analysis in Least Squares Regression with an Application to the Abortion-Crime Debate, 5 J. EMPIRICAL LEGAL STUD. 647, 649-64 (2008).
70. Jonathan P. Kastellec, The Statistical Analysis of Judicial Decisions and Legal Rules with Classification Trees, 7 J. EMPIRICAL LEGAL STUD. 202, 202-03 (2010).
71. See Theodore Eisenberg & Geoffrey P. Miller, Do Juries Add Value? Evidence from an Empirical Study of Jury Trial Waiver Clauses in Large Corporate Contracts, 4 J. EMPIRICAL LEGAL STUD. 539, 574 (2007).
72. Charles R. Shipan, Partisanship, Ideology, and Senate Voting on Supreme Court Nominees, 5 J. EMPIRICAL LEGAL STUD. 55, 63-64 (2008) (accounting for the nonindependence of senatorial voting on Supreme Court nominees).
73. Jonathan P. Kastellec & Jeffrey R. Lax, Case Selection and the Study of Judicial Politics, 5 J. EMPIRICAL LEGAL STUD. 407, 407-10 (2008);
74. Denise M. Keele et al., An Analysis of Ideological Effects in Published Versus Unpublished Judicial Opinions, 6 J. EMPIRICAL LEGAL STUD. 213, 213-16 (2009).
75. E.g., M. Hashem Pesaran & Ron Smith, The Role of Theory in Econometrics, 67 J. ECONOMETRICS 61, 63 (1995) (describing traditional strategy in which "the theorist provided the model and the econometrician estimated and tested it").
76. National Health and Nutrition Examination Survey, CTRS. DISEASE CONTROL & PREVENTION, http://www.cdc.gov/nchs/nhanes.htm (last updated July 19, 2011).
77. National Health Interview Survey, CTRS. DISEASE CONTROL & PREVENTION, http://www.cdc.gov/nchs/nhis.htm (last updated June 22, 2011).
78. Consumer Expenditure Survey, BUREAU LAB. STAT., http://www.bls.gov/cex/ (last visited July 25, 2011).
79. The Panel Study of Income Dynamics-PSID-Is the Longest Running Longitudinal Household Survey in the World, PSID: INST. FOR SOC. RES., http://psidonline.isr.umich.edu/(last visited July 25, 2011).
80. E.g., Theodore Eisenberg et al., The Decision to Award Punitive Damages: An Empirical Study, 2 J. LEGAL ANALYSIS 577, 578 (2010);
81. Theodore Eisenberg et al., Juries, Judges, and Punitive Damages: Empirical Analyses Using the Civil Justice Survey of State Courts 1992, 1996, and 2001 Data, 3 J. EMPIRICAL LEGAL STUD. 263, 263-66 (2006);
82. Theodore Eisenberg et al., The Predictability of Punitive Damages, 26 J. LEGAL STUD. 623, 632-33 (1997).
83. E.g., Kevin M. Clermont & Theodore Eisenberg, Xenophilia or Xenophobia in U. S. Courts? Before and After 9/11, 4 J. EMPIRICAL LEGAL STUD. 441, 452-53 (2007);
84. Federal Court Cases: Integrated Data Base, 1970-2000, ICPSR: INTER-UNIV. CONSORTIUM POL. & SOC. RES., http://www.icpsr.umich.edu/icpsrweb/ICPSR/ studies/08429 (last visited July 25, 2011).
85. Josh Wright, ELS, Technical Fetishization vs. Legal Relevance, and a Partial Defense of the Perfectly Proportional Mediocrity of Legal Empiricists, TRUTH ON MKT. (July 8, 2010, 9:14 AM), http://truthonthemarket.com/2010/07/08/ els-technical-fetishization-vs-legal-relevance-and-a-partial-defense-of-the- perfectly-proportional-mediocrity-of-legal-empiricists/.
86. Information & Resources, NCSC: NAT'L CTR. FOR CTS., http://www.ncsc.org/Information-and-Resources.aspx (last visited July 25, 2011).
87. Theodore Eisenberg et al., Israel's Supreme Court Appellate Jurisdiction: An Empirical Study, 96 CORNELL L. REV. 693 (2011).
88. Previous empirical studies conducted of the ISC include the following: GAD BARZILAI ET AL., THE ISRAELI SUPREME COURT AND THE ISRAELI PUBLIC (1994);
89. Yoav Dotan, Do the "Haves" Still Come Out Ahead? Resource Inequalities in Ideological Courts: The Case of the Israeli High Court of Justice, 33 LAW & SOC'Y REV. 1059 (1999);
90. Yoav Dotan & Menachem Hofnung, Interest Groups in the Israeli High Court of Justice: Measuring Success in Litigation and in Out-of-Court Settlements, 23 LAW & POL'Y 1 (2001);
91. Meron Gross & Yoram Shachar, How Are Supreme Court Panels Selected-A Quantitative Analysis, 29 HEBREW U. L. REV. 567 (1999) (in Hebrew);
92. Menachem Hofnung & Keren Weinshall Margel, Judicial Setbacks, Material Gains: Terror Litigation at the Israeli High Court of Justice, 7 J. EMPIRICAL LEGAL STUD. 664 (2010);
93. Eli Salzberger, Acting Justices in the Supreme Court and Judicial Independence-Theoretical Analysis and Empirical Findings, 19 BAR-ILAN L. REV. 541 (2003) (in Hebrew);
94. Yoram Shacher et al., 100 Leading Precedents-A Quantitative Analysis, 7 HAIFA U. L. REV. 243 (2003) (in Hebrew);
95. Yoram Shachar et al., Anatomy of Discourse and Dissent in the Supreme Court-Quantitative Analyses, 20 TEL AVIV U. L. REV. 749 (1997) (in Hebrew);
96. Yoram Shachar et al., Reference Patterns of the Supreme Court-A Quantitative Analysis, 27 HEBREW U. L. REV. 119 (1996) (in Hebrew).
97. Ido Baum, The Chances of Defeating the State in an Appeal to the ISC Are Slim, MARKER, June 15, 2010 (in Hebrew).
98. Ido Baum, Unequal Before the Law, HAARETZ. COM (June 16, 2010, 3:29 AM), http://www.haaretz.com/print-edition/business/unequal-before-the-law-1.296446.
99. E.g., Nathan Koppel, Empirical Holy War: Cornell Law Prof Takes on Chamber of Commerce, WALL ST. J. L. BLOG (Sept. 24, 2009, 4:51 PM), http://blogs.wsj.com/law/2009/09/24/empirical-holy-war-cornell-law-prof-takes- on-us-chamber-of-commerce/.
100. WILLIAM HALTOM & MICHAEL MCCANN, DISTORTING THE LAW: POLITICS, MEDIA, AND THE LITIGATION CRISIS 147-49 (2004).
101. Theodore Eisenberg, U. S. Chamber of Commerce Liability Survey: Inaccurate, Unfair, and Bad for Business, 6 J. EMPIRICAL LEGAL STUD. 969, 969-70 (2009).
102. HARRIS INTERACTIVE, 2010 U. S. CHAMBER OF COMMERCE STATE LIABILITY SYSTEMS RANKING STUDY 88 (2010), http://courts.delaware.gov/Courts/Superior/pdf/ harris-2010.pdf.
103. Punitive Damages Reform, ATRA: Am. TORT REFORM ASS'N, http://www.atra.org/show/7343 (last visited July 25, 2011).
104. SWISS RE, THE GLOBALISATION OF COLLECTIVE REDRESS: CONSEQUENCES FOR THE INSURANCE INDUSTRY 2 (2009), http://media.swissre.com/documents/Globalisation- of-Collective-Redress-en.pdf ("Well-publicized cases involving significant punitive damages have led to the commonly-held view that awards are becoming increasingly extreme.").
105. Theodore Eisenberg et al., Variability in Punitive Damages: Empirically Assessing Exxon Shipping Co. v. Baker, 166 J. INSTITUTIONAL & THEORETICAL ECON. (JITE) 5, 5-6 (2010).
106. Exxon Shipping Co. v. Baker, 554 U. S. 471, 497-99 (2008).
107. Alexander J. S. Colvin, An Empirical Study of Employment Arbitration: Case Outcomes and Processes, 8 J. EMPIRICAL LEGAL STUD. 1, 11-16 (2011).
108. Laura Beth Nielsen et al., Individual Justice or Collective Legal Mobilization? Employment Discrimination Litigation in the Post Civil Rights United States, 7 J. EMPIRICAL LEGAL STUD. 175, 175-77 (2010).
109. Daphna Hacker, The Gendered Dimensions of Inheritance: Empirical Food for Legal Thought, 7 J. EMPIRICAL LEGAL STUD. 322, 322-24 (2010).
110. Oren Gazal-Ayal & Raanan Sulitzeanu-Kenan, Let My People Go: Ethnic In-Group Bias in Judicial Decisions-Evidence from a Randomized Natural Experiment, 7 J. EMPIRICAL LEGAL STUD. 403, 403-05 (2010).