Evidence-based law

Cornell Law Review

Volumn 96, Issue 4, 2011, Pages 901-923

  Rachlinski, Jeffrey J ^a  

^a MIT Sloan School of Management   (United States)

Author keywords

[No Author keywords available]

Indexed keywords

EID: 79959444785     PISSN: 00108847     EISSN: None     Source Type: Journal    
DOI: None     Document Type: Conference Paper

References (141)

1. More and Bigger Clinical Trials Ahead, MED. WORLD NEWS, Sept. 1, 1972, at 45.
2. See id. at 46 (noting that, in 1972, the "multi-institutional approach to clinical research appear[ed] to be gaining popularity throughout medicine").
3. See generally David L. Sackett et al., Evidence Based Medicine: What It Is and What It Isn't, 312 BRIT. MED. J. 71 (1996) (describing evidence-based medicine).
4. See id. at 71 ("Evidence based medicine is the conscientious, explicit, and judicious use of current best evidence in making decisions about the care of individual patients.").
5. See JEFFREY PFEFFER & ROBERT I. SUTTON, HARD FACTS, DANGEROUS HALF-TRUTHS AND TOTAL NONSENSE: PROFITING FROM EVIDENCE-BASED MANAGEMENT 13 (2006) ("Evidence-based management proceeds from the premise that using better, deeper logic and employing facts to the extent possible permits leaders to do their jobs better.").
6. See id. at 14 ("In almost every field there are accepted truths, or conventional wisdom, that guide decisions and actions. And in almost every field... many practitioners and their advisers are unwilling or unable to observe the world systematically because they are trapped by dieir beliefs and ideologies.... The result is that much conventional wisdom is wrong.").
7. See id. at 126 ("[M]ost pay-for-performance programs fail to achieve their objectives....").
8. See id. at 136 (calling many businesses' focus on strategy a "half-truth []" that "can obscure as much as it illuminates").
9. See id. at 107 ("A survey of more than 200 human resource professionals... found that even though more than half of the companies used forced ranking, the respondents reported that forced ranking resulted in lower productivity....").
10. See id. at 57-187.
11. According to Professor Ian Ayres, "retailers like Wal-Mart have more information in their databases than the U. S. Library of Congress." Dana Flavelle, What the Data Crunchers Know About You, THESTAR. COM (Apr. 23, 2010), http://www.thestar.com/business/article/799986-what-the-data-crunchers-know- about-you;
12. see also IAN AYRES, SUPER CRUNCHERS: WHY THINKING-BY-NUMBERS IS THE NEW WAY TO BE SMART 10-11 (2007) ("Business and government professionals are relying more and more on databases to guide their decisions.").
13. See Ayres, supra note 11, at 11 (noting that "Wal-Mart's data warehouse... stores more than 570 terabytes" of data).
14. Multiple Nobel Prizes in Economics were awarded to practitioners of game theory in the past twenty years, including recipients such as John Nash, John Harsanyi, and Reinhard Selten in 1994;
15. Thomas Schelling and Robert Aumann in 2005;
16. and Roger Myerson, Leonid Hurwicz, and Eric Maskin in 2007.
17. See All Prizes in Economic Sciences, NOBELPRIZE. ORG, http://nobelprize.org/nobel-prizes/economics/laureates/ (last visited Jan. 20, 2011).
18. STEVEN D. LEVITT & STEPHEN J. DUBNER, FREAKONOMICS: A ROGUE ECONOMIST EXPLORES THE HIDDEN SIDE OF EVERYTHING (2005).
19. STEVEN D. LEVITT & STEPHEN J. DUBNER, SUPER FREAKONOMICS: GLOBAL COOLING, PATRIOTIC PROSTITUTES AND WHY SUICIDE BOMBERS SHOULD BUY LIFE INSURANCE (2009).
20. See, e.g., Shari Seidman Diamond & Pam Mueller, Empirical Legal Scholarship in Law Reviews, 6 ANN. REV. L. & SOC. SCI. 581, 592 (2010) (reviewing the content of law reviews and concluding that aldiough "original empirical work still accounts for only a modest portion of the work published in law reviews", "[t]he trend, however, continues to be upward");
21. Lee Epstein & Gary King, The Rules of Inference, 69 U. CHI. L. REV. 1, 2 (2002) ("The law reviews are replete with articles ranging from the normative to the descriptive, from narrow doctrinal analyses to large-sample-size (large-n) statistical investigations.... [M]any... evince a common characteristic: a concern, however implicit, with empiricism....");
22. Tracey E. George, An Empirical Study of Empirical Legal Scholarship: The Top Law Schools, 81 IND. L. J. 141, 147 (2006) (reporting that in law reviews, "the number of references to ELS phrases-empiric, empirical, quantitative, statistically significant, statistical significance... continues to grow").
23. See Anthony Sebok, Law Reviews Submission Guidelines, Fall '06 Version 1.0, http://www.concurringopinions.eom/archives/ Fall%2006%20Submission%20Guide%201.0.doc (last visited Mar. 11, 2011) (noting that for empirical pieces, the Yale Law Journal requests audiors to "please upload all materials required for replication (including computer programs and data sets) ").
24. For example, the Harvard Law Review now asserts that it conducts peer review of articles. See Submissions, HARV. L. REV., http://www.harvardlawreview. org/submissions.php (last visited Mar. 11, 2011) ("[A]t least two editors review every submission, and many pieces go through substantially more stages of review, including an Articles Committee vote, a preemption check, faculty peer review, and a vote by the body of the Review.').
25. See About Journal, J. LEGAL STUD., http://www.journals.uchicago.edu/page/ jls/brief.html (last visited Mar. 11, 2011) ("The Journal of Legal Studies is a journal of interdisciplinary academic research into law and legal institutions. It emphasizes social science approaches, especially those of economics, political science, and psychology, but it also publishes the work of historians, philosophers, and others who are interested in legal theory.");
26. see also William M. Landes, The Empirical Side of Law & Economics, 70 U. CHI. L. REV. 167, 170 tbl.1 (2003) (reporting that 38.5% of the articles in the Journal of Legal Studies are empirical). Professor Landes also reports a 2.7% yearly increase in the rate of empirical articles in the Journal of Law & Economics between 1972 and 2002. Landes, supra, at 172.
27. As a disclaimer, I am one of the editors, as are three others who have contributed to this Symposium: Theodore Eisenberg, Michael Heise, and Stewart Schwab.
28. See Journal of Empirical Legal Studies, WILEY-BLACKWELL, http://www.wiley.com/bw/aims.asp?ref=1740-1453&site=1 (last visited Mar. 11, 2011) ("The Journal of Empirical Legal Studies (JELS) is a peer-edited, peer-refereed, interdisciplinary journal that publishes highquality, emirically-oriented articles of interest to scholars in a diverse range of law and lawrelated fields, including civil justice, corporate law, criminal justice, domestic relations, economics, finance, health care, political science, psychology, public policy, securities regulation, and sociology.").
29. See Law Journals: Submissions and Rankings, WASH. & LEE U. SCH. L., http://lawlib.wlu.edu/lj/ (follow "older surveys" hyperlink, then select check box labeled "2009" and "CF", and click "Submit") (last visited Mar. 11, 2011) (noting that, in 2009, the Journal of Empirical Legal Studies ranked first in "currency factor" among all refereed journals and fifteenth among all print journals, with currency factor measuring how rapidly journals cite a particular journal's articles following publication).
30. See Cornell Law School Hosts Record-Breaking 2008 Conference on Empirical Legal Studies, CORNELL U. L. SCH., http://www.lawschool.cornell.edu/spodight. cfm?pageid=152185 (last visited Jan. 1, 2011) ("The two-day conference, which featured original empirical and experimental legal scholarship from a diverse range of fields, attracted more than 300 participants from all over the world.").
31. See George, supra note 16, at 144 ("Legal realism was the first movement likely to bring empiricism to law.");
32. Herbert M. Kritzer, Empirical Legal Studies Before 1940: A Bibliographic Essay, 6 J. EMPIRICAL LEGAL STUD. 925 (2009) (cataloging early empirical legal work).
33. See Ruth Bader Ginsburg, Muller v. Oregon: One Hundred Years Later, 45 WILLAMETTE L. REV. 359, 361-65 (2009)
34. (discussing how Louis Brandeis's use of social science influenced the Court's decision in Muller v. Oregon, 208 U. S. 412 (1908)).
35. -347 U. S. 483, 494 n. 11 (1954).
36. For discussion of the debate regarding the death penalty, see infra notes 50-74 and accompanying text.
37. The Law and Society Association was founded in 1964. See Law & Soc'y Ass'n, http://www.lawandsociety.org/ (last visited Nov. 12, 2010).
38. See Law & Society Review, Law & Soc'y Ass'n, http://www. lawandsociety.org/review.htm (last visited Mar. 11, 2011) (stating that the Law & Society Review, founded in 1966, has published volumes on an annual basis).
39. See Law and Human Behavior, SPRINGER, http://www.springerlink.com/ content/0147-7307 (last visited Nov. 12, 2010) (noting that the first volume of Law and Human Behaviorwas published in 1977 and the thirty-fourth volume in 2010).
40. Lawrence M. Friedman, The Law and Society Movement, 38 STAN. L. REV. 763, 777 (1986) ("Prestigious law schools offer courses in sociology, history, or philosophy of law; or in psychology or anthropology of law. But everybody knows that these are elegant frills, like thick rugs in the dean's office; they have nothing to do with 'real' legal education. ").
41. But see Roger C. Cramton, "The Most Remarkable Institution": The American Law Review, 36 J. LEGAL EDUC. 1, 9 (1986) ("Empirical studies dealing with legal institutions or the legal profession also find their way increasingly into new specialized faculty-edited journals."). Ironically, the first use of the term Empirical Legal Studies to mark a particular way of studying law also dates from roughly the same time.
42. See David M. Trubek, Where the Action Is: Critical Legal Studies and Empiricism, 36 STAN. L. REV. 575, 585 (1984) ("[N]ondoctrinalists, influenced by a pragmatic concept of law, aware of the fact that legal rules often were only of marginal impact in daily life, and affected by positivist concepts of knowledge imbibed through contact with their social science allies, tended to dhink of themselves as 'empiricists' and to champion empirical legal studies.").
43. For example, Chicago's well-known law and economics orientation has long boasted Bill Landes, a trained economist, and Judge Richard Posner, both of whom have consistendy produced empirical legal scholarship for many decades. See The Faculty, U. CHI.: L. SCH., http://www.law.uchicago.edu/people/faculty (follow hyperlinks "L" and "P") (last visited Mar. 11, 2011). The first volume of Chicago's Journal of Legal Studies, published in 1972 and cofounded by Judge Posner, included several notable empirical contributions.
44. See, e.g., Richard A. Posner, A Theory of Negligence, 1 J. LEGAL STUD. 29 (1972) (presenting a somewhat systematic review of cases in which courts relied on implicit economic theories). The University of Michigan Law School has long boasted two well-known empirical scholars in Phoebe Ellsworth, a psychologist, and Richard Lempert, a sociologist.
45. See Professors of Law, MICH. L.: U. MICH. L. SCH., http://web.law.umich. edu/-FacultyBioPage/index.asp (last visited Mar. 11, 2011);
46. see also Professors Emeriti, MICH. L.: U. MICH. L. SCH., http://web.law.umich.edu/-FacultyBioPage/index.aspPCategorp5 (last visited Mar. 11, 2011).
47. Northwestern University School of Law's web site boasts that the majority of its faculty have earned PhDs, eighteen of whom hold these degrees in social sciences that are empirically focused (e.g., political science, psychology, sociology, anthropology, and economics). See Faculty Research & Achievement, Nw. L., http://www.law.northwestern. edu/faculty/ (last visited Mar. 11, 2011). Not all of these are necessarily empirical scholars (political science and economics includes subdisciplines that are not primarily empirical), and the faculty includes scholars who produce empirical work but who do not hold PhDs. My identification of specific schools is meant only to be illustrative and is necessarily impressionistic. A specific accounting is obviously perilous, as it will inevitably be incomplete, and those omitted might feel slighted-for which I apologize. The listing in this Essay, for example, omits the law school faculties of the University of Southern California and New York University, which successfully hosted the fourth and the second Conference on Empirical Legal Studies, respectively, and which both have sizeable sets of empirical scholars. My listing also omits two of my own empirically minded coauthors: Mitu Gulati of Duke University School of Law and Chris Guthrie, the Dean of Vanderbilt University Law School, which has a large group of scholars devoted to studying human behavior and law.
48. See George, supra note 16, at 152-53 (reporting the percentage of faculty at top institutions who have social science PhDs);
49. Kathleen M. Sullivan, Foreword: Interdisciplinarity, 100 MICH. L. REV. 1217, 1222 (2002) ("[T]he increasingly empirical study of law is one of the most dramatic trends in recent legal scholarship. ").
50. See George, supra note 16, at 159-60 ("[T]he University of Pennsylvania, the University of Southern California, and Northwestern University have hired remarkable numbers of social scientists into entry-level posts.").
51. See id. at 152 tbl.2 (noting that for the 2003-04 academic year, over 20% of the tenure-track law faculty at the University of California, Berkeley; George Mason University; Northwestern University; the University of Pennsylvania; and Stanford University had doctorates in social science).
52. Id. at 149-50 ("Law professors with social science doctorates usually are better positioned to undertake empirical research than are other professors. Law schools generally do not teach courses in survey methodology, statistical analysis, or research design. Graduate social science programs do.").
53. See id. at 150 ("[A] law school with a greater proportion of its faculty holding social science doctorates is more likely to produce [empirical legal scholarship] than a law school with a lower proportion".).
54. See id. ("Non-social scientists benefit from the presence of social scientists for informal interactions, such as advice on how to undertake an empirical project or what method would be appropriate, and for formal collaborations, such as co-audiorship. ").
55. See Tracey E. George & Chris Gutiirie, Joining Forces: The Role of Collaboration in the Development of Legal Thought, 52 J. LEGAL EDUC. 559, 566 fig.4 (2002) (documenting the rate of coauthorship in various social science disciplines).
56. See George, supra note 16, at 150 ("[A] trend toward increased collaboration can be seen in law schools.").
57. See id.
58. PACER: PUB. ACCESS TO CT. ELECTRONIC RECS., http://www.pacer.gov/ (last visited Mar. 11, 2011) ("[PACER] is an electronic public access service that allows users to obtain case and docket information from federal appellate, district and bankruptcy courts....").
59. See George, supra note 16, at 150 (noting that law faculty can consult their colleagues trained in the social sciences for advice on appropriate methods, among other things).
60. See Cass R. Sunstein, Behavioral Analysis of Law, 64 U. CHI. L. REV. 1175, 1175 (1997) ("The future of economic analysis of law lies in new and better understandings of decision and choice."). (Pubitemid 127445777)
61. See Jeffrey J. Rachlinski, The "New" Law and Psychology: A Reply to Critics, Skeptics, and Cautious Supporters, 85 CORNELL L. REV. 739, 766 (2000) ("Economics provided law with a behavioral theory that is rigorous and precise, but lacks an empirical foundation. Psychology offers an empirical, scientific source for theories of human behavior. We have only begun to see how the scientific study of human behavior will reshape the study of law.").
62. See Sunstein, supra note 45, at 1175 ("[S]ocial scientists have learned a great deal about how people actually make decisions. Much of this work calls for qualifications of rational choice models. Those models are often wrong in the simple sense that they yield inaccurate predictions." (footnote omitted)).
63. See Sackett et al., supra note 3;
64. see also PFEFFER & SUTTON, supra note 5, at 13-14 (briefly discussing evidence-based medicine).
65. See PFEFFER & SUTTON, supra note 5, at 217 (arguing that many companies "are more interested in just copying odiers, doing what they've always done, and making decisions based on beliefs in what ought to work rather than on what actually works").
66. -408 U. S. 238, 240, 305, 310, 314, 370-71 (1972) (per curiam).
67. Id. at 370-71.
68. -428 U. S. 153, 207, 226-27 (1976).
69. See STUART BANNER, THE DEATH PENALTY: AN AMERICAN HISTORY 267 (2002) ("The Supreme Court's 1972 decision in Furman v. Georgia... touched off the biggest flurry of capital punishment legislation the nation had ever seen. ").
70. See John J. Donohue & Justin Wolfers, Uses and Abuses of Empirical Evidence in the Death Penalty Debate, 58 STAN. L. REV. 791, 792-94 (2005) (briefly reviewing some of the evidence on the death penalty).
71. See Carol S. Steiker, No, Capital Punishment Is Not Morally Required: Deterrence, Deontology, and the Death Penalty, 58 STAN. L. REV. 751, 752 (2005) (noting that one of the two principle arguments for the death penalty is retributive: "The first gambit is to consider in detail the facts of one or more capital murders and to propose that only the punishment of death is an adequate and proportional response to the terrible suffering of the victim intentionally inflicted by the perpetrator-a predominantly retributive argument").
72. See Donohue & Wolfers, supra note 54, at 792 (asserting that beliefs about deterrence affect public debate on the death penalty).
73. See BANNER, supra note 53, at 268 (stating that by 1976, supporters of the death penalty outnumbered opponents of the death penalty 65% to 28%).
74. See Donohue & Wolfers, supra note 54, at 792 (reviewing research on the deterrent effect of the death penalty).
75. See id. at 792-94.
76. See id.
77. See Hashem Dezhbakhsh et al., Does Capital Punishment Have a Deterrent Effect? New Evidence from Postmoratorium Panel Data, 5 AM. L. & ECON. REV. 344, 344 (2003).
78. Both Donohue and Wolfers and Dezhbakhsh used the Federal Bureau of Investigation's Uniform Crime Reports as primary sources of data to conclude that the deadi penalty does not and does deter violent crime, respectively.
79. See id at 360 ("The crime and arrest rates are from the Federal Bureau of Investigation's (FBI) Uniform Crime Reports.");
80. Donohue & Wolfers, supra note 54, at 796 ("[W]e present two series for homicides-one from the Uniform Crime Reports and the other compiled from Vital Statistics sources, based on death certificates."). Donohue and Wolfers explicitly attempted to replicate the results of Dezhbakhsh and his colleagues. See Donohue & Wolfers, supra note 54, at 805 ("Column 2 shows our replication attempt based on independendy collected data (but using the same sources).").
81. See Donohue & Wolfers, supra note 54, at 794 ("[T]he death penalty-at least as it has been implemented in the United States since Gregg ended the moratorium on executions-is applied so rarely that the number of homicides it can plausibly have caused or deterred cannot be reliably disentangled from the large year-to-year changes in the homicide rate caused by other factors.").
82. See Banner, supra note 53, at 281 ("Academic studies of deterrence had scarcely any impact, in any event, on the pervasive folk wisdom that the death penalty had to have a deterrent effect, simply because it was more severe than any other.").
83. See Alan Berlow, The Texas Clemency Memos, ATLANTIC MONTHLY, Jul./Aug. 2003, at 91, 91 ("During Bush's six years as governor 150 men and two women were executed in Texas-a record unmatched by any other governor in modern American history.").
84. The Third Gore-Bush Presidential Debate: October 17, 2000 Debate Transcript, COMMISSION ON PRESIDENTIAL DEBATES, http://www.debates.org/index. php?page=october-17-2000-debate-transcript (last visited Mar. 11, 2011) (noting that when asked if he believed the death penalty deterred crime, Bush responded: "I do. It's the only reason to be for it. Let me finish, sir. I don't think you should support the death penalty to seek revenge. I don't think that's right. I think the reason to support the death penalty is because it saves other people's lives.").
85. Id.
86. See Jeffrey M. Jones, Support for the Death Penalty 30 Years After the Supreme Court Ruling, GALLUP (June 30, 2006), http://www.gallup. com/poU/23548/Support-Death-Penalty-Years-After-Supreme-Court-Ruling.aspx (reporting that, in 2003, 11% of those who supported the death penalty cited deterrence as the primary reason for their support).
87. BANNER, supra note 53, at 275 (noting that "[c]apital punishment's popularity held steady" from the 1970s to the 1990s).
88. See Robert Ruby, Capital Punishment's Consistent Constituency: An American Majority, PEW RES. CENTER (June 26, 2007), http://pewresearch.org/pubs/ 523/capital-punishments-constant-constituency-an-american-majority (noting the rising number of exonerations and stating that "[m]eanwhile, support for the death penalty among the public has declined somewhat from its peak in the mid 1990s"). But see id. (noting that support for the death penalty "still remains at a substantial 64%").
89. See Theodore Eisenberg & Martin T. Wells, Deadly Confusion: Juror Instructions in Capital Cases, 79 CORNELL L. REV. 1, 9 (1993) ("In addition to jurors' imposed ignorance of the sentencing alternative, their lack of understanding of the standards of proof applicable to mitigating circumstances and the required level of interjuror agreement also hamper the decisionmaking process."). (Pubitemid 24783462)
90. See Bryan Myers & Edith Greene, The Prejudicial Nature of Victim Impact Statements: Implications for Capital Sentencing Policy, 10 PSYCHOL. PUB. POL'Y & L. 492, 506-07 (2004) (noting that "[t]here is mounting empirical evidence that the status and character of the victim have a significant effect on jurors' decisions about life and death" and recommending that such victim impact evidence be limited or eliminated).
91. But see Theodore Eisenberg et al., Victim Characteristics and Victim Impact Evidence in South Carolina Capital Cases, 88 CORNELL L. REV. 306, 340 (2003) ("VIE [victim impact evidence] has a modest effect, if any, on sentencing outcomes.").
92. See William C. Thompson, Death Qualification After Wainwright v. Witt and Lockhart v. McCree, 13 LAW & HUM. BEHAV. 185, 185 (1989) ("Critics of deadi qualification have argued for many years that deadi qualified jurors (those who survive deadi qualification) are more conviction-prone than those who are excluded....").
93. See DAVID C. BALDUS, GEORGE WOODWORTH & CHARLES A. PULASKI, JR., EQUAL JUSTICE AND THE DEATH PENALTY: A LEGAL AND EMPIRICAL ANALYSIS 399-404 (1990) (describing evidence of race-of-victim discrimination but noting that there was weaker evidence of race-of-defendant discrimination post-Furman).
94. Stephen S. Meinhold & David W. Neubauer, Exploring Attitudes About the Litigation Explosion, 22 Just. Sys. J. 105, 106 (2001) (stating that "[i]t is now conventional wisdom that the United States suffers from a litigation explosion" but that social scientists have debunked this myth);
95. see Marc Galanter, The Day After the Litigation Explosion, 46 MD. L. REV. 3, 3-5 (1986) (documenting observations of an increasingly litigious society).
96. See WALTER K. OLSON, THE LITIGATION EXPLOSION: WHAT HAPPENED WHEN AMERICA UNLEASHED THE LAWSUIT 152-77 (1991) (discussing wildly disparate jury verdicts).
97. See id. at 184 (noting that in the context of automobile insurance, for example, people pay higher rates when there is more lawsuit exposure).
98. See Peter Kerr, 'Ghost Riders' Are Target of an Insurance Sting, N. Y. TIMES, Aug. 18, 1993, at Al.
99. See TOM BAKER, THE MEDICAL MALPRACTICE MYTH 43, 45 (2005) (stating that the standard refrain of the American Medical Association is that '[t]he tort system is out of control" and arguing that "when it comes to medical malpractice, the organized interests who want to downplay the impact of malpractice and exaggerate the impact of lawsuits have outhusded, outspent, and just plain beat the other team in mobilizing public opinion").
100. Mary Margaret Penrose & Dace A. Caldwell, A Short and Plain Solution to the Medical Malpractice Crisis: Why Charles E. Clark Remains Prophetically Correct About Special Pleading and the Big Case, 39 GA. L. REV. 971, 983-84 (2005) (noting that many states have enacted legislation that "heighten [s] the standard of pleading required to bring a medical malpractice action").
101. Catherine M. Sharkey, Unintended Consequences of Medical Malpractice Damages Caps, 80 N. Y. U. L. REV. 391, 412 (2005) (stating that, in 2005, twenty-nine states had compensatory damage caps).
102. See Baker, supra note 79, at 118-39 (discussing defensive medicine).
103. See id. at 5-6 ("[PJublic opinion remains firmly anchored to the view that... medical malpractice lawsuits contribute significandy to the high cost of health care in the United States."). But see CONG. BUDGET OFFICE, LIMITING TORT LIABILITY FOR MEDICAL MALPRACTICE 1 (2004) ("[M]alpractice costs account for less than 2 percent of [health care] spending.").
104. Liebeck v. McDonald's, No. CV-93-02419, 1994 WL 16777703 (N. M. Dist. Ct Jul. 29, 1994) (trial order).
105. *1 (N. M. Dist. Ct. Aug. 18, 1994).
106. See Michael J. Saks, Do We Really Know Anything About the Behavior of the Tort Litigation System-and Why Noll 140 U. PA. L. REV. 1147, 1149 (1992) ("Much of what we think we know about the behavior of the tort litigation system is untrue, unknown, or unknowable.").
107. See Marc Galanter, Real World Torts: An Antidote to Anecdote, 55 MD. L. REV. 1093, 1103-05 (1996).
108. See id. at 1113-15 (suggesting that while the volume of awards has been increasing, this could be because "the composition of the pool of tried cases has changed" and "the cost of compensating victims has changed").
109. See Baker supra note 79, at 45 (stating that in the medical malpractice context, it is "[n]ot crazy, but not right" to "think that malpractice lawsuits are the reason for the insurance premium hikes").
110. See Theodore Eisenberg & Geoffrey P. Miller, Attorney Fees in Class Action Settlements: An Empirical Study, 1 J. EMPIRICAL LEGAL STUD. 27, 28 (2004) ("Contrary to popular belief, we find no robust evidence that either recoveries for plaintiffs or fees of their attorneys as a percentage of the class recovery increased during the time period studied.").
111. See Kevin M. Clermont & Theodore Eisenberg, Litigation Realities, 88 CORNELL L. REV. 119, 147-48 fig.5 (2002) (arguing that trial awards are not out of control and noting that the ratio of the mean personal injury tort award to the mean general contract award has actually been decreasing).
112. See Theodore Eisenberg et al., Juries, Judges, and Punitive Damages: Empirical Analyses Using the Civil Justice Survey of State Courts 1992, 1996, and 2001 Data, 3 J. EMPIRICAL LEGAL STUD. 263, 265 (2006).
113. See Theodore Eisenberg & Martin T. Wells, Trial Outcomes and Demographics: Is There a Bronx Effect?, 80 TEX. L. REV. 1839 (2002).
114. See Saks, supra note 86, at 1161 ("Anecdotes have a power to mislead us into thinking we know things that anecdotes simply cannot teach us.").
115. See David A. Hyman et al., Estimating the Effect of Damage Caps in Medical Malpractice Cases: Evidence from Texas, 1 J. LEGAL ANALYSIS 355, 356 (2009) (noting that thirty states have damage caps).
116. Prison Litigation Reform Act of 1995, Pub. L. No. 104-134, 110 Stat 1321-66 to 1321-77.
117. Private Securities Litigation Reform Act of 1995, Pub. L. No. 104-67, 109 Stat. 737 (codified in scattered sections of 15 U. S. C.).
118. See Kevin M. Clermont & Stephen C. Yeazell, Inventing Tests, Destabilizing Systems, 95 Iowa L. Rev. 821, 826-31 (2010) (reviewing recent changes to pleading practice).
119. Class Action Fairness Act of 2002, Pub L. No. 109-2, 119 Stat. 4 (codified as amended in scattered sections of 28 U. S. C.);
120. see Kevin M. Clermont & Theodore Eisenberg, CAFA Judicata: A Tale of Waste and Politics, 156 U. Pa. L. Rev. 1553, 1554-55 (2008) (noting that CAFA's intent was to "defeat[] the plaintiffs' bar's manipulation of state courts" by "funnel[ing] more class actions away from the state courts and into the federal courts, and perhaps thereby to discourage class actions").
121. See supra notes 75-93 and accompanying text.
122. See PFEFFER & SUTTON, supra note 5, at 13-14 (arguing that in almost any field, inefficient practices persist).
123. See Steiker, supra note 55, at 755.
124. See BANNER, supra note 53, at 280-81 (describing studies on the deterrent effect of the death penalty and stating that "[a] few studies found a deterrent effect, but most did not").
125. See Theodore Eisenberg et al., Juries, Judges, and Punitive Damages: An Empirical Study, 87 CORNELL L. REV. 743, 750-79 (2002) (subjecting claims regarding damage awards to empirical analysis).
126. See Theodore Eisenberg, U. S. Chamber of Commerce Liability Survey: Inaccurate, Unfair, and Bad for Business, 6 J. EMPIRICAL LEGAL STUD. 969, 1002 (2009) (arguing that "[t]he Chamber [of Commerce] needs devils to justify its tort reform budget" and to promote legislation).
127. See Robert L. Habush, The Insurance "Crisis": Reality or Myth? A Plaintiffs' Lawyer's Perspective, 64 DENY. U. L. REV. 641 (1988).
128. See Carol S. Steiker & Jordan M. Steiker, Abolition in Our Time, 1 OHIO ST. J. CRIM. L. 323, 342 (2003) ("Willie Horton's face [was] so powerfully used in the late 1980s as the symbol of what was wrong with American crime policy....").
129. See Emily Bazelon, Arguing Three Strikes, N. Y. TIMES, May 23, 2010, § 6 (Magazine), at 40 (describing the case of Norman Williams, who was sentenced to life in prison under California's repeat-offender law for stealing a floor jack from a tow truck).
130. See Adam Liptak, Long Term in Drug Case Fuels Debate on Sentencing: Mandatory Minimums Called Too Tough, N. Y. TIMES, Sept 12, 2004, at N20 (describing the case of a man facing over fifty years in prison for selling several hundred dollars worth of marijuana).
131. See Andrea Gerlin, A Matter of Degree: How a Jury Decided That a Coffee Spill Is Worth $2. 9 Million, Wall St. J., Sept. 1, 1994, at A1.
132. Id.
133. See Dan M. Kahan, The Cognitively Illiberal State, 60 STAN. L. REV. 115, 122-23 (2007) (noting that some "support a relatively hierarchical social order" and are more likely to worry about "forms of behavior that denigrate traditional, stratifying norms" (emphasis omitted)).
134. See id. at 122.
135. See id at 122-25.
136. See id. at 122-42.
137. Id.
138. Id.
139. Memorandum from Andrea Kuperman, Rules Law Clerk to Judge Lee H. Rosenthal, to the Civil Rules Comm. and the Standing Rules Comm. (July 26, 2010), available at http://www.uscourts.gov/uscourts/RulesAndPolicies/rules/ Iqbal-memo-072610.pdf.
140. U. S. SENTENCING COMM'N, FINAL REPORT ON THE IMPACT OF UNITED STATES V. BOOKER ON FEDERAL SENTENCING (2006).
141. Office of Management and Budget: About OIRA, WHITEHOUSE. GOV, http://www.whitehouse.gov/omb/inforeg-administrator/ (last visited Mar. 11, 2011).