When 10 trials are better than 1000: An evidentiary perspective on trial sampling

University of Pennsylvania Law Review

Volumn 160, Issue 4, 2012, Pages 955-965

  Cheng, Edward K ^a,b  

^a VANDERBILT UNIVERSITY LAW SCHOOL   (United States)

^b Och Spine at New York Presbyterian Hospitals   (United States)

Author keywords

[No Author keywords available]

Indexed keywords

EID: 84859566677     PISSN: 00419907     EISSN: 19428537     Source Type: Journal    
DOI: None     Document Type: Review

References (25)

1. 131 S. Ct. 2541 (2011).
2. See In re World Trade Ctr. Disaster Site Litig., 598 F. Supp. 2d 498, 503-05 (S.D.N.Y. 2009) (ordering a complex procedure to select a representative sample of cases for full discovery and early trial from the more than 9000 total cases filed).
3. Schwab v. Philip Morris USA, Inc., 449 F. Supp. 2d 992 (E.D.N.Y. 2006), rev'd sub nom.
4. McLaughlin v. Am. Tobacco Co., 522 F.3d 215 (2d Cir. 2008).
5. See, e.g., Shari Seidman Diamond, Survey Research (discussing the use of sampling surveys in a variety of contexts, including employment discrimination, juror bias, obscenity cases, and trademark litigation), in 1 MODERN SCIENTIFIC EVIDENCE: THE LAW AND SCIENCE OF EXPERT TESTIMONY 479, 483-86 (David L. Faigman et al. eds., 2010-2011 ed. 2010).
6. See generally 2 JOSEPH L. GASTWIRTH, STATISTICAL REASONING IN LAW AND PUBLIC POLICY 483-528 (1988) (describing other uses of sampling in the legal system, including the adjudication of false labeling and gender-based equal protection claims).
7. See, e.g., McLaughlin, 522 F.3d at 220, 231 (rejecting the district court's proposal to use sample trials to determine aggregate liability in litigation over whether "light" cigarettes were advertised deceptively);
8. Cimino v. Raymark Indus., 151 F.3d 297, 319-20 (5th Cir. 1998) (rejecting the use of sampling in asbestos cases).
9. But see Dukes v. Wal-Mart Stores, Inc., 603 F.3d 571, 625 n.53 (9th Cir. 2010) (noting an exception to the preference for individual hearings as a means to determine each claimant's damages when individualized evidence is difficult to obtain), rev'd, 131 S. Ct. 2541 (2011);
10. Hilao v. Estate of Marcos, 103 F.3d 767, 786 (9th Cir. 1996) (noting that the sampling "methodology in determining valid claims [was] unorthodox. . [but could] be justified by the extraordinarily unusual" circumstances).
11. See generally Laurens Walker & John Monahan, Sampling Evidence at the Crossroads, 80 S. CAL. L. REV. 969, 974-79 (2007) (providing an overview of mass tort cases that used sampling and arguing that Judge Weinstein's proposed use of sampling in McLaughlin marked a step toward more efficient damage awards).
12. See Alexandra D. Lahav, Bellwether Trials, 76 GEO. WASH. L. REV. 576, 581 (2008).
13. See Dukes, 603 F.3d at 625, 627 (expressing "no opinion" about the district court's specific sampling plan, but referring to sampling as "one method" to manage damage calculations in a large class action).
14. For two earlier cross-cutting discussions of these issues, see Robert G. Bone, Statistical Adjudication: Rights, Justice, and Utility in a World of Process Scarcity, 46 VAND. L. REV. 561, 598-617 (1993)
15. and Michael J. Saks & Peter David Blanck, Justice Improved: The Unrecognized Benefits of Aggregation and Sampling in the Trial of Mass Torts, 44 STAN. L. REV. 815, 833-41 (1992).
16. See FED. R. CIV. P. 23(a)(2) (allowing class actions to proceed where "there are questions of law or fact common to the class there are questions of law or fact common to the class");
17. see also 28 U.S.C. § 1407 (2006) (providing for coordinated or consolidated pretrial proceedings when "one or more common questions of fact are pending in different districts").
18. See 751 F. Supp. 649, 653 (E.D. Tex. 1990) (describing a system for determining damages in which sample cases from each of five disease categories went before a jury and all other class members received the average award of those sample cases according to their disease category), aff'd in part, vacated in part, 151 F.3d 297 (5th Cir. 1998).
19. Bradley Efron & Carl Morris, Stein's Paradox in Statistics, SCI. AM., May 1977, at 119.
20. See Charles Stein, Inadmissibility of the Usual Estimator for the Mean of a Multivariate Normal Distribution (mathematically proving the inadmissability of the usual estimator), in 1 PROCEEDINGS OF THE THIRD BERKELEY SYMPOSIUM ON MATHEMATICAL STATISTICS AND PROBABILITY 197, 199-200 (1956);
21. see also BRADLEY EFRON, LARGE-SCALE INFERENCE: EMPIRICAL BAYES METHODS FOR ESTIMATION, TESTING AND PREDICTION 1-12 (2010) (discussing James-Stein estimators generally).
22. The effectiveness of shrinkage turns out to apply even when one uses completely unrelated data. This "James-Stein Paradox" is the subject of various commentaries, although they are not applicable here. See, e.g., Stephen M. Stigler, The 1988 Neyman Memorial Lecture: A Galtonian Perspective on Shrinkage Estimators, 5 STAT. SCI. 147, 148-49 (1990) (recasting the James-Stein estimator as a regression problem). The key insight for our purposes is that an excessive focus on individualized determinations may be suboptimal in the long run. Considering evidence about other, seemingly unrelated, individuals or individual cases can improve the overall accuracy of results.
23. Alexandra D. Lahav, Rough Justice 17 (Aug. 9, 2010) (unpublished manuscript), available at http://ssrn.com/abstract=1562677;
24. Harry Zvi Davis, Hershey H. Friedman & Jianming Ye, An Ancient Sampling Technique: Flawed, Surprisingly Good, or Optimal?, CHANCE, Mar. 2011, at 19, 19-20.
25. For example, stratified sampling techniques can capture some of the gains of extreme value sampling while mitigating some of the dangers. Further, if one really wanted to maximize effective sample size for estimating the mean, the median would be a terrific estimator (although it is unclear how one could estimate the median without full litigation). See generally Gang Zheng & Joseph L. Gastwirth, Where Is the Fisher Information in an Ordered Sample?, 10 STATISTICA SINICA 1267, 1275 (2000) (discussing where most of the information about the mean comes from in an ordered sample).