Allocating the risk of error: Its Role in the Theory of Evidence Law

Legal Theory

Volumn 13, Issue 2, 2007, Pages 129-164

  Nance, Dale A ^a  

^a CASE WESTERN RESERVE UNIVERSITY   (United States)

Author keywords

[No Author keywords available]

Indexed keywords

EID: 85010659520     PISSN: 13523252     EISSN: 14698048     Source Type: Journal    
DOI: 10.1017/S135232520707005X     Document Type: Review

References (42)

1. The book is an elaboration'with significant modification'of an earlier article that is both shorter and for the most part better written. See Alex Stein, The Refoundation of Evidence Law, 9 Can. J.L. & Jurisprudence 279 (1996).
2. The most conspicuous problem in Stein's scope-of-inquiry analysis lies in his attempt to distinguish the evidential from the procedural (8-9). Stein does not attempt to characterize the core procedural concerns, so one cannot tell how evidence is distinct from procedure. Evidence as a field should be understood as a subdivision of procedure, not something wholly distinct from it. If so, then distinguishing evidence from procedure is a matter of identifying those procedural concerns that are not evidential concerns and analyzing how those nonevidential concerns qualify or otherwise interact with evidential goals. Stein does not attempt such an analysis.
3. A more extended explanation of the best evidence principle is given later in Stein's book (39-40). There he correctly articulates that principle in terms of party obligations; it “requires each party to a civil or criminal litigation to produce the best evidence available.” Stein generously cites the work of your humble reviewer on this matter, and I take the opportunity to add one feature of the best evidence principle that Stein does not mention but which is critically important if one is to take that principle as a plausible interpretation of our adversarial practices. That feature is a qualified adversarial privilege under which a party is generally not obligated to produce some item of evidence that would otherwise be required under the principle when that item is reasonably available to the opponent, with special qualifications for criminal cases. See Dale A. Nance, The Best Evidence Principle, 73 Iowa L. Rev. 227, 242-243 (1988).
4. As noted above, Stein places the best evidence principle at the center of the epistemology of trials. But the best evidence principle that he recognizes is certainly not amoral, at least not when it is applied to adversarial adjudication. In that context, it is not simply the principle that would govern a unified managerial authority facing no relevant moral issues'only prudential ones'governing the process of making a decision. Rather, it is a practical principle that governs human interaction. In structuring the relations among litigants and tribunal in an adversary system, the best evidence principle speaks the language of morality, a language of obligation and of sanctions for violation of obligation. See A more extended explanation of the best evidence principle is given later in Stein's book (39-40) note 5.
5. See, e.g., William Twining, Evidence and Legal Theory, 47 Mod. L. Rev. 261, 272 (1984). In an earlier article, Stein states his point more conservatively: “To determine the conditions under which litigants may be exposed to this risk [of error] is one of the central tasks of evidence law.” See Stein, As noted above, Stein places the best evidence principle at the center of the epistemology of trials note 2, at 294. Thus stated, the claim is both true and important.
6. This idea seems to be expressed in Stein's earlier work. See Stein, Evidence and Legal Theory note 2, at 322. It might be the case, for example, that procedural rules not falling within the domain of evidence law that Stein embraces'rules of discovery, for example'do most of the work of accuracy enhancement, and that evidence rules are thus freed to address the risk-allocation problem. Stein does not make such a claim explicitly, however. It would implicate the distinction between procedure and evidence law proper. See Evidence and Legal Theory note 3.
7. Part of the problem here is the ambiguous logical status of the two “specific requirements” constituting or derived from PMI. Drawing on a distinction articulated by Ronald Dworkin (with which Stein is clearly familiar; 11), are they to be regarded as principles that can be balanced against contending principles? Or are they rules to be applied according to their terms? See Ronald Dworkin, Taking Rights Seriously 14-130 (1978). Their collective name (the principle of maximal individualization) suggests the former. The phrase “specific requirements” suggests the latter, especially in connection with a specific procedural consequence for the violation of the second requirement. Does the omission of a specific procedural consequence with regard to the first requirement mean that Stein understands the first requirement as a principle and the second as a rule? As indicated in the text, on balance I conclude that Stein intends both “specific requirements” to be taken as principles in Dworkin's sense.
8. See Part of the problem here is the ambiguous logical status of the two “specific requirements” constituting or derived from PMI note 5.
9. Stein cites the familiar Smith v. Rapid Transit, 58 N.E.2d 754 (1945). For a case that tends in the opposite direction but is no more conclusive on the matter, see Kaminsky v. Hertz Corp., 288 N.W.2d 426 (Mich. App. 1980).
10. In several places Stein defines “quantitative sufficiency” in terms of whether the evidence “covers every segment of the relevant factual allegations,” and “qualitative sufficiency” in terms of whether the evidence “eliminates any dependency on the information that determines its credibility” (64, 121). The relationships between these parameters and the two requirements of PMI are not entirely obvious. For example, it would seem that parties might introduce all the case-specific evidence without covering every segment of the relevant allegations; conversely, it would seem that parties might introduce evidence covering every segment of the relevant allegations without introducing all case-specific evidence. All that Stein says about the matter is that the two PMI requirements constitute “ideal” versions of the two sufficiency standards (121).
11. I qualified the claim that such trace evidence could fill the gap because of Stein's analysis of DNA evidence, another kind of trace evidence (86-88). That analysis suggests that Stein might say that the combination of two kinds of probabilistic evidence (e.g., the base rates of blue and red buses in the area and tire-track evidence accompanied by a random match probability, assuming that there is any possibility of a match with tires from a bus not owned by the Blue Bus company) would remain insufficiently weighty. This problem points to an important difficulty for Stein's theory: it is not clear whether Stein would ever allow a verdict for the claimant based only on circumstantial evidence. Stein does not squarely address the issue. (The example he offers as satisfying weight demands involves DNA evidence coupled with direct evidence, specifically, eyewitness testimony of the claimant; 87.)
12. The concept of the burden of production, as distinct from the burden of persuasion and standards of proof, does not appear until late in Stein's book, well after his discussion of the Blue Bus case and similar problems. At that point, in two short paragraphs, Stein presents two quite different accounts of the content of the burden of production without any plausible explanation of the doctrinal difference or relationship between the two, except to say that they are motivated by common policy concerns. (He seems to say, quite implausibly, that the first account governs pretrial and the second governs trial; 154.) The usual articulation of the burden of production speaks in terms of whether the trier of fact can reasonably believe that the burden of persuasion has been satisfied. (This is essentially Stein's first account of the burden of production.) Because that seems to be satisfied in many cases of naked statistical evidence, burden of production may not seem to be the right doctrinal category for explaining the ostensibly conventional result (what Stein calls a “direct dismissal”) in such cases, and perhaps that is why Stein does not mention this burden in his discussions of weight. I have argued, however, that considerations of weight are properly a distinct component of the burden of production, which would at least explain the doctrinal basis of the ostensibly conventional result. See Dale A. Nance, Evidential Completeness and the Burden of Proof, 49 Hastings L. Rev. 621 (1998). Stein's second account of the burden is obliquely related to this argument. See infra note 25.
13. It is tempting to think that Stein here adopts a frequentist theory of probability in which probabilities may be defined only in relation to classes of events and no probability can be ascribed to a single nonreplicable event. This, however, is not consistent with his allowance that it is rational to ascribe a “slim prospect of victory for individual participants” in the lottery. Moreover, Stein elsewhere endorses a more plausible theory of probability, at least as pertains to trials, that is nonfrequentist (40-42).
14. See L. Jonathan Cohen, The Probable and the Provable (1977). Stein rejects Cohen's claims qua epistemology (48-49), but as we shall see below, Stein does seem to endorse important but dubious aspects of Cohen's conception of evidential weight.
15. See, e.g., David H. Kaye, Do We Need a Calculus of Weight to Understand Proof beyond a Reasonable Doubt, 66 B.U. L. Rev. 657 (1986) (explaining this incorporation by the fact-finder)
16. See Nance, Do We Need a Calculus of Weight to Understand Proof beyond a Reasonable Doubt note 15, at 633-639. Stein pursues something along these lines in an earlier article: “Weight also cannot be factored into the probability calculus, e.g., by reducing the probability of a proposition not sufficiently supported by the evidence. This would unwarrantedly increase the probability of this proposition's negation; unwarrantedly'because scanty evidential support for proposition A does not entail, ipso facto, the existence of massive support for not-A.” Stein, Do We Need a Calculus of Weight to Understand Proof beyond a Reasonable Doubt note 2, at 303 n. 113.
17. In this context, “cogency” is a synonym for weight (82).
18. See, e.g., L. Jonathan Cohen, The Role of Evidential Weight in Criminal Proof, 66 B.U. L. Rev. 635, 640, 642 (1986). One comment late in Stein's book (which is quoted later in the text of this review) confirms this position, at least in regard to criminal cases (199).
19. Stein does not suggest anything like the proxy-preferential explanation. See Cohen note 20.
20. Several years ago, I developed just such a theory in an article that also maintains that verdicts must satisfy both probability standards and a preferential weight standard. See Nance, Stein does not suggest anything like the proxy-preferential explanation note 15.
21. In various places late in the book, Stein does acknowledge certain doctrines that could be related to the idea of a preferential PMI. He states, for example, that “the production burden requires the party with the best access to evidence to produce that evidence. Failure to do so leads to adverse inferences and sometimes to adverse rulings” (154). Similarly, Stein discusses remedies for “evidential damage,” damage done to evidence by a party or third person, but his panoply of remedies includes only shifting the burden of persuasion and a separate cause of action in tort (167-171). Later in the book, when he disaggregates the problems of criminal and civil litigation, Stein discusses “missing evidence” problems in terms that suggest the kind of moral analysis that centers on the reasons for not presenting certain evidence, but again this discussion is not tied to the burden of production (199, 240-241). And none of these ideas is developed systematically as a part of PMI or any other explicit criterion of weight. Cf. Stein, Several years ago note 2, at 316 (discussing but rejecting the kind of weight analysis that I here call “preferential”).
22. Stein acknowledges this in a footnote (173 n. 6).
23. There are many reasons consistent with the decision-theoretic model for believing that no single threshold probability could be appropriate for every criminal case and for believing that fact-finders should be left some discretion to adjust that threshold in an ad hoc manner for the case at hand. See Erik Lillquist, Recasting Reasonable Doubt: Decision Theory and the Virtues of Variability, 36 U.C. Davis L. Rev. 85 (2002). Furthermore, one can endorse a rule of thumb (e.g., a jury instruction) that directs the fact-finder to examine every plausible scenario of innocence for substantiating evidence that one would expect to receive from the defense if the scenario were true and even authorizes the fact-finder to ignore the scenario if such evidence is not forthcoming, all without thereby committing oneself to the proposition that a decision-theoretic threshold probability has no substantial role in the assessment of reasonable doubt.
24. Stein appears to believe (183) and tries to argue (176-177) that a strict (nonpreferential) PMI is entailed by the equal-best standard, but this argument simply does not work. That the state must do its “best” cannot require that the state must do what cannot be done. Ought implies can. Of course, the state can give perfect protection against the risk of erroneous conviction by not prosecuting anyone, but that option is clearly not what Stein has in mind.
25. In an earlier article Stein insists that even in criminal cases, “probability of guilt will have to be determined independently of the evidential weight.” Stein, Stein appears to believe (183) and tries to argue (176-177) that a strict (nonpreferential) PMI is entailed by the equal-best standard, but this argument simply does not work note 2, at 304. It is unclear why he seems to have changed his opinion on this point.
26. Stein might be generous in allowing something “specifically” to refute the alibi. For example, Stein does not address the question of whether the prosecution could satisfy this burden by adducing evidence that the alibi witness and the defendant are friends or relatives. Is bias too “general” to rebut “specifically,” as one might presume from Stein's discussion of the defects of character evidence, for example (183-187, 227-228)? If not, then his theory as applied to alibis will not produce bizarre results very often, only when the alibi witness is a stranger or hostile to the defendant. But clearly, for Stein, the “abstract” (i.e., not “specifically” evidenced) possibility that a stranger might have misidentified the defendant would not be enough for a conviction, even in combination with overwhelming affirmative evidence identifying the defendant as the perpetrator. (Throw in inculpatory DNA if you like; it makes no difference for Stein.) Perhaps, however, all it would take specifically to rebut the alibi is a cross-examination question that gets W to acknowledge that it is just barely possible he mistakenly identified the defendant. If so, then the permissibility of a conviction, even in the presence of overwhelming affirmative evidence of guilt, would depend on the willingness of the alibi witness to make such a concession on cross-examination. That still seems quite arbitrary.
27. One new wrinkle on these doctrines that Stein articulates is both questionable and illustrative of Stein's approach. It concerns the explanation of “equipoise” cases, cases in which the trier's assessment places the pertinent probability at or indistinguishably near 0.5. In one chapter, Stein explains the rule favoring defendant in the traditional way as a concession to pragmatic considerations, such as cost-containment in the face of the nonzero costs of putting the legal machinery in motion for the enforcement of judgments (145). And in the chapter devoted to civil litigation, he struggles with the fact that such considerations do not explain the placement of this equipoise risk on defendants in regard to affirmative defenses. He tries to explain this doctrine as a consequence of the equality principle, arguing that placement of the equipoise risk on defendants with regard to affirmative defenses creates a rough equality with the placement of the equipoise risk on plaintiffs for the plaintiff's case (221-222). But the great majority of cases'I hazard the guess'do not turn on the validity of affirmative defenses, so even rough equality does not seem to follow. If such equality were really the driving force, why not assign the equipoise burden of parts of (what is now regarded as) the plaintiff's case'for example, the causation element'to the defendant and comparable parts of the affirmative defenses to the plaintiff? In that case equality would not depend on the fortuity that an affirmative defense is in play. Of course, part of the explanation of the existing rules may be simple symmetry and inertia: we prescribe as to affirmative defenses in direct analogy to how the plaintiff's case is treated because it is easy that way. And at least some of the assignment of the equipoise risk to defendants, namely, for affirmative defenses that allege serious wrongdoing on the part of the plaintiff, can be explained in terms of the principle of civility, the principle that the law presumes people act in conformity with serious social norms. For some reason, Stein does not see civility as speaking to the risk of nonpersuasion in equipoise cases (222-223 n. 16). That, however, is one of its functions in civil cases, at least as I developed it. See Dale A. Nance, Civility and the Burden of Proof, 17 Harv. J.L. & Pub. Pol'y 647, 659-672 (1994).
28. This is particularly troublesome in that, by Stein's definition of naked statistical evidence, all (relevant) evidence is naked statistical evidence: “A piece of evidence is nakedly statistical when it applies to an individual case by affiliating that case to a general category of cases” (43).
29. Once again, this could be a proxy-preferential weight rule (see This is particularly troublesome in that note 20) based on the idea that it is so inherently unlikely, for example, that a confession cannot be corroborated with additional evidence of the crime, if one occurred, that we express an inexcusable preference for such additional evidence. That is, we refuse to credit the prosecution's claim that investigators could not find any.
30. Stein correctly associates this particular preemptive strategy with the best evidence principle (134-135). This, however, falsifies a claim, which he makes early in the book, that the best evidence principle, being epistemological, does not perform the nonepistemological function of allocating the risk of error (40). To the contrary, it does so quite explicitly. See also Once again notes 5 and 6.
31. I am unsure what Stein means by the requirement that the evidence “survive” such individualized examination. It could mean that the probative value of the evidence in question was unchanged by the examination, though that seems highly implausible. Alternatively, it might simply mean that the evidence in question, even in light of such critical examination, still has nontrivial probative value. That seems better. Presumably, it will be a very rare case in which the court determines that no reasonable fact-finder could believe the evidence has no (or insignificant) probative value after the examination in question has occurred. So as a guide to admissibility, the principal force of PMI lies in its other demand, that the evidence be “exposed to” maximal individualized testing. (Elsewhere Stein says the evidence must be “susceptible to” such testing [72], which clarifies that it is the opponent's option in an adversary system.)
32. Actually, Stein says that PMI itself requires this generosity to the accused (193). But I can see no reading of his articulation of PMI that suggests this. To be sure, he emphasizes that risk allocation is a “zero-sum game” (132), so one might think that if PMI is strictly applied to the prosecution, then it must a fortiori be applied only weakly to the defense. Yet a review of Stein's second PMI requirement shows that it could easily be applied to defense evidence if that were desired, and as strictly as it is applied to prosecution evidence. Not to do so is a choice, one that cannot be derived directly from PMI. Apparently Stein believes that the accused should be subject to the epistemological constraints of the best evidence principle but not to any allocation of risk that is I am unsure what Stein means by the requirement that the evidence “survive” such individualized examinationepistemic. See I am unsure what Stein means by the requirement that the evidence “survive” such individualized examination note 35.
33. This distinction roughly and intentionally parallels the distinction between testimonial and nontestimonial statements developed for the Confrontation Clause in Crawford v. Washington, 541 U.S. 36 (2004). (189)
34. See, e.g., Chambers v. Mississippi, 410 U.S. 284 (1973) (holding that accused's due-process rights require admission of highly reliable exculpatory hearsay not otherwise admissible under state law, at least when necessary because the accused has unsuccessfully attempted to elicit the same information by calling the declarant to testify)
35. Actually, I think this distinction can only make sense in terms of an explanation about why a particular opportunity is unavailable. The want of an opportunity, such as the opportunity to cross-examine a witness, is morally relevant to the question of imposing adverse procedural consequences on the proponent of the witness only when the proponent can be rightly held responsible for the loss of that opportunity. If that is not the case, then the absence of such an opportunity is no different, in a morally relevant sense, from any other feature of the evidential situation (such as the accidental death of an important witness who might have been adverse to the proponent) that tends to immunize the proponent's case from rebuttal. But I need not develop that thesis here. See Dale A. Nance, Rethinking Confrontation after Crawford, 2:1 Int'l Comment. on Evidence (online ed. 2004), available at http://www.bepress.com/ice/vol2/iss1/art2 (developing this idea in the context of Confrontation Clause implications for hearsay law).
36. See, e.g., Fed. R. Evid. 806 (authorizing impeachment of hearsay declarants as if they were testifying witnesses)
37. Cf. Old Chief v. United States, 519 U.S. 172 (1997).
38. Juries, for example, almost inevitably are required to interpret the law, or at least judges’ instructions on the law. See, e.g., Daryl Brown, Plain Meaning, Practical Reason, and Culpability: Toward a Theory of Jury Interpretation of Criminal Statutes, 96 Mich. L. Rev. 1199 (1998). Even more extreme forms of value-implementation authority can be and sometimes have been delegated to juries. Outright jury nullification of the law, for example, has a respected place in our jurisprudence, though perhaps a less prominent role than it once enjoyed. See, e.g., United States v. Dougherty, 473 F.2d 1113 (D.C. Cir. 1972)
39. My view is that case weight assessments should generally be case-specific, requiring ad hoc analysis that does not lend itself well to precedents, and that they should be preferential in the sense articulated here. Because of their training, judges are in a better position to make those assessments than juries, and juries can be assisted by improvements in case weight that can result from interaction between the judge and the litigants regarding the weight issue, interaction of a kind that is not generally or easily allowed between juries and litigants. Consequently, assessments of the adequacy of the evidential weight should generally be done by trial judges, with little precedential value beyond fairly abstract principles. See Nance, Juries, for example, almost inevitably are required to interpret the law, or at least judges’ instructions on the law note 15.
40. See Jeremy Bentham, 3 Rationale of Judicial Evidence 393-593 (J.S. Mill ed., 1827).
41. See Nance, 3 Rationale of Judicial Evidence note 35 and accompanying text note 5, at 275.
42. See Jeremy Bentham, 4 Rationale of Judicial Evidence 477-645 (J.S. Mill ed., 1827). What Bentham was up against can be seen from a perusal of any standard evidence treatise of the early nineteenth century. See, e.g., S.M. Phillipps, A Treatise on the Law of Evidence 13-102, 173-203 (1st ed. 1814) (summarizing English cases and devoting three times more space to witness competency than to hearsay, for example); Thomas Peake, A Compendium of the Law of Evidence 21-40, 195-246 (U.S. ed., from the 5th U.K. ed., 1824) (summarizing American and English cases and devoting more than twice as much space to witness competency as to hearsay, for example).