Exploring the proof paradoxes

Legal Theory

Volumn 14, Issue 4, 2008, Pages 281-309

  Redmayne, Mike ^a  

^a LONDON SCHOOL OF ECONOMICS   (United Kingdom)

Author keywords

[No Author keywords available]

Indexed keywords

EID: 77955258198     PISSN: 13523252     EISSN: 14698048     Source Type: Journal    
DOI: 10.1017/S1352325208080117     Document Type: Article

References (98)

1. Good accounts include D.H. Kaye, Clarifying the Burden of Persuasion: What Bayesian Decision Rules Do and Do Not Do, 3 Evidence & Proof 1 (1999) D. Hamer, Probabilistic Standards of Proof, Their Complements, and the Errors That Are Expected to Flow from Them, 1 U. N. Eng. L.J. 71 (2004)
2. See, e.g., L.J. Cohen, The Probable and the Provable (1977), ch. 5; R.J. Allen & S.A. Jehl, Burdens of Persuasion in Civil Cases: Algorithms v. Explanations, 4 Mich. St. L. Rev. 893 (2003)
3. An excellent review of the literature is H.L. Ho, A Philosophy of Evidence Law: Justice in the Search for Truth (2008), at 135-143
4. This example originates in C.R. Nesson, Reasonable Doubt and Permissive Inferences: The Value of Complexity, 92 Harv. L. Rev. 1187 (1979)
5. The example is taken from R.A. Duff, Dangerousness and Citizenship, in Fundamentals of Sentencing Theory: Essays in Honour of Andrew von Hirsch (A. Ashworth & M. Wasik eds., 1998)
6. A variation on Summers v. Tice, 33 Cal.2d 80, 199 P.2d 1 (1948). In the actual case, the court found both defendants liable. This version is taken from J.J. Thomson, Liability and Individualized Evidence, in Rights, Restitution and Risk: Essays in Moral Theory (W.A. Parent ed., 1986)
7. The Shonubi case gave rise to a number of different judgments. For an introduction to the litigation, see A.J. Izenman, Introduction to Two Views on the Shonubi Case, in Statistical Science in the Courtroom (J.L. Gastwirth ed., 2000)
8. G.L. Wells, Naked Statistical Evidence of Liability: Is Subjective Probability Enough?, 62 J. Personality & Soc. Psychol. 739 (1992)
9. A useful review of the case law is J.J. Koehler, When Do Courts Think Base Rate Statistics Are Relevant?, 42 Jurimetrics J. 373 (2002)
10. Mass. 469, 58 N.E.2d 754 (1945)
11. United States v. Shonubi, 998 F.2d 84 (2d Cir. 1993); see also United States v. Shonubi, 103 F.3d 1085 (2d Cir. 1997)
12. Wells, United States v. Shonubi note 8
13. See, generally, F. Schauer, Profiles, Probabilities, and Stereotypes (2003), ch. 3
14. See Shonubi, 998 F.2d at 89
15. See, e.g., R.A. Posner, An Economic Approach to the Law of Evidence, 51 Stan. L. Rev. 1477 (1999), at 1509
16. See Wells, An Economic Approach to the Law of Evidence note 8
17. See, e.g., Posner, An Economic Approach to the Law of Evidence note note 15, at 1510
18. For an overview, see M. Redmayne, Objective Probability and the Law of Evidence, 2 Law, Probability & Risk 275 (2003) 10.1093/lpr/2.4.275
19. See, e.g., B. de Finetti, Probabilism: A Critical Essay on the Theory of Probability and on the Value of Science, 31 Erkenntnis 169 (1989)
20. See M. Kaplan, Decision Theory as Philosophy (1996), at 85-88
21. See, e.g., D. Gillies, Philosophical Theories of Probability (2000), at 119-125
22. While the reference-class problem is generally associated with frequentist theories of probability, it can be molded to fit other theories of probability, too, with the exception of the extreme subjectivism rejected above. See A. Hájek, The Reference Class Problem Is Your Problem Too, 156 Synthese 563 (2007)
23. R.J. Allen & M.S. Pardo, The Problematic Value of Mathematical Models of Evidence, 36 J. Legal Stud. 107 (2007) M. Colyvan, H.M. Regan & S. Ferson, Is It a Crime to Belong to a Reference Class? 9 J. Pol. Phil. 168 (2001), reprinted in Probability Is the Very Guide of Life: The Philosophical Uses of Chance (H.E. Kyburg & M. Thalos eds., 2003)
24. Allen & Pardo, The Problematic Value of Mathematical Models of Evidence note 24, at 109
25. Colyvan, Regan & Ferson, Allen & Pardo note 24, at 172
26. See M. Colyvan & H.M. Regan, Legal Decisions and the Reference Class Problem, 11 Evidence & Proof 274 (2007), at 279, where the toll-collector argument is described as a “bad defence.” This is not as obvious in the original article
27. Allen & Pardo, Legal Decisions and the Reference Class Problem note 24 at 109
28. See D.A. Schum, Evidential Foundations of Probabilistic Reasoning (1994), esp. ch. 3. See also Schauer, Allen & Pardo note 13
29. They appear to allow that the reference-class problem can be overcome if the reference class is “homogenous”; it is not clear what this means, given that no human classes will ever be completely homogenous. In reality, there is some evidence to suggest that organized drug smugglers have little choice about the amount they smuggle, and this is likely to undermine the importance of factors personal to the smuggler in determining the amount swallowed; A.J. Izenman, Assessing the Statistical Evidence in the Shonubi Case, in Statistical Science in the Courtroom (J.L. Gastwirth ed., 2000), at 423-424
30. See Gillies, They appear to allow that the reference-class problem can be overcome if the reference class is “homogenous”; note 22, at 122-125
31. See A. Hájek, Conditional Probability Is the Very Guide of Life, in Probability Is the Very Guide of Life: The Philosophical Uses of Chance (H.E. Kyburg & M. Thalos eds., 2003)
32. See, in particular, Cohen, Conditional Probability Is the Very Guide of Life note 2. Much of the debate sparked by this work can be found in Probability and Inference in the Law of Evidence (P. Tillers & E. Green eds., 1988), which collects papers originally published at 66 B.U. L. Rev. 377-952 (1986)
33. See L.J. Cohen, The Dialogue of Reason: An Analysis of Analytical Philosophy (1986), ch. 3; see also the discussion, followed by commentary, in L.J. Cohen, Can Human Irrationality Be Experimentally Demonstrated?, 4 Behav. & Brain Sci. 317 (1981)
34. See Cohen, Dialogue of Reason, The Dialogue of Reason: An Analysis of Analytical Philosophy note 34, at 165-192. See also R.J. Allen, On the Significance of Batting Averages and Strikeout Totals, 65 Tul. L. Rev. 1093 (1991)
35. Cohen, Dialogue of Reason, Dialogue of Reason note 34, at 165.
36. Another way of making the point is that the factors are explanatory; explanation plays a significant role in Allen & Pardo's discussion of the reference-class problem. (Allen & Pardo, Dialogue of Reason note 24); indeed, they would argue that the fundamental structure of proof is explanatory (see M.S. Pardo & R.J. Allen, Juridical Proof and the Best Explanation, 27 Law & Phil. 223 (2008)). This line of argument is not explored here, because in this context it adds nothing to causal accounts of the paradoxes
37. Cohen, Dialogue of Reason, Another way of making the point is that the factors are explanatory; explanation plays a significant role in Allen & Pardo's discussion of the reference-class problem note 34, at 167
38. Cohen comes close to accepting this: see id. at 166 for his “freshman” example; and discussion id. at 175, 177. But the precise point in the text is not explicitly discussed in Dialogue, where Cohen's emphasis is on psychological experiments that involve combining probabilities, and his argument is that one probability is more counterfactualizable than another and thus may rationally dominate calculations
39. Cohen does argue that the noncounterfactualizability of this probability is significant. When discussing experiments in which subjects see colored chips drawn from a bag and are asked to state the probability that the bag is the one in which chips of a particular color predominate, Cohen criticizes psychologists for presuming that subjects should think that the draws are random. Subjects, he argues, are right to “allow for the effect of any bias in how chips lie, or are drawable from, a particular bag,” and thus should let the prior probability of picking a bag containing mainly, e.g., red chips dominate their calculations, because “the drawing of the bag is stated to have been at random” (id. at 169). This is a rather gerrymandered argument: one can raise the same concerns about bias in how the bags lie or are chosen; and in any case, the possibility of bias in the bag does not tell us whether the bias will be towards red chips or blue chips (though there is presumably more likely to be a pocket of balls of the predominant color)
40. See T. McGrew, Direct Inference and the Problem of Induction, in Probability Is the Very Guide of Life: The Philosophical Uses of Chance (H.E. Kyburg & M. Thalos eds., 2003); S. Campbell & J. Franklin, Randomness and the Justification of Induction, 138 Synthese 79 (2004) J.E. Adler, Belief's Own Ethics (2002), at 250-254
41. In The Probable and the Provable (Cohen, Direct Inference and the Problem of Induction note 2), Cohen's discussion is in terms of weight, but by the time of An Introduction to the Philosophy of Induction and Probability, he suggests that weight can be analyzed in terms of counterfactualizability (L.J. Cohen, An Introduction to the Philosophy of Induction and Probability (1989), at 109 n. 22)
42. D.T. Wasserman, The Morality of Statistical Proof and the Risk of Mistaken Liability, 13 Cardozo L. Rev. 935 (1991), at 943
43. Duff, Id note 5
44. See V. Tadros, Criminal Responsibility (2005), at 47-53. Duff discusses this theory of character in more detail in R.A. Duff, Criminal Attempts (1996), at 175-192
45. Duff, Criminal Responsibility note 5, at 155. In this context, the idea of a presumption of harmlessness seems to be due to J. Floud & W. Young, Dangerousness and Criminal Justice (1981), at 43-45. While Floud and Young view the presumption as an important moral constraint on incapacitation, they see it as a restraint on practice (only those convicted of an offence should be considered for incapacitation on the basis of likely future offending) and do not discuss the issue considered here: whether the presumption should constrain the sort of evidence we use to judge dangerousness
46. Duff, Criminal Responsibility note 5 note 5, at 156.
47. Indeed, in one sense lawbreakers are more autonomous than the (literally) heteronomous law-abiding, and there is a certain romantic view of crime under which offending is an expression of freedom. Lurking here, however, are difficult questions about the meaning of autonomy. On a Kantian view of autonomy, the opposite conclusion would be reached, for lawbreakers lack autonomy because they cannot will their crimes as universal law. On this account of autonomy, see O. O'Neill, Autonomy, Coherence and Independence, in Liberalism, Citizenship, and Autonomy (D. Milligan & W.W. Miller eds., 1992). It is clear from context, however, that neither Wasserman nor Duff are using “autonomy” in this Kantian sense. Nor is there any need to settle debates about the meaning of autonomy for the purposes of the present discussion; Kantians are free to replace references to autonomy in the text with references to some other concept, such as independence.
48. See J. Raz, The Morality of Freedom (1986), at 155-156; Patricia Smith, Liberalism and Affirmative Obligation (1998), at 87-88
49. Peter Tillers also claims that a moral argument underlies Colyvan, Regan & Ferson's claims, though he suggests a slightly different one. See P. Tillers, If Wishes Were Horses: Discursive Comments on Attempts to Prevent Individuals from Being Unfairly Burdened by Their Reference Classes, 4 Law, Probability & Risk 33 (2005)
50. As they acknowledge; Colyvan, Regan & Ferson, Peter Tillers also claims that a moral argument underlies Colyvan note 24, at 175
51. See, e.g., J.M. Doris, Lack of Character: Personality and Moral Behavior (2005). Cf. D.C. Funder, Personality Judgment: A Realistic Approach to Person Perception (1999)
52. See, e.g., I.H. Dennis, The Law of Evidence (3d ed. 2007), esp. ch. 2; P. Roberts & A. Zuckerman, Criminal Evidence (2004), esp. ch. 4; H.L. Ho, Justice in the Pursuit of Truth: A Moral Defence of the Similar Facts Rule, 35 Common L. World Rev. 51 (2006). Cf. L. Laudan, Truth, Error and Criminal Law: An Essay in Legal Epistemology (2006), esp. chs. 7, 9
53. Thomson, The Law of Evidence note 6
54. Cf. R.W. Wright, Causation, Responsibility, Risk, Probability, Naked Statistics, and Proof, 73 Iowa L. Rev. 1001 (1998), at 1059-1061
55. Explaining just when a conclusion is due to luck is obviously difficult; see discussion in the following subsection; and J. Greco, Knowledge as Credit for True Belief, in Intellectual Virtue: Perspectives from Ethics and Epistemology (M. DePaul & L. Zagzebski eds., 2007)
56. Thomson acknowledges Gilbert Harman for comments on the paper; Harman had already discussed the Lottery paradox in Gilbert Harman, Thought (1973), at 161
57. J. Hawthorne, Knowledge and Lotteries (2004)
58. A good analysis of justification is S. Haack, Evidence and Inquiry: Towards Reconstruction in Epistemology (1993)
59. See, further, S. Brewer, Scientific Expert Testimony and Intellectual Due Process, 107 Yale L.J. 1535 (1998), at 1596-1601
60. The analysis of “inferential” accounts earlier in the paper could be thought of as exploring whether a particular type of justification, such as a causal one, is intrinsic to legal verdicts, but it is difficult to find support for any such requirement. The brief discussion below of contextualist approaches says a little more about the degree of justification required for a true belief to amount to knowledge
61. See E. Gettier, Is Justified True Belief Knowledge?, 23 Analysis 121 (1963). A good discussion is L. Zagzebski, The Inescapability of Gettier Cases, 44 Phil. Q. 65 (1994)
62. See, e.g., R.A. Duff et al., The Trial on Trial: Volume Three: Towards a Normative Theory of the Criminal Trial (2007), at 89-91. For a skeptical response to this claim, see J.F. Beltrán, Legal Proof and Fact Finders’ Beliefs, 12 Legal Theory 293 (2006) at 301-303
63. See M.S. Pardo, The Field of Evidence and the Field of Knowledge, 24 Law & Phil. 321 (2005), at 322-323
64. While the example used by Pardo (id.) is suggestive, there are more complex examples of Gettier cases, and it is not clear that these raise antiliability intuitions. For a skeptical view on importing the Gettier requirement to verdicts, see Sorensen, The Field of Evidence and the Field of Knowledge note 59
65. This does raise the question of how high the justification criterion for knowledge should be. Beliefs can be more or less justified, but exactly how well justified does a belief have to be to count as knowledge? It is not easy to find a cutoff point, and some of the more radical analyses might allow something like weak knowledge that could be applied to civil cases; see S. Hetherington, Good Knowledge, Bad Knowledge: On Two Dogmas of Epistemology (2001). Contextualists also admit of variable standards for knowledge; contextualism is discussed briefly below, but it is doubtful that contextualists would go so low as the balance of probabilities.
66. D.K. Nelkin, The Lottery Paradox, Knowledge, and Rationality, 109 Phil. Rev. 373 (2000). Nelkin's article also points to another way of analyzing the proof paradoxes. Nelkin draws parallels between Lottery and a similar paradox that involves belief rather than knowledge: you hold a ticket in a large lottery; it is initially plausible to suppose that you believe that the ticket will not win, as it is very improbable that it will. But you should then be prepared to form the same belief about any ticket in the lottery; the conjunction of your beliefs then entails that you believe that no ticket will win, hence the paradox. Nelkin resolves the belief version of the Lottery paradox by arguing that you do not in fact believe that your ticket will lose because, again, of the missing causal element. One might then argue that what is required for verdicts is belief rather than knowledge and that it is belief that is missing in the proof paradoxes. This possibility is not pursued at length here, largely because Nelkin's argument about the lack of belief in the belief version of the Lottery paradox has received little support. See, further, D. Christensen, Putting Logic in Its Place: Formal Constraints on Rational Belief (2004). For a skeptical view of a belief requirement for verdicts, see Beltrán, This does raise the question of how high the justification criterion for knowledge should be note 70
67. Duncan Pritchard, Epistemic Luck (2005). See also the discussion in Duncan Pritchard, Knowledge, Luck and Lotteries, in New Waves in Epistemology (D. Pritchard & V. Hendricks eds., 2008)
68. For objections to this account, see A. Hiller & R. Neta, Safety and Epistemic Luck, 158 Synthese 303 (2007) J. Comesaña, Unsafe Knowledge, 146 Synthese 395 (2005) J. Lackey, Pritchard's Epistemic Luck, 56 Phil. Q. 284 (2006)
69. See Pritchard, Knowledge, Luck and Lotteries, For objections to this account note 75, at 46-48. Pritchard also suggests that I do not know that I will not be able to afford the expensive house if I own a lottery ticket. It is unclear whether Pritchard would stand by his claim in respect of Heart Attack if, for example, I have a history of sudden heart attacks in my family
70. For an actual example, see R v. Smith, Court of Appeal Criminal Division (UK), 8 February 2000
71. See Hiller & Neta, For an actual example note 76
72. See D. Nute, Topics in Conditional Logic (1980), at 65-73; R. Nozick, Invariances: The Structure of the Objective World (2001), at 148-155
73. See, e.g., Hawthorne, Topics in Conditional Logic note 63 E. Sosa, Relevant Alternatives, Contextualism Included, 119 Phil. Stud. 35 (2004)
74. Or at least, the responses of the philosophically literate. There have been no surveys of the responses of the general public to Lottery cases. Cf. J.M. Weinberg, S. Nichols & S. Stich, Normativity and Epistemic Intuitions, 29 Phil. Topics 429 (2001)
75. See, e.g., J. Stanley, Knowledge and Practical Interests (2005); B. Weatherson, Questioning Contextualism, in Aspects of Knowing: Philosophical Essays (S. Hetherington ed., 2006); W.A. Davies, Knowledge Claims and Context: Loose Use, 132 Phil. Stud 395 (2007)
76. T. Williamson, Knowledge, Context and the Agent's Point of View, in Contextualism in Philosophy (G. Preyer & G. Peters eds., 2005); T. Williamson, Contextualism, Context-Sensitive Invariantism, and Knowledge of Knowledge, 55 Phil. Q. 213 (2005)
77. That said, quantification does sometimes seem to cause courts to refuse to convict even when the evidence is very strong: see, e.g., R v. Watters, Court of Appeal Criminal Division (UK), 19 October 2000. It may be that quantification alone makes doubt seem more vivid, and that particular forms of quantification (as in Blue Bus) have effects over and above this
78. See Wells, That said, quantification does sometimes seem to cause courts to refuse to convict even when the evidence is very strong note 8
79. See K.E. Neidermeier, Jurors’ Use of Naked Statistical Evidence: Exploring the Basis and Implications of the Wells Effect, 76 J. Personality & Soc. Psy. 533 (1999). See also K.J. Heller, The Cognitive Psychology of Circumstantial Evidence, 105 Mich. L. Rev. 241 (2006)
80. J.J. Koehler, The Psychology of Numbers in the Courtroom: How to Make DNA Match Statistics Seem Impressive or Insufficient, 74 S. Cal. L. Rev. 1275 (2001)
81. J.L. Austin, Other Minds, in Philosophical Papers (3d ed., J.O. Urmson & G.J. Warnock ed., 1979). It is not really accurate to describe Austin's account as a theory of knowledge; it hardly aims at completeness and is developed in the context of the problem of knowledge of other minds. However, Mark Kaplan is currently filling out the Austinian account. See M. Kaplan, If You Know You Can't be Wrong, in Epistemology Futures (S. Hetherington ed., 2006)
82. Austin, Other Minds note 90, at 85
83. One stream in the epistemological literature picks up on this insight and takes the nonexistence of “defeaters” as a condition for knowledge. Most tantalizingly, for our purposes, we find this comment: “A belief is epistemically justified for knowledge, according to the “criminal” standard that we endorse, when one has strong reasons in support of it, no undefeated epistemic reason to doubt it, and no undefeated epistemic reason to believe that one's evidence for it is unreliable”; E. Conee & R. Feldman, Evidentialism: Essays in Epistemology (2004), at 296. However, this does not really give us a clear answer to the puzzles that concern us: in Prisoners, what makes the slim possibility of the prisoner being innocent a defeater, when the same slim possibility of an eyewitness mistake is not?
84. Austin, One stream in the epistemological literature picks up on this insight and takes the nonexistence of “defeaters” as a condition for knowledge. Most tantalizingly, for our purposes, we find this comment note 90, at 88. There are obvious connections here with contextualist approaches, noted in M. Kaplan, Deciding What You Know, in Knowledge and Inquiry: Essays on the Pragmatism of Isaac Levi (E.J. Olsson ed., 2006)
85. See E. Craig, Knowledge and the State of Nature (1990), at 2; S. Haack, One stream in the epistemological literature picks up on this insight and takes the nonexistence of “defeaters” as a condition for knowledge. Most tantalizingly, for our purposes, we find this comment note 90 note 66, at 7
86. See M. Kaplan, It's Not What You Know That Counts, 82 J. Phil. 350 (1988)
87. D can, of course, challenge the facts that he was near the crime scene, that the thread matched, and so on, but likewise, the blue bus company can challenge the fact that 60 percent of buses are blue. For actual cases that obviously call for conviction but where the evidence seems unchallengeable in the sense explored here, see R v. Straffen, [1952] 2 Q.B. 911; R v. Smith (1916) 11 Cr. App. R. 229; United States v. Veysey, 334 F.3d 600 (7th Cir. 2003). There are obvious connections here with Alex Stein's account of legal proof, though Stein arrives at the significance of challengeability by a different route; see A. Stein, Foundations of Evidence Law (2005); cf. M. Redmayne, The Structure of Evidence Law, 26 Oxford J. Legal Stud. 805 (2006) D.A Nance, Allocating the Risk of Error, 13 Legal Theory 129 (2007)
88. See J.L. Kvanvig, The Value of Knowledge and the Pursuit of Understanding (2003)
89. See J.L. Pollock & J. Cruz, Contemporary Theories of Knowledge (2d ed. 1999), at 13-15; Haack, The Value of Knowledge and the Pursuit of Understanding note 66, at 7; Kaplan, The Value of Knowledge and the Pursuit of Understanding note 95
90. Pollock & Cruz, Contemporary Theories of Knowledge note 98, at 14
91. And note that (see note 64 Pollock & Cruz), our knowledge that the ticket lost may be less certain than our justified true belief that it did
92. Cf. Weinberg, Nichols & Stich, note that (see note 64 Pollock & Cruz) note 82
93. See, e.g., W. Lycan, On the Gettier Problem Problem, in Epistemology Futures (S. Hetherington ed., 2006), at 165 n. 22. Hetherington, Nichols & Stich note 73, at 104, describes the lottery proposition as “excellent, but not perfect, knowledge.” Olsson suggests that Isaac Levi is committed to the view that there is knowledge in Lottery, though he himself states that “our lack of knowledge [in Lottery] is as clear a part of common sense as any other claim I can think of”; E.J. Olsson, Levi and the Lottery, in Knowledge and Inquiry: Essays on the Pragmatism of Isaac Levi (E.J. Olsson ed., 2006), at 174
94. P. Unger, Living High and Letting Die: Our Illusion of Innocence (1996)
95. C. Sunstein, Moral Heuristics, 28 Behav. & Brain Sci. 531 (2005)
96. See also F. Spicer, Epistemic Intuitions and Epistemic Contextualism, 72 Phil. & Phenomenological Res. 366 (2006) B. Weatherson, What Good Are Counterexamples?, 115 Phil. Stud 1 (2003)
97. Thus note, for example, David Lewis's response to Unger in D. Lewis, Illusory Innocence? in Papers Ethics & Soc. Phil. (2000); and Barbara Fried's to Sunstein in B. Fried, Moral Heuristics and the Means/End Distinction, 28 Behav. & Brain Sci. 549 (2005)
98. A similar point is made in D. Shaviro, Statistical Probability Evidence and the Appearance of Justice, 103 Harv. L. Rev. 530 (1989), at 543