DNA database searches and the legal consumption of scientific evidence

Michigan Law Review

Volumn 97, Issue 4, 1997, Pages 931-983

  Donnelly, Peter ^a   Friedman, Richard D ^a  

^a NONE

Author keywords

[No Author keywords available]

Indexed keywords

DNA;

ARTICLE; EXPERT WITNESS; FACTUAL DATABASE; HUMAN; LEGAL ASPECT; ORGANIZATION; STATISTICAL ANALYSIS; UNITED KINGDOM; UNITED STATES; UTILIZATION REVIEW;

DATA INTERPRETATION, STATISTICAL; DATABASES, FACTUAL; DNA; EXPERT TESTIMONY; GREAT BRITAIN; HUMANS; NATIONAL ACADEMY OF SCIENCES (U.S.); UNITED STATES;

EID: 0000630688     PISSN: 00262234     EISSN: None     Source Type: Journal    
DOI: 10.2307/1290377     Document Type: Article

References (156)

1. 509 U.S. 579 (1993).
2. 118 S. Ct. 512 (1997).
3. 293 F. 1013 (D.C. Cir. 1923).
4. 131 F.3d 1433 (11th Cir. 1997), cert. granted sub nom. Kumho Tire Co., Ltd. v. Carmichael, 118 S. Ct. 2339 (1998).
5. See Carey Goldberg, DNA Databanks Giving Police a Powerful Weapon, and Critics, N.Y. TIMES, Feb. 19, 1998, at A1: Of all the new thrills that DNA analysis offers forensic scientists, nothing seems to beat what they call a "cold hit": when a computer discovers the identity of a killer or rapist by matching DNA from blood, semen or saliva left at a crime scene with a DNA profile in a database. A criminal is fingered by his own genes. Until now, cold hits have come sporadically, mainly in several states where DNA forensic work is most advanced, totaling about 200 nationwide. But Federal and state experts say they will soon be cropping up much more often.
6. See Nicholas Wade, F.B.I. Set to Open Its DNA Database for Fighting Crime, N.Y. TIMES, Oct. 12, 1998, at A1; see also infra text accompanying note 27.
7. See Wade, supra note 6.
8. COMMITTEE ON DNA TECHNOLOGY IN FORENSIC SCIENCE, NATIONAL RESEARCH COUNCIL, DNA TECHNOLOGY IN FORENSIC SCIENCE (1992) [hereinafter NRC I].
9. Id. at 124, 129.
10. COMMITTEE ON DNA FORENSIC SCIENCE, NATIONAL RESEARCH COUNCIL, THE EVALUATION OF FORENSIC DNA EVIDENCE (1996) [hereinafter NRC II].
11. See N.E. Morton, The Forensic DNA Endgame, 37 JURIMETRICS J. 477, 487-92 (1997); Anders Stockmarr, Likelihood ratios for evaluating DNA evidence when the suspect is found through a database search (Research Rept. 98/1, Dept. of Biostatistics, University of Copenhagen) (to be published, in slightly revised form, in Biometrics (Sept. 1999)); Aidan Sudbury, Comment to David J. Balding & Peter Donnelly, Inference in Forensic Identification, 158 J. ROYAL STAT. SOCY. A, Part 1, at 21, 48-49 (1995); cf. Richard Lempert, After the DNA Wars: Skirmishing with NRC II, 37 JURIMETRICS J. 439, 461-62 (1997) [hereinafter Lempert, After the DNA Wars] (endorsing NRC I and criticizing NRC II with respect to database searches); Richard Lempert, Some Caveats Concerning DNA as Criminal Identification Evidence: With Thanks to the Reverend Bayes, 13 CARDOZO L. REV. 303, 333-34 (1991) [hereinafter Lempert, Some Caveats] (anticipating NRC I and proposing retesting requirement).
12. See N.E. Morton, The Forensic DNA Endgame, 37 JURIMETRICS J. 477, 487-92 (1997); Anders Stockmarr, Likelihood ratios for evaluating DNA evidence when the suspect is found through a database search (Research Rept. 98/1, Dept. of Biostatistics, University of Copenhagen) (to be published, in slightly revised form, in Biometrics (Sept. 1999)); Aidan Sudbury, Comment to David J. Balding & Peter Donnelly, Inference in Forensic Identification, 158 J. ROYAL STAT. SOCY. A, Part 1, at 21, 48-49 (1995); cf. Richard Lempert, After the DNA Wars: Skirmishing with NRC II, 37 JURIMETRICS J. 439, 461-62 (1997) [hereinafter Lempert, After the DNA Wars] (endorsing NRC I and criticizing NRC II with respect to database searches); Richard Lempert, Some Caveats Concerning DNA as Criminal Identification Evidence: With Thanks to the Reverend Bayes, 13 CARDOZO L. REV. 303, 333-34 (1991) [hereinafter Lempert, Some Caveats] (anticipating NRC I and proposing retesting requirement).
13. See N.E. Morton, The Forensic DNA Endgame, 37 JURIMETRICS J. 477, 487-92 (1997); Anders Stockmarr, Likelihood ratios for evaluating DNA evidence when the suspect is found through a database search (Research Rept. 98/1, Dept. of Biostatistics, University of Copenhagen) (to be published, in slightly revised form, in Biometrics (Sept. 1999)); Aidan Sudbury, Comment to David J. Balding & Peter Donnelly, Inference in Forensic Identification, 158 J. ROYAL STAT. SOCY. A, Part 1, at 21, 48-49 (1995); cf. Richard Lempert, After the DNA Wars: Skirmishing with NRC II, 37 JURIMETRICS J. 439, 461-62 (1997) [hereinafter Lempert, After the DNA Wars] (endorsing NRC I and criticizing NRC II with respect to database searches); Richard Lempert, Some Caveats Concerning DNA as Criminal Identification Evidence: With Thanks to the Reverend Bayes, 13 CARDOZO L. REV. 303, 333-34 (1991) [hereinafter Lempert, Some Caveats] (anticipating NRC I and proposing retesting requirement).
14. See N.E. Morton, The Forensic DNA Endgame, 37 JURIMETRICS J. 477, 487-92 (1997); Anders Stockmarr, Likelihood ratios for evaluating DNA evidence when the suspect is found through a database search (Research Rept. 98/1, Dept. of Biostatistics, University of Copenhagen) (to be published, in slightly revised form, in Biometrics (Sept. 1999)); Aidan Sudbury, Comment to David J. Balding & Peter Donnelly, Inference in Forensic Identification, 158 J. ROYAL STAT. SOCY. A, Part 1, at 21, 48-49 (1995); cf. Richard Lempert, After the DNA Wars: Skirmishing with NRC II, 37 JURIMETRICS J. 439, 461-62 (1997) [hereinafter Lempert, After the DNA Wars] (endorsing NRC I and criticizing NRC II with respect to database searches); Richard Lempert, Some Caveats Concerning DNA as Criminal Identification Evidence: With Thanks to the Reverend Bayes, 13 CARDOZO L. REV. 303, 333-34 (1991) [hereinafter Lempert, Some Caveats] (anticipating NRC I and proposing retesting requirement).
15. See N.E. Morton, The Forensic DNA Endgame, 37 JURIMETRICS J. 477, 487-92 (1997); Anders Stockmarr, Likelihood ratios for evaluating DNA evidence when the suspect is found through a database search (Research Rept. 98/1, Dept. of Biostatistics, University of Copenhagen) (to be published, in slightly revised form, in Biometrics (Sept. 1999)); Aidan Sudbury, Comment to David J. Balding & Peter Donnelly, Inference in Forensic Identification, 158 J. ROYAL STAT. SOCY. A, Part 1, at 21, 48-49 (1995); cf. Richard Lempert, After the DNA Wars: Skirmishing with NRC II, 37 JURIMETRICS J. 439, 461-62 (1997) [hereinafter Lempert, After the DNA Wars] (endorsing NRC I and criticizing NRC II with respect to database searches); Richard Lempert, Some Caveats Concerning DNA as Criminal Identification Evidence: With Thanks to the Reverend Bayes, 13 CARDOZO L. REV. 303, 333-34 (1991) [hereinafter Lempert, Some Caveats] (anticipating NRC I and proposing retesting requirement).
16. Indeed, it may be that most of the non-DNA evidence in the trawl case is exculpatory. See, e.g., Regina v. Adams. [1998] 1 Crim. App. 377 (Eng. C.A. 1997), discussed infra note 74.
17. The arguments made here are consistent with, but fuller in some respects than, those made in IAN W. EVETT & BRUCE S. WEIR, INTERPRETING DNA EVIDENCE: STATISTICAL GENETICS FOR FORENSIC SCIENTISTS 219-22 (1998); David J. Balding, Errors and Misunderstandings in the Second NRC Report, 37 JURIMETRICS J. 469 (1997); David J. Balding & Peter Donnelly, Evaluating DNA Profile Evidence When the Suspect Is Identified Through a Database Search, 41 J. FORENSIC SCI. 603 (1996) [hereinafter Balding & Donnelly, Evaluating DNA Profile Evidence]; David J. Balding & Peter Donnelly, Inference in Forensic Identification, 158 J. ROYAL STAT. SOCY. A, Part 1, at 21, 29 (1995) [hereinafter Balding & Donnelly, Inference in Forensic Identification]; A.P. Dawid & J. Mortera, Coherent Analysis of Forensic Identification Evidence, 58 J. ROYAL STAT. SOCY. B, 425 (1996).
18. The arguments made here are consistent with, but fuller in some respects than, those made in IAN W. EVETT & BRUCE S. WEIR, INTERPRETING DNA EVIDENCE: STATISTICAL GENETICS FOR FORENSIC SCIENTISTS 219-22 (1998); David J. Balding, Errors and Misunderstandings in the Second NRC Report, 37 JURIMETRICS J. 469 (1997); David J. Balding & Peter Donnelly, Evaluating DNA Profile Evidence When the Suspect Is Identified Through a Database Search, 41 J. FORENSIC SCI. 603 (1996) [hereinafter Balding & Donnelly, Evaluating DNA Profile Evidence]; David J. Balding & Peter Donnelly, Inference in Forensic Identification, 158 J. ROYAL STAT. SOCY. A, Part 1, at 21, 29 (1995) [hereinafter Balding & Donnelly, Inference in Forensic Identification]; A.P. Dawid & J. Mortera, Coherent Analysis of Forensic Identification Evidence, 58 J. ROYAL STAT. SOCY. B, 425 (1996).
19. The arguments made here are consistent with, but fuller in some respects than, those made in IAN W. EVETT & BRUCE S. WEIR, INTERPRETING DNA EVIDENCE: STATISTICAL GENETICS FOR FORENSIC SCIENTISTS 219-22 (1998); David J. Balding, Errors and Misunderstandings in the Second NRC Report, 37 JURIMETRICS J. 469 (1997); David J. Balding & Peter Donnelly, Evaluating DNA Profile Evidence When the Suspect Is Identified Through a Database Search, 41 J. FORENSIC SCI. 603 (1996) [hereinafter Balding & Donnelly, Evaluating DNA Profile Evidence]; David J. Balding & Peter Donnelly, Inference in Forensic Identification, 158 J. ROYAL STAT. SOCY. A, Part 1, at 21, 29 (1995) [hereinafter Balding & Donnelly, Inference in Forensic Identification]; A.P. Dawid & J. Mortera, Coherent Analysis of Forensic Identification Evidence, 58 J. ROYAL STAT. SOCY. B, 425 (1996).
20. The arguments made here are consistent with, but fuller in some respects than, those made in IAN W. EVETT & BRUCE S. WEIR, INTERPRETING DNA EVIDENCE: STATISTICAL GENETICS FOR FORENSIC SCIENTISTS 219-22 (1998); David J. Balding, Errors and Misunderstandings in the Second NRC Report, 37 JURIMETRICS J. 469 (1997); David J. Balding & Peter Donnelly, Evaluating DNA Profile Evidence When the Suspect Is Identified Through a Database Search, 41 J. FORENSIC SCI. 603 (1996) [hereinafter Balding & Donnelly, Evaluating DNA Profile Evidence]; David J. Balding & Peter Donnelly, Inference in Forensic Identification, 158 J. ROYAL STAT. SOCY. A, Part 1, at 21, 29 (1995) [hereinafter Balding & Donnelly, Inference in Forensic Identification]; A.P. Dawid & J. Mortera, Coherent Analysis of Forensic Identification Evidence, 58 J. ROYAL STAT. SOCY. B, 425 (1996).
21. The arguments made here are consistent with, but fuller in some respects than, those made in IAN W. EVETT & BRUCE S. WEIR, INTERPRETING DNA EVIDENCE: STATISTICAL GENETICS FOR FORENSIC SCIENTISTS 219-22 (1998); David J. Balding, Errors and Misunderstandings in the Second NRC Report, 37 JURIMETRICS J. 469 (1997); David J. Balding & Peter Donnelly, Evaluating DNA Profile Evidence When the Suspect Is Identified Through a Database Search, 41 J. FORENSIC SCI. 603 (1996) [hereinafter Balding & Donnelly, Evaluating DNA Profile Evidence]; David J. Balding & Peter Donnelly, Inference in Forensic Identification, 158 J. ROYAL STAT. SOCY. A, Part 1, at 21, 29 (1995) [hereinafter Balding & Donnelly, Inference in Forensic Identification]; A.P. Dawid & J. Mortera, Coherent Analysis of Forensic Identification Evidence, 58 J. ROYAL STAT. SOCY. B, 425 (1996).
22. The arguments made here are consistent with, but fuller in some respects than, those made in IAN W. EVETT & BRUCE S. WEIR, INTERPRETING DNA EVIDENCE: STATISTICAL GENETICS FOR FORENSIC SCIENTISTS 219-22 (1998); David J. Balding, Errors and Misunderstandings in the Second NRC Report, 37 JURIMETRICS J. 469 (1997); David J. Balding & Peter Donnelly, Evaluating DNA Profile Evidence When the Suspect Is Identified Through a Database Search, 41 J. FORENSIC SCI. 603 (1996) [hereinafter Balding & Donnelly, Evaluating DNA Profile Evidence]; David J. Balding & Peter Donnelly, Inference in Forensic Identification, 158 J. ROYAL STAT. SOCY. A, Part 1, at 21, 29 (1995) [hereinafter Balding & Donnelly, Inference in Forensic Identification]; A.P. Dawid & J. Mortera, Coherent Analysis of Forensic Identification Evidence, 58 J. ROYAL STAT. SOCY. B, 425 (1996).
23. The arguments made here are consistent with, but fuller in some respects than, those made in IAN W. EVETT & BRUCE S. WEIR, INTERPRETING DNA EVIDENCE: STATISTICAL GENETICS FOR FORENSIC SCIENTISTS 219-22 (1998); David J. Balding, Errors and Misunderstandings in the Second NRC Report, 37 JURIMETRICS J. 469 (1997); David J. Balding & Peter Donnelly, Evaluating DNA Profile Evidence When the Suspect Is Identified Through a Database Search, 41 J. FORENSIC SCI. 603 (1996) [hereinafter Balding & Donnelly, Evaluating DNA Profile Evidence]; David J. Balding & Peter Donnelly, Inference in Forensic Identification, 158 J. ROYAL STAT. SOCY. A, Part 1, at 21, 29 (1995) [hereinafter Balding & Donnelly, Inference in Forensic Identification]; A.P. Dawid & J. Mortera, Coherent Analysis of Forensic Identification Evidence, 58 J. ROYAL STAT. SOCY. B, 425 (1996).
24. For helpful descriptions of the underlying genetics of DNA evidence, see NRC II, supra note 10, at 12-14, 70-74.
25. Sperm and eggs contain only half the DNA that is in an organism's other cells.
26. The description here applies only to nuclear DNA. Mitochondrial DNA, found in the energy-producing material surrounding the nucleus, is inherited solely from the mother, and (apart from possible mutations) all offspring of the same mother share the same mitochondrial DNA. For this reason, among others, most forensic DNA testing is done on nuclear DNA, though mitochondrial DNA can be useful when the trace evidence has little or no nuclear DNA. See Mark Hansen, A Comeback for Hair Evidence, A.B.A. J., May 1998, at 66.
27. If, however, the individual is homozygous at a given locus, meaning that the measured fragments from the mother's and father's sides are the same length, then in profiling systems only a single measurement, of the common fragment length, will be returned.
28. In older systems, length measurements are only approximate, and the profiles are said to match at a given locus if the pairs of measurements from each are sufficiently close - that is, if they fall within some pre-specified tolerance of each other. There is thus a slight chance that, even if two samples from the same source are measured properly, the test results will nevertheless be sufficiently different that the samples will be declared not to match.
29. This possibility can never be eliminated. We discuss lab error further below. See infra note 57.
30. There are theoretical objections to this aspect of the standard approach in the systems that depend on approximate measurement, for the criteria for determining a match are necessarily arbitrary. See, e.g., BERNARD ROBERTSON & G.A. VIGNAUX, INTERPRETING EVIDENCE: EVALUATING FORENSIC SCIENCE IN THE COURTROOM 115-20 (1995); D.A. Berry et al., Statistical Inference in Crime Investigations Using Deoxyribonucleic Acid Profiling, 41 APPLIED STAT. 499 (1992). A more theoretically satisfactory approach might be to present the likelihood ratio of the evidence - the ratio of the probability that the actual test results would have occurred if the two samples came from the same source to the probability that the results would have arisen if someone else from a given population were the source of the sample of unknown origin. But such an approach presents practical difficulties, not the least of which is that it may be baffling to a jury. We are inclined to believe that the binary approach, either declaring a match or not according to previously prescribed criteria, is a satisfactory method of summarizing the data, even though some information is lost in this way. In any event, this is the way the data is usually presented, and we will not challenge it here.
31. There are theoretical objections to this aspect of the standard approach in the systems that depend on approximate measurement, for the criteria for determining a match are necessarily arbitrary. See, e.g., BERNARD ROBERTSON & G.A. VIGNAUX, INTERPRETING EVIDENCE: EVALUATING FORENSIC SCIENCE IN THE COURTROOM 115-20 (1995); D.A. Berry et al., Statistical Inference in Crime Investigations Using Deoxyribonucleic Acid Profiling, 41 APPLIED STAT. 499 (1992). A more theoretically satisfactory approach might be to present the likelihood ratio of the evidence - the ratio of the probability that the actual test results would have occurred if the two samples came from the same source to the probability that the results would have arisen if someone else from a given population were the source of the sample of unknown origin. But such an approach presents practical difficulties, not the least of which is that it may be baffling to a jury. We are inclined to believe that the binary approach, either declaring a match or not according to previously prescribed criteria, is a satisfactory method of summarizing the data, even though some information is lost in this way. In any event, this is the way the data is usually presented, and we will not challenge it here.
32. See, e.g., NRC II, supra note 10, at 97-108 (arguing that this assumption of independence may not be precisely accurate, but that if the markers are chosen carefully, it is accurate enough for use in forensic casework).
33. See, e.g., Lempert, After the DNA Wars, supra note 11; Lempert, Some Caveats, supra note 11; Kathryn Roeder, DNA Fingerprinting: A Review of the Controversy, 9 STAT. SCI. 222 (1994). See generally NRC I, supra note 8, passim; NRC II, supra note 10, passim.
34. See, e.g., Lempert, After the DNA Wars, supra note 11; Lempert, Some Caveats, supra note 11; Kathryn Roeder, DNA Fingerprinting: A Review of the Controversy, 9 STAT. SCI. 222 (1994). See generally NRC I, supra note 8, passim; NRC II, supra note 10, passim.
35. See, e.g., Lempert, After the DNA Wars, supra note 11; Lempert, Some Caveats, supra note 11; Kathryn Roeder, DNA Fingerprinting: A Review of the Controversy, 9 STAT. SCI. 222 (1994). See generally NRC I, supra note 8, passim; NRC II, supra note 10, passim.
36. See, e.g., Regina v. Doheny, [1997] 1 Crim. App. 369, 375 (Eng. C.A. 1996); EVETT & WEIR, supra note 13, at 222-38; Richard D. Friedman, Assessing Evidence, 94 MICH. L. REV. 1810, 1835-37 (1996); Jonathan J. Koehler, Error and Exaggeration in the Presentation of DNA Evidence at Trial, 34 JURIMETRICS J. 21 (1993).
37. See, e.g., Regina v. Doheny, [1997] 1 Crim. App. 369, 375 (Eng. C.A. 1996); EVETT & WEIR, supra note 13, at 222-38; Richard D. Friedman, Assessing Evidence, 94 MICH. L. REV. 1810, 1835-37 (1996); Jonathan J. Koehler, Error and Exaggeration in the Presentation of DNA Evidence at Trial, 34 JURIMETRICS J. 21 (1993).
38. As is most common, we use the term match probability to refer to the probability of a match as assessed after the profile of the crime sample is known.
39. Because p is a number between zero and one, presumably so small that it has several zeroes immediately to the right of the decimal point, it is often easier to deal with R, a number greater than one.
40. See Jonathan J. Koehler, On Conveying the Probative Value of DNA Evidence: Frequencies, Likelihood Ratios, and Error Rates, 67 U. COLO. L. REV. 859, 877-80 (1996). See generally Doheny, 1 Crim. App. 369 (Eng. C.A. 1996).
41. See Jonathan J. Koehler, On Conveying the Probative Value of DNA Evidence: Frequencies, Likelihood Ratios, and Error Rates, 67 U. COLO. L. REV. 859, 877-80 (1996). See generally Doheny, 1 Crim. App. 369 (Eng. C.A. 1996).
42. The statutory authority is the Police and Criminal Evidence Act, 1984, ch. 63(3)(b) (Eng.). If the person is subsequently found innocent, in most circumstances the profile is removed from the database. The English database now contains approximately 350,000 samples and is growing at a rate of 200,000 samples per year. Personal communication with Dr. Peter Gill, Forensic Science Service (August 1998).
43. DNA Identification Act of 1994, Pub. L. No. 103-322, §§ 210301-210306, 108 Stat. 2065 (codified at 42 U.S.C. §§ 3796kk to 3796kk-6, 14131-14134 (1994) and codified as amended at 42 U.S.C. §§ 3751, 3753, 3793, 3797 (1994)); see also Harold J. Krent, Of Diaries and Data Banks: Use Restrictions Under the Fourth Amendment, 74 TEX. L. REV. 49, 86 & nn. 173-74 (1995).
44. All but one of the citations are provided in Statutes and Legislation Regarding Mandatory Submission of Blood Samples for DNA Identification Purposes, in FORENSIC DNA STATUTES 50 (assembled by Federal Bureau of Investigation, 1998); see also Krent, supra note 28. Most of the state laws require samples only from persons convicted of a narrow set of felonies, including homicide and sexual assault. Four states (Alabama, New Mexico, Virginia, and Wyoming) require samples from all convicted felons. See Wade, supra note 6. Louisiana requires samples from, among others, persons arrested for felony sex offenses or other specified offenses after September 1, 1999. See LA. REV. STAT. ANN. § 15:609(A) (West Supp. 1999) (effective September 1, 1999); Wade, supra note 6. So far, the constitutionality of the state laws has largely been upheld. See, e.g., Schlicher v. Peters, 103 F.3d 940 (10th Cir. 1996) (upholding a Kansas law requiring persons convicted of designated felonies to submit blood and saliva samples for compilation of a database of genetic markers): Boling v. Romer, 101 F.3d 1336 (10th Cir. 1996) (upholding a Colorado statute requiring an inmate convicted of an offense involving sexual assault to provide the state with a DNA sample before release on parole); Rise v. Oregon, 59 F.3d 1556 (9th Cir. 1995) (upholding a state law that requires persons convicted of murder, a sexual offense, or conspiracy or attempt to commit a sexual offense to submit a blood sample for preparation of a DNA database), cert. denied, 517 U.S. 1160 (1996); Jones v. Murray, 962 F.2d 302 (4th Cir. 1992) (upholding a Virginia statute requiring convicted felons to submit blood samples for creation of a DNA database, but holding part of the enforcement mechanism invalid as an ex post facto law). But see Landry v. Harshbarger, No. CIV.A.98-462 (Mass. Super. Ct., Aug. 12, 1998) (unpublished order preliminarily enjoining enforcement of a Massachusetts statute requiring any person convicted of any of an array of enumerated offenses to submit a DNA sample).
45. See Wade, supra note 6.
46. Personal communication with M. Dawn Herkenham, Chief, Forensic Science Systems Unit, Federal Bureau of Investigation (Aug. 12 & Nov. 3, 1998).
47. Id. on Nov. 2, 1998. See also the web page of the FBI's Forensic Science Research and Training Center (visited Dec. 4, 1998) 〈http://www.fbi.gov/lab/report/research.htm〉.
48. Note the comment made by NRC II in 1996: As the number and size of DNA databanks increase, the identification of suspects by this means will become more common. Already, more than 20 suspects have been identified by searches through databases maintained by various states. The number and sizes of these databases are sure to increase. NRC II, supra note 10, at 134.
49. See Wade, supra note 6, at A1 (quoting M. Dawn Herkenham, chief of the FBI's Forensic Science Systems Unit: "I think the trend is that 10 years from now all felonies will be covered . . . . We recommend that all violent felonies, burglaries, juveniles and retroactivity for people on parole be included."). Whether such broad databases are appropriate raises issues of policy and constitutional law on which we do not mean to express an opinion.
50. See Wade, supra note 6.
51. NRC I, supra note 8, at 124.
52. Id.
53. Id. at 129.
54. Id. at 124.
55. See infra notes 81-82 and accompanying text (views of Professors Morton and Lempert).
56. NRC II, supra note 10, at 134.
57. Id.
58. Id.
59. See id.
60. Id. at 135.
61. (D+1). If D + 1 is much smaller than 1/p, which we have called R, then this quantity is very close to (D + 1)p.
62. Recommendation 5.1, NRC II, supra note 10, at 40.
63. NRC II, supra note 10, at 40. The argument is developed in detail in Stockmarr, supra note 11, at 6-9.
64. We show how to derive this ratio in the Appendix. See also Stockmarr, supra note 11, at 8.
65. These differences are small when the databases are small, and they disappear altogether in the limiting case in which there is only one profile in the database.
66. This appears rather obvious: The more profiles there are in the database, the more probable it is that at least one of them will match that of the crime sample but be from someone other than the source of that sample. That is, the number of false positives will tend to increase with the size of the database. It is even true that the larger the database is, up to a point, the more probable it is that there will be exactly one false positive. (Our analysis has focused, for simplicity, on the most important case, in which the database search turns up exactly one match with the crime sample.) One attempting to support the NRC analyses might well point out that, the larger the database is, up to a point, the more probable it is that there will be exactly one false positive. That is, the more probable it is that both the following propositions are true: (1) A search of the database will turn up exactly one sample that matches the crime sample, and (2) in fact the source of the crime sample is someone who has not contributed a sample to the database. After a point, this conjunction of possibilities becomes less likely because the second part of it - that the source was someone outside the database - becomes very unlikely, and the first part may also become less likely.
67. See FED. R. EVID. 401 (defining "relevant evidence" as "evidence having any tendency to make the existence of any fact that is of consequence to the determination of the action more probable or less probable than it would be without the evidence").
68. See, e.g., Stockmarr, supra note 11, at 6-8.
69. The suspect population is the population from which the crime sample is believed to originate. The notion of a suspect population is a useful simplification for some analyses, but its limitations are important to recognize. It seems to suggest that all people within a defined group are equally likely, before any other evidence is revived, to have committed the crime, and that anyone outside that group could not have committed it. But clearly this is not so. In the typical stranger rape case, for example, a man who lives within five miles of the crime scene is more likely, absent other evidence, to have committed the crime than a man who lives 300 miles away, and that man is a more likely suspect than one who lives on the other side of the world - but even the man from the antipodes cannot be altogether excluded. For further comments, see EVETT & WEIR, supra note 13, at 27-28, 233-34.
70. See supra section I.B.
71. This effect is demonstrated in Balding & Donnelly, Evaluating DNA Profile Evidence, supra note 13, at 604; EVETT & WEIR, supra note 13, at 220.
72. Some perceptive scholars have emphasized the importance of the possibility of laboratory error, which is often far more plausible than the possibility of a match by chance. See, e.g., Jonathan J. Koehler, Why DNA Likelihood Ratios Should Account for Error (Even When a National Research Council Report Says They Should Not), 37 JURIMETRICS J. 425 (1997); Lempert, After the DNA Wars, supra note 11, at 446-54; William C. Thompson & S. Ford, The Meaning of a Match: Sources of Ambiguity in the Interpretation of DNA Prints, in FORENSIC DNA TECHNOLOGY 93 (Mark A. Farley & James J. Harrington eds., 1991). We agree that jurors should take the possibility of lab error into account in their subjective assessment of the evidence. It is important to emphasize, however, that what matters is not the probability of any laboratory error, but rather only the probability of those errors that would lead to the false declaration of a match in the given case - a probability that will vary widely with the circumstances of the DNA testing. For a fuller treatment, see David J. Balding & Peter Donnelly. Inferring Identity from DNA Profile Evidence, PROC. NATL. ACAD. SCI. (USA) 11,741 (1995). See also infra notes 59-60 and accompanying text (analyzing "paradox of the lottery" in similar terms); A. Philip Dawid & Julia Mortera, Forensic Identification with Imperfect Evidence, 85 BIOMETRIKA 835, 844 (1998) (drawing analogy to lottery paradox but inaccurately ascribing erroneous argument to Balding and Donnelly). In particular, the probability of this type of error will tend to be smaller in a trawl case than in many confirmation cases because in the trawl case the DNA profiling of the crime sample and the defendant's sample are performed at different times and often in different laboratories. In addition to providing information that will assist the jury in assessing the probability of a false report of a match in the case at hand, an expert should provide information bearing on the probability of an actual match on the assumption that the defendant is innocent. We focus in this article on how the latter probability - which is also the focus of most expert testimony in this area - is affected by the fact that the match is the result of a database search.
73. Some perceptive scholars have emphasized the importance of the possibility of laboratory error, which is often far more plausible than the possibility of a match by chance. See, e.g., Jonathan J. Koehler, Why DNA Likelihood Ratios Should Account for Error (Even When a National Research Council Report Says They Should Not), 37 JURIMETRICS J. 425 (1997); Lempert, After the DNA Wars, supra note 11, at 446-54; William C. Thompson & S. Ford, The Meaning of a Match: Sources of Ambiguity in the Interpretation of DNA Prints, in FORENSIC DNA TECHNOLOGY 93 (Mark A. Farley & James J. Harrington eds., 1991). We agree that jurors should take the possibility of lab error into account in their subjective assessment of the evidence. It is important to emphasize, however, that what matters is not the probability of any laboratory error, but rather only the probability of those errors that would lead to the false declaration of a match in the given case - a probability that will vary widely with the circumstances of the DNA testing. For a fuller treatment, see David J. Balding & Peter Donnelly. Inferring Identity from DNA Profile Evidence, PROC. NATL. ACAD. SCI. (USA) 11,741 (1995). See also infra notes 59-60 and accompanying text (analyzing "paradox of the lottery" in similar terms); A. Philip Dawid & Julia Mortera, Forensic Identification with Imperfect Evidence, 85 BIOMETRIKA 835, 844 (1998) (drawing analogy to lottery paradox but inaccurately ascribing erroneous argument to Balding and Donnelly). In particular, the probability of this type of error will tend to be smaller in a trawl case than in many confirmation cases because in the trawl case the DNA profiling of the crime sample and the defendant's sample are performed at different times and often in different laboratories. In addition to providing information that will assist the jury in assessing the probability of a false report of a match in the case at hand, an expert should provide information bearing on the probability of an actual match on the assumption that the defendant is innocent. We focus in this article on how the latter probability - which is also the focus of most expert testimony in this area - is affected by the fact that the match is the result of a database search.
74. Some perceptive scholars have emphasized the importance of the possibility of laboratory error, which is often far more plausible than the possibility of a match by chance. See, e.g., Jonathan J. Koehler, Why DNA Likelihood Ratios Should Account for Error (Even When a National Research Council Report Says They Should Not), 37 JURIMETRICS J. 425 (1997); Lempert, After the DNA Wars, supra note 11, at 446-54; William C. Thompson & S. Ford, The Meaning of a Match: Sources of Ambiguity in the Interpretation of DNA Prints, in FORENSIC DNA TECHNOLOGY 93 (Mark A. Farley & James J. Harrington eds., 1991). We agree that jurors should take the possibility of lab error into account in their subjective assessment of the evidence. It is important to emphasize, however, that what matters is not the probability of any laboratory error, but rather only the probability of those errors that would lead to the false declaration of a match in the given case - a probability that will vary widely with the circumstances of the DNA testing. For a fuller treatment, see David J. Balding & Peter Donnelly. Inferring Identity from DNA Profile Evidence, PROC. NATL. ACAD. SCI. (USA) 11,741 (1995). See also infra notes 59-60 and accompanying text (analyzing "paradox of the lottery" in similar terms); A. Philip Dawid & Julia Mortera, Forensic Identification with Imperfect Evidence, 85 BIOMETRIKA 835, 844 (1998) (drawing analogy to lottery paradox but inaccurately ascribing erroneous argument to Balding and Donnelly). In particular, the probability of this type of error will tend to be smaller in a trawl case than in many confirmation cases because in the trawl case the DNA profiling of the crime sample and the defendant's sample are performed at different times and often in different laboratories. In addition to providing information that will assist the jury in assessing the probability of a false report of a match in the case at hand, an expert should provide information bearing on the probability of an actual match on the assumption that the defendant is innocent. We focus in this article on how the latter probability - which is also the focus of most expert testimony in this area - is affected by the fact that the match is the result of a database search.
75. Some perceptive scholars have emphasized the importance of the possibility of laboratory error, which is often far more plausible than the possibility of a match by chance. See, e.g., Jonathan J. Koehler, Why DNA Likelihood Ratios Should Account for Error (Even When a National Research Council Report Says They Should Not), 37 JURIMETRICS J. 425 (1997); Lempert, After the DNA Wars, supra note 11, at 446-54; William C. Thompson & S. Ford, The Meaning of a Match: Sources of Ambiguity in the Interpretation of DNA Prints, in FORENSIC DNA TECHNOLOGY 93 (Mark A. Farley & James J. Harrington eds., 1991). We agree that jurors should take the possibility of lab error into account in their subjective assessment of the evidence. It is important to emphasize, however, that what matters is not the probability of any laboratory error, but rather only the probability of those errors that would lead to the false declaration of a match in the given case - a probability that will vary widely with the circumstances of the DNA testing. For a fuller treatment, see David J. Balding & Peter Donnelly. Inferring Identity from DNA Profile Evidence, PROC. NATL. ACAD. SCI. (USA) 11,741 (1995). See also infra notes 59-60 and accompanying text (analyzing "paradox of the lottery" in similar terms); A. Philip Dawid & Julia Mortera, Forensic Identification with Imperfect Evidence, 85 BIOMETRIKA 835, 844 (1998) (drawing analogy to lottery paradox but inaccurately ascribing erroneous argument to Balding and Donnelly). In particular, the probability of this type of error will tend to be smaller in a trawl case than in many confirmation cases because in the trawl case the DNA profiling of the crime sample and the defendant's sample are performed at different times and often in different laboratories. In addition to providing information that will assist the jury in assessing the probability of a false report of a match in the case at hand, an expert should provide information bearing on the probability of an actual match on the assumption that the defendant is innocent. We focus in this article on how the latter probability - which is also the focus of most expert testimony in this area - is affected by the fact that the match is the result of a database search.
76. Some perceptive scholars have emphasized the importance of the possibility of laboratory error, which is often far more plausible than the possibility of a match by chance. See, e.g., Jonathan J. Koehler, Why DNA Likelihood Ratios Should Account for Error (Even When a National Research Council Report Says They Should Not), 37 JURIMETRICS J. 425 (1997); Lempert, After the DNA Wars, supra note 11, at 446-54; William C. Thompson & S. Ford, The Meaning of a Match: Sources of Ambiguity in the Interpretation of DNA Prints, in FORENSIC DNA TECHNOLOGY 93 (Mark A. Farley & James J. Harrington eds., 1991). We agree that jurors should take the possibility of lab error into account in their subjective assessment of the evidence. It is important to emphasize, however, that what matters is not the probability of any laboratory error, but rather only the probability of those errors that would lead to the false declaration of a match in the given case - a probability that will vary widely with the circumstances of the DNA testing. For a fuller treatment, see David J. Balding & Peter Donnelly. Inferring Identity from DNA Profile Evidence, PROC. NATL. ACAD. SCI. (USA) 11,741 (1995). See also infra notes 59-60 and accompanying text (analyzing "paradox of the lottery" in similar terms); A. Philip Dawid & Julia Mortera, Forensic Identification with Imperfect Evidence, 85 BIOMETRIKA 835, 844 (1998) (drawing analogy to lottery paradox but inaccurately ascribing erroneous argument to Balding and Donnelly). In particular, the probability of this type of error will tend to be smaller in a trawl case than in many confirmation cases because in the trawl case the DNA profiling of the crime sample and the defendant's sample are performed at different times and often in different laboratories. In addition to providing information that will assist the jury in assessing the probability of a false report of a match in the case at hand, an expert should provide information bearing on the probability of an actual match on the assumption that the defendant is innocent. We focus in this article on how the latter probability - which is also the focus of most expert testimony in this area - is affected by the fact that the match is the result of a database search.
77. Professor Morton's approach is founded on the frequentist argument. See Morton, supra note 11, at 488. Dr. Stockmarr explicitly rejects that argument, giving the likelihood ratio argument instead. See Stockmarr, supra note 11, at 9.
78. Two early presentations of the problem are RICHARD PRICE, FOUR DISSERTATIONS 410-12 (1767), and ISAAC TODHUNTER, A HISTORY OF THE MATHEMATICAL THEORY OF PROBABILITY FROM THE TIME OF PASCAL TO THAT OF LAPLACE 400 (1865). Todhunter's presentation is quoted in Lea Brilmayer & Lewis Kornhauser, Quantitative Methods and Legal Decisions, 46 U. CHI. L. REV. 116, 148 n.114 (1978). See also L. Jonathan Cohen, Can Human Irrationality Be Experimentally Demonstrated? 4 BEHAV. & BRAIN SCI. 317, 329 (1981).
79. Two early presentations of the problem are RICHARD PRICE, FOUR DISSERTATIONS 410-12 (1767), and ISAAC TODHUNTER, A HISTORY OF THE MATHEMATICAL THEORY OF PROBABILITY FROM THE TIME OF PASCAL TO THAT OF LAPLACE 400 (1865). Todhunter's presentation is quoted in Lea Brilmayer & Lewis Kornhauser, Quantitative Methods and Legal Decisions, 46 U. CHI. L. REV. 116, 148 n.114 (1978). See also L. Jonathan Cohen, Can Human Irrationality Be Experimentally Demonstrated? 4 BEHAV. & BRAIN SCI. 317, 329 (1981).
80. Two early presentations of the problem are RICHARD PRICE, FOUR DISSERTATIONS 410-12 (1767), and ISAAC TODHUNTER, A HISTORY OF THE MATHEMATICAL THEORY OF PROBABILITY FROM THE TIME OF PASCAL TO THAT OF LAPLACE 400 (1865). Todhunter's presentation is quoted in Lea Brilmayer & Lewis Kornhauser, Quantitative Methods and Legal Decisions, 46 U. CHI. L. REV. 116, 148 n.114 (1978). See also L. Jonathan Cohen, Can Human Irrationality Be Experimentally Demonstrated? 4 BEHAV. & BRAIN SCI. 317, 329 (1981).
81. Two early presentations of the problem are RICHARD PRICE, FOUR DISSERTATIONS 410-12 (1767), and ISAAC TODHUNTER, A HISTORY OF THE MATHEMATICAL THEORY OF PROBABILITY FROM THE TIME OF PASCAL TO THAT OF LAPLACE 400 (1865). Todhunter's presentation is quoted in Lea Brilmayer & Lewis Kornhauser, Quantitative Methods and Legal Decisions, 46 U. CHI. L. REV. 116, 148 n.114 (1978). See also L. Jonathan Cohen, Can Human Irrationality Be Experimentally Demonstrated? 4 BEHAV. & BRAIN SCI. 317, 329 (1981).
82. For fuller analyses see Richard D. Friedman, Route Analysis of Credibility and Hearsay, 96 YALE L.J. 667, 683-85, 736-39 (1987); Jonathan J. Koehler & Daniel Shaviro, Veridical Verdicts: Increasing Verdict Accuracy Through the Use of Overtly Probabilistic Evidence and Methods, 75 CORNELL L. REV. 247, 270-71 (1990).
83. For fuller analyses see Richard D. Friedman, Route Analysis of Credibility and Hearsay, 96 YALE L.J. 667, 683-85, 736-39 (1987); Jonathan J. Koehler & Daniel Shaviro, Veridical Verdicts: Increasing Verdict Accuracy Through the Use of Overtly Probabilistic Evidence and Methods, 75 CORNELL L. REV. 247, 270-71 (1990).
84. We present a derivation of this likelihood ratio in Section D of the Appendix.
85. That proposition would also have higher prior odds the more profiles there are in the database, and these two factors would tend to balance out. See infra section II.D.2.
86. See NRC II, supra note 10, at 40 ("In the extreme case, a search of the whole population should, of course, provide a definitive answer.")
87. Id.
88. Expression (1), which id the key quantity for both of the NRC's arguments, diminishes constantly as the size of the database increases, indicating diminishing value of the evidence. There is no point at which it begins to move in the other direction - obviously so, for the size of the database is the denominator of the Expression and is unrelated to the numerator. NRC II offers no hints at what point, or under what circumstances, the arguments leading to Expression (1) lose force and some other argument becomes dominant. NRC II does suggest that the likelihood ration argument leads to a proper assessment of posterior odds, see NRC II, supra note 10, at 164-65, but as we indicate infra section II.D.2, this is no consolation for the argument. Stockmarr offers an extended discussion of the problem of the large database, see Stockmarrm, supra note 11, at 10-14, but his argumant is irretrievably flawed. Stockmarr's argument, like the NRC's, appears to be based on the fact that he is able to produce a calculation yielding appropriate posterior odds for the case in which the database trawl yields one match with the crime sample; the argument therefore suffers the same flaws as does the NRC's. Stockmarr calculates the posterior odds for the the trawl case (which we show in the Appendix to equal R/N, in our notation), and then divides them by the likelihood ratio for the confirmation case (R in our notation) to yield a factor, equal to the reciprocal of the population not represented in the database (that is, 1/N in our notation), that he calls a "sliding correction" to take the size of the database into account. But the "correction" merely means that he is able to reach the correct posterior odds. It is unclear what significance Stockmarr attaches to the factor, given that he still maintains that the same likelihood ration prevails for the evidence no matter how large the database. Stockmarr's interpretation also produces bizarre results - not surprising, given that he is comparing the posterior odds from the trawl case with the likelihood ratio from the confirmation case, which, to borrow a phrase from the English legal scholar Peter Mirfield, seems to us be like comparing apples and Tuesday. Thus, he contends that if the database contains profiles from all but one member of the population the evidence of a match has just the same strength as it would in the confirmation case, and that if the database is at all smaller the evidence is weaker than in that case. See id. at 14. He seems not to recognize fully that a single match after the search of such a nearly all-inclusive database would be virtually conclusive evidence against the source of the single matching sample.
89. R would also be relatively low if the evidence were of some other kind of profile, such as standard blood groupings, which are not as rare as DNA profiles; the validity of the statistical analysis does not depend on the identification technique used, and nothing in the derivation of the NRC's likelihood ratio (Expression (1)) depends on either the size of the database or the value of the match probability. Just as it is untenable to adjust the likelihood ratio prescribed by NRC II for large databases, it is untenable to make another adjustment for relatively undiscriminating tests.
90. See Balding & Donnelly, Evaluating DNA Profile Evidence, supra note 13, at 605.
91. Indeed, if the database is large enough, the defendant might - under this theory -even be able to make the match appear exculpatory. See supra section II.C.3.b.
92. Note that the prior probability that Matcher is the source is less in some confirmation cases than in some trawl cases. The degree of suspicion of Matcher might be very slight in a confirmation case, and in a trawl case it may be that everyone outside a very small database has been eliminated as a suspect.
93. That is, a factor equal to the size of the database in assessing the match probability or the likelihood ratio.
94. Particularly if it is extended to include the possibility that the police simultaneously send samples from several suspects for profiling. This change does not alter our argument.
95. The frequentist argument in the trawl case is that the probability of interest is that of the event that the database search procedure will yield a match if indeed all of the individuals in the database are innocent. In the sequential search case, the analogy would be the probability that the sequential search procedure would yield a match if indeed all of the individuals sequentially investigated are innocent. This is effectively the classical statistical approach to sequential hypothesis testing.
96. See Balding & Donnelly, Inference in Forensic Identification, supra note 13, at 27-29.
97. Regina v. Adams, [1998] 1 Crim. App. 377 (Eng. C.A. 1997), a rape case, is the textbook example. Effectively the only evidence linking the defendant to the crime was a DNA match between him and the crime sample, together with proof that he lived and worked in the general area where the rape took place. The victim, having originally told police that she had a good view of her attacker and that she remembered him clearly, did not identify Adams in a lineup. At the committal proceedings, she testified that he did not look like her description of her attacker; she stated that Adams looked to be in his forties while her attacker was in his early twenties. At trial, Adams testified that he had spent the whole of the night in question with his girlfriend, who corroborated his alibi.
98. Part C of the Appendix shows that, assuming that Matcher is just as likely, apart from the DNA evidence, as every other member of the population to be the source of the crime scene sample, then the posterior odds that he is the source are R/N. Part C of the Appendix also shows that, for database relatively small compared to the suspect population, these odds are much lower than the posterior odds in the confirmation case, even if the members of the population.
99. See Lempert, Some Caveats, supra note 11, at 319. Of course suggestiveness can cut both ways. A confirmation case, the non-DNA evidence can create suggestiveness that might cast doubt both on subsequently discovered non-DNA evidence such as an eyewitness identification (if, for example, the police suggest that they have "got the man") and, given that the DNA test is performed by humans at the behest of the authorities, on the DNA evidence itself. We mean only to point out that the possibility of suggestiveness might weaken the force of the non-DNA evidence more in a given trawl case than in a comparable confirmation case.
100. See Stockmarr, supra note 11, at 11. Stockmarr says the posterior odds are k/(N - n)p, where k is the number of matches, N the population, n the size of the database, and p the random match probability. For one match, this is identical to our R/N.
101. See supra section I.B.
102. This phrasing does not indicate when the sample was taken from Matcher. This is sometimes preferable, because revealing that the sample was taken before suspicion was cast on Matcher in this case, or that his profile was part of a database on file with the police, might suggest that he had a previous criminal record. Cf. Commmonwealth v. Claffey, 400 A.2d 173 (Pa. Super. Ct. 1979) (emphasizing that jury did not learn that defendant was arrested after a fingerprint comparison proved positive, and thus had no reasonable basis for an inference of prior crime). If the defendant's prior record is appearent to the jury anyway, there should be no problem. Df. State v. Lozano, 588 P.2d 841, 842-43 (Ariz. Ct. App. 1978) ("Any prejudice was removed . . . when appellant testified in his defense and admitted a prior felony conviction that adequately explained the existence of the master [fingerprint] file card."); People v. Garlin, 428 N.E.2d 697, 702 (Ill. App. Ct. 1981). Also, the defendant may affirmatively wish the jury to know that the DNA identification was made before the police had any other significant evidence against him, so that he can argue that the other evidence was tainted by suggestion from the DNA identification. Df. State v. Montgomery, 46 S.E.2d 732, 738-39 (N.C. 1995) (ruling that defendant opened the door to evidence concerning dtae of fingerprinting). Even absent factors such as these, there is some reason to allow the prosecution to indicate, without flourishes gratuitously suggesting a prior record or its nature, that the defendant's profile was part of a databse. This allows the prosecution to tell a coherent story of how the defendant's profile came to be compared to that of the crime scene sample. Furthermore, if the evidence that the other members of the database were eliminated as suspects is deemed significantly probative - which probably should be so only if the database is relatiely large - it probably makes sense not to present the evidence as if the examination of the defendant's profile and the database trwal were two separate operations. Cf., e.g., Garlin, 428 N.E.2d at 701 (acknowledging that fingerprint card suggested prior arrest record, but holding that the evidence was necessary to prove identity). The problem will probably be eased as DNA testing becomes more common, so that knowing that the police had access to a person's profile will not necessarily suggest that he had a prior record, or at least - as is already true in England - will not necessarily suggest that he had previously come under suspicion for a particularly serious crime. Cf., e.g., Riley v. Sigler, 320 F. Supp. 96, 99 (D. Neb. 1970) ("The mere fact that the jury knows that police officers have taken fingerprints of a defendant does not prejudice the defense. . . . [T]here was no assertion that the petitioner has previously been charged with a crime or as to why the fingerprint was on profile. the jury could only speculate that petitioner has a record of other crimes." And the jury was told not to speculate. So there.); People v. McGhee, 605 N.E.2d 1039, 1048 (Ill. App. Ct. 1992) (holding, in a fingerprint case, that "the mere inference that defendant had committed some offense in the past, without some suggestion as to the nature and circumstances of that offense, fails to rise to a level of prejudice sufficient to overcome the obvious probative value such evidence provided in connecting defendant to this crime"); Commenwealth v. Reiss, 468 A.2d 451, 453 (Pa. 1983) ("Where there is no indication . . . that the photographs were 'mugshots' or that they came from police files, it is not error for a witness to testify that he identified a defendant from photographs shown to him by police."); State v. Emrick, 282 A.2d 821, 826 (Vt. 1971) (refusing to find prejudice, because of common knowledge that fingerprints are taken for many purposes other than in connection with criminal proceedings).
103. See supra note 79.
104. See Morton, supra note 11, at 489. Morton agrees in principle with the analysis of NRC II. But he expresses concern that, for the defendant to gain the benefit of the diminution of probative value that he believes is appropriate, the defendant would have to reveal the fact that his profile was in a preexisting database, which may suggest a prior criminal record. This issue is addressed in note 79, supra. To avoid this problem - and others as well that are predicated on agreement with the fundamental analysis of NRC II - he advocates a retesting requirement.
105. See Lempert, After the DNA Wars, supra note 11, at 461. Lempert, like NRC II, believes that the probative value of evidence of a match found after a database trawl is diminished by the fact that, the larger the database, the more likely it is that a match will be found even though no one profiled in the database is actually the source of the crime sample. But then he criticizes the statistical adjustment offered by NRC II in an attempt to address this problem - for example, he discusses incisively the problem of the all-inclusive database - and so concludes that this solution is "a step backward" from the retesting requirement of NRC I. (Professor Lempert was a member of the Committee that produced NRC I.)
106. See NRC II, supra note 10, at 134 ("If the amount of DNA in the evidence sample is too small, following the recommendation in the 1992 report could leave too few additional loci for computing a match probability of LR."). The problem, where it exists, is substantially the same - insufficient additional DNA for a second test - whether the same or additional loci are used.
107. Any testing technique requires a minimum amount of DNA for it to work satisfactorily. This minimum amount is much smaller for more modern techniques than for older ones. But whatever the threshold may be for a given technique, there will be some cases in which the available DNA is barely above the threshold. In such cases, the first test will exhaust the DNA, leaving little or
108. Note, for example, the discussion of quality assurance and quality control in NRC II, supra note 10, at 76-78, (among other things endorsing the recommendations of NRC I, supra note 8, at 104-05, for thorough testing before implementation in casework). To gauge the extent of effort necessary to validate one particular collection of markers, see R. Sparkes et al., The validation of a 7-locus multiplex STR test for use in forensic casework (I): Mixtures, ageing, degradation and species studies, 109 INTL. J. LEGAL MED. 186 (1996); R. Sparkes et al., The validation of a 7-locus multiplex STR test for use in forensic casework (II): Artefacts, casework studies and success rates, 109 INTL. J. LEGAL MED. 195 (1996). In the words of Dr. Peter Gill, Head of Research at the United Kingdom Forensic Science Service and one of the authors of these papers, the development of a system of satisfactory markers involves "many person-years of work." Personal Communication with Peter Gill (Aug. 1998).
109. Note, for example, the discussion of quality assurance and quality control in NRC II, supra note 10, at 76-78, (among other things endorsing the recommendations of NRC I, supra note 8, at 104-05, for thorough testing before implementation in casework). To gauge the extent of effort necessary to validate one particular collection of markers, see R. Sparkes et al., The validation of a 7-locus multiplex STR test for use in forensic casework (I): Mixtures, ageing, degradation and species studies, 109 INTL. J. LEGAL MED. 186 (1996); R. Sparkes et al., The validation of a 7-locus multiplex STR test for use in forensic casework (II): Artefacts, casework studies and success rates, 109 INTL. J. LEGAL MED. 195 (1996). In the words of Dr. Peter Gill, Head of Research at the United Kingdom Forensic Science Service and one of the authors of these papers, the development of a system of satisfactory markers involves "many person-years of work." Personal Communication with Peter Gill (Aug. 1998).
110. For a modern advocacy of the "best evidence" principle as the force behind much of evidentiary law, see Dale A. Nance, The Best Evidence Principle, 73 IOWA L. REV. 227 (1988).
111. See, e.g., Larry Laudan, Why do (natural) scientists agree?, in INTERPRETING THE WORLD 89 (William R. Shea & Antonio Spadafora eds., 1992). Although controversy remains among philosophers of science over the extent to which such objectivity is attainable, we do not intend to address this issue here. For our purposes, the general aspiration of science towards objectivity is the critical point.
112. 5, or 1/32, of all repetitions.
113. 10, which equals approximately 73%.
114. See generally JOHN EARMAN, BAYES OR BUST (1992); GERD GIGERENZER ET AL., THE EMPIRE OF CHANCE (1989).
115. See generally JOHN EARMAN, BAYES OR BUST (1992); GERD GIGERENZER ET AL., THE EMPIRE OF CHANCE (1989).
116. NRC I, supra note 8, at 124.
117. See id. at 129; supra text accompanying note 38.
118. Thus, although an American court in a civil case has the power to throw out a verdict as against the great weight of the evidence, if the case goes to the jury then a verdict for either side will usually be upheld. See, e.g., Wilkerson v. McCarthy, 336 U.S. 53, 55 (1949) ("This Court has previously held in many cases that where jury trials are required, courts must submit the issues of negligence to a jury if evidence might justify a finding either way on those issues.").
119. FED. R. EVID. 401.
120. The presumption of innocence demands that, before the presentation of evidence, the jury assign a very low probability to the proposition that the defendant was the perpetrator: in particular, the jury must not allow the fact that the prosecution has chosen to bring charges against the defendant, alone among all other people on earth, to elevate that probability.
121. See, e.g., PETER HUBER, GALILEO'S REVENGE: JUNK SCIENCE IN THE COURTROOM (1991).
122. 293 F. 1013, 1014 (D.C. Cir. 1923).
123. See, e.g., People v. Leahy, 882 P.2d 321 (Cal. 1994) (declining to follow Daubert and continuing to apply California's version of Frye, commonly referred to as Kelly-Frye); People v. Peterson, 537 N.W.2d 857 (Mich. 1995).
124. FED. R. EVID. 702.
125. 509 U.S. 579 (1993).
126. 509 U.S. at 589.
127. 509 U.S. at 597, 592.
128. 509 U.S. at 590 & n.8.
129. 509 U.S. at 590.
130. 509 U.S. at 592-95.
131. See, e.g., Heidi Li Feldman, Science and Uncertainty in Mass Exposure Litigation, 74 TEXAS L. REV. 1 (1995); Brian Leiter, The Epistemology of Admissibility: Why Even Good Philosophy of Science Would Not Make for Good Philosophy of Evidence, 1997 B.Y.U. L. REV. 803 (1997).
132. See, e.g., Heidi Li Feldman, Science and Uncertainty in Mass Exposure Litigation, 74 TEXAS L. REV. 1 (1995); Brian Leiter, The Epistemology of Admissibility: Why Even Good Philosophy of Science Would Not Make for Good Philosophy of Evidence, 1997 B.Y.U. L. REV. 803 (1997).
133. FED. R. EVID. 702. Justice Blackmun did purport to focus on the "helpfulness" standard - but he did so by melding that issue into the one of scientific validity. See Daubert, 509 U.S. at 591-92 ("Rule 702's 'helpfulness' standard requires a valid scientific connection to the pertinent inquiry as a precondition to admissibility.").
134. 118 S. Ct. 512 (1997).
135. 118 S. Ct. at 522 n.4 (quoting David Teitelbaum).
136. 118 S. Ct. at 522 (Stevens, J., dissenting).
137. 118 S. Ct. at 516.
138. 118 S. Ct. at 519.
139. See, e.g., EVETT & WEIR, supra note 13, at 240-41.
140. But see, e.g., Peitzmeier v. Hennessy Indus., Inc., 97 F.3d 293, 297 (8th Cir. 1996) (applying Daubert criteria, over plaintiffs' protests that the expert's testimony was to matters of basic engineering), cert. denied, 117 S. Ct. 1552 (1997). Cf. Gier v. Educational Serv. Unit No. 16, 66 F.3d 940, 943-44 (8th Cir. 1995) (applying Daubert to psychological evaluations in child custody cases).
141. See, e.g., United States v. Starzecpyzel, 880 F. Supp. 1027 (S.D.N.Y. 1995) (holding that the work of forensic document examiners is not genuine science, but neverthe ess admitting an opinion as a non-scientific one in a case of forgery detection because it could help the jury in focusing on minute similarities and dissimilarities between two documents that laypeople might have missed).
142. 131 F.3d 1433 (11th Cir. 1997), cert. granted sub nom. Kumho Tire Co., Ltd. v. Carmichael, 118 S. Ct. 2339 (1998).
143. Daubert v. Merrell-Dow Pharmaceuticals, 509 U.S. 579, 597 (1993).
144. See Richard D. Friedman, The Death and Transfiguration of Frye, 34 JURIMETRICS J. 133, 147 (1994) ("[S]ometimes courts ought to be willing to allow juries to take advantage of scientific information even when the scientific establishment is unwilling to declare a conclusion.").
145. Stockmarr, supra note 11, at 15.
146. See generally Leiter, supra note 106, at 805, 814 (arguing that the questions "What is the best account of scientific method?" and "What is the best criterion for judges to use in deciding the admissibility of science evidence?" must be kept separate, and posing the question of "social epistemology:" "[U]nder the real-world epistemic limits of a particular social process for the acquisition of knowledge, what epistemic norms actually work the best?").
147. The law sometimes attaches serious consequences to the matter of an individual's mental condition or constitution. Where those criteria do not accord well with criteria ordinarily used by psychiatrists, or where the law asks psychiatrists for predictions that the profession believes cannot be made with confidence, many psychiatrists naturally chafe. See, e.g., Barefoot v. Estelle, 463 U.S. 880 (1983) (over dissent, rejecting position of American Psychiatric Association that psychiatric predictions of long-term future dangerousness are too unreliable to be admissible); Statement of Dr. Lawrence C. Kolb, Director, N.Y.S. Psychiatric Institute, at Second Circuit Conference, 37 F.R.D. 365, 387 (1964) ("[M]any of our younger people have very little interest today in offering themselves as experts in court in view of the constricted framework in which they believe their testimony may be cast, a framework which is considered irrelevant to their areas of competence."). At times, the response of the courts in the context of the insanity defense has been to defer, allowing testimony in terms of criteria approved by the psychiatric profession. See, e.g., 2 PAUL H. ROBINSON, CRIMINAL LAW DEFENSES § 173(b)(3) (1984) (noting and criticizing "a common tendency to define the mental illness disability in terms of psychiatric labels"). We do not wish to minimize the difficulties; if, for example, predictions of violence cannot be made to an acceptable degree of confidence, then it is probably unwise to create a legal rule that depends on such predictions. But determining what constitutes sufficient mental capacity for a person to be held responsible for a crime is ultimately a normative judgment. See, e.g., United States v. Brawner, 471 F.2d 969, 983, 981 (D.C. Cir. 1972) (en banc) (rejecting the Durham rule - which asked whether the crime was the "product" of a mental disease or defect and "was adopted in large part to permit experts to testify in their own terms" - principally because the rule was responsible for the "undesirable characteristic . . . of undue dominance by the experts giving testimony"): McDonald v. United States, 312 F.2d 847, 851 (D.C. Cir. 1962) (en banc) ("[N]either the court nor the jury is bound by ad hoc definitions or conclusions as to what experts state is a disease or defect. What psychiatrists may consider a 'mental disease or defect' for clinical purposes, where their concern is treatment, may or may not be the same as mental disease or defect for the jury's purpose in determining criminal responsibility."); 312 F.2d at 861 (Miller, C.J., concurring in part) ("Until now, this court has allowed the shifting wind of expert nomenclature to control its decisions."); ROBINSON, supra, § 173(b)(3) (emphasizing "the community's view" as "central to the proper operation of a system of excuses"). Psychiatrists can be enormously helpful, both in advising on the practicality and consequences of any test and in providing information and opinions to help apply it. At bottom, though, crafting the test is a matter of law.
148. For example, scientific experts sometimes testify that a given result is "consistent" with a proposition at issue. See, e.g., ROBERTSON & VIGNAUX, supra note 20, at 56. This tendency may well be a result of a phenomenon on which we have commented, the hesitancy of scientists to reach definitive conclusions. It may also reflect a habit of thinking consonant with classical statistics, in which an analyst seeks to determine the validity of a proposition by subjecting it to one or more tests, rejecting the proposition if it is inconsistent with the test results. In any event, this manner of testifying is unhelpful for the law. An adjudicative factfinder needs to know the relative likelihood of the result given the truth of the proposition at issue and given its falsity. This information is obscured by testimony in terms of consistency. We offer one more example from the realm of statistics. Meta-analysis is the use of formal statistical techniques to provide a quantitative synthesis of multiple studies, each of which may in itself be insufficient to yield significant results. The use of meta-analysis has been controversial in various disciplines, see, e.g., Jack W. Snyder, Silicone Breast Implants, 18 J. LEG. MED. 133, 209 (1997). We do not wish here to suggest criteria for the use of metaanalysis in adjudication, or to minimize the difficulties of drawing conclusions from a body of studies that may use somewhat different methodologies. But our model of the law as an aggressive consumer suggests that courts should probably be willing - under the necessity of drawing the best conclusion possible given the available data - to allow inferences to be drawn from meta-analyses under criteria that in non-forensic contexts would appear unduly lax.
149. For example, scientific experts sometimes testify that a given result is "consistent" with a proposition at issue. See, e.g., ROBERTSON & VIGNAUX, supra note 20, at 56. This tendency may well be a result of a phenomenon on which we have commented, the hesitancy of scientists to reach definitive conclusions. It may also reflect a habit of thinking consonant with classical statistics, in which an analyst seeks to determine the validity of a proposition by subjecting it to one or more tests, rejecting the proposition if it is inconsistent with the test results. In any event, this manner of testifying is unhelpful for the law. An adjudicative factfinder needs to know the relative likelihood of the result given the truth of the proposition at issue and given its falsity. This information is obscured by testimony in terms of consistency. We offer one more example from the realm of statistics. Meta-analysis is the use of formal statistical techniques to provide a quantitative synthesis of multiple studies, each of which may in itself be insufficient to yield significant results. The use of meta-analysis has been controversial in various disciplines, see, e.g., Jack W. Snyder, Silicone Breast Implants, 18 J. LEG. MED. 133, 209 (1997). We do not wish here to suggest criteria for the use of metaanalysis in adjudication, or to minimize the difficulties of drawing conclusions from a body of studies that may use somewhat different methodologies. But our model of the law as an aggressive consumer suggests that courts should probably be willing - under the necessity of drawing the best conclusion possible given the available data - to allow inferences to be drawn from meta-analyses under criteria that in non-forensic contexts would appear unduly lax.
150. See Generally Peter H. Schuck, Multi-Culturalism Redux: Science, Law, and Politics, 11 YALE L. & POL. REV. 1 (1993). Schuck "advocate[s] a criterion of cultural competence for allocating decisional authority over multi-cultural issues," id. at 3, and cautions: Scientists must remember that it is lawyers and politicians who formulate the public rules for our complex society. Lawyers must remember that the project of science demands, and generally deserves, a kind of freedom to which legal controls will often be inimical. Politicians must remember that science . . . is to some irreducible degree an elite enterprise that cannot flourish under the incubus of a militant populism.
151. For analyses of more complex and general versions of the problem, see Balding & Donnelly, Evaluating DNA Profile Evidence, supra note 13, and Balding & Donnelly, Inference in Forensic Identification, supra note 13.
152. For analyses of more complex and general versions of the problem, see Balding & Donnelly, Evaluating DNA Profile Evidence, supra note 13, and Balding & Donnelly, Inference in Forensic Identification, supra note 13.
153. In reality, learning information about the profiles of some members of the population gives new information bearing on the probability that any other member of the population will have the profile in question. We will disregard this complexity for purposes of this discussion.
154. 2), yields Equation (An1).
155. For a more general derivation, see Balding & Donnelly, Evaluating DNA Profile Evidence, supra note 13, at 607.
156. In xl(D+N), which is multiplied by R to form the posterior odds in the confirmation case, the numerator of x reflects the elevated prior odds, while the denominator of (D+N) reflects the fact that the DNA evidence has not eliminated any other suspects. By contrast, in 1/N, which is multiplied by R to form the posterior odds in the variant of the trawl case being considered here, the numerator of 1 reflects prior odds that did not distinguish Matcher from the rest of the population, while the denominator of N reflects the fact that the DNA evidence has eliminated D alternative suspects.