Signature identification in the light of science and experience

Hastings Law Journal

Volumn 59, Issue 5, 2008, Pages 1101-1157

  Park, Roger C ^a  

^a UNIVERSITY OF CALIFORNIA   (United States)

Author keywords

[No Author keywords available]

Indexed keywords

EID: 49649101895     PISSN: 00178322     EISSN: None     Source Type: Journal    
DOI: None     Document Type: Conference Paper

References (252)

1. Daubert v. Merrill Dow Pharm., Inc., 509 U.S. 579, 593 (1993). See also David L. Faigman, Is Science Different for Lawyers?, 297 SCIENCE 339, 339-40 (2002);
2. David L. Faigman, Making the Law Safe for Science: A Proposed Rule for the Admission of Expret Testimony, 35 WASHBURN L.J. 401 (1996).
3. See Michael J. Saks & Jonathan J. Koehler, The Coming Paradigm Shift in Forensic Identification Science, 309 SCIENCE 892, 893 (2005).
4. For interesting accounts of human progress in technology and agriculture in prehistoric times, see JARED DIAMOND, GUNS, GERMS, AND STEEL: THE FATES OF HUMAN SOCIETIES (2005),
5. and NICHOLAS WADE, BEFORE THE DAWN: RECOVERING THE LOST HISTORY OF OUR ANCESTORS (2006).
6. I am using the term "experience" because it is familiar. I intend to exclude expertise that is based on personal experience, on the reported experience of others, and on introspection. Other terms that might describe what I am talking about are "fireside inductions" or "unsystematic inductions."
7. Fireside inductions are needed when we make judgments about the internal and external validity of experiments. For example, common-sense inductions tell us that handwriting experts might try harder on proficiency tests than lay subjects, so that differences in performance could be due to effort instead of expertise; and that studies showing superior expert performance on signature authentication might not generalize to other tasks, such as attribution of authorship in cases of disguised hand printing. D. Michael Risinger. Handwriting Identification, in 4 MODERN SCIENTIFIC EVIDENCE: THE LAW AND SCIENCE OF EXPERT TESTIMONY 379, §33:14 (David L. Faigman et al. eds., 2007-2008 ed.) [hereinafter MODERN SCIENTIFIC EVIDENCE].
8. Cf. Old Chief v. United States, 519 U.S. 172, 180, 182-84 (1997) (describing an analogous process under Rule 403 of weighing prejudice and other costs against probative value while considering alternative means of proof).
9. See authorities cited infra notes 38-47.
10. See United States v. Starzecpyzel, 880 F. Supp. 1027, 1041, 1046 (S.D.N.Y. 1995).
11. This is a supposition on my part. It seems plausible that harbor pilots would get negative feedback upon going aground or damaging the ship. For an example, see Henry K. Lee & Carl Nolte, Cosco Busan Pilot Charged with Pair of Crimes, S.F. CHRONICLE, Mar. 18, 2008, at B1 (reporting that harbor pilot who crashed into Bay Bridge was indicted for misdemeanors based on negligent conduct, including sailing in the fog, speeding, failing to check navigation equipment, and not using radar). However, they may make other mistakes that go undetected or unpunished. For example, in the absence of systematic record-keeping and controlled testing, experience-based experts can have mistaken beliefs that go undetected because they take the form of unnecessary precautions.
12. For examples of unnecessary precautions, see Jack Foisie, U.S. Pilots Fly Modern Jets but Cling to Superstitions, 30 W. FOLKLORE 140 (1971).
13. Mark P. Denbeaux & D. Michael Risinger, Kumho Tire and Expert Reliability: How the Question You Ask Gives the Answer You Get, 34 SETON HALL L. REV. 15, 56 (2003).
14. Id.; see also Randolph N. Jonakait, The Assessment of Expertise: Transcending Construction, 37 SANTA CLARA L. REV. 301, 342-43 (1997).
15. Andre A. Moenssens, Meeting the Daubert Challenge to Handwriting Evidence: Preparing for a Daubert Hearing, http://forensic-evidence.com/ site/ID/ID_FBI.html (last visited Apr. 20, 2008). When the subject of testing and validity comes up, it should also be pointed out that no research has ever surfaced that denies the existence of the skill of competent handwriting examiners or that proves that such skill does not exist! In other words, the only critical publications are the Risinger-Denbeaux-Saks articles, which do not deny explicitly the existence of the skill, but state only that they have not been convinced the skills exists. Their disbelief does not constitute proof of the non-existence of the skill of handwriting examiners. There are no studies showing that the skill of competent forensic document examiners in identifying authors of handwritings does not exist. The critics have it backwards. Id.
16. See infra Part III.
17. See David Black et al., The Frequency of the Occurrence of Handwriting Performance Features Used to Predict Whether Questioned Signatures Are Simulated, 15 J. FORENSIC DOCUMENT EXAMINATION 17 (2003). The investigators had thirty-one participants create 620 simulated signatures. Id. These were compared to 177 genuine signatures produced by one participant. Id. An FDE was asked to examine the genuine and simulated signatures and count blunt endings. Id. at 22. The FDE found blunt endings in 16.9% of the genuine signatures and 72.7% of the simulated signatures. Id. at 23 tbl. 1. The value of the study is diminished by the fact that a single subject created all of the genuine signatures; it is possible that her genuine signature was unusually devoid of blunt endings.
18. Michael J. Saks, Forensic Identification: From a Faith-Based "Science" to a Scientific Science. FORENSIC SCI. INT'L (forthcoming 2008).
19. See United States v. Starzecpyzel, 880 F. Supp. 1027, 1048 (S.D.N.Y. 1995) ("With regard to scientific experts, a major rationale for Frye, and now Daubert, is that scientific testimony may carry an 'aura of infallibility.' . . . Skilled experts generally present less of a problem, as, with all due respect, accountants are unlikely bearers of an aura of infallibility." (citation omitted)).
20. See Jonakait, supra note 11, at 308 (citing Anthony Champagne et al., An Empirical Examination of the Use of Expert Witnesses in American Courts, 31 JURIMETRICS J. 375, 380 (1991) (proposing that "jurors more willingly accept experts who present information nontechnically and give firm conclusions than those who do not")).
21. For this "translation," see John I. Thornton & Joseph L. Peterson, The General Assumptions and Rationale of Forensic Identification, in MODERN SCIENTIFIC EVIDENCE, supra note 5, § 29:21. Cf. Jonakait, supra note 11, at 310 & n.30 ("[T]he fact-finder need never take a scientific expert witness' 'word for it.'" (quoting MICHAEL J. SAKS & RICHARD VAN DUIZEND, THE USE OF SCIENTIFIC EVIDENCE IN LITIGATION 5 (1983)) (alteration in original)).
22. Thornton & Peterson, supra note 18.
23. D. Michael Risinger et al., The Daubert/Kumho Implications of Observer Effects in Forensic Science: Hidden Problems of Expectation and Suggestion, 90 CAL. L. REV. 1, 21 (2002).
24. Jennifer L. Mnookin, The Achilles' Heel of Fingerprints, WASH. POST, May 29, 2004, at A27;
25. see also Robert B. Stacey, Report on the Erroneous Fingerprint Individualization in the Madrid Train Bombing Case, 7 FORENSIC SCI. COMM. (2005), http://www.fbi.gov/hq/lab/fsc/backissu/ jan2005/special_report/2005_special_report.htm. Mr. Stacey, Unit Chief of the FBI Quality Assurance and Training Unit, concluded that context effect influenced the initial identification (though he did not specifically state that the examiners were aware of Mr. Mayfield's Muslim connections), and that subsequent identifications were "tainted" because, among other things, the subsequent examiners were aware of the previous examiner's conclusion and "[t]o disagree was not an expected response." Id.
26. D. Michael Risinger, Functional Taxonomy of Expertise, in 1 MODERN SCIENTIFIC EVIDENCE, supra note 5, § 2:3.
27. Id.
28. See generally Marvin L. Simner et al., A Comparison of the Arabic Numerals One Through Nine, Written by Adults from Native English-Speaking vs. Non-Native English Speaking Countries, 15 J. FORENSIC DOCUMENT EXAMINATION 1 (2003) (offering an example of a study providing information about this sort of class characteristic). The Simner et al. study found, among other things, that among writers of Arabic numbers, 97% of the 113 German university students surveyed wrote crossed sevens sometimes also adding a hook, whereas 98% of the 86 Japanese students surveyed wrote hooked sevens without a cross. Id. at 14 tbl.7.
29. United States v. Fujii, 152 F. Supp. 2d 939, 941-42 (N.D. Ill. 2000).
30. Id. at 939-40.
31. Id. at 941 ("Michael Saks, who testified for the defense, testified that he was aware of only one study of the reliability of handprinting identification, and in that study, only 13% of the handwriting experts tested got the right answer; 45% identified the wrong person."); Risinger, supra note 5, at 513 (describing the hand printing proficiency test). It seems to have been a particularly hard test, in which a professional document examiner wrote a mock holdup note simulating the hand printing of another subject, and 45% of the subjects assigned authorship to the forger instead of to the actual author. See id.
32. Risinger, supra note 5, at 505 (citing ALBERT S. OSBORN, QUESTIONED DOCUMENTS 13-14 (1910) [hereinafter OSBORN, 1910 edition]).
33. Fujii, 152 F. Supp. 2d at 941.
34. Id.
35. 526 U.S. 137, 141, 153, 157-58 (1999).
36. See id. at 137.
37. See Edward J. Imwinkelried, Should the Courts Incorporate a Best Evidence Rule into the Standard Determining the Admissibility of Scientific Testimony?: Enough Is Enough Even When It Is Not the Best, 50 CASE W. RES. L. REV. 19, 33 (1999).
38. See David L. Faigman et al., How Good Is Good Enough?: Expert Evidence Under Daubert and Kumho, 50 CASE W. RES. L. REV. 645, 667 (2000); Imwinkelried. supra note 33, at 28.
39. Moreover, observational studies cannot necessarily fill the gap in the behavioral area, because of problems with determining the ground truth. Rape trauma syndrome is an example. In determining the effects of rape for purposes of distinguishing between rape victims and complainants making false claims of rape, the best comparison would be between women who accurately claim to have been raped and women who falsely claim to have been raped. This comparison is not feasible, so a comparison of the symptoms of women who report being raped and those who report not being raped is substituted for it.
40. Edward J. Imwinkelried, Flawed Expert Testimony: Striking the Right Balance in Admissibility Standards, 18 CRIM. JUST. 28, 29 (2003).
41. For studies on helpfulness of expert testimony about eyewitnessing conditions, see BRIAN L. CUTLER & STEVEN D. PENROD, MISTAKEN IDENTIFICATION: THE EYEWITNESS, PSYCHOLOGY AND THE LAW 19-54 (1995).
42. See also Roger C. Park, Eyewitness Identification: Expert Witnesses Are Not the Only Solution, 2 LAW PROBABILITY & RISK 305, 306-07 (2003).
43. Jennifer L. Mnookin, Scripting Expertise: The History of Handwriting Identification Evidence and the Judicial Construction of Reliability, 87 VA. L. REV. 1723, 1764-66 (2001) (citing nineteenth century cases excluding handwriting exemplars offered solely for the purpose of allowing the jury to make comparisons of handwriting).
44. See Old Chief v. United States, 519 U.S. 172, 188 (1997) (citing Stephen A. Saltzburg, A Special Aspect of Relevance: Countering Negative Inferences Associated with the Absence of Evidence, 66 CAL. L. REV. 1011, 1019 (1978) ("If [jurors'] expectations are not satisfied, triers of fact may penalize the party who disappoints them by drawing a negative inference against that party.")).
45. Federal Rule of Evidence 901(b)(3) provides that authentication may be accomplished by "[c]omparison by the trier of fact or by expert witnesses with specimens which have been authenticated." The Advisory Committee Note expresses disapproval of "common law restrictions upon the technique of proving or disproving the genuineness of a disputed specimen of handwriting through comparison with a genuine specimen, by either the testimony of expert witnesses or direct viewing by the triers themselves." FED. R. EVID. 901 advisory committee's note. The restrictions referred to were reservation to the judge of the question of genuineness of exemplars and "imposition of an unusually high standard of persuasion." Id. Of course, the advisory committee note was not enacted by Congress, and is merely a guide; moreover, its permissive attitude towards experts is arguably obsolete in light of the requirements of Federal Rule of Evidence 702, codifying the Daubert case. See United States v. Saelee, 162 F. Supp. 2d 1097, 1101 (D. Alaska 2001). However, it would seem odd to prevent comparisons by the trier of fact in view of this language and the provision in Federal Rule of Evidence 901(b)(2) allowing nonexperts to compare handwriting and give opinions. Federal Rule of Evidence 901(b)(2) specifically provides for authentication of handwriting by "[n]onexpert opinion as to the genuineness of handwriting, based upon familiarity not acquired for purposes of the litigation."
46. See United States v. Hines, 55 F. Supp. 2d 62, 70-71 (D. Mass. 1999); United States v. Starzecpyzel, 880 F. Supp. 1027, 1030, 1051 (S.D.N.Y. 1995).
47. Risinger et al., supra note 20, at 45-47.
48. Id. at 47-50.
49. D. Michael Risinger et al., Exorcism of Ignorance as a Proxy for Rational Knowledge: The Lessons of Handwriting Identification "Expertise," 137 U. PA. L. REV. 731 (1989).
50. Id. at 747.
51. Id. at 748.
52. See Risinger, supra note 5, at 379; D. Michael Risinger & Michael J. Saks, Science and Nonscience in the Courts: Daubert Meets Handwriting Identification Expertise, 82 IOWA L. REV. 21, 41-64 (1996).
53. Risinger, supra note 5, at 379.
54. See Kumho Tire Co. v. Carmichael, 526 U.S. 137, 141 (1999); Risinger, supra note 5, at 389.
55. As Professor Risinger notes, standard FDE theory holds that it is much more difficult to attribute authorship to a forgery than it is to determine that a signature is not genuine. Risinger, supra note 5, at 394 n.13 (citing ALBERT S. OSBORN, QUESTIONED DOCUMENTS 286-87 (2d ed. 1929) [hereinafter OSBORN, 1929 edition]); see also WILSON R. HARRISON, SUSPECT DOCUMENTS: THEIR SCIENTIFIC EXAMINATION 374 (1966);
56. Ordway Hilton, Can the Forger Be Identified from His Handwriting?, 43 J. CRIM. L. & CRIMINOLOGY 547, 547, 548, 555 (1953); sources cited infra note 215.
57. These are indeed just examples. Other conceivable subtasks would vary according to the number of exemplars available, whether the exemplars were demand exemplars (written with knowledge that they might be used for forensic identification) or course-of-business exemplars, or whether the person named in the questioned signature was in a different physical condition (older, sicker, in more pain) than when that person's exemplars were collected.
58. I say "named in the signature" despite the fact that some signatures are logos that cannot really be said to contain discernible letters of a name. The reports on the studies that I will describe give me no reason to think that logo signatures were predominant or even strongly represented in the studies.
59. This might occur because he lacks a model or because he does not think the signature will ever be questioned.
60. One could argue for different terminology. For example, Moshe Kam uses the terms "false authentication" and "false simulation- detection" for the two types of error. Moshe Kam et al., Signature Authentication by Forensic Document Examiners, 46 J. FORENSIC SCIS. 884, 886 (2001). I have opted for the term "false elimination" for two reasons. First, it is shorter. Second, the studies that I will be describing (with a notable exception) do not specifically ask the subjects to distinguish between genuine signatures and "simulated" signatures.
61. See Jodi Sita et al., Forensic Handwriting Examiners' Expertise for Signature Comparison, 47 J. FORENSIC SCIS. 1117, 1118 (2002). Instead, they ask whether the signature is genuine or not, or some very similar question. In addition, there is a difference between the concepts of "false elimination" and "false simulation-detection." A signature might be forged without simulation (the forger does not know what the genuine signature looks like, only the name of the person he is seeking to stand for) or simulated without forgery (seeking deniability, the person named in the signature makes an obvious tracing of his own signature, or creates a freehand disguise of his handwriting while writing his own name - for example, a right-handed person writes his name with his left hand, and tries to inject differences in letter slant and character formation).
62. See COLLABORATIVE TESTING SERVS., INC., CRIME LABORATORY PROFICIENCY TESTING PROGRAM: QUESTIONED DOCUMENTS ANALYSIS, REPORT NO. 88-5 (1988) [hereinafter CTS REPORT NO. 88-5] (on file with author);
63. COLLABORATIVE TESTING SERVS., INC., CRIME LABORATORY PROFICIENCY TESTING PROGRAM: QUESTIONED DOCUMENTS ANALYSIS, REPORT NO. 85-8 (1985) [hereinafter CTS REPORT NO. 85-8] (on file with author). CTS is a private company that tests the performance of forensic and industrial labs. When the Forensic Science Foundation received grants from the U.S. Justice Department in the 1970s to test the proficiency of crime labs, CTS became a subcontractor and developed tests. See Thornton & Peterson, supra note 18, at 41 n.4. After the end of the federal grant, it continued on a fee-for-service basis. Id. The Exorcism article examined a number of additional proficiency tests in reaching the conclusions in the quoted paragraphs above. See supra notes 44-46 and accompanying text. While the Exorcism article was examining all tested aspects of handwriting expertise; I am limiting myself to the specific task of signature authentication.
64. See infra Part III.
65. For examples of difficult tasks on which experts performed no better than what would be expected by chance, see Oliver Galbraith III et al., The Principle of the "Drunkard's Search" as a Proxy for Scientific Analysis: The Misuse of Handwriting Test Data in a Law Journal Article, 1 INT'L J. FORENSIC DOCUMENT EXAMINERS 7, 14-16 (1995).
66. CTS REPORT NO. 85-8, supra note 55. Professor Michael Risinger kindly provided me with copies of the CTS summary reports on this test and on the 1988 CTS test discussed below. The summary reports are the feedback given collectively to participating laboratories. They contain brief introductory remarks, the scenarios posed in the tests, the correct answers, and the results for individual laboratories (laboratory anonymity is protected by a coding system). Reports on the tests are not otherwise published. Risinger, supra note 5, at 521-22.
67. Risinger et al., supra note 44, at 745-76.
68. Galbraith III et al., supra note 57, at 7.
69. Risinger, supra note 5, at 383.
70. CTS REPORT NO. 85-8, supra note 55, at 4-9. The proposition in text is this Author's inference from reading the narrative answers of the text takers.
71. Id. at 2 tbl.1.
72. This is the Author's inference from the description of the task in the manufacturer's information. See id.
73. Id. at 2 tbl.2.
74. Id.
75. Id.
76. See Galbraith III et al., supra note 57, at 11, 14. The Galbraiths calculate the percentage correct in alternative ways, first by discarding inconclusives (their method 1, yielding 59% correct), then by analyzing the inconclusive results to see whether the document examiners gave qualified opinions of authorship (their method 2, yielding 55% correct). Id.
77. See CTS REPORT NO. 85-8, supra note 55, at 1.
78. See supra note 66.
79. See CTS REPORT NO. 85-8, supra note 55, at 2 tbl.1.
80. See id. at 1.
81. See id.
82. Galbraith III et al., supra note 57, at 14.
83. See CTS REPORT NO. 85-8, supra note 55, at 2-3 tbl.2. I have excluded responses that declined to express an opinion (e.g., "inconclusive" answers). This eliminated the data from ten responding labs, leaving twenty-two respondents. That decision increased both the error rate and the correct answer rate (compared with the alternatives of giving percentages for three categories: "correct," "inconclusive," and "error"). Where an answer said certain pairs of signatures came from the same source and certain other pairs "possibly" came from the same source, I have treated the "possibly" answers as being authentications. That was a decision that disfavored the document examiners. Had the "possibly" answers been treated as exclusions, there would have been eight fewer same-source errors.
84. See id.
85. Technically, this should be described as "false authentication of handwriting of someone signing another's name as the same handwriting as that contained on specimens provided by another person who was signing a name not her own," an arguably harder task. See infra note 161.
86. Technically, this should be described as "false authentication of the handwriting of someone simulating specimens signed by a person who was not signing her own name," an arguably harder task. See infra note 161.
87. Each of the twenty-two responding FDEs faced five such pairs. CTS REPORT NO. 85-8, supra note 55, at 2-3 tbl.2. Depending on the test package, #6 was a simulation of either #3 or #8, and #11 was a simulation of either #3 or #8. Id. at 2-3 tbls.1 & 2. Since #3 and #8 were written by the same person, I am treating a simulation of #3 as also being a simulation of #8, and vice versa. Also, simulations of a signature from the same source are treated as simulations of each other (if you disagree with this judgment, count four pairs instead of five). Thus, the five simulation pairs were 3-6, 3-11, 6-8, 6-11, and 8-11.
88. See id. at 2-3 tbl.2.
89. See id. at 2-3 tbls.1 & 2. In order to have two signatures as known exemplars, the FDEs needed first to determine that Drinkwalter Signature #3 was written by the same person as Drinkwalter signature no. 8.
90. In fact, considering the lack of exemplars, one can make a case for treating "inconclusive" answers as also being correct, in which case the error rate would shrink to 11%.
91. Id. at 1.
92. Id. at 9 (answer of respondent 829).
93. Id. at 1.
94. Id.
95. CTS REPORT NO. 88-5, supra note 55.
96. Id. at 40.
97. Id.
98. Id.
99. Id.
100. Id.
101. I checked my work against Professor Risinger's tabulation of names. Our results were the same. See Risinger, supra note 5, at 522.
102. For a more detailed tabulation, see Appendix 3. "Called" answers are ones that state whether the signature is authentic or not, as opposed to "inconclusive" answers and other answers that do not decide the authentication issue. In calculating the percentage of correct answers, my inclination is to exclude the "inconclusive" and "other" answers. With one exception, the "other" answers either leaned toward the right answer or called for more evidence. The "inconclusive" answers may have also been based on the view that the evidence was insufficient. That is not necessarily a wrong answer. For example, if Ms. Neuman did in fact have a simple signature that she signed with significant variation, it seems reasonable to call for more exemplars or to conclude that the evidence was not enough for a conclusion.
103. CTS REPORT NO. 88-5, supra note 55, at 34-39 tbl.5.
104. Id. at 34-38 tbl.5.
105. See id.
106. See supra note 55.
107. See Risinger, supra note 5, at 558-65 (discussin COLLABORATIVE TESTING SERVS., QUESTIONED DOCUMENTS ANALYSIS, REPORT NO. 92-6 (1992); COLLABORATIVE TESTING SERVS., QUESTIONED DOCUMENTS EXAMINATION, REPORT NO. 9406 (1994);
108. COLLABORATIVE TESTING SERVS., QUESTIONED DOCUMENTS EXAMINATION, REPORT NO. 9606 (1996);
109. COLLABORATIVE TESTING SERVS., FORENSIC TESTING PROGRAM, HANDWRITING EXAMINATION, REPORT NO. 9714 (1997);
110. COLLABORATIVE TESTING SERVS., FORENSIC TESTING PROGRAM: HANDWRITING EXAMINATION, REPORT NO. 9814 (1998);
111. COLLABORATIVE TESTING SERVS., FORENSIC TESTING PROGRAM: HANDWRITING EXAMINATION, TEST NO. 99-524 (1999);
112. COLLABORATIVE TESTING SERVS., FORENSIC TESTING PROGRAM: HANDWRITING EXAMINATION, TEST NO. 00-524 (2000);
113. COLLABORATIVE TESTING SERVS., FORENSIC TESTING PROGRAM: HANDWRITING EXAMINATION, TEST NO. 01-524 (2001) [hereinafter CTS TEST NO. 01-524] (on file with author);
114. COLLABORATIVE TESTING SERVS., FORENSIC TESTING PROGRAM: HANDWRITING EXAMINATION, TEST NO. 02-524 (2003)).
115. See CTS TEST NO. 01-524, supra note 99. In my description, I have extracted the parts of the task that relate to the narrow task of signature authentication as I have defined it (determining whether the person named in the signature wrote the signature), and not to the other task presented in the test - that of deciding whether a simulated signature should be attributed to someone else. The attribution task may have made the authentication task more difficult on Q2, because none of the exemplars of other persons apparently suspected of simulating the signature were exemplars of the person who did simulate the signature, so the examinee who assumed that either the real Kenny Bania or one of the other suspects was the one who wrote the signature would have been led astray. This problem could, of course, also cause error in ordinary casework.
116. Id. at 2-4.
117. Id. at 2.
118. Id.
119. Id.
120. On Q1, 94% (123/131) accurately responded that Kenny Bania was the writer, and 6% (8/131) responded that he probably wrote the entry. Id. at 7. On Q3, 93% (122/131) accurately responded that Kenny Bania was the writer, and 7% (9/131) responded that he probably wrote the entry. Id. at 13.
121. Twenty FDEs positively eliminated him, fifty-five opted for probable elimination. Id. at 10. One reason why it may have been harder for the FDEs to eliminate than it was for them to identify is the possibility that the person named in the signature actually wrote the signature, but in a disguised hand so as to be able to later deny writing it. Id. at 3. Thus the detection of signs of simulation does not conclusively exclude the possibility that the person whose name appears in the signature did in fact write the signature. Sometimes the other case facts might make writing in a disguised hand highly unlikely, and this may be a situation in which it would be appropriate, at some point, to give the FDE access to other case facts even if those facts are initially screened so as to prevent observer bias. For example, cases involving allegedly forged wills are probably ones in which it seems highly unlikely that the real decedent will have signed her own signature in a disguised hand.
122. I am describing it as a questioned signature even though the signature was actually accompanied by a few numbers. To be precise, the task was whether to authenticate the signature and its accompanying numbers, but the signature seems to have provided most of the relevant information for performing the task.
123. See CTS TEST NO. 01-524, supra note 99, at 10.
124. My description of the Found and Rogers study comes from examination of a printout of a presentation setting forth their results. See BRYAN FOUND & DOUG ROGERS, REVISION AND CORRECTIVE ACTION PACKAGE: SIGNATURE TRIAL 2001 (on file with author). This study was distributed on CD-ROM by the Forensic Expertise Profiling Laboratory, School of Human Biosciences, La Trobe University, Australia, and provided to this Author by Michael Risinger.
125. Id. at 26 tbl.2. The raw scores were 1,628 correct, 30 wrong, and 105 inconclusive. Id. at 25 tbl.1.
126. Id. at 26 tbl.2. The raw scores were 2,840 correct, 265 wrong, and 3455 inconclusive.
127. Id. at 12.
128. Id. at 26 tbl.2.
129. Id. at 8.
130. Id. at 9.
131. Id. at 26 tbl.2.
132. Id. at 26 tbl.2.
133. See Risinger et al., supra note 20, at 21.
134. See, e.g., CTS REPORT NO. 88-5, supra note 55, at intro. Since it is the laboratory's option how the samples are to be used (e.g., training exercise, known or blind proficiency testing, research and development of new techniques), the results compiled in the summary report are not intended to be an overview of the quality of work performed in the profession , and cannot be interpreted as such. . . . They are included for the benefit of participating laboratory directors to assist them with maintaining or enhancing the quality of results from their individual laboratories. These comments are not intended to reflect the general state of the art within the profession. Id.
135. See CTS TEST NO. 01-524, supra note 99, at 3.
136. Id.
137. See Joseph L. Peterson et al., The Feasibility of External Blind DNA Proficiency Testing: Background and Findings, 48 J. FORENSIC SCIS. 21, 26 (2003) (presenting data indicating that in a variety of forensic fields, blind testing yields fewer positive calls than open testing).
138. For similar problems with blind proficiency testing by DNA labs, see Margaret A. Berger, Laboratory Error Seen Through the Lens of Science and Policy, 30 U.C. DAVIS L. REV. 1081, 1088-89 (1997).
139. See Risinger et al., supra note 20.
140. I have not included the pilot study comparing expert and lay performance set forth in Galbraith III et al., supra note 57, at 7, on grounds that it did not involve a signature authentication task.
141. Kam et al., supra note 54. Dr. Kam has done other studies comparing expert and lay performance on assignment of authorship to naturally written nonsignature handwriting, but those studies are not directly relevant to signature authentication, and they had methodological problems involving differential lay and expert incentives that were not present in the same degree in the 2001 study. For a description of these earlier studies, see Risinger, supra note 5, at 527-49.
142. Kam et al., supra note 54, at 884.
143. Id. at 885-86.
144. Id. at 885.
145. Id.
146. Id.
147. Id. at 885 (citing W. HARRISON, FORGERY DETECTION: A PRACTICAL GUIDE (1964)).
148. Id. at 885.
149. Id.
150. Id.
151. Id. at 885-86.
152. Id. at 886.
153. Id.
154. Id.
155. Id.
156. Id.
157. Id.
158. Id. at 887. The differences in error rates for the FDE and lay subjects were statistically significant. Id.
159. Bryan Found et al., The Development of a Program for Characterizing Forensic Handwriting Examiners' Expertise: Signature Examination Pilot Study, 12 J. FORENSIC DOCUMENT EXAMINATION 69, 75-76 (1999).
160. Id. at 72.
161. Id. at 76.
162. Id.
163. Id.
164. Id.
165. Id. at 71.
166. The lay subjects had a higher percentage of both correct answers and wrong answers because the experts made more frequent use of the option of answering "inconclusive."
167. Sita et al., supra note 54, at 1117.
168. Id. at 1118.
169. Id.
170. Id.
171. Id.
172. Id.
173. Id.
174. Id. at 1119.
175. Id.
176. Id. at 1118-19. For consistency of terminology, I am continuing to use the categories of "false authentication" and "false elimination," even though technically the errors might be described as "false missing of simulation" and "false detection of simulation." The reason is that the study asked the subjects to distinguish between "genuine" signatures and "simulated" signatures. They were instructed to answer "genuine" when "[t]he questioned signature is in your opinion written by the same person who wrote the standard signature group" and to answer "simulated" when "[t]he questioned signature is inconsistent with the standard signature group and displays features that you consider to be indicative of a copying process. Note that this term does not imply that the standard signature writer did not write it." Id. at 1118. This instruction seems hard to follow, even illogical (a person who traced his own signature would fall in both categories), and I am assuming that subjects called signatures "genuine" when they thought the signature had been written by the person named in the signature, and "simulated" when they thought the signature was not genuine.
177. Id. at 1120 tbl.5. The comparable called error rates in the 2001 Kam study were 7.59% for false elimination and 0.50% for false authentication. See Kam et al., supra note 54, at 887.
178. Sita et al., supra note 54, at 1118.
179. Id.
180. Id.
181. FOUND & ROGERS, supra note 109, at 8.
182. Sita et al., supra note 54, at 1120-21.
183. CTS REPORT NO. 85-8, supra note 55, at 34.
184. See Sita et al., supra note 54, at 1122.
185. On the other hand, unless the test makers were intentionally asking FDEs whether signatures were genuine or not without providing enough information to make the judgment, the experts that the test makers consulted in devising the test must have thought that there was enough data to justify an answer. Of course, it is possible that the test makers simply devised a plan for making the stimulus materials (have people sign their signatures and other people imitate them) without giving any subsequent thought to whether the results provided enough information for a judgment of genuineness.
186. Kam et al., supra note 54, at 886.
187. Id.
188. Sita et al., supra note 54, at 1123.
189. See D.H. Kaye, Revising Dreyfus: A More Complete Account of a Trial by Mathematics, 91 MINN. L. REV. 825, 827-28.
190. See supra tbl.2. If inconclusives are discarded, the lay subjects were correct on 93.4% of their calls, and the FDEs were correct on 99.5% of their calls. See supra tbl.3.
191. See supra tbl.4. The error rate for FDEs was 0.49%, for lay subjects 6.47%. The reason why the correct answers and errors do not sum to 100% is that subjects were also allowed to answer "inconclusive." On this task, 3.45% of the FDE answers were inconclusive, compared to 1.40% of the lay answers. See supra tbl.2.
192. See infra note 182.
193. See discussion infra Part IV.
194. See discussion infra Part IV.
195. Wolfgang Conrad, Empirische Untersuchungen über die Urteilsgate verschiedener Gruppen von Laien und Sachverständigen bei der Unterscheidung authentischer und gefälschter Unterschriften, 156 ARCHIV FÜR KRIMINOLOGIE 169-83 (1975), translated in Empirical Studies Regarding the Quality of Assessments of Various Groups of Lay Persons and Experts in Differentiating Between Authentic and Forged Signatures (Peter Bernhardt trans.) (unpublished manuscript on file with author). I am grateful to Michael Saks for providing me with the English translation by Peter Bernhardt of Dr. Conrad's article.
196. Id. at 2 (English translation).
197. Id. at 3.
198. Id. at 4 tbl.1.
199. A small group of six specially screened experts did substantially better than the generally-qualified experts, and a group of sixteen specially qualified experts did somewhat better. Id. at 4 tbl.1.
200. Id. at 2.
201. Id. at 6.
202. Id. at 6 tbl.1. Inconclusives were counted as errors on this comparison, and the experts may have suffered from use of this decision criteria, if, like the FDEs in the Kam and Sita studies, they were more prone to give inconclusive answers than were lay participants.
203. The educational level of the twenty-five experts was as follows: three had finished elementary school, six had received their general certificate of education from secondary school, and sixteen had passed their "Abitur," or graduation exam from Gymnasium qualifying them for university entrance. Of the "Abitur" experts, eleven had completed university studies. Id. at 2.
204. Id. at 6 tbl.1 (counting inconclusives as wrong answers).
205. See id. at 6 tbl.1. I am using Conrad's "error grade II" criteria, under which opinions were called with a safety grade of probable or higher. In other words, I have excluded inconclusives but accepted called opinions with any degree of certainty.
206. Id.
207. Id. at 8-9 tbls.2 & 3 (error grade II comparison).
208. Each group had a total of nine errors. Outlier performance by one or two members of either group could have had a material impact on the percentage performance figures.
209. United States, v. Hines. 55 F. Supp. 2d 62, 70-71 (D. Mass. 1999). I was not a case involving signature authentication, but rather a case in which the expert would have attributed authorship of a robbery note used in a bank robbery to the defendant. Id. at 62. Signatures are regarded as special by the document examiner community, and at any rate there is likely to be a greater danger of disguise when composing a note that one knows that police investigators will scrutinize (such as a "stick-up" note). Assigning authorship to disguised handwriting is a difficult task.
210. These are among the signs of simulation mentioned in the FDE literature. See ORDWAY HILTON, SCIENTIFIC EXAMINATION OF QUESTIONED DOCUMENTS (rev. ed. 1982);
211. OSBORN, 1929 edition. supra note 50; SCIENTIFIC EXAMINATION OF QUESTIONED DOCUMENTS 94 (Jan Seaman Kelly & Brian S. Lindbloom eds., 2d ed. 2006). Kelly and Lindbloom, sum up by saying that "[hesitation, unnatural pen lifts, patching, tremor, uncertainty of movement as portrayed by abrupt changes in the direction of the line, and a stilted, drawn quality devoid of free, normal writing movements combine to reveal the defective nature of a poor-quality simulation." Id. Their more complete catalogue of suspicious signs also includes: pooling of ink, id. at 82; uniformly heavy stroke, id. at 89; presence of a lightly drawn outline, id. at 90; and smearing caused by erasure of an outline, id. at 91. When one of the exemplars known to be genuine is suspected as being the model for a tracing, then wandering away from the stroke of the genuine signature and returning to the common track is indicative of tracing, as is indentation in the genuine signature thought to have been used as a model. Id. at 91-92. Suspicious signs noted in OSBORN, 1910 edition, supra note 28, include: evidence of erasure, id. at 45; pencil outline or impression on paper under the signature, id. at 77; uneven, slow, wiggly, rough, haltering, clumsy, hesitating, irregular writing in questioned document when authentic signatures were smooth and natural, id. at 109-11; blunt as opposed to tapering terminal strokes, id. at 114-15; deviation from uniform strokes (tremors), id. at 117; interruptions of movement in direct curves or straight lines, id. at 117; deviation from pen pressure or alignment of genuine exemplars, id. at 123-25, 133; and hesitation, abnormal changes of direction, inconsistent pen pressure, unnatural movement interruption, pen lifts, retouching, id. at 267-69. Where a genuine signature may have been used as a model, exact replication of the genuine signature in the questioned one is a sign of tracing. Id. at 276-77. See also OSBORN, 1929 edition, supra note 50, where signs of forgery include: tremor not attributed to age, weakness, or illiteracy; id. at 110; disconnections or pen-lifts, id. at 114; alignment not consistent with rest of document, id. at 115; and patching, id. at 129. For an empirical study of the frequency of some of these features in forged signatures, see infra note 209.
212. United States v. Starzecpyzel, 880 F. Supp. 1027, 1029, 1036 (S.D.N.Y. 1995).
213. Id. at 1048. The Starzecpyzel court prohibited the expert from using a misleadingly precise nine-point scale to express degrees of certainty.
214. Id. at 1051.
215. Id. at 1050.
216. See studies cited supra notes 126, 144, 180 (Kam, Sita, and Conrad studies).
217. See Risinger et al., supra note 20, at 10.
218. For skeptical assessments of demeanor evidence, see Roger C. Park, Empirical Evaluation of the Hearsay Rule, in ESSAYS FOR COLIN TAPPER 91 (Peter Mirfield & Roger Smith eds., 2003);
219. ALDERT VRII, DETECTING LIES AND DECEIT (2000);
220. and Olin Guy Wellborn III, Demeanor, 76 Cornell L. Rev. 1075 (1991).
221. Risinger, supra note 5, at 449.
222. Kumho Tire calls for a task-specific judgment. See Kumho Tire Co. v. Carmichael, 526 U.S. 137, 153-54 (1999). [T]he specific issue before the court was not the reasonableness in general of a tire expert's use of a visual and tactile inspection to determine whether overdeflection had caused the tire's tread to separate from its steel-belted carcass. Rather, it was the reasonableness of using such an approach, along with Carlson's particular method of analyzing the data thereby obtained, to draw a conclusion regarding the particular matter to which the expert testimony was directly relevant. That matter concerned the likelihood that a defect in the tire at issue caused its tread to separate from its carcass. Id.
223. Leading FDE authors have identified signature authentication as a special subtask. See HILTON, supra note 195, at 173 ("[T]he identification of signatures constitutes a specialized branch of handwriting examination."); JOE NICKELL, DETECTING FORGERY: FORENSIC INVESTIGATION OF DOCUMENTS 59 (1996); OSBORN, 1929 edition, supra note 50, at 18-94, 384; see also sources cited supra note 49.
224. See OSBORN, 1929 edition, supra note 50. In fact, cases in which it is possible that the questioned signature was written by the person named in the signature, but in a disguised hand (so that the person could later deny signing the document) also present this different, more difficult task-that of assigning true authorship to a disguised hand. That is another reason to limit testimony to similarities and differences and to signs of simulation.
225. HILTON, supra note 195, at 185.
226. See id.; see also S.C. Leung et al.. Forgery II-Tracing, 38 J. FORENSIC SCIS. 413, 423-24 (1993) (reporting the results of an empirical study, using a writing pressure meter, of signatures traced by 189 subjects). All the signatures traced were "highlighted by the pressure of a slow measured stoke accompanied with hesitation, pen pause and absence of vigor and spontaneity": the investigators measured " superimposability" (overlapping strokes of traced simulations with the model used) and found that "the probability that a questioned signature has been produced by tracing from another (genuine) signature is related to the superimposability of the two." Id.
227. See Black et al., supra note 14. and authorities cited therein. Black et al. collected 177 genuine signatures and 620 simulated signatures, and asked an FDE to count pen lifts, blunt endings, tremor. splices, and touch ups. Id. at 20. These signs of simulation occurred much more frequently in the simulations than in the genuine signatures. Id. at 21. The comparative percentages of suspicious features in the genuine to simulated signatures were: 18% to 52% (pen lifts), 17% to 73% (blunt endings), 10% to 62% (tremor), 0% to 99% (splices), 0% to 19% (touch ups). Id. at 21. The value of the study is somewhat diminished by the fact that all of the genuine signatures were produced by one person (the simulations were produced freehand by thirty-one different participants). Id. at 19. On superimposability of tracings, see Leung et al., supra note 208, at 408-10.
228. OSBORN, 1929 edition, supra note 50, at 107. For other references to tremor as a suspicious sign, see id. at 110; HILTON, supra note 195. at 185; and NICKELL, supra note 205, at 68.
229. See sources cited supra note 209.
230. See, e.g., United States v. Paul, 175 F.3d 906, 910-11, (11th Cir. 1999); United States v. Prime, 220 F. Supp. 2d 1203, 1205-06 (W.D. Wash. 2002), aff'd, 363 F.3d 1028 (9th Cir. 2004). These cases presented tasks other than the harrowing task of signature authentication scrutinized in this Article; however, they are examples of global or near-global endorsement and FDE expertise in terms that would include signature authentication expertise. In his presentation for this symposium, Professor Risinger noted that during the past four years, trial judges have shown a marked proclivity to admit FDE testimony after string citing appellate court decisions that upheld admission of the testimony as within trial court discretion (and treating those appellate opinions as if they were ones that mandated the admission of FDE testimony). He concluded that the battle to exclude FDE testimony had been lost. Michael Risinger, Professor, Presentation at Hastings Law Journal Symposium: "Faces of Forensics: Identification and Behavior" (Mar. 21, 2008).
231. United States v. Starzecpyzel, 880 F. Supp. 1027, 1042 (S.D.N.Y. 1995).
232. United States v. Brewer, No. 01CR892, 2002 WL 596365, at *6-8 (N.D. Ill. Apr. 16, 2002). The document examiner would have testified that a signature was a traced forgery, the model for which was one of the exemplars that had been located. Id. at *6-7. The court held that the expert testimony was inadmissible. Id. at *8 (citing United States v. Saelee, 162 F. Supp. 2d 1077 (D. Alaska 2000); United States v. Fujii, 152 F. Supp. 2d 939 (D. 111. 2000); United States v. Hines, 55 F. Supp. 2d. 62, 67-68 (D. Mass. 1999)).
233. Examples of exposure include the Dreyfus trial, see D.H. Kaye, Revisiting Dreyfus: A More Complete Account of Trial by Mathematics, 91 MINN. L. REV. 825 (2007), and the Clifford Irving/Howard Hughes hoax, see NICKELL, supra note 205. Identification of the signature on the Bush National Guard records also raised dangers of being proven wrong.
234. See Howard Kurtz, Rather Defends CBS over Memos on Bush, WASH. POST, Sept. 11, 2004, at A07.
235. See, e.g., CTS TEST NO. 01-524, supra note 99, at 1 (containing standard statement that test samples may, in the option of the receiving labs, be used in a "training exercise").
236. See sources cited supra notes 213-14.
237. For example, Dean Wigmore wrote the introduction to Osborn's first edition. See OSBORN, 1910 edition, supra note 28, at vii-ix. Also, many courts have endorsed the expertise. See also sources cited supra note 212.
238. See generally Black et al., supra note 14.
239. See id.
240. See supra Part III.
241. See Berger, supra note 123, at 1138. Is the defendant better protected when jurors during deliberation compare the uncrossed t's or undotted i's in the crime scene samples and the provided specimens, as they have probably seen done on some TV show, than if the court allows a prosecution expert to testify who is then demolished by Professor Saks? Id.
242. Risinger & Saks, supra note 47, at 67-74.
243. See Roger C. Park & Michael J. Saks, Evidence Scholarship Reconsidered: Results of the Interdisciplinary Turn, 47 B.C. L. REV. 949, 960-64 (2006).
244. See, e.g., RICHARD A. LEO, POLICE INTERROGATION AND AMERICAN JUSTICE (2008);
245. Richard A. Leo et al., Bringing Reliability Back in: False Confessions and Legal Safeguards in the Twenty-First Century, 2006 WIS. L. REV. 479.
246. David Faigman, Professor, Presentation at Hastings Law Journal Symposium: "Faces of Forensics: Identification and Behavior" (Mar. 21, 2008);
247. see also David L Faigman, Anecdotal Forensics, Phrenology, and Other Abject Lessons from the History of Science, 59 HASTINGS L. J. 979 (2008).
248. See supra note 197 and accompanying text.
249. This appendix has been excerpted, with permission, from American Jurisprudence Proof of Facts 3d. 27 AM. JUR. 3D Proof of Facts § 108.
250. This appendix has been excerpted, with permission, from American Jurisprudence Proof of Facts 3d. 27 AM. JUR. 3D Proof of Facts § 110.
251. CTS REPORT NO. 88-5, supra note 55; see also supra note 93 and accompanying text.
252. In tabulating the answers, I have excluded the answers of lab 517, which lodged a blanket protest against every question on the test, writing that "Due to the quality of the photographs and the type of known handwriting specimens submitted, a definite conclusion could not be reached . . . ." I think it is more appropriate to treat this statement as a nonresponse than as an "inconclusive." CTS REPORT NO. 88-5, supra note 55, at 24.