Proficiency tests to estimate error rates in the forensic sciences

Law, Probability and Risk

Volumn 12, Issue 1, 2013, Pages 89-98

  Koehler, Jonathan J ^a  

^a NORTHWESTERN UNIVERSITY SCHOOL OF LAW   (United States)

Author keywords

Base rates; Blind testing; Error rates; Forensic science; Proficiency tests

Indexed keywords

EID: 84875346850     PISSN: 14708396     EISSN: 1470840X     Source Type: Journal    
DOI: 10.1093/lpr/mgs013     Document Type: Note

References (48)

1. AMERICAN STATISTICAL ASSOCIATION. (2010). ASA Statement on Strengthening Forensic Science. Available at http://www.amstat.org/outreach/pdfs/Forensic_Science_Endorsement.pdf. Accessed 1 May 2012.
2. BALDING, D. J. (1997). Errors and misunderstandings in the second NRC report. Jurimetrics, 37, 469-476.
3. BALDING, D. J. (2005). Weight-of-Evidence for Forensic DNA Profiles. John Wiley and Sons Ltd, West Sussex, UK.
4. BALDING, D. J. & DONNELLY, P. (1995). Inferring identity from DNA profile evidence. PNAS, 92, 11741-11745.
5. BONO, J. P. (2011). Commentary on "The need for a research culture in the forensic sciences". UCLA Law Rev., 58, 781-787.
6. BOONE, D. J.,HANSEN, H. J.,HEARN, T. L., LEWIS, D.S.&DUDLEY, D. (1982). Laboratory evaluation and assistance efforts:Mailed, on-site and blind PT surveys conducted by the Centers for Disease Control. Am. J. Public Health, 72, 1364-1368.
7. BUDOWLE, B., BOTTRELL, M. C., BUNCH, S.G., FRAM, R., HARRISON, D.,MEAGHER, S., OIEN, C. T., PETERSON, P. E., SEIGER, D. P., SMITH, M.B., SMRZ, M.A., SOLTIS, G. L.&STACEY, R. B. (2009). A perspective on errors, bias, and interpretation in the forensic sciences and direction for continuing advancement. J. Forensic Sci., 54, 798-809.
8. BUNCH, S. G., SMITH, E. D., GIROUX, B. N.&MURPHY, P. D. (2009). Is a match really a match? A primer on the procedures and validity of firearm and toolmark identification. Forensic Sci. & Communication, 11, 1-10.
9. CEMBROWSKI, G.S.&VANDERLINDE, R. E. (1988). Survey of special practices associated with College of American Pathologists Proficiency Testing in the Commonwealth of Pennsylvania. Archives of Pathology and Laboratory Medicine, 112, 374-376.
10. DAWES, R. M., FAUST, D.&MEEHL, P. E. (1989). Clinical versus actuarial judgment. Science, 243, 1668-1674.
11. DUNN, J. E. JR & GREENHOUSE, S. W. (1950). Cancer Diagnostic Tests: Principles and Criteria for Development and Evaluation. Federal Security Agency, Psublic Health Service, #9, Government Printing Office.
12. EFRON, B. (2010). The future of indirect evidence. Stat. Sci., 25, 145-157.
13. EFRON, B.&MORRIS, C. (1977). Stein's paradox in statistics. Scientific American, 236, 119-127.
14. EXPERT WORKING GROUP ON HUMAN FACTORS IN LATENT PRINT ANALYSIS (2012) Latent Print Examination and Human Factors: Improving the Practice through a Systems Approach. U.S. Department of Commerce, National Institute of Standards and Technology.
15. GASTWIRTH, J. L. (1987). The statistical precision of medical screening procedures: Application to polygraph and aids antibodies test data. Stat. Sci., 2, 213-238.
16. GIANNELLI, P. C. (2009). The NRC report and its implications for criminal litigation. Jurimetrics J., 50, 53-66.
17. GUTOWSKI, S. (2005). Error rates in the identification sciences. Forensic Bulletin, 23, 23-29.
18. HANSEN, H. J., CAUDILL, S. P. & BOONE, J. D. (1985). Crisis in drug testing: Results of a CDC blind study. J. Am. Med. Ass., 253, 2382-2387.
19. HONMA, M., YOSHII, T., ISHIYAMA, I., MITANI, K., KOMINAMI, R. & MURAMATSU, M. (1989). Individual identification from semen by the deoxyribonucleic acid (DNA) fingerprint technique. J. Forensic Sci., 34, 222-227.
20. KAHNEMAN, D.& TVERSKY, A. (1973). On the psychology of prediction. Psychol. Rev., 80, 237-251.
21. KAYE, D. H. (1993). DNA evidence: Probability, population genetics, and the courts. Harvard. J. Law Technol., 7, 101-172.
22. KOEHLER, J. J. (1993). DNA matches and statistics: Important questions, surprising answers. Judicature, 76, 222-229.
23. KOEHLER, J. J. (1996). The base rate fallacy reconsidered: Normative, descriptive and methodological challenges. Behavioral & Brain Sci., 19, 1-53.
24. KOEHLER, J. J. (1997). Why DNA likelihood ratios should account for error (even when a National Research Council report says they should not). Jurimetrics J., 37, 425-437.
25. KOEHLER, J. J. (2008). Fingerprint error rates and proficiency tests: What they are and why they matter. Hastings Law J., 59, 1077-1098.
26. KOEHLER, J. J. (2010). Forensic science reform in the 21st century: A major conference, a blockbuster report, and reasons to be pessimistic. Law, Probability & Risk, 9, 1-6.
27. KOEHLER, J. J. (2011). If the shoe fits, they might acquit: The value of shoeprint testimony. J. Empirical Legal Studies, 8, 21-48.
28. KOEHLER, J. J., CHIA, A. & LINDSEY, J. S. (1995). The random match probability (RMP) in DNA evidence: Irrelevant and prejudicial? Jurimetrics J., 35, 201-219.
29. KUO, M. (1988). California Association of Crime Laboratory Directors: DNA Committee Report #6. Orange County Sheriff Coroners Crime Laboratory.
30. KUO, M. (1990). California Association of Crime Laboratory Directors. DNA Committee - Results of Blind Trial #2. Orange County Sheriff Coroners Crime Laboratory.
31. LEMPERT, R. (1991). Some caveats concerning DNA as criminal identification evidence: With thanks to the Reverend Bayes. Cardozo Law Rev., 13, 303-341.
32. LEMPERT, R. (1997). After the DNA wars: Skirmishing with NRC II. Jurimetrics J., 37, 439-468.
33. MEEHL, P. E. & ROSEN, A. (1955). Antecedent probability and the efficiency of psychometric signs, patterns, or cutting scores. Psychol. Bulletin, 52, 194-216.
34. MNOOKIN, J. (2009). Error rates for latent fingerprinting as a function of visual complexity and cognitive difficulty. National Institute of Justice Award 2009-DN-BX-K225.
35. NATIONAL ACADEMY OF SCIENCES, NATIONAL RESEARCH COUNCIL, COMMITTEE ON DNA TECHNOLOGY IN FORENSIC SCIENCE (1992). DNA Technology in Forensic Science. National Academies Press, Washington D.C.
36. NATIONAL ACADEMY OF SCIENCES, NATIONAL RESEARCH COUNCIL, COMMITTEE ON DNA FORENSIC SCIENCE: AN UPDATE (1996). The Evaluation of Forensic DNA Evidence. National Academies Press, Washington D.C.
37. NATIONAL ACADEMY OF SCIENCES, NATIONAL RESEARCH COUNCIL, COMMITTEE ON IDENTIFYING THE NEEDS OF THE FORENSIC SCIENCE COMMUNITY (2009). Strengthening Forensic Science in the United States: A Path Forward. National Academies Press, Washsington D.C.
38. PETERSON, J. L.& GAENSSLEN, R. E. (2001). Developing Criteria for Model External DNA Proficiency Testing - Final Report. National Institute of Justice, Grant No. 96-DN-VX-0001.
39. PETERSON, J. L., LIN, G., HO, M., CHEN, Y. & GAENSSLEN, R. E. (2003a). The feasibility of external blind DNA proficiency testing. I. Background and findings. J. Forensic Sci., 48, 21-31.
40. PETERSON, J. L., LIN, G., HO, M., CHEN, Y. & GAENSSLEN, R. E. (2003b). The feasibility of external blind DNA proficiency testing. II. Experience with actual blind tests. J. Forensic Sci., 48, 32-40.
41. RAND, S., SCHURENKAMP, W.&BRINKMANN, B. (2002). The GEDNAP (German DNA Profiling Group) blind trial concept. Int. J. Legal Med., 116, 199-206.
42. RAND, S., SCHURENKAMP,W.,HOHOFF, C.&BRINKMANN, B. (2004). The GEDNAP (German DNA Profiling Group) blind trial concept, Part II, trends and developments. Int. J. Legal Med., 118, 83-89.
43. RISINGER, D. M., SAKS, M. J., THOMPSON, W. C. & ROSENTHAL, R. (2002). The Daubert/Kumho implications of observer effects in forensic science: Hidden problems of expectation and suggestion. California Law Rev., 90, 1-56.
44. SCIENTIFICWORKING GROUP FOR SHOEPRINT AND TIRE TREAD EVIDENCE (2009). "Standard for expressing conclusions of forensic footwear and tire impression examinations". http://www.swgtread.org/standards/draft, accessed 31 July 2012.
45. THOMPSON,W. C. (1993). Evaluating the admissibility of new genetic identification tests: Lessons from the "DNA War." J. Criminal Law & Criminol., 84, 22-104.
46. THOMPSON,W. C. (1997). Accepting lower standards: The National Research Council's second report on forensic DNA evidence. Jurimetrics, 37, 405-424.
47. THOMPSON,W.C.,TARONI, F.&AITKEN, C.G.G. (2003). How the probability of a false positive affects the value of DNA evidence. J. Forensic Sci., 48, 47-54.
48. WALSH, P. S., FILDES, N.,LOUIE, A. S.&HIGUCHI, R. (1991). Report of the blind trial of the Cetus AmpliType HLA DQ-alpha forensic deoxyribonucleic acid (DNA) amplification and typing kit. J. Forensic Sci., 36, 1551-1556.