Reliable support: Measuring calibration of likelihood ratios

Forensic Science International

Volumn 230, Issue 1-3, 2013, Pages 156-169

  Ramos, Daniel ^a   Gonzalez Rodriguez, Joaquin ^a  

^a Universidad Autonoma de Madrid   (Spain)

Author keywords

Accuracy; Calibration; Empirical Cross Entropy; Evidence evaluation; Likelihood ratio; Performance

Indexed keywords

ANALYTICAL ERROR; ARTICLE; CALIBRATION; DATA BASE; DISCRIMINANT ANALYSIS; EMPIRICAL CROSS ENTROPY; FORENSIC SCIENCE; HUMAN; INTERMETHOD COMPARISON; LEGAL EVIDENCE; PERFORMANCE; PRIORITY JOURNAL; PROBABILITY; RELIABILITY; SCORING SYSTEM; SPEECH ANALYSIS; STATISTICAL ANALYSIS; STATISTICAL MODEL;

EID: 84879600985     PISSN: 03790738     EISSN: 18726283     Source Type: Journal    
DOI: 10.1016/j.forsciint.2013.04.014     Document Type: Article

References (42)

1. Evett I. Expressing evaluative opinions: a position statement. Sci. Justice 2011, 51:1-2. several signatories.
2. Royall R. On the probability of observing statistical misleading evidence. J. Am. Stat. Assoc. 2000, 95(451):760-768.
3. Curran J.M., Buckleton J.S., Triggs C.M., Weir B.S. Assessing uncertainty in DNA evidence caused by sampling effects. Sci. Justice 2002, 42(1):29-37.
4. Saunders C., Gantz D.T., Buscaglia J., Kalendra E. The effect of uncertainty about the background population on the forensic value of the evidence. European Academy of Forensic Science Conference, EAFS 2012 2012.
5. Zadora G., Ramos D. Evaluation of glass samples for forensic purposes - an application of likelihood ratio model and information-theoretical approach. Chemometr. Intell. Lab. Syst. 2010, 102:62-63.
6. Morrison G.S., Ochoa F., Thiruvaran T. Database selection for forensic voice comparison. Proc. of Odyssey 2012, Singapore 2012.
7. Gilbert N. DNA's identity crisis. Nature 2010, 464:347-348.
8. Gill P., Whitaker J., Flaxman C., Brown N., Buckleton J. An investigation of the rigor of interpretation rules for STRs derived from less than 100 pg of DNA. Forensic Sci. Int. 2000, 112:17-40.
9. Perez-Gomez S., Ramos D., Gonzalez-Dominguez J., Gonzalez-Rodriguez J. Score-level compensation of extreme speech duration variability in speaker verification. Proc. of Interspeech 2010, Makuhari, Japan 2010, 374-377.
10. Savage L. The elicitation of personal probabilities and expectations. J. Am. Stat. Assoc. 1971, 66(336):783-801.
11. deGroot M.H., Fienberg S.E. The comparison and evaluation of forecasters. Statistician 1983, 32:12-22.
12. Gneiting T., Raftery A. Strictly proper scoring rules, prediction and estimation. J. Am. Stat. Assoc. 2007, 102:359-378.
13. Aitken C.G.G., Taroni F. Statistics and the Evaluation of Evidence for Forensic Scientists 2004, John Wiley & Sons, Chichester.
14. Dawid A.P. The well-calibrated Bayesian. J. Am. Stat. Assoc. 1982, 77(379):605-610.
15. Jaynes E.T. Probability Theory: The Logic of Science 2003, Cambridge University Press, Cambridge, UK.
16. Lindley D.V., Tversky A., Brown R.V. On the reconciliation of probability assessments. J. Roy. Stat. Soc. A 1979, 142(2):146-180.
17. Garthwaite P.H., Kadane J.B., O'Hagan A. Statistical methods for eliciting probability distributions. J. Am. Stat. Assoc. 2005, 100(470):680-701.
18. Lindley D.V. Understanding Uncertainty 2006, Wiley, Hoboken, New Jersey, USA.
19. O'Hagan T. Dicing with the unknown. Significance 2004, 1(3):132-133.
20. Brümmer N., du Preez J. Application independent evaluation of speaker detection. Comput. Speech Lang. 2006, 20(2-3):230-275.
21. N. Brümmer, Measuring, refining and calibrating speaker and language information extracted from speech, Ph.D. Thesis, School of Electrical Engineering, University of Stellenbosch, Stellenbosch, South Africa, available at: , 2010. http://sites.google.com/site/nikobrummer/.
22. D. Ramos, Forensic evaluation of the evidence using automatic speaker recognition systems, Ph.D. Thesis, Depto. de Ingenieria Informatica, Escuela Politecnica Superior, Universidad Autonoma de Madrid, Madrid, Spain, available at: , 2007. http://atvs.ii.uam.es.
23. vanLeeuwen D., Brümmer N. An introduction to application-independent evaluation of speaker recognition systems. Speaker Classification, vol. 4343 of Lecture Notes in Computer Science/Artificial Intelligence 2007, Springer, Heidelberg, Berlin, New York. C. Müller (Ed.).
24. D. Ramos, J. Gonzalez-Rodriguez, G. Zadora, C. Aitken, Information-theoretical assessment of the performance of likelihood ratio computation methods, J. Forensic Sci., in press.
25. Cook R., Evett I.W., Jackson G., Jones P.J., Lambert J.A. A model for case assessment and interpretation. Sci. Justice 1998, 38:151-156.
26. Cook R., Evett I.W., Jackson G., Jones P.J., Lambert J.A. A hierarchy of propositions: deciding which level to address in casework. Sci. Justice 1998, 38(4):231-239.
27. Robertson B., Vignaux G.A. Interpreting Evidence Evaluating Forensic Science in the Courtroom 1995, Wiley, UK.
28. Evett I.W. Avoiding the transposed conditional. Sci. Justice 1995, 35:127-131.
29. Cover T.M., Thomas J.A. Elements of Information Theory 2006, Wiley Interscience, United States of America. 2nd ed.
30. Cohen I., Goldszmidt M. Properties and benefits of calibrated classifiers. Proc. of European Conference on Machine Learning ECML/PKDD 2004.
31. Evett I.W., Buckleton J. Statistical Analysis of STR Data. Advances in Forensic Haemogenetics 1996, 6:79-86. Springer-Verlag, Heidelberg.
32. Gonzalez-Rodriguez J., Rose P., Ramos D., Toledano D.T., Ortega-Garcia J. Emulating DNA. Rigorous quantification of evidential weight in transparent and testable forensic speaker recognition, IEEE Transactions on Audio. Speech Lang. Process. 2007, 15(7):2072-2084.
33. Cambridge International Dictionary of English 1995, Cambridge University Press, Cambridge, UK. P. Procter (Ed.).
34. D. Meuwly, Reconaissance de Locuteurs en Sciences Forensiques: L'apport d'une Approache Automatique, Ph.D. Thesis, IPSC-Universite de Lausanne, 2001.
35. Meuwly D. Forensic individualisation from biometric data. Sci. Justice 2007, 46(4):205-213.
36. Egli N., Champod C., Margot P. Evidence evaluation in fingerprint comparison and automated fingerprint identification systems - modelling within finger variability. Forensic Sci. Int. 2007, 167(2-3):189-195.
37. N. Egli, Interpretation of partial fingermarks using an automated fingerprint identification system, Ph.D. Thesis, Institute de Police Scientifique, Ecole de Sciences Criminelles, 2009.
38. Neumann C., Evett I., Skerret J. Quantifying the weight of evidence from a forensic fingerprint comparison: a new paradigm. J. Roy. Stat. Soc. Ser. A: Stat. Soc. 2012, 175(2):371-415.
39. Hepler A.B., Saunders C.P., Davis L.J., Buscaglia J. Score-based likelihood ratios for handwriting evidence. Forensic Sci. Int. 2012, 219(1-3):129-140.
40. D. Ramos, J. Franco-Pedroso, J. Gonzalez-Rodriguez, Calibration and weight of the evidence by human listeners: the ATVS-UAM submission to NIST human-aided speaker recognition 2010, in: Proc. of ICASSP 2011, Pague, Czeck Republic, 2011, pp. 5908-5911.
41. Gonzalez-Dominguez J., Lopez-Moreno I., Franco-Pedroso J., Ramos D., Toledano D.T., Gonzalez-Rodriguez J. ATVS-UAM NIST SRE 2010 system. Proc. of FALA 2010 2010.
42. Silverman B.W. Density Estimation for Statistics and Data Analysis 1986, Chapman and Hall, London, UK.