Another Look at the (Im-)Precision of Individual Risk Estimates Made Using Actuarial Risk Assessment Instruments

Behavioral Sciences and the Law

Volumn 31, Issue 1, 2013, Pages 81-102

  Hart, Stephen D ^a   Cooke, David J ^b  

^a UNIVERSITY OF BERGEN   (Norway)

^b GLASGOW CALEDONIAN UNIVERSITY   (United Kingdom)

Author keywords

[No Author keywords available]

Indexed keywords

ARTICLE; CANADA; CONFIDENCE INTERVAL; HUMAN; MALE; METHODOLOGY; RISK ASSESSMENT; SEXUAL CRIME; STANDARD; STATISTICAL ANALYSIS; STATISTICAL MODEL; VIOLENCE;

ACTUARIAL ANALYSIS; CANADA; CONFIDENCE INTERVALS; HUMANS; LOGISTIC MODELS; MALE; RISK ASSESSMENT; SEX OFFENSES; VIOLENCE;

EID: 84874289020     PISSN: 07353936     EISSN: 10990798     Source Type: Journal    
DOI: 10.1002/bsl.2049     Document Type: Article

References (59)

1. Agresti, A., & Coull, B. A. (1998). Approximate is better than 'exact' for interval estimation of binomial proportions. American Statistician, 52, 119-126.
2. Altman, D. G., & Royston, P. (2000). What do we mean by validating a prognostic model? Statistics in Medicine, 19, 453-473.
3. Boer, D. P., & Hart, S. D. (2008). Sexual Violence Risk-20 (SVR-20). In B. L. Cutler (Ed.), Encyclopedia of psychology and law (pp. 743-744). Thousand Oaks, CA: Sage.
4. Boer, D. P., Hart, S. D., Kropp, P. R., & Webster, C. D. (1997). Manual for the Sexual Violence Risk - 20: Professional guidelines for assessing risk of sexual violence. Vancouver, British Columbia: British Columbia Institute on Family Violence and Mental Health, Law, & Policy Institute, Simon Fraser University.
5. Cooke, D. J. & Michie, C. (2010). Limitations of diagnostic precision and predictive utility in the individual case: A challenge for forensic practice. Law and Human Behavior, 34, 259-274.
6. Cooke, D. J., & Michie, C. (2011). Violence risk assessment: Challenging the illusion of certainty. In B. McSherry & P. Keyzer (Eds.), Dangerous people: Policy, prediction and practice. Oxford, UK: Routledge, pp. 147-161.
7. Cooke, D. J., & Michie, C. (2012). Violence Risk Assessment: From Prediction to understanding, from what? to why? In C. Logan & E. Johnstone (Eds.), Managing clinical risk: A guide to effective practice. London: Routledge.
8. Cooke, D. J., Michie, C., & Hart, S. D. (2012). Precision of estimates do matter: Weighing the evidence in individual predictions of recidivism. Unpublished Manuscript.
9. COVR, Inc., & PAR Staff (2005). Score report. Retrieved August 3, 2012 from http://www4.parinc.com/WebUploads/samplerpts/COVR.pdf
10. Craig, L., & Beech, A. R. (2009). Best practice in conducting actuarial risk assessments with adult sexual offenders. Journal of Sexual Aggression, 15(2), 193-211.
11. Cunningham, M. D., Sorensen, J. R., & Reidy, T. J. (2005). An actuarial model for assessment of prison violence risk among maximum security inmates. Assessment, 12, 40-49.
12. Daubert v. Merrell Dow Pharmaceuticals, 509 U.S. 579 (1993).
13. Doren, D. M. (2002). Evaluating sex offenders: A manual for civil commitments and beyond. Thousand Oaks, CA: Sage.
14. Einstein, A. (1954). Ideas and opinions. New York: Crown.
15. Fazel, S., Singh, J. P., Doll, H., & Grann, M. (2012). Use of risk assessment instruments to predict violence and antisocial behaviour in 73 samples involving 24 827 people: Systematic review and meta-analysis. BMJ, 345, e4692. DOI: 10.1136/bmj.e4692
16. Grisso, T., & Appelbaum, P. S. (1992). Is it unethical to offer predictions of future violence? Law and Human Behavior, 16, 621-633.
17. Grove, W. M., & Meehl, P. E. (1996). Comparative efficiency of informal (subjective, impressionistic) and formal (mechanical, algorithmic) prediction procedures: The clinical-statistical controversy. Psychology, Public Policy, and Law, 2, 293-323.
18. Hájek, A. (2003). Interpretations of probability. In E. N. Zalta (Ed.), The Stanford encyclopedia of philosophy, Summer 2003 ed. Retrieved July 27, 2006 from http://plato.stanford.edu/entries/probability-interpret.
19. Hájek, A., & Hall, N. (2002). Induction and probability. In P. Machamer & M. Silberstein (Eds.), The Blackwell Guide to the philosophy of science (pp. 149-172). London: Blackwell.
20. Hanson, R. K., Helmus, L., & Thornton, D. (2010). Predicting recividism amongst sexual offenders: A multi-site study of Static-2002. Law and Human Behavior, 34, 198-211.
21. Hanson, R. K., & Howard, P. D. (2010). Individual confidence intervals do not inform decision makers about the accuracy of risk assessment evaluations. Law and Human Behavior, 34, 275-281.
22. Hanson, R. K., & Morton-Bourgon, K. E. (2007). The accuracy of recidivism risk assessments for sexual offenders: A meta-analysis. Ottawa: Public Safety and Emergency Preparedness Canada.
23. Hare, R. D. (2003). The Hare Psychopathy Checklist - Revised (Vol. 2nd). Toronto, Ontario: Multi-Health Systems.
24. Harris, G. T., Rice, M. E., & Quinsey, V. L. (1993). Violent recidivism of mentally disordered offenders: The development of a statistical prediction instrument. Criminal Justice and Behavior, 20, 315-335.
25. Harris, A. J., Phenix, A., Hanson, R. K., & Thornton, D. (2003). STATIC-99 coding rules revised - 2003. Ottawa: Public Safety and Emergency Preparedness Canada.
26. Harris, G. T., Rice, M. E., & Quinsey, V. L. (2008). Shall evidence-based risk assessment be abandoned? [Letter]. The British Journal of Psychiatry, 192, 154.
27. Hart, S. D. (1998). The role of psychopathy in assessing risk for violence: Conceptual and methodological issues. Legal and Criminological Psychology, 3, 121-137.
28. Hart, S. D. (2003). Actuarial risk assessment: Commentary on Berlin et al. Sexual Abuse: A Journal of Research and Treatment, 15, 383-388.
29. Hart, S. D. (2009). Evidence-based assessment of risk for sexual violence. Chapman Journal of Criminal Justice, 1, 143-165.
30. Hart, S. D., Cox, D. N., & Hare, R. D. (1995). Manual for the Hare Psychopathy Checklist: Screening Version (PCL:SV). Toronto: Multi-Health Systems.
31. Hart, S. D. (2011). Complexity, uncertainty, and the reconceptualization of violence risk assessment. In R. Abrunhosa (Ed.), Victims and offenders: Chapters on psychology and law (pp. 57-69). Brussels: Politeia. (Original work published in 2004.)
32. Hart, S. D., & Logan, C. (2011). Formulation of violence risk using evidence-based assessments: The structured professional judgment approach. In P. Sturmey & M. McMurran (Eds.), Forensic case formulation (pp. 83-106). Chichester, UK: Wiley-Blackwell.
33. Hart, S. D., Michie, C., & Cooke, D. J. (2007a). Precision of actuarial risk assessment instruments: Evaluating the "margins of error" of group v. individual predictions of violence. The British Journal of Psychiatry, 190, s60-s65.
34. Hart, S. D., Michie, C., & Cooke, D. J. (2007b). Authors' reply: Margins of error for individual risk estimates: Large, unknown, or incalculable [Letter]. The British Journal of Psychiatry, 191, 561-562.
35. Hart, S. D., Michie, C., & Cooke, D. J. (2008). Authors' reply: Evaluating the precision (and accuracy!) of criticisms [Letter]. The British Journal of Psychiatry, 192, 154.
36. Henderson, R., & Keiding, N. (2005). Individual survival time prediction using statistical models. Journal of Medical Ethics, 31, 703-706.
37. Hilton, N. Z., Harris, G. T., & Rice, M. E. (2006). Sixty-six years of research on the clinical versus actuarial prediction of violence. The Counseling Psychologist, 34, 400-409.
38. Hosmer, D. W., & Lemeshow, S. (2000). Applied logistic regression. New York: Wiley.
39. Hosmer, D. W., Taber, S., & Lemeshow, S. (1991). The importance of assessing the fit of logistic regression models: A case study. American Journal of Public Health, 81, 1630-1635.
40. Janus, E., & Prentky, R. (2003). Forensic use of actuarial risk assessment with sex offenders: Accuracy, admissibility and accountability. American Criminal Law Review, 40, 1443-1499.
41. Kahneman, D., Slovic, P., & Tversky, A. (Eds.) (1982). Judgment under uncertainty: Heuristics and biases. New York: Cambridge University Press.
42. Kleinbaum, D. G., & Klein, M. (2002). Logistic regression: A self-learning text, 2nd ed. New York: Springer.
43. Kumho Tire Co. v. Carmichael, 526 U.S. 137 (1999).
44. Litwack, T. R., Zapf, P. A., Groscup, J. L., & Hart, S. D. (2006). Violence risk assessment: Research, legal, and clinical considerations. In I. B. Weiner & A. K. Hess (Eds.), The handbook of forensic psychology, 3rd ed. (pp. 487-533). New York: Wiley.
45. Long, J. S., & Freese, J. (2006). Regression models for categorical dependent variables using Stata. College Station, Texas: Stata Press.
46. Meehl, P. E. (1996). Clinical versus statistical prediction: A theoretical analysis and a review of the literature. Northvale, NJ: Jason Aronson. (Original work published in 1954.)
47. Meehl, P. E. (2006). The power of quantitative thinking. In N. G. Waller, L. J. Yonce, W. M. Grove, D. R. Faust, & M. F. Lenzenweger (Eds.), A Paul Meehl Reader: Essays on the practice of scientific psychology (pp. 433-444). Mahwah, NJ: Erlbaum.
48. Monahan, J. A., Steadman, H. J., Appelbaum, P. S., Grisso, T., Mulvey, E. P., Roth, L. H., ... Silver, E. (2005). Classification of Violence Risk (COVR). Lutz, FL: Psychological Assessment Resources, Inc.
49. Monahan, J., Steadman, H., Silver, E., Appelbaum, P., Robbins, P. C., Mulvey, E. P., ... Banks, S. (2001). Rethinking risk assessment: The MacArthur study of mental disorder and violence (Vol. 1). New York: Oxford University Press.
50. Mossman, D., & Sellke, T. (2007). Avoiding errors about "margins of error" [Letter]. The British Journal of Psychiatry, 191, 561.
51. Pampel, F. C. (2000). Logistic regression: A primer. Thousand Oaks, CA: Sage.
52. Quinsey, V. L., Harris, G. T., Rice, M. E., & Cormier, C. A. (1998). Violent offenders: Appraising and managing risk. Washington, DC: American Psychological Association.
53. Quinsey, V. L., Harris, G. T., Rice, M. E., & Cormier, C. A. (2006). Violent offenders: Appraising and managing risk, 2nd ed. Washington, DC: American Psychological Association.
54. Rose, G. (1992). The strategy of preventative medicine. Oxford: Oxford Medical Publications.
55. Skeem, J. L., & Monahan, J. (2011). Current directions in violence risk assessment. Current Directions in Psychological Science, 20, 38-42.
56. Sommerer, J. C. (2005). A view of future APL science and technology: Guest Editor's introduction. Johns Hopkins APL Technical Digest, 26, 298-305.
57. StataCorp. (2007). Stata/SE, Version 10.0 [Software]. College Station, Texas: Author.
58. Wald, N. J., Hackshaw, A. K., & Frost, C. D. (1999). When can a risk factor be used as a worthwhile screening test. British Medical Journal, 319, 1562-1565.
59. Wilson, E. B. (1927). Probable inference, the law of succession, and statistical inference. Journal of the American Statistical Association, 22, 209-212.