Measurement of Predictive Validity in Violence Risk Assessment Studies: A Second-Order Systematic Review

Behavioral Sciences and the Law

Volumn 31, Issue 1, 2013, Pages 55-73

  Singh, Jay P ^a   Desmarais, Sarah L ^b   Van Dorn, Richard A ^c  

^a MOLDE UNIVERSITY COLLEGE   (Norway)

^b NORTH CAROLINA STATE UNIVERSITY   (United States)

^c RTI INTERNATIONAL   (United States)

Author keywords

[No Author keywords available]

Indexed keywords

FORECASTING; FORENSIC PSYCHIATRY; HUMAN; RESEARCH; REVIEW; RISK ASSESSMENT; STANDARD; VIOLENCE;

FORECASTING; FORENSIC PSYCHIATRY; HUMANS; RESEARCH; RISK ASSESSMENT; VIOLENCE;

EID: 84874323697     PISSN: 07353936     EISSN: 10990798     Source Type: Journal    
DOI: 10.1002/bsl.2053     Document Type: Article

References (88)

1. Altman, D. G. (1991). Practical statistics for medical research. London: Chapman and Hall.
2. Altman, D. G., & Bland, J. M. (1994a). Diagnostic tests 1: Sensitivity and specificity. British Medical Journal, 308, 1552.
3. Altman, D. G., & Bland, J. M. (1994b). Diagnostic tests 2: Predictive values. British Medical Journal, 309, 102.
4. Andrews, D. A., & Bonta, J. (1995). LSI-R: The Level of Service Inventory-Revised. Toronto, ON: Multi-Health Systems.
5. Andrews, D. A., Bonta, J., & Wormith, J. S. (2004). Level of Service/Case Management Inventory (LS/CMI): An offender assessment system. Toronto, ON: Multi-Health Systems.
6. Archer, R. P., Buffington-Vollum, J. K., Stredny, R. V., & Handel, R. W. (2006). A survey of psychological test use patterns among forensic psychologists. Journal of Personality Assessment, 87, 84-94.
7. Blair, P. R., Marcus, D. K., & Boccaccini, M. T. (2008). Is there an allegiance effect for assessment instruments? Actuarial risk assessment as an exemplar. Clinical Psychology: Science & Practice, 15, 346-360. DOI:10.1111/j.1468-2850.2008.00147.x
8. Boer, D. P., Hart, S. D., Kropp, P. R., & Webster, C. D. (1997). Manual for the Sexual Violence Risk-20. Professional guidelines for assessing risk of sexual violence. Burnaby, BC: Simon Fraser University, Mental Health, Law, and Policy Institute.
9. Bonta, J. (2002). Offender risk assessment: Guidelines for selection and use. Criminal Justice & Behavior, 29, 355-379. DOI:10.1177/0093854802029004002
10. Borum, R., Bartel, P., & Forth, A. (2002). Manual for the structured assessment of violence risk in youth (SAVRY). Tampa: University of South Florida.
11. Borum, R., Bartel, P., & Forth, A. (2003). Manual for the structured assessment of violence risk in youth (SAVRY). Version 1.1. Tampa: University of South Florida.
12. Bossuyt, P. M., Reitsma, J. B., Bruns, D. E., Gatsonis, C. A., Glaziou, P. P., Irwig, L. M., et al. (2003). Towards complete and accurate reporting of studies of diagnostic accuracy: The STARD initiative. Clinical Chemistry, 49, 1-6. DOI:10.1373/49.1.1
13. Buchanan, A., & Leese, M. (2001). Detention of people with dangerous severe personality disorders: A systematic review. Lancet, 358, 1955-1959.
14. Buchanan, A., Binder, R., Norko, M., & Swartz, M. (2012). Resource document on psychiatric violence risk assessment. American Journal of Psychiatry, 169, S1-S10. DOI:10.1176/appi.ajp.2012.169.3.340
15. Cohen, J. (1988). Statistical power analysis for the behavioral sciences (2nd ed.). Hillsdale, NJ: Lawrence Erlbaum.
16. Copas, J., & Marshall, P. (1998). The Offender Group Reconviction Scale: A statistical reconviction score for use by probation officers. Applied Statistics, 47, 159-171. DOI:10.1111/1467-9876.00104
17. Dahle, K. P., Schneider, V., & Ziethen, F. (2007). Standardisierte Instrumente zur Kriminalprognose [Standerized instruments for criminal prediction]. Forensische Psychiatrie, Psychologie, Kriminologie, 1, 15-26. DOI:10.1007/s11757-006-0004-6
18. Department of Health. (2007). Best practice in managing risk. London: Department of Health.
19. Douglas, K. S., & Weir, J. (2003).HCR-20 violence risk assessment scheme: Overview and annotated bibliography. Burnaby, BC: Simon Fraser University. Retrieved from http://www.sfu.ca/psychology/groups/faculty/hart/violink.htm
20. Douglas, K. S., Guy, L. S., & Weir, J. (2005). HCR-20 violence risk assessment scheme: Overview and annotated bibliography. Burnaby, BC: Simon Fraser University. Retrieved from www.sfu.ca/psychology/faculty/hart/violink.htm
21. Douglas, K. S., Guy, L. S., & Weir, J. (2006). HCR-20 violence risk assessment scheme: Overview and annotated bibliography. Burnaby, BC: Simon Fraser University. Retrieved from http://www.sfu.ca/psyc/faculty/hart/HCR-20%20Annotated%20Bibliograph, %202006.pdf
22. Douglas, K. S., Ogloff, J. R. P., & Hart, S. D. (2003). Evaluation of a model of violence risk assessment among forensic psychiatric patients. Psychiatric Services, 54, 1372-1379. DOI:10.1176/api.ps.54.10.1372
23. Douglas, K. S., Otto, R. K., Desmarais, S. L., & Borum, R. (2012). Clinical forensic psychology. In I. B. Weiner, J. A. Schinka, & W. F. Velicer (Eds.), Handbook of psychology, volume 2: Research methods in psychology (pp. 213-244). Hoboken, NJ: John Wiley & Sons.
24. Dunlap, W. P. (1999). A program to compute McGraw and Wong's common language effect size indicator. Behavior Research Methods, Instruments, & Computers, 31, 706-709.
25. Epperson, D. L., Kaul, J. D., Huot, S. J., Hesselton, D., Alexander, W., & Goldman, R. (1998). Minnesota Sex Offender Screening Tool-Revised (MnSOST-R). St. Paul, MN: Minnesota Department of Corrections.
26. Fazel, S., Singh, J. P., Doll, H., & Grann, M. (2012). The prediction of violence and antisocial behaviour: A systematic review and meta-analysis of the utility of risk assessment instruments in 73 samples involving 24, 827 individuals. British Medical Journal, 345, e4692.
27. Ferguson, C. J. (2009). An effect size primer: A guide for clinicians and researchers. Professional Psychology, Research and Practice, 40, 532-538. DOI:10.1037/a0015808
28. Forth, A. E., Kosson, D. S., & Hare, R. D. (2003). The Psychopathy Checklist: Youth Version manual. Toronto: Multi-Health Systems.
29. Gerhold, C. K., Browne, K. D., & Beckett, R. (2007). Predicting recidivism in adolescent sexual offenders. Aggression & Violent Behavior, 12, 427-438. DOI:10.1016/j.avb.2006.10.004
30. Glas, A. S., Lijmer, J. G., Prins, M. H., Bonsel, G. J., & Bossuyt, P. M. M. (2003). The diagnostic odds ratio: A single indicator of test performance. Journal of Clinical Epidemiology, 56, 1129-1135. DOI:10.1016/S0895-4356(03)00177-X
31. Grann, M.., Haggård, U., Hiscoke, U. L., Sturidsson, K., Lovstrom, L., Siverson, E., et al. (2000). The SORM manual. Stockholm: Karolinska Institute.
32. Guy, L. (2008). Performance indicators of the structured professional judgement approach for assessing risk for violence to others: A meta-analytic survey. Unpublished doctoral dissertation, Simon Fraser University, Burnaby, BC.
33. Hanson, R. K. (1997). The development of a brief actuarial risk scale for sexual offense recidivism. Ottawa, ON: Department of the Solicitor General of Canada.
34. Hanson, R. K., & Thornton, D. (1999). Static-99: Improving actuarial risk assessments for sex offenders (User Report 99-02). Ottawa, ON: Department of the Solicitor General of Canada.
35. Hanson, R. K., & Thornton, D. (2003). Notes on the development of Static-2002. Ottawa, ON: Department of the Solicitor General of Canada.
36. Hare, R. D. (1991). The Hare Psychopathy Checklist-Revised. North Tonawanda, NY: Multi-Health Systems.
37. Hare, R. D. (2003). The Hare Psychopathy Checklist-Revised (2nd ed.). Toronto, ON: Multi-Health Systems.
38. Harris, A. J. R., Phenix, A., Hanson, R. K., & Thornton, D. (2003). Static-99 coding rules: Revised 2003. Ottawa, ON: Solicitor General Canada.
39. Hart, S. D., & Logan, C. (2011). Formulation of violence risk using evidence-based assessments: The structured professional judgment approach. In P. Sturmey, & M. McMurran (Eds.), Forensic case formulation. Chichester, UK: Wiley-Blackwell.
40. Hart, S. D., Michie, C., & Cooke, D. J. (2007). Precision of actuarial risk assessment instruments: Evaluating the 'margins of error' of group v. individual predictions of violence. British Journal of Psychiatry, 49, S60-S65.
41. Hart, S. D., Cox, D., & Hare, R. D. (1995). Manual for the Screening Version of the Hare Psychopathy Checklist-Revised. Toronto: Multi-Health Systems.
42. Hartvig, P., Østberg, B., Alfarnes, S., Moger, T., Skjønberg, M., et al. (2007). Violence Risk Screening-10 (V-RISK-10). Oslo: Centre for Research and Education in Forensic Psychiatry.
43. Hildebrand, M., Hesper, B. L., Spreen, M., & Nijman, H. L. I. (2005). De waarde van gestructureerde risicotaxatie en van de diagnose psychopathie: Een onderzoek naar de betrouwbaarheid en predictieve validiteit van de HCR-20, HKT-30 en PCL-R. The value of structured risk assessment and the diagnosis psychopathy: Research into the reliability and predictive validity of the HCR-20, HKT-30 and the PCL-R. Utrecht: Expertisecentrum Forensische Psychiatrie.
44. Holtfreter, K., & Cupp, R. (2007). Gender and risk assessment: The empirical status of the LSI-R for women. Journal of Contemporary Criminal Justice, 23, 363-382. DOI:10.1177/1043986207309436
45. Howells, K., Watt, B., Hall, G., & Baldwin, S. (1997). Developing programmes for violent offenders. Legal & Criminological Psychology, 2, 117-128. DOI:10.1111/j.2044-8333.1997.tb00337.x
46. Janus, E. (2004). Sexually violent predator laws: Psychiatry in service to a morally dubious enterprise. Lancet, 3664, 50-51. DOI:10.1016/S0140-6736(04)17641-1
47. Jovanović, A. A., Toševski, D. L., Ivković, M., Damjanović, A., & Gašić, M. J. (2009). Predicting violence in veterans with posttraumatic stress disorder. Vojnosanitetski Pregled, 66, 13-21.
48. Khiroya, R., Weaver, T., & Maden, T. (2009). Use and perceived utility of structured violence risk assessments in English medium secure forensic units. Psychiatrist, 33, 129-132. DOI:10.1192/pb.bp.108.019810
49. Kropp, P. R., Hart, S. D., Webster, C. W., & Eaves, D. (1994). Manual for the Spousal Assault Risk Assessment guide. Vancouver, BC: British Columbia Institute on Family Violence.
50. Kropp, P. R., Hart, S. D., Webster, C. D., & Eaves, D. (1995) Manual for the Spousal Assault Risk Assessment guide (2nd ed.). Vancouver, BC: British Columbia Institute on Family Violence.
51. Kropp, P. R., Hart, S. D., Webster, C. D., & Eaves, D. (1999). Spousal Assault Risk Assessment guide (SARA). Toronto, ON: Multi-Health Systems.
52. Landis, J. R., & Koch, G. G. (1977). The measurement of observer agreement for categorical data. Biometrics, 33, 159-174.
53. MedCalc Software. (2010). MedCalc: Version 11.3.8.0. Mariakerke. Belgium: MedCalc Software.
54. Meyers, J. R., & Schmidt, F. (2008). Predictive validity of the Structured Assessment for Violence Risk in Youth (SAVRY) with Juvenile Offenders. Criminal Justice & Behavior, 35, 344-355. DOI:10.1177/0093854807311972
55. Mills, J. F., Jones, M. N., & Kroner, D. G. (2005). An examination of the generalizability of the LSI-R and VRAG probability bins. Criminal Justice & Behavior, 32, 565-585. DOI:10.1177/0093854805278417
56. Monahan, J. (2006). A jurisprudence of risk assessment: Forecasting harm among prisoners, predators and patients. Virginia Law Review, 92, 391-435.
57. Monahan, J., Steadman, H., Appelbaum, P., Grisso, T., Mulvey, E., Roth, L. et al. (2005). The Classification of Violence Risk. Lutz, FL: Psychological Assessment Resources.
58. Mossman, D., & Somoza, E. (1991). ROC curves, test accuracy, and the description of diagnostic tests. Diagnostic Testing in Neuropsychiatry, 3, 330-333.
59. Mossman, D. (1994). Assessing predictions of violence: Being accurate about accuracy. Journal of Consulting & Clinical Psychology, 62, 783-792.
60. Munro, E. (2004). A simpler way to understand the results of risk assessment instruments. Children & Youth Services Review, 26, 873-883. DOI:10.1016/j.childyouth.2004.02.026
61. Nuffield, J. (1982). Parole decision-making in Canada: Research towards decision guidelines. Ottawa, ON: Ministry of Supply and Services Canada.
62. Nursing and Midwifery Council. (2004). Standards of proficiency for specialist community public health nurses. London: Nursin and Midwifery Council.
63. Quinsey, V. L., Harris, G. T., Rice, M. E., & Cormier, C. A. (1998). Violent offenders: Appraising and managing risk. Washington, DC: American Psychological Association.
64. Quinsey, V. L., Harris, G. T., Rice, M. E., & Cormier, C. A. (2006). Violent offenders: Appraising and managing risk (2nd ed.). Washington, DC: American Psychological Association.
65. Rice, M. E., & Harris, G. T. (1995). Violent recidivism: Assessing predictive validity. Journal of Consulting & Clinical Psychology, 63, 737-748.
66. Rice, M. E., & Harris, G. T. (2005). Comparing effect sizes in follow-up studies: ROC area, Cohen's d, and r. Law & Human Behavior, 29, 615-620. DOI:10.1007/s10979-005-6832-7
67. Roaldset, J., Hartvig, P., & Bjørkly, S. (2011). V-RISK-10: Validation of a screen for risk of violence after discharge from acute psychiatry. European Psychiatry, 26, 85-91. DOI:10.1016/j.eurpsy.2010.04.002
68. Singh, J. P., & Fazel, S. (2010). Forensic risk assessment: A metareview. Criminal Justice & Behavior, 37, 965-988. DOI:10.1177/0093854810374274
69. Singh, J. P., Grann, M., & Fazel, S. (2011). A comparative study of risk assessment tools: A systematic review and metaregression analysis of 68 studies involving 25, 980 participants. Clinical Psychology Review, 499-513. DOI:10.1016/j.cpr.2010.11.00931
70. Singh, J. P., Serper, M., Reinharth, J., & Fazel, S. (2011). Structured assessment of violence risk in schizophrenia and other disorders: A systematic review of the validity, reliability, and item content of 10 available instruments. Schizophrenia Bulletin, 37, 899-912. DOI:10.1093/schbul/sbr093
71. Sjöstedt, G., & Grann, M. (2002). Risk assessment: What is being predicted by actuarial prediction instruments? International Journal of Forensic Mental Health, 1, 179-183. DOI:10.1080/14999013.2002.10471172
72. Skeem, J. L., & Monahan, J. (2011). Current directions in violence risk assessment. Current Directions in Psychological Science, 20, 38-42. DOI:10.1177/0963721410397271
73. Smits, N. (2010). A note on Youden's J and its cost ratio. BMC Medical Research Methodology, 10, 89-93. DOI:10.1186/1471-2288-10-89
74. Sreenivasan, S., Garrick, T., Norris, R., Cusworth-Walker, S., Weinberger, L. E., Essres, G., et al. (2007). Predicting the likelihood of future sexual recidivism: Pilot study findings from a California sex offender risk project and cross-validation of the Static-99. Journal of the American Academy of Psychiatry & the Law, 35, 454-468.
75. StataCorp (2007). STATA statistical software: Release 10.1. College Station, TX: StataCorp LP.
76. Swets, J. (1988). Measuring the accuracy of diagnostic systems. Science, 240, 1285-1293. DOI:10.1126/science.3287615
77. Tape, T. (2006). The area under the ROC curve. In Interpreting diagnostic tests. Retrieved from http://gim.unmc.edu/dxtests/ROC3.htm
78. Thomson, L., Davidson, M., Brett, C., Steele, J., & Darjee, R. (2008). Risk assessment in forensic patients with schizophrenia: The predictive validity of actuarial scales and symptom severity for offending and violence over 8-10 years. International Journal of Forensic Mental Health, 7, 173-189. DOI:10.1080/14999013.2008.9914413
79. Thornton, D., Mann, R. E., Webster, S., Blud, L., Travers, R., Friendship, C. et al. (2003). Distinguishing and combining risks for sexual and violent recidivism. In R. A. Prentky, E. S. Janus, & M. C. Seto (Eds), Sexually coercive behavior: Understanding and management (pp. 225-235). Annals of the New York Academy of Science, Vol. 989. New York: New York Academy of Sciences.
80. Tyrer, P., Duggan, C., Cooper, S., Crawford, M., Seivewright, H., Rutter, D., et al. (2010). The successes and failures of the DSPD experiment: The assessment and management of severe personality disorder. Medicine, Science and the Law, 50, 95-99. DOI:10.1258/msl.2010.010001
81. Viljoen, J. L., McLachlan, K., & Vincent, G. M. (2010). Assessing violence risk and psychopathy in juvenile and adult offenders: A survey of clinical practices. Assessment, 17, 377-395. DOI:10.1177/1073191109359587
82. Webster, C. D., Douglas, K. S., Eaves, D., & Hart, S. D. (1997). HCR- 20: Assessing risk for violence. Version 2. Burnaby, BC: Simon Fraser University, Mental Health, Law, and Policy Institute.
83. Webster, C. D., Eaves, D., Douglas, K. S., & Wintrup, A. (1995). The HCR-20 scheme: The assessment of dangerousness and risk. Vancouver, BC: Mental Health Law and Policy Institute, and Forensic Psychiatric Services Commission of British Columbia.
84. Webster, C. D., Nicholls, T. L., Martin, M. L., Desmarais, S. L., & Brink, J. (2006). Short-Term Assessment of Risk and Treatability (START): The case for a new violence risk structured professional judgment scheme. Behavioral Sciences & the Law, 24, 747-766.
85. Werkgroep Pilotstudy Risicotaxatie Forensische Psychiatrie. (2002). Bevindingen van een landelijke pilotstudy naar de HKT-30. [Findings of a nationwide pilot study on the HKT-30]. The Hague: Ministerie van Justitie.
86. Wong, S., & Gordon, A. (2009) Manual for the Violence Risk Scale. Saskatoon, SK: University of Saskatchewan.
87. Woods, P., & Ashley, C. (2007). Violence and aggression: A literature review. Journal of Psychiatric & Mental Health Nursing, 14, 652-660. DOI:10.1111/j.1365-2850.2007.01149.x
88. Wootton, L., Buchanan, A., Leese, M., Tyrer, P., Burns, T., Creed, F., et al. (2008). Violence in psychosis: Estimating the predictive validity of readily accessible clinical information in a community sample. Schizophrenia Research, 101, 176-184. DOI:10.1016/j.schres.2007.12.490