Using dynamic risk and protective factors to predict inpatient aggression: Reliability and validity of START assessments

Psychological Assessment

Volumn 24, Issue 3, 2012, Pages 685-700

  Desmarais, Sarah L ^a   Nicholls, Tonia L ^b,c,d   Wilson, Catherine M ^d   Brink, Johann ^b,c,d  

^a UNIVERSITY OF SOUTH FLORIDA   (United States)

^b BC Mental Health & Addiction Services   (Canada)

^c UNIVERSITY OF BRITISH COLUMBIA   (Canada)

^d SIMON FRASER UNIVERSITY   (Canada)

Author keywords

Dynamic factors; Protective factors; START; Structured professional judgment; Violence risk assessment

Indexed keywords

ADULT; AGGRESSION; ARTICLE; CONTROLLED STUDY; FOLLOW UP; FORENSIC PSYCHIATRY; HARE PSYCHOPATHY CHECKLIST: SCREENING VERSION; HISTORICAL CLINICAL RISK MANAGEMENT 20; HOSPITAL PATIENT; HUMAN; INCREMENTAL VALIDITY; MAJOR CLINICAL STUDY; MALE; MEDICAL DECISION MAKING; MEDICAL RECORD; MENTAL DISEASE; MENTAL PATIENT; NAMED INVENTORIES, QUESTIONNAIRES AND RATING SCALES; OVERT AGGRESSION SCALE; PERSONALITY DISORDER; PREDICTION; PSYCHOLOGIC ASSESSMENT; RELIABILITY; RISK ASSESSMENT; SHORT-TERM ASSESSMENT OF RISK AND TREATABILITY; STRUCTURED PROFESSIONAL JUDGMENT; SUBSTANCE ABUSE; VALIDITY; VIOLENCE; DISEASE MANAGEMENT; INSTRUMENTATION; METHODOLOGY; OFFENDER; PSYCHIATRIC DEPARTMENT; PSYCHOLOGICAL ASPECT; PSYCHOLOGICAL RATING SCALE; PSYCHOMETRY; REPRODUCIBILITY; RETROSPECTIVE STUDY; RISK; SEVERITY OF ILLNESS INDEX; STANDARD; TIME;

ADULT; AGGRESSION; CRIMINALS; DISEASE MANAGEMENT; FOLLOW-UP STUDIES; FORENSIC PSYCHIATRY; HUMANS; INPATIENTS; MALE; MENTAL DISORDERS; PSYCHIATRIC DEPARTMENT, HOSPITAL; PSYCHIATRIC STATUS RATING SCALES; PSYCHOMETRICS; REPRODUCIBILITY OF RESULTS; RETROSPECTIVE STUDIES; RISK; SEVERITY OF ILLNESS INDEX; TIME FACTORS;

EID: 84869437965     PISSN: 10403590     EISSN: 1939134X     Source Type: Journal    
DOI: 10.1037/a0026668     Document Type: Article

References (87)

1. Alterman, A. L., Cacciola, J. S., & Rutherford, M. J. (1993). Reliability of the Revised Psychopathy Checklist in substance abuse patients. Psychological Assessment, 5, 442-448. doi:10.1037/1040-3590.5.4.442.
2. American Psychological Association Presidential Task Force on EvidenceBased Practice. (2006). Evidence-based practice in psychology. American Psychologist, 61, 271-285. doi:10.1037/0003-066X.61.4.271.
3. Andrews, D. A., & Bonta, J. (1995). The Level of Service InventoryRevised. Toronto, Ontario, Canada: Multi-Health Systems.
4. Belfrage, H., Fransson, G., & Strand, S. (2000). Prediction of violence using the HCR-20: A prospective study in two maximum security correctional institutions. Journal of Forensic Psychiatry, 11, 167-175.
5. Braithwaite, E., Charette, Y., Crocker, A. G., & Reyes, A. (2010). The predictive validity of clinical ratings of the Short-Term Assessment of Risk and Treatability (START). The International Journal of Forensic Mental Health, 9, 271-281. doi:10.1080/14999013.2010.534378.
6. Brown, S. L., St. Amand, M. D., & Zamble, E. (2009). The dynamic-wave prospective study. Law and Human Behavior, 33, 25-45. doi:10.1007/s10979-008-9139-7.
7. Campbell, M. B., French, S., & Gendreau, P. (2009). The prediction of violence in adult offenders: A meta-analytic comparison of instruments and methods of assessment. Criminal Justice and Behavior, 36, 567-590. doi:10.1177/0093854809333610.
8. Chu, C. M., Thomas, S. D. M., Ogloff, J. R. P., & Daffern, M. (2011). The shortto medium-term predictive accuracy of static and dynamic risk assessment measures in a secure forensic hospital. Assessment. Advance online publication. doi:10.1177/1073191111418298.
9. Cicchetti, D., Bronen, R., Spencer, S., Haut, S., Berg, A., Oliver, P., & Tyrer, P. (2006). Rating scales, scales of measurement, issues of reliability: Resolving some critical issues for clinicians and researchers. Journal of Nervous and Mental Disease, 194, 557-564. doi:10.1097/01.nmd.0000230392.83607.c5.
10. Cohen, J. (1988). Statistical power analysis for the behavioral sciences. Hillsdale, NJ: Erlbaum.
11. Crocker, A. G., Braithwaite, E., Laferrière, D., Gagnon, D., Venegas, C., & Jenkins, T. (2011). START changing practice: Implementing a risk assessment and management tool in a civil psychiatric setting. The International Journal of Forensic Mental Health, 10, 13-28. doi: 10.1080/14999013.2011.553146.
12. Daffern, M. (2007). The predictive validity and practical utility of structured schemes used to assess risk for aggression in psychiatric inpatient settings. Aggression and Violent Behavior, 12, 116-130. doi:10.1016/j.avb.2006.03.005.
13. Department of Health, National Risk Management Programme. (2007). Best practice in managing risk: Principles and evidence for best practice in the assessment and management of risk to self and others in mental health services. London, England: Author.
14. Desmarais, S. L., Collins, M. J., Nicholls, T. L., & Brink, J. (2011). Measuring clinician attitudes to bridge the gap between violence risk assessment science and practice: Perceived utility and acceptability of START. Manuscript submitted for publication.
15. Desmarais, S. L., Hucker, S., Brink, J., & De Freitas, K. (2008). A Canadian example of insanity defence reform: Accused found not criminally responsible before and after the Winko decision. The International Journal of Forensic Mental Health, 7, 1-14. doi:10.1080/14999013.2008.9914399.
16. Desmarais, S. L., Webster, C. D., Martin, M. L., Dassinger, C., Brink, J., & Nicholls, T. L. (2007). Short-Term Assessment of Risk and Treatability (START): Instructors' guide and workbook (Version 2). Coquitlam, Canada: Forensic Psychiatric Services Commission.
17. de Vogel, V., & de Ruiter, C. (2004). Differences between clinicians and researchers in assessing risk of violence in forensic psychiatric patients. The Journal of Forensic Psychiatry & Psychology, 15, 145-164. doi: 10.1080/14788940410001655916.
18. de Vogel, V., & de Ruiter, C. (2006). Structured professional judgment of violence risk in forensic clinical practice: A prospective study into the predictive validity of the Dutch HCR-20. Psychology, Crime & Law, 12, 321-336. doi:10.1080/10683160600569029.
19. de Vogel, V., de Ruiter, C., Bouman, Y., & de Vries Robbé, M. (2009). SAPROF. Guidelines for the assessment of protective factors for violence risk (English version). Utrecht, the Netherlands: Forum Educatief.
20. Douglas, K. S., Guy, L. S., Reeves, K. A., & Weir, J. (2008, November). HCR-20 violence risk assessment scheme: Overview and annotated bibliography. Retrieved from http://kdouglas.files.wordpress.com/2006/04/annotate10-24nov2008.pdf.
21. Douglas, K. S., & Kropp, P. R. (2002). A prevention-based paradigm for violence risk assessment: Clinical and research applications. Criminal Justice and Behavior, 2, 617-658.
22. Douglas, K. S., Ogloff, J. R. P., & Hart, S. D. (2003). Evaluation of a model of violence risk assessment among forensic psychiatric patients. Psychiatric Services, 54, 1372-1379. doi:10.1176/appi.ps.54.10.1372.
23. Douglas, K. S., & Skeem, J. (2005). Violence risk assessment: Getting specific about being dynamic. Psychology, Public Policy, and Law, 11, 347-383. doi:10.1037/1076-8971.11.3.347.
24. Douglas, K. S., Strand, S., Belfrage, H., Fransson, G., & Levander, S. (2005). Reliability and validity evaluation of the Psychopathy Checklist: Screening Version (PCL:SV) in Swedish correctional and forensic psychiatric samples. Assessment, 12, 145-161. doi:10.1177/1073191105275455.
25. Douglas, K. S., Yeomans, M., & Boer, D. P. (2005). Comparative validity analysis of multiple measures of violence risk in a sample of criminal offenders. Criminal Justice and Behavior, 32, 479-510. doi:10.1177/0093854805278411.
26. Doyle, M., Dolan, M., & McGovern, J. (2002). The validity of North American risk assessment tools in predicting in-patient violent behav-iour in England. Legal and Criminological Psychology, 7, 141-154. doi:10.1348/135532502760274756.
27. Doyle, M., Lewis, G., & Brisbane, M. (2008). Implementing the ShortTerm Assessment of Risk and Treatability (START) in a forensic mental health service. Psychiatric Bulletin, 32, 406-408. doi:10.1192/pb.bp.108.019794.
28. Duckworth, A. L., Steen, T. A., & Seligman, M. E. P. (2005). Positive psychology in clinical practice. Annual Review of Clinical Psychology, 1, 629-651. doi:10.1146/annurev.clinpsy.1.102803.144154.
29. Edens, J. F., Boccaccini, M. T., & Johnson, D. W. (2010). Inter-rater reliability of the PCL-R total and factor scores among psychopathic sex offenders: Are personality features more prone to disagreement than behavioral features? Behavioral Sciences & the Law, 28, 106-119. doi:10.1002/bsl.918.
30. Gadon, L., Johnstone, L., & Cooke, D. (2006). Situational variables and institutional violence: A systematic review of the literature. Clinical Psychology Review, 26, 515-534. doi:10.1016/j.cpr.2006.02.002.
31. Gagliardi, G. J., Lovell, D., Peterson, P. D., & Jemelka, R. (2004). Forecasting recidivism in mentally ill offenders released from prison. Law and Human Behavior, 28, 133-155. doi:10.1023/B:LAHU.0000022319.03637.45.
32. Gilgun, J., Klein, C., & Pranis, K. (2000). The significance of resources in models of risk. Journal of Interpersonal Violence, 15, 631-650. doi: 10.1177/088626000015006005.
33. Grann, M., Långström, N., Tengströem, A., & Stålenheim, E. G. (1998). The reliability of file-based retrospective ratings of psychopathy with the PCL-R. Journal of Personality Assessment, 70, 416-426. doi:10.1207/s15327752jpa7003_2.
34. Gray, S. N., Bonson, R., Craig, R., Davies, H., Fitzgerald, S., Huckle, P.,Snowden, R. J. (in press). The Short-Term Assessment of Risk and Treatability (START): A prospective study of inpatient behaviour. International Journal of Forensic Mental Health.
35. Grove, W. M., & Meehl, P. E. (1996). Comparative efficiency of informal (subjective, impressionistic) and formal (mechanical, algorithmic) prediction procedures: The clinical-statistical controversy. Psychology, Public Policy, and Law, 2, 293-323. doi:10.1037/1076-8971.2.2.293.
36. Grove, W. M., Zald, D. H., Lebow, B. S., Snitz, B. E., & Nelson, C. (2000). Clinical versus mechanical prediction: A meta-analysis. Psychological Assessment, 12, 19-30. doi:10.1037/1040-3590.12.1.19.
37. Guy, L. S. (2008). Performance indicators of the structured professional judgment approach for assessing risk for violence to others: A metaanalytic survey. Unpublished doctoral thesis, Simon Fraser University, Burnaby, Canada.
38. Guy, L. S., & Douglas, K. S. (2006). Examining the utility of the PCL:SV as a screening measure using competing factor models of psychopathy. Psychological Assessment, 18, 2, 225-230. doi:10.1037/1040-3590.18.2.225.
39. Hamrin, V., Iennaco, J., & Olsen, D. (2009). A review of ecological factors affecting inpatient psychiatric unit violence: Implications for relational and unit cultural improvements. Issues in Mental Health Nursing, 30, 214-226. doi:10.1080/01612840802701083.
40. Hanson, R. K. (2005). Twenty years of progress in violence risk assessment. Journal of Interpersonal Violence, 20, 212-217. doi:10.1177/0886260504267740.
41. Hanson, R. K., & Morton-Bourgon, K. E. (2009). The accuracy of recidivism risk assessments for sexual offenders: A meta-analysis. Psychological Assessment, 21, 1-21. doi:10.1037/a0014421.
42. Haque, Q., Cree, A., Webster, C., & Hasnie, B. (2008). Best practice in managing violence and related risks. Psychiatric Bulletin, 32, 403-405. doi:10.1192/pb.bp.108.019927.
43. Hare, R. D. (2003). The Revised Psychopathy Checklist. Toronto, Canada: Multi-Health Systems.
44. Hart, S. D., Cox, D. N., & Hare, R. D. (1995). The Hare Psychopathy Checklist: Screening Version. Toronto, Canada: Multi Health Systems.
45. Hart, S. D., Webster, C. D., & Douglas, K. S. (2001). Risk management using the HCR-20: A general overview of focusing on historical factors. In K. S. Douglas, C. D. Webster, S. D. Hart, D. Eaves, & J. R. P. Ogloff (Eds.), HCR-20 violence risk management companion guide (pp. 27-40). Burnaby, Canada/Tampa, FL: Simon Fraser University, Mental Health, Law & Policy Institute/University of South Florida, Department of Mental Health Law & Policy.
46. Heilbrun, K., Yasuhara, K., & Shah, S. (2010). Violence risk assessment tools: Overview and critical analysis. In R. K. Otto & K. S. Douglas, Handbook of violence risk assessment (pp. 1-17). New York, NY: Routledge/Taylor & Francis.
47. Higgins, N., Watts, D. Bindman, J., Slade, M., & Thornicroft, G. (2005). Assessing violence in general adult psychiatry. Psychiatric Bulletin, 29, 131-133. doi:10.1192/pb.29.4.131.
48. Hilton, N. Z., Harris, G. T., & Rice, M. E. (2006). Sixty-six years of research on the clinical versus actuarial prediction of violence. The Counseling Psychologist, 34, 400-409. doi:10.1177/0011000005285877.
49. Johnstone, L., & Cooke, D. J. (2010). PRISM: A promising paradigm for assessing and managing institutional violence: Findings from a multiple case study analysis of five Scottish prisons. The International Journal of Forensic Mental Health, 9, 180-191. doi:10.1080/14999013.2010.526477.
50. Khiroya, R., Weaver, T., & Maden, T. (2009). Use and perceived utility of structured violence risk assessments in English medium secure forensic units. Psychiatric Bulletin, 33, 129-132. doi:10.1192/pb.bp.108.019810.
51. Kroppan, E., Nesset, M. B., Nonstad, K., Pedersen, T. W., Almvik, R., & Palmstierna, T. (2011). Implementation of the Short-Term Assessment of Risk and Treatability (START) in a forensic high secure unit. The International Journal of Forensic Mental Health, 10, 7-12. doi:10.1080/14999013.2011.552368.
52. Landis, J. R., & Koch, G. G. (1977). The measurement of observer agreement for categorical data. Biometrics, 33, 159-174. doi:10.2307/2529310.
53. Lussier, P., Verdun-Jones, S., Deslauriers-Varin, N., Nicholls, T. L., & Brink, J. (2010). Chronic violent patients in an inpatient psychiatric hospital: Prevalence, description, and identification. Criminal Justice and Behavior, 37, 5-28. doi:10.1177/0093854809347738.
54. McDermott, B. E., Edens, J. F., Quanbeck, C. D., Busse, D., & Scott, C. L. (2008). Examining the role of static and dynamic risk factors in the prediction of inpatient violence: Variableand person-focused analyses. Law and Human Behavior, 32, 325-338. doi:10.1007/s10979-007-9094-8.
55. Monahan, J., Steadman, H. J., Appelbaum, P. S., Grisso, T., Mulvey, E. P., Roth, L. H., et al. (2005). Classification of Violence Risk (COVR). Lutz, FL: Psychological Assessment Resources.
56. Monahan, J., Steadman, H. J., Silver, E., Appelbaum, P., Robbins, P., Mulvey, E.,Banks, S. (2001). Rethinking risk assessment: The Mac Arthur study of mental disorder and violence. New York, NY: Oxford University Press.
57. Nicholls, T. L., Brink, J., Desmarais, S. L., Webster, C. D., & Martin, M.-L. (2006). The Short-Term Assessment of Risk and Treatability (START): A prospective validation study in a forensic psychiatric sample. Assessment, 13, 313-327. doi:10.1177/1073191106290559.
58. Nicholls, T. L., Brink, J., Greaves, C., Lussier, P., & Verdun-Jones, S. (2009). Forensic psychiatric inpatients and aggression: An exploration of incidence, prevalence, severity, and interventions by patient gender. International Journal of Law and Psychiatry, 32, 23-30. doi:10.1016/j.ijlp.2008.11.007.
59. Nicholls, T. L., Ogloff, J. R. P., & Douglas, K. S. (2004). Assessing risk for violence among female and male civil psychiatric patients: The HCR-20, PCL:SV, and McNiel & Binder's VSC. Behavioral Sciences & the Law, 22, 127-158. doi:10.1002/bsl.579.
60. Nicholls, T. L., Petersen, K., Brink, J., & Webster, C. (2011). A clinical risk profile of forensic psychiatric patients: Treatment team STARTs in a Canadian service. The International Journal of Forensic Mental Health, 10, 187-199. doi:10.1080/14999013.2011.600234.
61. Nonstad, K., Nesset, M. B., Kroppan, E., Pedersen, T. W., Nøttestad, J. A., Almvik, R., & Palmstierna, T. (2010). Predictive validity and other psychometric properties of the Short-Term Assessment of Risk and Treatability (START) in a Norwegian high secure hospital. The International Journal of Forensic Mental Health, 9, 294-299. doi:10.1080/14999013.2010.534958.
62. O'Shaughnessy, R. J. (2007). AAPL practice guideline for the forensic psychiatric evaluation of competence to stand trial: A Canadian legal perspective. Journal of the American Academy of Psychiatry and the Law, 35, 505-508.
63. Otto, R. K., & Douglas, K. S. (Eds.). (2010). Handbook of violence risk assessment. New York, NY: Routledge.
64. Petersen, K. L., Douglas, K. S., & Nicholls T. L. (2011, March). Gender differences in the psychometric properties of the Short-Term Assessment of Risk and Treatability (START) in an acute civil psychiatric sample. Paper presented at the American Psychology-Law Society conference, Miami, FL.
65. Philipse, M. W. G., Koeter, M. W. J., van der Staak, C. P. F., & van den Brink, W. (2006). Static and dynamic patient characteristics as predictors of criminal recidivism: A prospective study in a Dutch forensic psychiatric sample. Law and Human Behavior, 30, 309-327.
66. Quinsey, V. L., Harris, G. T., Rice, M. E., & Cormier, C. A. (2006). Violent offenders: Appraising and managing risk (2nd ed.). Washington, DC: American Psychological Association. doi:10.1037/11367-000.
67. Rice, M. E., Harris, G. T., & Quinsey, V. L. (2002). The appraisal of violence risk. Current Opinion in Psychiatry, 15, 589-593. doi:10.1097/00001504-200211000-00005.
68. Rogers, R. (2000). The uncritical acceptance of risk assessment in forensic practice. Law and Human Behavior, 24, 595-605. doi:10.1023/A: 1005575113507.
69. Rutter, M. (1987). Psychosocial resilience and protective mechanisms. American Journal of Orthopsychiatry, 57, 316-331.
70. Ryba, N. L. (2008). The other side of the equation: Considering risk state and protective factors in violence risk assessment. Journal of Forensic Psychology Practice, 8, 413-423. doi:10.1080/15228930802199382.
71. Saleebey, D. (1996). The strengths perspective n social work practice (2nd ed.). New York, NY: Addison-Wesley.
72. Scott, P. D. (1977). Assessing dangerousness in criminals. British Journal of Psychiatry, 131, 127-142. doi:10.1192/bjp.131.2.127.
73. Simourd, D. J. (2004). Use of dynamic risk/need assessment instruments among long-term incarcerated offenders. Criminal Justice and Behavior, 31, 306-323. doi:10.1177/0093854803262507.
74. Singh, J. P., Grann, M., & Fazel, S. (2011). A comparative study of violence risk assessment tools: A systematic review and metaregression analysis of 68 studies involving 25,980 participants. Clinical Psychology Review, 31, 499-513. doi:10.1016/j.cpr.2010.11.009.
75. Skeem, J. L., & Monahan, J. (2011). Current directions in violence risk assessment. Current Directions in Psychological Science, 20, 38-42. doi:10.1177/0963721410397271.
76. Swets, J. A. (1988). Measuring the accuracy of diagnostic systems. Science, 240, 285-1293.
77. Verdun-Jones, S., Brink, J., Lussier, P., & Nicholls, T. L. (2006). Aggression and violence at the B.C. Forensic Psychiatric Hospital: Description, prediction, management and legal issues. Vancouver, Canada: Forensic Psychiatric Services Commission.
78. Viljoen, S., Launeanu, M., Hendry, M., Nicholls, T. L., & Brink, J. (2011, March). Comparing ratings of subject matter experts and clinicians on the START in a forensic clinic. Poster presented at the American Psychology-Law Society conference, Miami, FL.
79. Webster, C. D., Douglas, K. S., Eaves, D., & Hart, S. D. (1997). HCR-20: Assessing risk for violence (Version 2). Vancouver, Canada: Mental Health, Law, & Policy Institute, Simon Fraser University.
80. Webster, C. D., Martin, M. L., Brink, J., Nicholls, T. L., & Desmarais, S. L. (2009). Manual for the Short-Term Assessment of Risk and Treatability (START) (Version 1.1). Coquitlam, Canada: British Columbia Mental Health & Addiction Services.
81. Webster, C. D., Martin, M. L., Brink, J., Nicholls, T. L., & Middleton, C. (2004). Manual for the Short Term Assessment of Risk and Treatability (START) (Version 1.0, Consultation ed.). Hamilton, Canada/Coquitlam, Canada: St. Joseph's Healthcare Hamilton/Forensic Psychiatric Services Commission.
82. Webster, C. D., Nicholls, T. L., Martin, M. L., Desmarais, S. L., & Brink, J. (2006). Short-Term Assessment of Risk and Treatability (START): The case for a new violence risk structured professional judgment scheme. Behavioral Sciences & the Law, 24, 747-766. doi:10.1002/bsl.737.
83. Wilson, C. M., Desmarais, S. L., Nicholls, T. L., & Brink, J. (2010). The role of client strengths in assessments of violence risk using the ShortTerm Assessment of Risk and Treatability (START). The International Journal of Forensic Mental Health, 9, 282-293. doi:10.1080/14999013.2010.534694.
84. Wilson, C. M., Desmarais, S. L., Nicholls, T. L., Hart, S. D., & Brink, J. (2011). Incremental validity of dynamic factors in the assessment of violence risk using the HCR-20 and START. Manuscript submitted for publication.
85. Wong, S. (1988). Is Hare's Psychopathy Checklist reliable without the interview? Psychological Reports, 62, 931-934. doi:10.2466/pr0.1988.62.3.931.
86. Yang, M., Wong, S. C., & Coid, J. (2010). The efficacy of violence prediction: A meta-analytic comparison of nine risk assessment tools. Psychological Bulletin, 136, 740-767.
87. Yudofsky, S. C., Silver, J. M., Jackson, W., Endicott, J., & Williams, D. W. (1986). The Overt Aggression Scale for the objective rating of verbal and physical aggression. The American Journal of Psychiatry, 143, 35-39.