Valuing Structured Professional Judgment: Predictive Validity, Decision-making, and the Clinical-Actuarial Conflict

Behavioral Sciences and the Law

Volumn 31, Issue 1, 2013, Pages 40-54

  Falzer, Paul R ^a  

^a VA CONNECTICUT HEALTHCARE SYSTEM   (United States)

Author keywords

[No Author keywords available]

Indexed keywords

ARTICLE; CONFLICT; DECISION MAKING; EXPERT WITNESS; FORECASTING; FORENSIC PSYCHIATRY; HUMAN; METHODOLOGY; RISK ASSESSMENT; STANDARD; STATISTICAL ANALYSIS; VIOLENCE;

ACTUARIAL ANALYSIS; CONFLICT (PSYCHOLOGY); DECISION MAKING; EXPERT TESTIMONY; FORECASTING; FORENSIC PSYCHIATRY; HUMANS; JUDGMENT; RISK ASSESSMENT; VIOLENCE;

EID: 84874392248     PISSN: 07353936     EISSN: 10990798     Source Type: Journal    
DOI: 10.1002/bsl.2043     Document Type: Article

References (91)

1. Ægisdǿttir, S., White, M. J., Spengler, P. M., Maugherman, A. S., Anderson, L. A., Cook, R. S., ... Rush, J. D. (2006). The Meta-Analysis of Clinical Judgment project: Fifty-six years of accumulated research on clinical versus statistical prediction. The Counseling Psychologist, 34(3), 341-382.
2. Beach, L. R., & Connolly, T. (2005). The psychology of decision making: People in organizations (2nd edn). Thousand Oaks, CA: Sage Publications.
3. Belfrage, H., Fransson, G., & Strand, S. (2004). Management of violent behaviour in the correctional system using qualified risk assessments. Legal and Criminological Psychology, 9(1), 11-22.
4. Birnbaum, A. (1977). The Neyman-Pearson theory as decision theory, and as inference theory; with a criticism of the Lindley-Savage argument for Bayesian theory. Synthese: An International Journal for Epistemology, Methodology and Philosophy of Science 36(1), 19-49.
5. Boer, D. P., Hart, S. D., Kropp, P. R., & Webster, C. D. (1997). Manual for the Sexual Violence Risk-20. Professional guidelines for assesisng risk of sexual violence. Burnaby, B.C.: Simon Fraser University, Mental Health, Law, and Policy Institute.
6. Bornstein, B. H., & Emler, A. C. (2001). Rationality in medical decision making: A review of the literature on doctors' decision-making biases. Journal of Evaluation in Clinical Practice, 7(2), 97-107.
7. Borum, R. (1996). Improving the clinical practice of violence risk assessment: Technology, guidelines, and training. American Psychologist, 51(9), 945-956.
8. Borum, R., Bartel, P. A., & Forth, A. E. (2005). Structured Assessment of Violence Risk in Youth. In Grisso T., Vincent G., & Seagrave D. (Eds.), Mental health screening and assessment in juvenile justice (pp. 311-323). New York: The Guilford Press.
9. Buchanan, A. (2008). Risk of violence by psychiatric patients: Beyond the "actuarial versus clinical" assessment debate. Psychiatric Services, 59(2), 184-190.
10. Cook, N. R. (2007). Use and misuse of the receiver operating characteristic curve in risk prediction. Circulation, 115(7), 928-935.
11. Cook, N. R. (2008a). Comentary on 'Evaluating the added predictive ability of a new marker: From area under the ROC curve to reclassification and beyond'. Statistics in Medicine, 27(2), 191-195.
12. Cook, N. R. (2008b). Statistical evaluation of prognostic versus diagnostic models: Beyond the ROC curve. Clinical Chemistry, 54(1), 17-23.
13. Cook, N. R., & Paynter, N. P. (2011). Performance of reclassification statistics in comparing risk prediction models. Biometrical Journal, 53(2), 237-258.
14. Dawes, R. M., Faust, D., & Meehl, P. E. (1989). Clinical versus actuarial judgment. Science, 243(4899), 1668-1674.
15. Diamond, G. A. (1992). What price perfection? Calibration and discrimination of clinical prediction models. Journal of Clinical Epidemiology, 45(1), 85-89.
16. Douglas, K. S., & Skeem, J. L. (2005). Violence risk assessment: Getting specific about being dynamic. Psychology, Public Policy, and Law, 11(3), 347-383.
17. Douglas, K. S., Ogloff, J. R. P., Nicholls, T. L., & Grant, I. (1999). Assessing risk for violence among psychiatric patients: The HCR-20 violence risk assessment scheme and the psychopathy checklist: screening version. Journal of Consulting and Clinical Psychology, 67(6), 917-930.
18. Douglas, K. S., Yeomans, M., & Boer, D. P. (2005). Comparative validity analysis of multiple measures of violence risk in a sample of criminal offenders. Criminal Justice and Behavior, 32(5), 479-510.
19. Elstein, A. S. (1999). Heuristics and biases: Selected errors in clinical reasoning. Academic Medicine, 74(7), 791-794.
20. Falzer, P. R. (2010). Expertise in assessing and managing risk of violence: The contribution of naturalistic decision making. In Mosier K. L., & Fischer U. M. (Eds.), Informed by knowledge: Expert performance in complex situations (pp. 313-328). New York: Taylor and Francis.
21. Fawcett, T. (2006). An introduction to ROC analysis. Pattern Recognition Letters, 27, 861-874.
22. Fazel, S., Singh, J. P., Doll, H., & Grann, M. (2012). Use of risk assessment instruments to predict violence and antisocial behaviour in 73 samples involving 24, 827 people: Systematic review and meta-analysis. BMJ: British Medical Journal, 345(7868), 1-12.
23. Feinstein, A. R. (1975). Clinical biostatistics. XXXIV. The other side of "statistical significance": Alpha, beta, delta, and the calculation of sample size. Clinical Pharmacology and Therapeutics, 18(4), 491-505.
24. Glas, A. S., Lijmer, J. G., Prins, M. H., Bonsel, G. J., & Bossuyt, P. M. M. (2003). The diagnostic odds ratio: a single indicator of test performance. Journal of Clinical Epidemiology, 56(11), 1129-1135.
25. Goldberg, L. R. (1967). Research on the clinical judgment process. Anthropology & Medicine, 15(4-5), 220-242.
26. Goldberg, L. R. (1970). Five models of clinical judgment: An empirical comparison between linear and nonlinear representations of the human inference process. Organizational Behavior and Human Performance, 6, 458-479.
27. Goodenough, D. J., Rossmann, K., & Lusted, L. B. (1974). Radiographic applications of receiver operating characteristic (ROC) curves. Radiology, 110(1), 89-95.
28. Gottfredson, S. D., & Moriarty, L. J. (2006). Statistical risk assessment: Old problems and new applications. Crime and Delinquency, 52(1), 178-200.
29. Green, D. M. (1960). Psychoacoustics and detection theory. The Journal of the Acoustical Society of America, 32(10), 1189-1203.
30. Grove, W. M., Zald, D. H., Lebow, B. S., Snitz, B. E., & Nelson, C. (2000). Clinical versus mechanical prediction: A meta-analysis. Psychological Assessment, 12(1), 19-30.
31. Guthrie, C., Rachlinski, J. L., & Wistrich, A. J. (2000). Inside the judicial mind. Cornell Law Review, 86, 777-830.
32. Hacking, I. (1975). The emergence of probability: A philosophical study of early ideas about probability, induction and statistical inference. London: Cambridge University Press.
33. Hanley, J. A., & McNeil, B. J. (1982). The meaning and use of the area under a receiver operating characteristic (ROC) curve. Radiology, 143(1), 29-36.
34. Hanson, R. K., & Howard, P. D. (2010). Individual confidence intervals do not inform decision-makers about the accuracy of risk assessment evaluations. Law and Human Behavior, 34(4), 275-281.
35. Harris, G. T., & Rice, M. E. (2007). Characterizing the value of actuarial violence risk assessments. Criminal Justice and Behavior, 34(12), 1638-1658.
36. Harris, G. T., Rice, M. E., & Quinsey, V. L. (1993). Violent recidivism of mentally disordered offenders: The development of a statistical prediction instrument. Criminal Justice and Behavior, 20(4), 315-335.
37. Hart, S. D., Michie, C., & Cooke, D. J. (2007a). Authors' reply to comments on "Precision of actuarial risk assessment instruments: Evaluating the 'margins of error' of group v. individual predictions of violence". The British Journal of Psychiatry, 191(6), 561-562.
38. Hart, S. D., Michie, C., & Cooke, D. J. (2007b). Precision of actuarial risk assessment instruments: Evaluating the 'margins of error' of group v. individual predictions of violence. The British Journal of Psychiatry, 190(49), s60-65.
39. Hathaway, S. R., & Meehl, P. E. (1951). An atlas for the clinical use of the MMPI. Minneapolis MN: University of Minnesota Press.
40. Hoffman, P. J. (1960). The paramorphic representation of clinical judgment. Psychological Bulletin, 57, 116-131.
41. Janes, H., Pepe, M. S., & Gu, W. (2008). Assessing the value of risk predictions by using risk stratification tables. Annals of Internal Medicine, 149(10), 751-760.
42. Janus, E. S., & Prentky, R. A. (2003). Forensic use of actuarial risk assessment with sex offenders: Accuracy, admissibility and accountability. American Criminal Law Review, 40(4), 1443-1499.
43. Klein, G. A., Snowden, D., & Pin, C. L. (2007). Anticipatory Thinking. Paper presented at the 8th International Conference on Naturalistic Decision Making, Asilomar Conference Center, Pacific Grove, CA.
44. Krakowski, M. I., Czobor, P., Citrome, L., Bark, N., & Cooper, T. B. (2006). Atypical antipsychotic agents in the treatment of violent patients with schizophrenia and schizoaffective disorder. Archives of General Psychiatry, 63(6), 622-629.
45. Lassiter, G. D. (2002). Illusory causation in the courtroom. Current Directions in Psychological Science, 11(6), 204-208.
46. Lodewijks, H. P., Doreleijers, T. A., & de Ruiter, C. (2008). SAVRY risk assessment in violent Dutch adolescents: Relation to sentencing and recidivism. Criminal Justice and Behavior, 35(6), 696-709.
47. Mayo, D. G. (1992). Did Pearson reject the Neyman-Pearson philosophy of statistics? Synthese: An International Journal for Epistemology, Methodology and Philosophy of Science 90(2), 233-262.
48. Meehl, P. E. (1957). When shall we use our heads instead of the formula? Journal of Counseling Psychology, 4, 268-273.
49. Meehl, P. E. (1967). Theory-testing in psychology and physics: A methodological paradox. Philosophy of Science, 34(2), 103-115.
50. Meehl, P. E. (1978). Theoretical risks and tabular asterisks: Sir Karl, Sir Ronald, and the slow progress of soft psychology. Journal of Consulting and Clinical Psychology, 46(4), 806-834.
51. Meehl, P. E. (1986). Causes and effects of my disturbing little book. Journal of Personality Assessment, 50(3), 370-375.
52. Meyers, J. R., & Schmidt, F. (2008). Predictive validity of the Structured Assessment for Violence Risk in Youth (SAVRY) with juvenile offenders. Criminal Justice and Behavior, 35(3), 344-355.
53. Moldoveanu, M., & Langer, E. (2002). False memories of the future: A critique of the applications of probabilistic reasoning to the study of cognitive processes. Psychological Review, 109(2), 358-375.
54. Monahan, J. (1981). The clinical prediction of violent behavior Crime and Delinquency Issues: A Monograph Series (pp. 134). Rockville, MD: National Institute of Mental Health.
55. Monahan, J. (1997). Actuarial support for the clinical assessment of violence risk. International Review of Psychiatry, 9(2-3), 167-169.
56. Monahan, J. (2004, Nov. 1). Mandated Community Treatment for People with Mental Disorder. Paper presented at the Yale School of Medicine, Department of Psychiatry, Division of Mental Health Services and Treatment Outcomes Research, West Haven, CT.
57. Monahan, J., Steadman, H. J., Appelbaum, P. S., Grisso, T., Mulvey, E. P., Roth, L. H., ... Silver, E. (2006). The classification of violence risk. Behavioral Sciences & the Law, 24(6), 721-730.
58. Monahan, J., Steadman, H. J., Appelbaum, P. S., Robbins, P. C., Mulvey, E. P., Silver, E., Grisso, T., ... (2000). Developing a clinically useful actuarial tool for assessing violence risk. British Journal of Psychiatry, 176, 312-319.
59. Monahan, J., Steadman, H. J., Robbins, P. C., Appelbaum, P., Banks, S., Grisso, T., Silver, E. (2005). An actuarial model of violence risk assessment for persons with mental disorders. Psychiatric Services, 56(7), 810-815.
60. Mossman, D. (1994). Assessing predictions of violence: Being accurate about accuracy. Journal of Consulting and Clinical Psychology, 62(4), 783-792.
61. Neyman, J. (1950). First course in probability and statistics. New York: Henry Holt.
62. Neyman, J., & Pearson, E. S. (1928). On the use and interpretation of certain test criteria for purposes of statistical inference: Part I. Biometrika, 20A(1/2), 175-240.
63. Neyman, J., & Pearson, E. S. (1933a). On the problem of the most efficient tests of statistical hypotheses. Philosophical Transactions of the Royal Society of London. Series A, Containing Papers of a Mathematical or Physical Character, 231, 289-337.
64. Neyman, J., & Pearson, E. S. (1933b). The testing of statistical hypotheses in relation to probabilities a priori. Mathematical Proceedings of the Cambridge Philosophical Society, 29, 491-510.
65. Ogloff, J. R. P., & Daffern, M. (2006). The dynamic appraisal of situational aggression: an instrument to assess risk for imminent aggression in psychiatric inpatients. Behavioral Sciences & the Law, 24(6), 799-813.
66. Otto, R. K. (1992). The prediction of dangerous behavior: A review and analysis of 'second generation' research. Forensic Reports, 5(1), 103-133.
67. Otto, R. K. (2000). Assessing and managing violence risk in outpatient settings. Journal of Clinical Psychology, 56(10), 1239-1262.
68. Pencina, M. J., D' Agostino, R. B., D' Agostino, R. B., & Vasan, R. S. (2008). Evaluating the added predictive ability of a new marker: From area under the ROC curve to reclassification and beyond. Statistics in Medicine, 27(2), 157-172.
69. Quinsey, V. L., Harris, G. T., Rice, M. E., & Cormier, C. A. (2006). Violent offenders: Appraising and managing risk (2nd edn). Washington, D.C.: American Psychological Association.
70. Ramsey, F. P. (1931). The foundations of mathematics and other logical essays. New York: Harcourt, Brace and Company.
71. Rice, M. E., & Harris, G. T. (1995). Violent recidivism: Assessing predictive validity. Journal of Consulting and Clinical Psychology, 63(5), 737-748.
72. Rofman, E. S., Askinazi, C., & Fant, E. (1980). The prediction of dangerous behavior in emergency civil committment. The American Journal of Psychiatry, 137(9), 1061-1064.
73. Schwalbe, C. (2004). Re-visioning risk assessment for human service decision making. Children and Youth Services Review, 26(6), 561-576.
74. Simon, H. A. (1976). From substantive to procedural rationality. In Latsis S. J. (Ed.), Method and appraisal in economics (pp. 129-148). New York: Cambridge University Press.
75. Singh, J. P. (2013). Predictive validity performance indicators in violence risk assessment: a methodological primer. Behavioral Sciences & the Law, 31, 8-22.
76. Singh, J. P., Grann, M., & Fazel, S. (2011). A comparative study of violence risk assessment tools: A systematic review and metaregression analysis of 68 studies involving 25, 980 participants. Clinical Psychology Review, 31(3), 499-513.
77. Steadman, H. J., Silver, E., Monahan, J., Appelbaum, P. S., Clark Robbins, P., Mulvey, E. P., ... Banks, S. (2000). A classification tree approach to the development of actuarial violence risk assessment tools. Law and Human Behavior, 24(1), 83-100.
78. Steyerberg, E. W., Vickers, A. J., Cook, N. R., Gerds, T., Gonen, M., Obuchowski, N., ... Kattan, M. W. (2010). Assessing the performance of prediction models: A framework for traditional and novel measures. Epidemiology, 21(1), 128-138.
79. Swanson, J. W. (2008). Preventing the unpredicted: Managing violence risk in mental health care. Psychiatric Services, 59(2), 191-193.
80. Tabachnick, B. G., & Fidell, L. S. (2001). Using multivariate statistics (Vol. 4th). Boston, MA: Allyn and Bacon.
81. Taktak, A. F. G., Eleuteri, A., Lake, S. P., & Fisher, A. C. (2008). A web-based tool for the assessment of discrimination and calibration properties of prognostic models. [Adobe Acrobat Document]. Computers in Biology and Medicine, 38, 785-791.
82. Toulmin, S. E. (1958). The uses of argument. Cambridge: Cambridge University Press.
83. Vincent, G. M., Chapman, J., & Cook, N. E. (2011). Risk-needs assessment in juvenile justice: Predictive validity of the SAVRY, racial differences, and the contribution of needs factors. Criminal Justice and Behavior, 38(1), 42-62.
84. Vitacco, M. J., Gonsalves, V., Tomony, J., Smith, B. E. R., & Lishner, D. A. (2012). Can standardized measures of risk predict inpatient violence? Criminal Justice and Behavior, 39(5), 589-606.
85. de Vogel, V., de Vries Robbe, M., de Ruiter, C., & Bouman, Y. H. (2011). Assessing protective factors in forensic psychiatric practice: Introducing the SAPROF. International Journal of Forensic Mental Health, 10(3), 171-177.
86. Webster, C. D., Douglas, K. S., Eaves, D., & Hart, S. (1997). HCR-20: Assessing risk for violence, Version 2. Burnaby, BC: Simon Fraser University University and Forensic Psychiatric Services Commission of British Columbia.
87. Webster, C. D., Muller-Isberner, R., & Fransson, G. (2002). Violence risk assessment: Using structured clinical guides professionally. International Journal of Forensic Mental Health, 1(2), 185-193.
88. Webster, C. D., Nicholls, T. L., Martin, M.-L., Desmarais, S. L., & Brink, J. (2006). Short-Term Assessment of Risk and Treatability (START): the case for a new structured professional judgment scheme. Behavioral Sciences & the Law, 24(6), 747-766.
89. Wilson, P. W. F., D'Agostino, R. B., Levy, D., Belanger, A. M., Silbershatz, H., & Kannel, W. B. (1998). Prediction of coronary heart disease Using risk factor categories. Circulation, 97(18), 1837-1847.
90. Wong, S. C. P., & Gordon, A. (2006). The validity and reliability of the Violence Risk Scale: A treatment-friendly violence risk assessment tool. Psychology, Public Policy, and Law, 12(3), 279-309.
91. Zou, K. H., O'Malley, A. J., & Mauri, L. (2007). Receiver-operating characteristic analysis for evaluating diagnostic tests and predictive models. Circulation, 115(5), 654-657.