Implementing routine outcome monitoring in clinical practice: Benefits, challenges, and solutions

Psychotherapy Research

Volumn 25, Issue 1, 2015, Pages 6-19

  Boswell, James F ^a   Kraus, David R ^b   Miller, Scott D ^c   Lambert, Michael J ^d  

^a STATE UNIVERSITY OF NEW YORK   (United States)

^b Outcome Referrals   (United States)

^c ISTC   (United States)

^d BRIGHAM YOUNG UNIVERSITY   (United States)

Author keywords

dissemination and implementation; feedback; routine outcome monitoring; science practice integration

Indexed keywords

HUMAN; OUTCOME ASSESSMENT; PROCEDURES; PSYCHOTHERAPY; STANDARDS;

HUMANS; OUTCOME ASSESSMENT (HEALTH CARE); PSYCHOTHERAPY;

EID: 84914664715     PISSN: 10503307     EISSN: 14684381     Source Type: Journal    
DOI: 10.1080/10503307.2013.817696     Document Type: Article

References (89)

1. Aarons, G. A. (2005). Measuring provider attitudes toward evidence-based practice: Consideration of organizational context and individual differences. Child and Adolescent Psychiatric Clinics of North America, 14, 255–271. http://dx.doi.org/10.1016/j.chc.2004.04.008
2. Adelman, R. (2005). Reducing adolescent clients' anger in a residential substance abuse treatment facility. Journal on Quality and Patient Safety, 31, 325–327.
3. American Psychological Association. (2009). Criteria for the Evaluation of Quality Improvement Programs and the Use of Quality Improvement Data. American Psychologist, 64, 551–557. doi:10.1037/a0016744
4. Anker, M. G., Duncan, B. L., & Sparks, J.A. (2009). Using client feedback to improve couple therapy outcomes: a randomized clinical trial in a naturalistic setting. Journal of Consulting & Clinical Psychology, 77, 693–704. doi:10.1037/a0016062
5. Barkham, M., Margison, F., Leach, C., Lucock, M., Mellor-Clark, J., Evans, C., McGrath, G. (2001). Service profiling and outcomes benchmarking using the CORE-OM: Towards practice-based evidence in the psychological therapies. Journal of Consulting and Clinical Psychology, 69, 184–196. doi:10.1037/0022-006X.69.2.184
6. Barkham, M., Mellor-Clark, J., Connell, J., & Cahill, J. (2006). A core approach to practice-based evidence: A brief history of the origins and applications of the CORE-OM and CORE System. Counselling & Psychotherapy Research, 6, 3–15. doi:10.1080/14733140600581218
7. Bickman, L. (2000). Summing up program theory. New Directions for Evaluation, 103–112. doi:10.1002/ev.1186
8. Bohanske, R. T., & Franczak, M. (2010). Transforming public behavioral health care: A case example of consumer-directed services, recovery, and the common factors. In B.L. Duncan, S.D. Miller, B.E. Wampold, & M.A. Hubble (Eds.), The heart and soul of change: Doing what works in therapy (2nd ed., pp. 299–322). Washington DC: American Psychological Association.
9. Boswell, J. F., & Castonguay, L. G. (2007). Psychotherapy training: Suggestions for core ingredients and future research. Psychotherapy: Theory, Research, Practice, Training, 44, 378–383. doi:10.1037/0033-3204.44.4.378
10. Brownson, R. C., Kreuter, M. W., Arrington, B. A., & True, W. R. (2006). Translating scientific discoveries into public health action: How can schools of public health move us forward? Public Health Reports, 121, 97–103.
11. Castonguay, L. G., Barkham, M., Lutz, W., & McAleavey, A. A. (2013). Practice-oriented research: Approaches and applications. In M.J. Lambert (Ed.), Bergin and Garfield's handbook of therapeutic change (6th ed.). New York: Wiley & Sons.
12. Castonguay, L. G., Boswell, J. F., Constantino, M. J., Goldfried, M.R., & Hill, C.E. (2010). Training implications of harmful effects of psychological treatments. American Psychologist, 65, 34–49. doi:10.1037/a0017330
13. Castonguay, L. G., Boswell, J. F., Zack, S. E., Baker, S., Boutselis, M. A., Chiswick, N. R. Grosse Holtforth, M. (2010). Helpful and hindering events in psychotherapy: A practice research network study. Psychotherapy: Theory, Research, Practice, Training, 47, 327–344. doi:10.1037/a0021164
14. Castonguay, L. G., Pincus, A. L., & McAleavey, A. A. (in press). Practice-research networks in training clinics. Psychotherapy Research.
15. Chin, R., & Benne, K. D. (1969). General strategies for effecting changes in human systems. In W.G. Bennis, K.D. Benne, & R. Chin (Eds.), The planning of change (pp. 32–59). New York: Holt, Rinehart & Winston.
16. Cook, I. A., Balasubramani, G. K., Eng, H., Friedman, E., Young, E. A., Martin, J., Wisniewski, S. R. (2007). Electronic source materials in clinical research: Acceptability and validity of symptom self-rating in major depressive disorder. Journal of Psychiatry Research, 41, 737–743. doi:10.1016/j.jpsychires.2006.07.015
17. De Jong, K., van Sluis, P., Nugter, M. A., Heiser, W. J., & Spinhoven, P. (2012). Understanding the differential impact of outcome monitoring: Therapist variables that moderate feedback effects in a randomized clinical trial. Psychotherapy Research, 22, 464–474. doi:10.1080/10503307.2012.673023
18. Diamond, G. A., & Kaul, S. (2009). Evidence-based financial incentives for healthcare reform: Putting it together. Circulation: Cardiovascular quality and outcomes, 2, 134–140. doi:10.1161/CIRCOUTCOMES.108.825695
19. Drapeau, M. (2012). The value of tracking in psychotherapy. Integrating Science & Practice, 2, 2–6.
20. Eby, L. T., Burk, H., & Maher, C. P. (2010). How serious of a problem is staff turnover in substance abuse treatment? A longitudinal study of actual turnover. Journal of Substance Abuse Treatment, 39, 264–271. doi:10.1016/j.jsat.2010.06.009
21. Ellsworth, J. R., Lambert, M. J., & Johnson, J. (2006). A comparison of the Outcome Questionnaire-45 and Outcome Questionnaire-30 in classification and prediction of treatment outcome. Clinical Psychology & Psychotherapy, 13, 380–391. doi:10.1002/cpp.503
22. Fernández-Álvarez, H., Gómez, B., & García, F. (in press). Bridging the gap between research and practice in a clinical and training network. Psychotherapy Research
23. Finch, R. A., & Phillips, K. (2005). Center for Prevention and Health Services. An employer's guide to behavioral health services: A roadmap and recommendations for evaluating, designing, and implementing behavioral health services. Washington DC: National Business Group on Health.
24. Fixsen, D. L., Naoom, S. F., Blase, K. A., Friedman, R. M., & Wallace, F. (2005). Implementation research: A synthesis of the literature. Tampa, FL: University of South Florida, Louis de la Parte Florida Mental Health Institute, The National Implementation Research Network (FMHI Publication #231).
25. Garb, H. N. (2005). Clinical judgment and decision making. Annual Review of Clinical Psychology, 1, 67–89. doi:10.1146/annurev.clinpsy.1.102803.143810
26. Gilbody, S., House, A., & Sheldon, T. (2002). Psychiatrists in the United Kingdom do not use outcomes measures. British Journal of Psychiatry, 180, 101–103. doi:10.1192/bjp.180.2.101
27. Greenhalgh, T., Robert, G., Macfarlane, F., Bate, P., & Kyriakidou, O. (2004). Diffusion of innovations in service organizations: Systematic review and recommendations. Milbank Quarterly, 82, 581–629. doi:10.1111/j.0887-378X.2004.00325.x
28. Hannan, C., Lambert, M. J., Harmon, C., Nielsen, S. L., Smart, D. W., Shimokawa, K., & Sutton, S. W. (2005). A lab test and algorithms for identifying clients at risk for treatment failure. Journal of Clinical Psychology: In Session, 61, 155–163. doi:10.1002/jclp.20108
29. Hansen, N. B., Lambert, M. J., & Forman, E. V. (2002). The psychotherapy dose-response effect and its implications for treatment delivery services. Clinical Psychology: Science and Practice, 9, 329–343. doi:10.1093/clipsy.9.3.329
30. Harford, T. (2011). Adapt: Why success always starts with failure. London: Macmillan.
31. Harmon, S. C., Lambert, M. J., Smart, D. W., Hawkins, E. J., Nielsen, S. L., Slade, K., & Lutz, W. (2007). Enhancing outcome for potential treatment failures: Therapist/client feedback and clinical support tools. Psychotherapy Research, 17, 379–392. doi:10.1080/10503300600702331
32. Hatfield, D., McCullough, L., Plucinski, A., & Krieger, K. (2010). Do we know when our clients get worse? An investigation of therapists' ability to detect negative client change. Clinical Psychology & Psychotherapy, 17, 25–32.
33. Hatfield, D. R., & Ogles, B. M. (2004). The current climate of outcome measures use in clinical practice. Professional Psychology: Research and Practice, 35, 485–491. doi:10.1037/0735-7028.35.5.485
34. Hatfield, D. R., & Ogles, B. M. (2007). Why some clinicians use outcome measures and others do not. Administrative Policy Mental Health and Mental Health Services Research, 34, 283–291. doi:10.1007/s10488-006-0110-y
35. Hawkins, E. J., Lambert, M. J., Vermeersch, D. A., Slade, K., & Tuttle, K. (2004). The effects of providing patient progress information to therapists and patients. Psychotherapy Research, 14, 308–327. doi:10.1093/ptr/kph027
36. Holmqvist, R., Philips, B., & Barkham, M. (in press). Developing and delivering practice-based evidence: Observations, tensions, and challenges. Psychotherapy Research.
37. Howard, K. I., Moras, K., Brill, P. L., Martinovich, Z., & Lutz, W. (1996). Evaluation of psychotherapy. American Psychologist, 51, 1059–1064. doi:10.1037/0003-066X.51.10.1059
38. Kazdin, A. E. (1996). Dropping out of child therapy. Issues for research and implications for practice. Clinical Child Psychology and Psychiatry, 1, 133–156. doi:10.1177/1359104596011012
39. Kazdin, A. E. (2005). Evidence-based assessment for children and adolescents: Issues in measurement development and clinical application. Journal of Clinical Child & Adolescent Psychology, 34, 548–558. doi:10.1207/s15374424jccp3403_10
40. Kiefe, C. I., Allison, J. J., Williams, O. D., Person, S. D., Weaver, M. T., & Weissman, N. W. (2001). Improving quality improvement using achievable benchmarks for physician feedback: a randomized controlled trial. JAMA, 285, 2871–2879. doi:10.1001/jama.285.22.2871
41. Kluger, A. N., & DeNisi, A. (1996). The effects of feedback interventions on performance: A historical review, a meta-analysis, and a preliminary feedback intervention theory. Psychological Bulletin, 119, 254–284. doi:10.1037/0033-2909.119.2.254
42. Koestner, R. (2008). Reaching one's personal goals: A motivational perspective focused on autonomy. Canadian Psychology, 49, 60–67. doi:10.1037/0708-5591.49.1.60
43. Koocher, G. P., & Keith-Spiegel, P. (2008). Ethics in psychology and the mental health professions: Standards and cases. New York: Oxford University Press.
44. Kraus, D. R., Boswell, J. F., Wright, A. G. C., Castonguay, L. G., & Pincus, A. L. (2010). Factor structure of the treatment outcome package for children. Journal of Clinical Psychology, 66, 627–640.
45. Kraus, D. R., Castonguay, L. G., Boswell, J. F., Nordberg, S. S., & Hayes, J. A. (2011). Therapist effectiveness: Implications for accountability and patient care. Psychotherapy Research, 21, 267–276. doi:10.1080/10503307.2011.563249
46. Kraus, D. R., Seligman, D., & Jordan, J. R. (2005). Validation of a behavioral health treatment outcome and assessment tool designed for naturalistic settings: The Treatment Outcome Package. Journal of Clinical Psychology, 61, 285–314. doi:10.1002/jclp.20084
47. Kraus, D. R., Wolf, A., & Castonguay, L. (2006). A kinder philosophy to the management of outcomes. Psychotherapy Bulletin, 41, 23–31.
48. Lambert, M. J. (2010). Prevention of treatment failure: The use of measuring, monitoring, and feedback in clinical practice. Washington DC: American Psychological Association Press.
49. Lambert, M. J. (2012). Helping clinicians to use and learn from research-based systems: The OQ-Analyst. Psychotherapy, 49, 109–114. doi:10.1037/a0027110
50. Lambert, M. J. (2013). The efficacy and effectiveness of psychotherapy. In M. J. Lambert (Ed.) Bergin & Garfield's Handbook of psychotherapy & behavior change (6th ed.). New York: Wiley.
51. Lambert, M. J., Burlingame, G. M., Umphress, V., Hansen, N. B., Vermeersch, D. A., Clouse, G. C., & Yanchar, S. C. (1996). The reliability and validity of the outcome questionnaire. Clinical Psychology & Psychotherapy, 3, 249–258. doi:10.1002/(SICI)1099-0879(199612)3:4<249::AID-CPP106>3.0.CO;2-S
52. Lambert, M. J. & Shimokawa, K. (2011). Collecting client feedback. In J.C. Norcross (Ed.), Psychotherapy relationships that work (2nd ed., pp. 203–223). New York: Oxford University Press.
53. Lambert, M. J., Whipple, J. L., Bishop, M. J., Vermeersch, D. A., Gray, G. V., & Finch, A. E. (2002). Comparison of empirically derived and rationally derived methods for identifying clients at risk for treatment failure. Clinical Psychology and Psychotherapy, 9, 149–164. doi:10.1002/cpp.333
54. Lambert, M. J., Whipple, J. L., Smart, D. W., Vermeersch, D. A., Nielsen, S.L., & Hawkins, E.J. (2001). The effects of providing therapists with feedback on client progress during psychotherapy: Are outcomes enhanced? Psychotherapy Research, 11, 49–68. doi:10.1080/713663852
55. Lambert, M. J., Whipple, J. L., Vermeersch, D. A., Smart, D. W., Hawkins, E. J., Nielsen, S. L., & Goates, M. K. (2002). Enhancing psychotherapy outcomes via providing feedback on client progress: A replication. Clinical Psychology and Psychotherapy, 9, 91–103. doi:10.1002/cpp.324
56. Latham, G. P., & Locke, E. A. (2006). Enhancing the benefits and overcoming the pitfalls of goal setting. Organizational Dynamics, 35, 332–340. doi:10.1016/j.orgdyn.2006.08.008
57. Locke, E. A., & Latham, G. P. (2006). New directions in goal-setting theory. Current Directions in Psychological Science, 15, 265–268. doi:10.1111/j.1467-8721.2006.00449.x
58. Marty, D., Rapp, C. A., McHugo, G. J., & Whitley, R. (2007). Factors influencing consumer outcome monitoring in implementation of evidence-based practices: Results from the National EBP Implementation Project. Administration and Policy in Mental Health, 35, 204–211. doi:10.1007/s10488-007-0157-4
59. McAleavey, A. A., Lockard, A. J., Castonguay, L. G., Hayes, J. A., & Locke, B.D. (in press). Research by, with, and for clinicians: Lessons from a large-scale PRN for University and College Counseling Centers. Psychotherapy Research.
60. McAleavey, A. A., Nordberg, S. S., Kraus, D. R., & Castonguay, L.G. (2012). Errors in treatment outcome monitoring: Implications of multidimensional and general measurements for real-world psychotherapy. Canadian Psychology, 53, 105–114. doi:10.1037/a0027833
61. McHugh, R. K., & Barlow, D. H. (2012). Dissemination and implementation of evidence-based psychological interventions. New York: Oxford University Press.
62. Merten, T., & Ruch, W. (1996). A comparison of computerized and conventional administration of the German versions of the Eysenck Personality Questionnaire and the Carroll Rating Scale for Depression. Personality and Individual Differences, 20, 281–291. http://dx.doi.org/10.1016/0191-8869(95)00185-9
63. Miller, S. D., Duncan, B. L., & Hubble, M. A. (2004). Beyond integration: The triumph of outcome over process in clinical practice. Psychotherapy in Australia, 10, 32–43.
64. Miller, S. D., Duncan, B. L., Sorrell, R., & Brown, G. S. (2005). The Partners for Change Outcome Management System. Journal of Clinical Psychology: In-Session, 61, 199–208. doi:10.1002/jclp.20111
65. Miller, S. D., Hubble, M. A., Chow, D. L., & Seidel, J. A. (2013). The outcome of psychotherapy: Yesterday, today, and tomorrow. Psychotherapy, 50, 88–97. doi:10.1037/a0031097
66. Nordberg, S. S., Boswell, J. F., Kraus, D. R., Castonguay, L. G., Hayes, J., & Wampold, B. (2010, June). Therapist effectiveness treating depression with and without co-morbid substance abuse. Paper presented at the Society for Psychotherapy Research Conference, Asilomar, CA.
67. Ogles, B. M., France, C. R., Lunnen, K. M., Bell, M. T., & Goldfarb, M. (1998). Computerized depression screening and awareness. Community Mental Health Journal, 34, 27–38. doi:10.1023/A:1018760128239
68. Okiishi, J., Lambert, M. J., Eggett, D., Nielsen, S. L., Dayton, D.D., & Vermeersch, D.A. (2006). An analysis of therapist treatment effects: Toward providing feedback to individual therapists on their clients’ psychotherapy outcome; Journal of Clinical Psychology, 62, 1157–1172. doi:10.1002/jclp.20272
69. Peterson, L., Johannsson, V., & Carlsson, S. (1996). Computerized testing in a hospital setting: Psychometric and psychological effects. Computers in Human Behavior, 12, 339–350. http://dx.doi.org/10.1016/0747-5632(96)00012-X
70. Pope, K. S., Tabachnick, B. G., & Keith-Spiegel, P. (1987). Ethics of practice: The beliefs and behaviors of psychologists as therapists. American Psychologist, 42, 993–1006. doi:10.1037/0003-066X.42.11.993
71. Riemer, M., Rosof-Williams, J., & Bickman, L. (2005). Theories related to changing clinician practice. Child and Adolescent Psychiatric Clinics of North America, 14, 241–254. doi:10.1016/j.chc.2004.05.002
72. Rogers, E. (2003). The diffusion of innovation (5th ed.). New York: The Free Press.
73. Rosenthal, M. B. (2008). Beyond pay for performance—emerging models of provider-payment reform. New England Journal of Medicine, 359, 1197–1200. doi:10.1056/NEJMp0804658
74. Shimokawa, K., Lambert, M. J., & Smart, D. (2010). Enhancing treatment outcome of patients at risk of treatment failure: Meta-analytic and mega-analytic review of a psychotherapy quality assurance system. Journal of Consulting & Clinical Psychology, 78, 298–311. doi:10.1037/a0019247
75. Slade, K., Lambert, M. J., Harmon, S. C., Smart, D. W., & Bailey, R. (2008). Improving psychotherapy outcome: The use of immediate electronic feedback and revised clinical support tools. Clinical Psychology & Psychotherapy, 15, 287–303. doi:10.1002/cpp.594
76. Slade, M., Thornicroft, G., & Glover, G.S.O. (1999). The feasibility of routine outcome measures in mental health. Social Psychiatry and Epidemiology, 34, 243–249. doi:10.1007/s001270050139
77. Strauss, B., Lutz, W., Steffanowski, A., Wittmann, W., Böhnke, J.R., Rubel, J., Kirchmann, H. (in press). Benefits and challenges in practice-oriented psychotherapy research in Germany: The TK and the QS-PSY-BAY projects of quality assurance in outpatient psychotherapy. Psychotherapy Research.
78. Stricker, G., & Trierweiler, S. J. (1995). The local clinical scientist: A bridge between science and practice. American Psychologist, 50, 995–1002. doi:10.1037/0003-066X.50.12.995
79. Strupp, H. H., Horowitz, L. M., & Lambert, M. J. (1997). Measuring patient changes in mood, anxiety, and personality disorders: Toward a core battery. Washington DC: American Psychological Association Press.
80. Trauer, T., Gill, L., Pedwell, G., & Slattery, P. (2006). Routine outcome measurement in public mental health—what do clinicians think? Australian Health Review, 30, 144–147. doi:10.1071/AH060144
81. Tremblay, M. C., Hevner, A. R., & Berndt, D. J. (2012). Design of an information volatility measure for health care decision making. Decision Support Systems, 52, 331–341. doi:10.1016/j.dss.2011.08.009
82. Walfish, S., McAlister, B., O'Donnell, P., & Lambert, M. J. (2012). An investigation of self-assessment bias in mental health providers. Psychological Reports, 110, 639–644. doi:10.2466/02.07.17
83. Warren, J. S., Nelson, P. L., Mondragon, S. A., Baldwin, S. A., & Burlingame, G.M. (2010). Youth psychotherapy change trajectories & outcome in usual care: Community mental health versus managed care. Journal of Clinical & Consulting Psychology, 78, 144–155. doi:10.1037/a0018544
84. Weissman, N. W., Allison, J. J., Kiefe, C. I., Farmer, R., Weaver, M.T., Williams, O.D., Baker, S. (1999). Achievable benchmarks of care: The ABCs of benchmarking. Journal of Evaluation in Clinical Practice, 5, 269–281. doi:10.1046/j.1365-2753.1999.00203.x
85. West, J. C., Mościcki, E. K., Duffy, F. F., Wilk, J.E., Countis, L., Narrow, W.E., & Regier, D.A. (in press). APIRE Practice Research Network: Accomplishments, challenges, and lessons learned. Psychotherapy Research.
86. Whipple, J. L., Lambert, M. J., Vermeersch, D. A., Smart, D. W., Nielsen, S.L., & Hawkins, E.J. (2003). Improving the effects of psychotherapy: The use of early identification of treatment failure and problem solving strategies in routine practice. Journal of Counseling Psychology, 58, 59–68. doi:10.1037/0022-0167.50.1.59
87. Wierzbicki, M. & Pekarik, G. (1993). A meta-analysis of psychotherapy dropout. Professional Psychology: Research and Practice, 24, 190–195. doi:10.1037/0735-7028.24.2.190
88. Youn, S.J., Kraus, D. R., & Castonguay, L.G. (2012). The Treatment Outcome Package: Facilitating practice and clinically relevant research. Psychotherapy, 49, 115–122. doi:10.1037/a0027932
89. Zimmerman, M., & McGlinchey, J. B. (2008). Why don't psychiatrists use scales to measure outcome when treating depressed patients? Journal of Clinical Psychiatry, 69, 1916–1919.