Testing a theory of strategic implementation leadership, implementation climate, and clinicians' use of evidence-based practice: A 5-year panel analysis

Implementation Science

Volumn 15, Issue 1, 2020, Pages

  Williams, Nathaniel J ^a   Wolk, Courtney Benjamin ^b   Becker Haimes, Emily M ^b,e   Beidas, Rinad S ^b,c,d  

^a BOISE STATE UNIVERSITY   (United States)

^b UNIVERSITY OF PENNSYLVANIA   (United States)

^c UNIVERSITY OF PENNSYLVANIA   (United States)

^d UNIVERSITY OF PENNSYLVANIA   (United States)

^e PENNSYLVANIA HOSPITAL   (United States)

Author keywords

Behavioral health; Evidence based practice; Implementation climate; Implementation leadership; Mechanism; Mediation

Indexed keywords

ADULT; CHILD; CHILD HEALTH CARE; EVIDENCE BASED PRACTICE; FEMALE; HUMAN; LEADERSHIP; MALE; MENTAL DISEASE; MENTAL HEALTH SERVICE; MIDDLE AGED; ORGANIZATION; ORGANIZATION AND MANAGEMENT; ORGANIZATIONAL CULTURE; PSYCHOTHERAPY; STATISTICAL MODEL;

ADULT; CHILD; CHILD HEALTH SERVICES; COMMUNITY MENTAL HEALTH SERVICES; EVIDENCE-BASED PRACTICE; FEMALE; HUMANS; IMPLEMENTATION SCIENCE; LEADERSHIP; MALE; MENTAL DISORDERS; MIDDLE AGED; MODELS, ECONOMETRIC; ORGANIZATIONAL CULTURE; ORGANIZATIONAL INNOVATION; PSYCHOTHERAPY;

EID: 85079081786     PISSN: None     EISSN: 17485908     Source Type: Journal    
DOI: 10.1186/s13012-020-0970-7     Document Type: Article

References (103)

1. Insel TR. Translating scientific opportunity into public health impact: a strategic plan for research on mental illness. Arch Gen Psychiatry. 2009;66(2):128-33.
2. Collins PY, Patel V, Joestl SS, March D, Insel TR, Daar AS, et al. Grand challenges in global mental health. Nature. 2011;475(7354):27-30.
3. Garland AF, Haine-Schlagel R, Brookman-Frazee L, Baker-Ericzen M, Trask E, Fawley-King K. Improving community-based mental health care for children: translating knowledge into action. Admin Pol Ment Health. 2013;40(1):6-22.
4. Weisz JR. Agenda for child and adolescent psychotherapy research: on the need to put science into practice. Arch Gen Psychiatry. 2000;57(9):837-8.
5. Wensing M, Grol R. Knowledge translation in health: how implementation science could contribute more. BMC Med. 2019;17 https://doi.org/10.1186/s12916-019-1322-9.
6. Williams NJ. Multilevel mechanisms of implementation strategies in mental health: integrating theory, research, and practice. Admin Pol Ment Health. 2016;43(5):783-98.
7. Lewis CC, Klasnja P, Powell BJ, Tuzzio L, Jones S, Walsh-Bailey C, et al. From classification to causality: advancing understanding of mechanisms of change in implementation science. Front Public Health. 2018;6 https://doi.org/10.3389/fpubh.2018.00136.
8. Williams NJ, Beidas RS. Annual research review: the state of implementation science in child psychology and psychiatry: a review and suggestions to advance the field. J Child Psychol Psychiatry. 2019;60(4):430-50.
9. Kazdin AE. Mediators and mechanisms of change in psychotherapy research. Annu Rev Clin Psychol. 2007;3:1-27.
10. Kraemer HC, Kazdin AE, Offord DR, Kessler RC, Jensen PS, Kupfer DJ. Coming to terms with the terms of risk. Arch Gen Psychiatry. 1997;54(4):337-43.
11. Tabak RG, Khoong EC, Chambers DA, Brownson RC. Bridging research and practice: models for dissemination and implementation research. Am J Prev Med. 2012;43(3):337-50.
12. Breimaier HE, Heckemann B, Halfens RJ, Lohrmann C. The consolidated framework for implementation research (CFIR): a useful theoretical framework for guiding and evaluating a guideline implementation process in a hospital-based nursing practice. BMC Nurs. 2015;14 https://doi.org/10.1186/s12912-015-0088-4.
13. Michie S, Johnston M, Abraham C, Lawton R, Parker D, Walker A. Making psychological theory useful for implementing evidence based practice: a consensus approach. BMJ Qual Saf. 2005;14(1):26-33.
14. Rabin BA, Lewis CC, Norton WE, Neta G, Chambers D, Tobin JN, et al. Measurement resources for dissemination and implementation research in health. Implement Sci. 2015;11 https://doi.org/10.1186/s13012-016-0401-y.
15. Chaudoir SR, Dugan AG, Barr CH. Measuring factors affecting implementation of health innovations: a systematic review of structural, organizational, provider, patient, and innovation level measures. Implement Sci. 2013;8 https://doi.org/10.1186/1748-5908-8-22.
16. Beidas RS, Aarons GA, Barg F, Evans AC, Hadley T, Hoagwood KE, et al. Policy to implementation: evidence-based practice in community mental health-study protocol. Implement Sci. 2013;8 https://doi.org/10.1186/1748-5908-8-38.
17. Damschroder LJ, Aron DC, Keith RE, Kirsh SR, Alexander JA, Lowery JC. Fostering implementation of health services research findings into practice: a consolidated framework for advancing implementation science. Implement Sci. 2009;4 https://doi.org/10.1186/1748-5908-4-50.
18. Aarons GA, Hurlburt M, Horwitz SM. Advancing a conceptual model of evidence-based practice implementation in public service sectors. Admin Pol Ment Health. 2011;38(1):4-23.
19. Stetler CB, Ritchie JA, Rycroft-Malone J, Charns MP. Leadership for evidence-based practice: strategic and functional behaviors for institutionalizing EBP. Worldviews Evid-Based Nurs. 2014;11(4):219-26.
20. Flottorp SA, Oxman AD, Krause J, Musila NR, Wensing M, Godycki-Cwirko M, et al. A checklist for identifying determinants of practice: a systematic review and synthesis of frameworks and taxonomies of factors that prevent or enable improvements in healthcare professional practice. Implement Sci. 2013;8 https://doi.org/10.1186/1748-5908-8-35.
21. Birken SA, Lee S-YD, Weiner BJ. Uncovering middle managers' role in healthcare innovation implementation. Implement Sci. 2012;7 https://doi.org/10.1186/1748-5908-7-28.
22. Guerrero EG, Frimpong J, Kong Y, Fenwick K, Aarons GA. Advancing theory on the multilevel role of leadership in the implementation of evidence-based health care practices. Health Care Manag Rev. 2018; https://doi.org/10.1097/HMR.0000000000000213.
23. Aarons GA, Wells RS, Zagursky K, Fettes DL, Palinkas LA. Implementing evidence-based practice in community mental health agencies: a multiple stakeholder analysis. Am J Public Health. 2009;99(11):2087-95.
24. Aarons GA, Ehrhart MG, Farahnak LR, Sklar M. Aligning leadership across systems and organizations to develop a strategic climate for evidence-based practice implementation. Annu Rev Public Health. 2014;35:255-74.
25. O'Reilly CA, Caldwell DF, Chatman JA, Lapiz M, Self W. How leadership matters: the effects of leaders' alignment on strategy implementation. Leadersh Q. 2010;21(1):104-13.
26. Birken S, Clary A, Tabriz AA, Turner K, Meza R, Zizzi A, et al. Middle managers' role in implementing evidence-based practices in healthcare: a systematic review. Implement Sci. 2018;13 https://doi.org/10.1186/s13012-018-0843-5.
27. Aarons GA, Ehrhart MG, Farahnak LR. The implementation leadership scale (ILS): development of a brief measure of unit level implementation leadership. Implement Sci. 2014;9 https://doi.org/10.1186/1748-5908-9-45.
28. Kozlowski SW, Doherty ML. Integration of climate and leadership: examination of a neglected issue. J Appl Psychol. 1989;74(4):546-53.
29. Birken SA, Lee S-YD, Weiner BJ, Chin MH, Schaefer CT. Improving the effectiveness of health care innovation implementation: middle managers as change agents. Med Care Res Rev. 2013;70(1):29-45.
30. Aarons GA, Ehrhart MG, Moullin JC, Torres EM, Green AE. Testing the leadership and organizational change for implementation (LOCI) intervention in substance abuse treatment: a cluster randomized trial study protocol. Implement Sci. 2017;12 https://doi.org/10.1186/s13012-017-0562-3.
31. Ehrhart MG, Aarons GA, Farahnak LR. Assessing the organizational context for EBP implementation: the development and validity testing of the implementation climate scale (ICS). Implement Sci. 2014;9 https://doi.org/10.1186/s13012-014-0157-1.
32. Weiner BJ, Belden CM, Bergmire DM, Johnston M. The meaning and measurement of implementation climate. Implement Sci. 2011;6 https://doi.org/10.1186/1748-5908-6-78.
33. Klein KJ, Sorra JS. The challenge of innovation implementation. Acad Manag Rev. 1996;21(4):1055-80.
34. Jacobs SR, Weiner BJ, Bunger AC. Context matters: measuring implementation climate among individuals and groups. Implement Sci. 2014;9 https://doi.org/10.1186/1748-5908-9-46.
35. Schneider B, Ehrhart MG, Macey WH. Organizational climate and culture. Annu Rev Psychol. 2013;64:361-88.
36. Hong Y, Liao H, Hu J, Jiang K. Missing link in the service profit chain: a meta-analytic review of the antecedents, consequences, and moderators of service climate. J Appl Psychol. 2013;98(2):237-67.
37. Glisson C. The organizational context of children's mental health services. Clin Child Fam Psychol Rev. 2002;5(4):233-53.
38. Kuenzi M, Schminke M. Assembling fragments into a lens: a review, critique, and proposed research agenda for the organizational work climate literature. J Manage. 2009;35(3):634-717.
39. Powell BJ, Mandell DS, Hadley TR, Rubin RM, Evans AC, Hurford MO, et al. Are general and strategic measures of organizational context and leadership associated with knowledge and attitudes toward evidence-based practices in public behavioral health settings? A cross-sectional observational study. Implement Sci. 2017;12 https://doi.org/10.1186/s13012-017-0593-9.
40. Shuman CJ, Liu X, Aebersold ML, Tschannen D, Banaszak-Holl J, Titler MG. Associations among unit leadership and unit climates for implementation in acute care: a cross-sectional study. Implement Sci. 2018;13 https://doi.org/10.1186/s13012-018-0753-6.
41. Williams NJ, Ehrhart MG, Aarons GA, Marcus SC, Beidas RS. Linking molar organizational climate and strategic implementation climate to clinicians' use of evidence-based psychotherapy techniques: cross-sectional and lagged analyses from a 2-year observational study. Implement Sci. 2018;13 https://doi.org/10.1186/s13012-018-0781-2.
42. Guerrero EG, Fenwick K, Kong Y. Advancing theory development: exploring the leadership-climate relationship as a mechanism of the implementation of cultural competence. Implement Sci. 2017;12 https://doi.org/10.1186/s13012-017-0666-9.
43. Bass BM, Riggio RE. Transformational leadership. Mahwah, NJ: Lawrence Erlbaum Associates, Inc.; 2006.
44. Klein KJ, Conn AB, Sorra JS. Implementing computerized technology: an organizational analysis. J Appl Psychol. 2001;86(5):811-24.
45. Schein EH. Organizational culture and leadership. 4th ed. San Francisco, CA: Wiley; 2010.
46. Lyon AR, Cook CR, Brown EC, Locke J, Davis C, Ehrhart M, et al. Assessing organizational implementation context in the education sector: confirmatory factor analysis of measures of implementation leadership, climate, and citizenship. Implement Sci. 2018;13 https://doi.org/10.1186/s13012-017-0705-6.
47. Torres EM, Ehrhart MG, Beidas RS, Farahnak LR, Finn NK, Aarons GA. Validation of the implementation leadership scale (ILS) with supervisors' self-ratings. Community Ment Health J. 2017;54(1):49-53.
48. Beidas RS, Williams NJ, Becker-Haimes E, Aarons G, Barg F, Evans A, et al. A repeated cross-sectional study of clinicians' use of psychotherapy techniques during 5 years of a system-wide effort to implement evidence-based practices in Philadelphia. Implement Sci. 2019;14 https://doi.org/10.1186/s13012-019-0912-4.
49. Wing C, Simon K, Bello-Gomez RA. Designing difference in difference studies: best practices for public health policy research. Annu Rev Public Health. 2018;39:453-69.
50. Lechner M. The estimation of causal effects by difference-in-difference methods. Foundations Trends® Econometrics. 2011;4(3):165-224.
51. Allison PD. Fixed effects regression models. Thousand Oaks, CA: SAGE Publications; 2009.
52. von Elm E, Altman DG, Egger M, Pocock SJ, Gotzsche PC, Vandenbroucke JP. The strengthening the reporting of observational studies in epidemiology (STROBE) statement: guidelines for reporting observational studies. Lancet. 2007;370(9596):1453-7.
53. Beidas RS, Marcus S, Aarons GA, Hoagwood KE, Schoenwald S, Evans AC, et al. Predictors of community therapists' use of therapy techniques in a large public mental health system. JAMA Pediatr. 2015;169(4):374-82.
54. Beidas RS, Stewart RE, Adams DR, Fernandez T, Lustbader S, Powell BJ, et al. A multi-level examination of stakeholder perspectives of implementation of evidence-based practices in a large urban publicly-funded mental health system. Admin Pol Ment Health. 2016;43(6):893-908.
55. Powell BJ, Beidas RS, Rubin RM, Stewart RE, Wolk CB, Matlin SL, et al. Applying the policy ecology framework to Philadelphia's behavioral health transformation efforts. Admin Pol Ment Health. 2016;43(6):909-26.
56. Stirman SW, Buchhofer R, McLaulin JB, Evans AC, Beck AT. Public-academic partnerships: the Beck initiative: a partnership to implement cognitive therapy in a community behavioral health system. Psychiatr Serv. 2009;60(10):1302-4.
57. Foa EB, Hembree EA, Cahill SP, Rauch SA, Riggs DS, Feeny NC, et al. Randomized trial of prolonged exposure for posttraumatic stress disorder with and without cognitive restructuring: outcome at academic and community clinics. J Consult Clin Psychol. 2005;73(5):953-64.
58. Cohen JA, Deblinger E, Mannarino AP, Steer RA. A multisite, randomized controlled trial for children with sexual abuse-related PTSD symptoms. J Am Acad Child Adolesc Psychiatry. 2004;43(4):393-402.
59. Linehan MM, Comtois KA, Murray AM, Brown MZ, Gallop RJ, Heard HL, et al. Two-year randomized controlled trial and follow-up of dialectical behavior therapy vs therapy by experts for suicidal behaviors and borderline personality disorder. Arch Gen Psychiatry. 2006;63(7):757-66.
60. Thomas R, Abell B, Webb HJ, Avdagic E, Zimmer-Gembeck MJ. Parent-child interaction therapy: a meta-analysis. Pediatrics. 2017;140(3):e20170352.
61. Weersing VR, Weisz JR, Donenberg GR. Development of the therapy procedures checklist: a therapist-report measure of technique use in child and adolescent treatment. J Clin Child Adolesc Psychol. 2002;31(2):168-80.
62. Kolko DJ, Cohen JA, Mannarino AP, Baumann BL, Knudsen K. Community treatment of child sexual abuse: a survey of practitioners in the national child traumatic stress network. Admin Pol Ment Health. 2009;36(1):37-49.
63. Weisz JR, Kuppens S, Eckshtain D, Ugueto AM, Hawley KM, Jensen-Doss A. Performance of evidence-based youth psychotherapies compared with usual clinical care: a multilevel meta-analysis. JAMA Psychiatry. 2013;70(7):750-61.
64. Klein JB, Jacobs RH, Reinecke MA. Cognitive-behavioral therapy for adolescent depression: a meta-analytic investigation of changes in effect-size estimates. J Am Acad Child Adolesc Psychiatry. 2007;46(11):1403-13.
65. Reynolds S, Wilson C, Austin J, Hooper L. Effects of psychotherapy for anxiety in children and adolescents: a meta-analytic review. Clin Psychol Rev. 2012;32(4):251-62.
66. McCart MR, Sheidow AJ. Evidence-based psychosocial treatments for adolescents with disruptive behavior. J Clin Child Adolesc Psychol. 2016;45(5):529-63.
67. Chorpita BF, Daleiden EL, Ebesutani C, Young J, Becker KD, Nakamura BJ, et al. Evidence-based treatments for children and adolescents: an updated review of indicators of efficacy and effectiveness. Clin Psychol (New York). 2011;18(2):154-72.
68. Abbass AA, Rabung S, Leichsenring F, Refseth JS, Midgley N. Psychodynamic psychotherapy for children and adolescents: a meta-analysis of short-term psychodynamic models. J Am Acad Child Adolesc Psychiatry. 2013;52(8):863-75.
69. De Nadai AS, Storch EA. Design considerations related to short-term psychodynamic psychotherapy. J Am Acad Child Adolesc Psychiatry. 2013;52(11):1213-4.
70. Chan D. Functional relations among constructs in the same content domain at different levels of analysis: a typology of composition models. J Appl Psychol. 1998;83(2):234-46.
71. Ehrhart MG, Raver JL. The effects of organizational climate and culture on productive and counterproductive behavior. In: Schneider B, Barbera K, editors. The Oxford handbook of organizational climate and culture. New York: Oxford University Press; 2014. p. 153-76.
72. James LR, Choi CC, Ko CE, McNeil PK, Minton MK, Wright MA, et al. Organizational and psychological climate: a review of theory and research. Eur J Work Organ Psychol. 2008;17(1):5-32.
73. Glisson C, Landsverk J, Schoenwald SK, Kelleher K, Hoagwood KE, Mayberg S, et al. Assessing the organizational social context (OSC) of mental health services: implications for research and practice. Admin Pol Ment Health. 2008;35:98-113.
74. Olin SS, Williams NJ, Pollock M, Armusewicz K, Kutash K, Glisson C, et al. Quality indicators for family support services and their relationship to organizational social context. Admin Pol Ment Health. 2014;41(1):43-54.
75. Glisson C, Green P, Williams NJ. Assessing the organizational social context (OSC) of child welfare systems: implications for research and practice. Child Abuse Negl. 2012;36(9):621-32.
76. Aarons GA, Glisson C, Green PD, Hoagwood KE, Kelleher KJ, Landsverk JA. The organizational social context of mental health services and clinician attitudes toward evidence-based practice: a United States national study. Implement Sci. 2012;7 https://doi.org/10.1186/1748-5908-7-56.
77. Aarons GA. Transformational and transactional leadership: association with attitudes toward evidence-based practice. Psychiatr Serv. 2006;57(8):1162-9.
78. Aarons GA, Sommerfeld DH. Leadership, innovation climate, and attitudes toward evidence-based practice during a statewide implementation. J Am Acad Child Adolesc Psychiatry. 2012;51(4):423-31.
79. Brimhall KC, Fenwick K, Farahnak LR, Hurlburt MS, Roesch SC, Aarons GA. Leadership, organizational climate, and perceived burden of evidence-based practice in mental health services. Admin Pol Ment Health. 2016;43(5):629-39.
80. Aarons GA, Green AE, Trott E, Willging CE, Torres EM, Ehrhart MG, et al. The roles of system and organizational leadership in system-wide evidence-based intervention sustainment: a mixed-method study. Admin Pol Ment Health. 2016;43(6):991-1008.
81. Bass BM, Avolio BJ. Multifactor leadership questionnaire. Mind Garden: Redwood City, CA; 2000.
82. Avolio BJ. Full range leadership development. 2nd ed. Thousand Oaks, CA: SAGE Publications; 2011.
83. Antonakis J, Avolio BJ, Sivasubramaniam N. Context and leadership: an examination of the nine-factor full-range leadership theory using the multifactor leadership questionnaire. Leadersh Q. 2003;14(3):261-95.
84. Aarons GA. Mental health provider attitudes toward adoption of evidence-based practice: The Evidence-Based Practice Attitude Scale (EBPAS). Mental health services research. 2004;6(2):61-74.
85. Klein KJ, Kozlowski SW. Multilevel theory, research, and methods in organizations: foundations, extensions, and new directions. San Francisco, CA: Jossey-Bass; 2000.
86. Brown RD, Hauenstein NM. Interrater agreement reconsidered: an alternative to the rwg indices. Organ Res Methods. 2005;8(2):165-84.
87. LeBreton JM, Senter JL. Answers to 20 questions about interrater reliability and interrater agreement. Organ Res Methods. 2008;11(4):815-52.
88. Croissant Y, Millo G. Panel data econometrics in R: the plm package. J Stat Softw. 2008;27(2):1-43.
89. Cohen J. Statistical power analysis for the behavioral sciences. 2nd ed. Hillsdale, NJ: Lawrence Erlbaum Associates; 1988.
90. Hedeker D, Gibbons RD. Longitudinal data analysis. Hoboken, NJ: Wiley; 2006.
91. MacKinnon DP. Introduction to statistical mediation analysis. New York: Taylor & Francis Group; 2008.
92. Preacher KJ. Advances in mediation analysis: a survey and synthesis of new developments. Annu Rev Psychol. 2015;66:825-52.
93. Preacher KJ, Selig JP. Advantages of Monte Carlo confidence intervals for indirect effects. Commun Methods Meas. 2012;6(2):77-98.
94. Hayes AF, Scharkow M. The relative trustworthiness of inferential tests of the indirect effect in statistical mediation analysis: does method really matter? Psychol Sci. 2013;24(10):1918-27.
95. Hayes AF. Beyond baron and Kenny: statistical mediation analysis in the new millennium. Commun Monogr. 2009;76(4):408-20.
96. MacKinnon DP, Fairchild AJ, Fritz MS. Mediation analysis. Annu Rev Psychol. 2007;58:593-614.
97. Powell BJ, Fernandez ME, Williams NJ, Aarons GA, Beidas RS, Lewis CC, et al. Enhancing the impact of implementation strategies in healthcare: a research agenda. Front Public Health. 2019;7 https://doi.org/10.1097/HCO.0000000000000067.
98. Aarons GA, Ehrhart MG, Farahnak LR, Hurlburt MS. Leadership and organizational change for implementation (LOCI): a randomized mixed method pilot study of a leadership and organization development intervention for evidence-based practice implementation. Implement Sci. 2015;10(1) https://doi.org/10.1186/s13012-014-0192-y.
99. Soo S, Berta W, Baker GR. Role of champions in the implementation of patient safety practice change. Healthc Q. 2009;12:123-8.
100. Proctor E, Ramsey AT, Brown MT, Malone S, Hooley C, McKay V. Training in implementation practice leadership (TRIPLE): evaluation of a novel practice change strategy in behavioral health organizations. Implement Sci. 2019;14 https://doi.org/10.1186/s13012-019-0906-2.
101. Egeland KM, Skar A-MS, Endsjø M, Laukvik EH, Bækkelund H, Babaii A, et al. Testing the leadership and organizational change for implementation (LOCI) intervention in Norwegian mental health clinics: a stepped-wedge cluster randomized design study protocol. Implement Sci. 2019;14 https://doi.org/10.1186/s13012-019-0873-7.
102. Brookman-Frazee L, Stahmer AC. Effectiveness of a multi-level implementation strategy for ASD interventions: study protocol for two linked cluster randomized trials. Implement Sci. 2018;13 https://doi.org/10.1186/s13012-018-0757-2.
103. Hogue A, Dauber S, Lichvar E, Bobek M, Henderson CE. Validity of therapist self-report ratings of fidelity to evidence-based practices for adolescent behavior problems: correspondence between therapists and observers. Admin Pol Ment Health. 2015;42(2):229-43.