A repeated cross-sectional study of clinicians' use of psychotherapy techniques during 5 years of a system-wide effort to implement evidence-based practices in Philadelphia

Implementation Science

Volumn 14, Issue 1, 2019, Pages

  Beidas, Rinad S ^a   Williams, Nathaniel J ^b   Becker Haimes, Emily M ^a,l   Aarons, Gregory A ^c   Barg, Frances K ^a   Evans, Arthur C ^d   Jackson, Kamilah ^e   Jones, David ^f   Hadley, Trevor ^a   Hoagwood, Kimberly ^g   Marcus, Steven C ^h   Neimark, Geoffrey ^e   Rubin, Ronnie M ^e   Schoenwald, Sonja K ⁱ   Adams, Danielle R ^j   Walsh, Lucia M ^k   Zentgraf, Kelly ^a   Mandell, David S ^a  

^a UNIVERSITY OF PENNSYLVANIA   (United States)

^b BOISE STATE UNIVERSITY   (United States)

^c UNIVERSITY OF CALIFORNIA   (United States)

^d AMERICAN PSYCHOLOGICAL ASSOCIATION   (United States)

^e Perelman School of Medicine ^*  (United States)

^f American Psychological Association ^*  (United States)

^g NEW YORK UNIVERSITY MEDICAL CENTER   (United States)

^h UNIVERSITY OF PENNSYLVANIA   (United States)

ⁱ OREGON SOCIAL LEARNING CENTER   (United States)

^j UNIVERSITY OF CHICAGO   (United States)

^k Department of Biology   (United States)

^l Hall Mercer Community Mental Health Center   (United States)

more..

Author keywords

Cognitive behavioral therapy; Organizational factors; System level implementation; Targets

Indexed keywords

ADOLESCENT; ADULT; ARTICLE; CHECKLIST; CLINICIAN; COGNITIVE BEHAVIORAL THERAPY; CONTROLLED STUDY; CROSS-SECTIONAL STUDY; EVIDENCE BASED PRACTICE CENTER; FEMALE; HUMAN; JUVENILE; MAJOR CLINICAL STUDY; MALE; ORGANIZATION; OUTPATIENT; PENNSYLVANIA; CLINICAL PRACTICE; EVIDENCE BASED PRACTICE; FAMILY THERAPY; MENTAL HEALTH SERVICE; ORGANIZATION AND MANAGEMENT; ORGANIZATIONAL CULTURE; STATISTICS AND NUMERICAL DATA;

ADULT; COMMUNITY MENTAL HEALTH SERVICES; CROSS-SECTIONAL STUDIES; EVIDENCE-BASED PRACTICE; FAMILY THERAPY; FEMALE; HUMANS; MALE; ORGANIZATIONAL CULTURE; PHILADELPHIA; PRACTICE PATTERNS, PHYSICIANS';

EID: 85068531698     PISSN: None     EISSN: 17485908     Source Type: Journal    
DOI: 10.1186/s13012-019-0912-4     Document Type: Article

References (86)

1. Hoagwood KE, Olin SS, Horwitz S, McKay M, Cleek A, Gleacher A, et al. Scaling up evidence-based practices for children and families in New York state: toward evidence-based policies on implementation for state mental health systems. J Clin Child Adolesc Psychol. 2014;43(2):145-57. https://doi.org/10.1080/15374416.2013.869749.
2. Bruns EJ, Kerns SE, Pullmann MD, Hensley SW, Lutterman T, Hoagwood KE. Research, data, and evidence-based treatment use in state behavioral health systems, 2001-2012. Psychiatr Serv. 2015;67(5):496-503. https://doi.org/10.1176/appi.ps.201500014.
3. Rubin RM, Hurford MO, Hadley T, Matlin SL, Weaver S, Evans AC. Synchronizing watches: the challenge of aligning implementation science and public systems. Admin Pol Ment Health. 2016;43(6):1023-8. https://doi.org/10.1007/s10488-016-0759-9.
4. Regan J, Lau AS, Barnett M, Stadnick N, Hamilton A, Pesanti K, et al. Agency responses to a system-driven implementation of multiple evidence-based practices in children's mental health services. BMC Health Serv Res. 2017;17(1):671. https://doi.org/10.1186/s12913-017-2613-5.
5. Walker SC, Lyon AR, Aos S, Trupin EW. The consistencies and vagaries of the Washington state inventory of evidence-based practice: the definition of "evidence-based" in a policy context. Admin Pol Ment Health. 2017;44(1):42-54. https://doi.org/10.1007/s10488-015-0652-y.
6. Mark DD, Rene'e WL, White JP, Bransford D, Johnson KG, Song VL. Hawaii's statewide evidence-based practice program. Nurs Clin. 2014;49(3):275-90. https://doi.org/10.1016/j.cnur.2014.05.002.
7. Cristofalo MA. Implementation of health and mental health evidence-based practices in safety net settings. Soc Work Health Care. 2013;52(8):728-40. https://doi.org/10.1080/00981389.2013.813003.
8. Hemmelgarn AL, Glisson C, James LR. Organizational culture and climate: implications for services and interventions research. Clin Psychol. 2006;13(1):73-89. https://doi.org/10.1111/j.1468-2850.2006.00008.x.
9. Okamura KH, Wolk B, Courtney L, Kang-Yi CD, Stewart R, Rubin RM, et al. The price per prospective consumer of providing therapist training and consultation in seven evidence-based treatments within a large public behavioral health system: an example cost-analysis metric. Front Public Health. 2018;5:356. https://doi.org/10.3389/fpubh.2017.00356.
10. Powell BJ, Beidas RS, Rubin RM, Stewart RE, Wolk CB, Matlin SL, et al. Applying the policy ecology framework to Philadelphia's behavioral health transformation efforts. Admin Pol Ment Health. 2016;43(6):909-26. https://doi.org/10.1007/s10488-016-0733-6.
11. Stewart RE, Marcus SC, Hadley TR, Hepburn BM, Mandell DS. State adoption of incentives to promote evidence-based practices in behavioral health systems. Psychiatr Serv. 2018;69(6):685-8. https://doi.org/10.1176/appi.ps.201700508.
12. Brown CH, Curran G, Palinkas LA, Aarons GA, Wells KB, Jones L, et al. An overview of research and evaluation designs for dissemination and implementation. Annu Rev Public Health. 2017;38:1-22. https://doi.org/10.1146/annurev-publhealth-031816-044215.
13. Starin AC, Atkins MS, Wehrmann KC, Mehta T, Hesson-McInnis MS, Marinez-Lora A, et al. Moving science into state child and adolescent mental health systems: Illinois' evidence-informed practice initiative. J Clin Child Adolesc Psychol. 2014;43(2):169-78. https://doi.org/10.1080/15374416.2013.848772.
14. Beidas RS, Aarons GA, Barg F, Evans AC, Hadley T, Hoagwood KE, et al. Policy to implementation: evidence-based practice in community mental health-study protocol. Implement Sci. 2013;8(1):38. https://doi.org/10.1186/1748-5908-8-38.
15. Brookman-Frazee L, Stadnick N, Roesch S, Regan J, Barnett M, Bando L, et al. Measuring sustainment of multiple practices fiscally mandated in children's mental health services. Admin Pol Ment Health. 2016;43(6):1009-22. https://doi.org/10.1007/s10488-016-0731-8.
16. Aarons GA, Hurlburt M, Horwitz SM. Advancing a conceptual model of evidence-based practice implementation in public service sectors. Admin Pol Ment Health. 2011;38(1):4-23. https://doi.org/10.1007/s10488-010-0327-7.
17. Nilsen P. Making sense of implementation theories, models and frameworks. Implement Sci. 2015;10(1):53. https://doi.org/10.1186/s13012-015-0242-0.
18. Damschroder LJ, Aron DC, Keith RE, Kirsh SR, Alexander JA, Lowery JC. Fostering implementation of health services research findings into practice: a consolidated framework for advancing implementation science. Implement Sci. 2009;4(1):50. https://doi.org/10.1186/1748-5908-4-50.
19. Aarons GA, Ehrhart MG, Farahnak LR, Sklar M. Aligning leadership across systems and organizations to develop a strategic climate for evidence-based practice implementation. Annu Rev Public Health. 2014;35:255-74. https://doi.org/10.1146/annurev-publhealth-032013-182447.
20. Cameron KS, Quinn RE. Diagnosing and changing organizational culture: based on the competing values framework. San Francisco: Wiley; 2011.
21. Hartnell CA, Ou AY, Kinicki A. Organizational culture and organizational effectiveness: a meta-analytic investigation of the competing values framework's theoretical suppositions. J Appl Psychol. 2011;96(4):677. https://doi.org/10.1037/a0021987.
22. Brownson RC, Royer C, Ewing R, McBride TD. Researchers and policymakers: travelers in parallel universes. Am J Prev Med. 2006;30(2):164-72. https://doi.org/10.1016/j.amepre.2005.10.004.
23. Beidas RS, Marcus S, Aarons GA, Hoagwood KE, Schoenwald S, Evans AC, et al. Predictors of community therapists' use of therapy techniques in a large public mental health system. JAMA Pediatr. 2015;169(4):374-82. https://doi.org/10.1001/jamapediatrics.2014.3736.
24. Locke J, Lawson GM, Beidas RS, Aarons GA, Xie M, Lyon AR, et al. Individual and organizational factors that affect implementation of evidence-based practices for children with autism in public schools: a cross-sectional observational study. Implement Sci. 2019;14(1):29. https://doi.org/10.1186/s13012-019-0877-3.
25. Guerrero EG, Fenwick K, Kong Y. Advancing theory development: exploring the leadership-climate relationship as a mechanism of the implementation of cultural competence. Implement Sci. 2017;12(1):133. https://doi.org/10.1186/s13012-017-0666-9.
26. Williams NJ, Glisson C, Hemmelgarn A, Green P. Mechanisms of change in the ARC organizational strategy: increasing mental health clinicians' EBP adoption through improved organizational culture and capacity. Admin Pol Ment Health. 2017;44(2):269-83. https://doi.org/10.1007/s10488-016-0742-5.
27. Saldana L, Chamberlain P, Wang W, Brown CH. Predicting program start-up using the stages of implementation measure. Admin Pol Ment Health. 2012;39(6):419-25. https://doi.org/10.1007/s10488-011-0363-y.
28. Lewis CC, Klasnja P, Powell BJ, Tuzzio L, Jones S, Walsh-Bailey C, et al. From classification to causality: advancing understanding of mechanisms of change in implementation science. Front Public Health. 2018;6:136. https://doi.org/10.3389/fpubh.2018.00136.
29. Powell BJ, Fernandez ME, Williams NJ, Aarons GA, Beidas RS, Lewis CC, et al. Enhancing the impact of implementation strategies in healthcare: a research agenda. Front Public Health. 2019;7:3. https://doi.org/10.1097/HCO.0000000000000067.
30. Williams NJ, Beidas RS. Annual research review: the state of implementation science in child psychology and psychiatry: a review and suggestions to advance the field. J Child Psychol Psychiatry. 2018; https://doi.org/10.1111/jcpp.12960.
31. Chambless DL, Ollendick TH. Empirically supported psychological interventions: controversies and evidence. Annu Rev Psychol. 2001;52(1):685-716. https://doi.org/10.1146/annurev.psych.52.1.685.
32. Garland AF, Brookman-Frazee L, Hurlburt MS, Accurso EC, Zoffness RJ, Haine-Schlagel R, et al. Mental health care for children with disruptive behavior problems: a view inside therapists' offices. Psychiatr Serv. 2010;61(8):788-95. https://doi.org/10.1176/ps.2010.61.8.788.
33. Hofmann SG, Asnaani A, Vonk IJ, Sawyer AT, Fang A. The efficacy of cognitive behavioral therapy: a review of meta-analyses. Cognit Ther Res. 2012;36(5):427-40. https://doi.org/10.1007/s10608-012-9476-1.
34. Weisz JR, Kuppens S, Ng MY, Eckshtain D, Ugueto AM, Vaughn-Coaxum R, et al. What five decades of research tells us about the effects of youth psychological therapy: a multilevel meta-analysis and implications for science and practice. Am Psychol. 2017;72(2) https://doi.org/10.1037/a0040360.
35. Zhou X, Hetrick SE, Cuijpers P, Qin B, Barth J, Whittington CJ, et al. Comparative efficacy and acceptability of psychotherapies for depression in children and adolescents: a systematic review and network meta-analysis. World Psychiatry. 2015;14(2):207-22. https://doi.org/10.1002/wps.20217.
36. Abbass AA, Rabung S, Leichsenring F, Refseth JS, Midgley N. Psychodynamic psychotherapy for children and adolescents: a meta-analysis of short-term psychodynamic models. J Am Acad Child Adolesc Psychiatry. 2013;52(8):863-75. https://doi.org/10.1016/j.jaac.2013.05.014.
37. De Nadai AS, Storch EA. Design considerations related to short-term psychodynamic psychotherapy. J Am Acad Child Adolesc Psychiatry. 2013;52(11):1213-4. https://doi.org/10.1016/j.jaac.2013.08.012.
38. Steel D. Repeated cross-sectional design. In: Encyclopedia of survery research methods. Edited by Layrakas PJ, vol. 2. Thousand Oaks, CA: SAGE Publications; 2008. p. 714-6.
39. Stirman SW, Buchhofer R, McLaulin JB, Evans AC, Beck AT. Public-academic partnerships: the Beck initiative: a partnership to implement cognitive therapy in a community behavioral health system. Psychiatr Serv. 2009;60(10):1302-4. https://doi.org/10.1176/ps.2009.60.10.1302.
40. Foa EB, Hembree EA, Cahill SP, Rauch SA, Riggs DS, Feeny NC, et al. Randomized trial of prolonged exposure for posttraumatic stress disorder with and without cognitive restructuring: outcome at academic and community clinics. J Consult Clin Psychol. 2005;73(5):953. https://doi.org/10.1037/0022-006X.73.5.953.
41. Beidas RS, Adams DR, Kratz HE, Jackson K, Berkowitz S, Zinny A, et al. Lessons learned while building a trauma-informed public behavioral health system in the city of Philadelphia. Eval Program Plann. 2016;59:21-32. https://doi.org/10.1016/j.evalprogplan.2016.07.004.
42. Linehan M. DBT skills training manual. New York: Guilford Publications; 2014.
43. Eyberg S. Parent-child interaction therapy: integration of traditional and behavioral concerns. Child Fam Behav Ther. 1988;10(1):33-46. https://doi.org/10.1300/J019v10n01-04.
44. Skriner LC, Wolk CB, Stewart RE, Adams DR, Rubin RM, Evans AC, et al. Therapist and organizational factors associated with participation in evidence-based practice initiatives in a large urban publicly-funded mental health system. J Behav Health Serv Res. 2017;45(2):174-86. https://doi.org/10.1007/s11414-017-9552-0.
45. Benjamin Wolk C, Marcus SC, Weersing VR, Hawley KM, Evans AC, Hurford MO, et al. Therapist-and client-level predictors of use of therapy techniques during implementation in a large public mental health system. Psychiatr Serv. 2016;67(5):551-7. https://doi.org/10.1176/appi.ps.201500022.
46. Babbar S, Adams DR, Becker-Haimes EM, Skriner LC, Kratz HE, Cliggitt L, et al. Therapist turnover and client non-attendance. Child Youth Serv Rev. 2018;93:12-6. https://doi.org/10.1016/j.childyouth.2018.06.026.
47. Weersing VR, Weisz JR, Donenberg GR. Development of the therapy procedures checklist: a therapist-report measure of technique use in child and adolescent treatment. J Clin Child Adolesc Psychol. 2002;31(2):168-80. https://doi.org/10.1207/S15374424JCCP3102-03.
48. Kolko DJ, Cohen JA, Mannarino AP, Baumann BL, Knudsen K. Community treatment of child sexual abuse: a survey of practitioners in the National Child Traumatic Stress Network. Admin Pol Ment Health. 2009;36(1):37-49. https://doi.org/10.1007/s10488-008-0180-0.
49. Do M, Warnick E, Weersing V. Examination of the psychometric properties of the Therapy Procedures Checklist-Family Revised. In: Poster presented at the annual meeting of the Association for Behavioral and Cognitive Therapies (ABCT). MD: National Harbor; 2012.
50. James LR, Demaree RG, Wolf G. Estimating within-group interrater reliability with and without response bias. J Appl Psychol. 1984;69(1):85. https://doi.org/10.1037/0021-9010.69.1.85.
51. Brown RD, Hauenstein NM. Interrater agreement reconsidered: an alternative to the rwg indices. Organ Res Methods. 2005;8(2):165-84. https://doi.org/10.1177/1094428105275376.
52. Bliese PD. Within-group agreement, non-independence, and reliability: implications for data aggregation and analysis. In: Klein KJ, SWJ K, editors. Multilevel theory, research, and methods in organizations: foundations, extensions, and new directions. San Francisco: Jossey-Bass; 2000. p. 349-81.
53. Glisson C, Landsverk J, Schoenwald SK, Kelleher K, Hoagwood KE, Mayberg S, et al. Assessing the organizational social context (OSC) of mental health services: implications for research and practice. Admin Pol Ment Health. 2008;35:98-113. https://doi.org/10.1007/s10488-007-0148-5.
54. Williams NJ, Glisson C. Testing a theory of organizational culture, climate and youth outcomes in child welfare systems: a United States national study. Child Abuse Negl. 2014;38(4):757-67. https://doi.org/10.1016/j.chiabu.2013.09.003.
55. Aarons GA, Glisson C, Green PD, Hoagwood KE, Kelleher KJ, Landsverk JA. The organizational social context of mental health services and clinician attitudes toward evidence-based practice: a United States national study. Implement Sci. 2012;7(1):56. https://doi.org/10.1186/1748-5908-7-56.
56. Olin SS, Williams NJ, Pollock M, Armusewicz K, Kutash K, Glisson C, et al. Quality indicators for family support services and their relationship to organizational social context. Admin Pol Ment Health. 2014;41(1):43-54. https://doi.org/10.1007/s10488-013-0499-z.
57. Aarons GA, Ehrhart MG, Farahnak LR. The implementation leadership scale (ILS): development of a brief measure of unit level implementation leadership. Implement Sci. 2014;9(1):45. https://doi.org/10.1186/1748-5908-9-45.
58. Ehrhart MG, Aarons GA, Farahnak LR. Assessing the organizational context for EBP implementation: the development and validity testing of the implementation climate scale (ICS). Implement Sci. 2014;9(1):157. https://doi.org/10.1186/s13012-014-0157-1.
59. Beidas RS, Stewart RE, Adams DR, Fernandez T, Lustbader S, Powell BJ, et al. A multi-level examination of stakeholder perspectives of implementation of evidence-based practices in a large urban publicly-funded mental health system. Admin Pol Ment Health. 2015; https://doi.org/10.1007/s10488-015-0705-2.
60. Beidas RS, Skriner L, Adams D, Wolk CB, Stewart RE, Becker-Haimes E, et al. The relationship between consumer, clinician, and organizational characteristics and use of evidence-based and non-evidence-based therapy strategies in a public mental health system. Behav Res Ther. 2017;99:1-10. https://doi.org/10.1016/j.brat.2017.08.011.
61. Aarons GA, Glisson C, Hoagwood KE, Kelleher K, Landsverk J, Cafri G. Psychometric properties and US national norms of the evidence-based practice attitude scale (EBPAS). Psychol Assess. 2010;22(2):356. https://doi.org/10.1037/a0019188.
62. Garner BR, Godley SH, Bair CM. The impact of pay-for-performance on therapists' intentions to deliver high-quality treatment. J Subst Abus Treat. 2011;41(1):97-103. https://doi.org/10.1016/j.jsat.2011.01.012.
63. Henggeler SW, Chapman JE, Rowland MD, Halliday-Boykins CA, Randall J, Shackelford J, et al. Statewide adoption and initial implementation of contingency management for substance-abusing adolescents. J Consult Clin Psychol. 2008;76(4):556-67. https://doi.org/10.1037/0022-006X.76.4.556.
64. Jensen-Doss A, Hawley KM, Lopez M, Osterberg LD. Using evidence-based treatments: the experiences of youth providers working under a mandate. Prof Psychol Res Pr. 2009;40(4):417. https://doi.org/10.1037/a0014690.
65. Williams JR, Williams WO, Dusablon T, Blais MP, Tregear SJ, Banks D. Evaluation of a randomized intervention to increase adoption of comparative effectiveness research by community health organizations. J Behav Health Serv Res. 2014;41(3):308-23. https://doi.org/10.1007/s11414-013-9369-4.
66. Enders CK. Applied missing data analysis. New York: Guildford Press; 2010.
67. Raudenbush SW, Bryk AS. Hierarchical linear models: applications and data analysis methods, vol. 1. Thousand Oaks, CA: Sage Publications; 2002.
68. Raudenbush SW, Bryk AS, Congdon R. HLM 6 for windows. In., 6.08 edn. Lincolnwood: Scientific software international, Inc.; 2004.
69. Raftery AE. Bayesian model selection in social research. Sociol Methodol. 1995;25:111-64. https://doi.org/10.2307/271063.
70. Rothman K: No adjustments are needed for multiple comparisons. 1990; 1:43-46.
71. Michels K, Rosner B. Data trawling: to fish or not to fish. Lancet. 1996;348:1152-3. https://doi.org/10.1016/S0140-6736(96)05418-9.
72. Benjamini Y, Hochberg Y. Controlling the false discovery rate: a practical and powerful approach to multiple testing. J R Stat Soc Ser B Methodol. 1995;57(1):289-300. https://doi.org/10.1111/j.2517-6161.1995.tb02031.x.
73. Salsberg E, Quigley L, Mehfoud N, Acquaviva KD, Wyche K, Silwa S: Profile of the social work workforce. The George Washington University; 2017. https://www.cswe.org/Centers-Initiatives/Initiatives/National-Workforce-Initiative/SW-Workforce-Book-FINAL-11-08-2017.aspx. Accessed: April 23, 2019.
74. Williams NJ. Multilevel mechanisms of implementation strategies in mental health: integrating theory, research, and practice. Admin Pol Ment Health. 2016;43(5):783-98. https://doi.org/10.1007/s10488-015-0693-2.
75. Curran GM, Bauer M, Mittman B, Pyne JM, Stetler C. Effectiveness-implementation hybrid designs: combining elements of clinical effectiveness and implementation research to enhance public health impact. Med Care. 2012;50(3):217-26. https://doi.org/10.1097/MLR.0b013e3182408812.
76. Glisson C, Schoenwald SK, Kelleher K, Landsverk J, Hoagwood KE, Mayberg S, et al. Therapist turnover and new program sustainability in mental health clinics as a function of organizational culture, climate, and service structure. Admin Pol Ment Health. 2008;35(1-2):124-33. https://doi.org/10.1007/s10488-007-0152-9.
77. Glisson C, Hemmelgarn A, Green P, Williams NJ. Randomized trial of the availability, responsiveness and continuity (ARC) organizational intervention for improving youth outcomes in community mental health programs. J Am Acad Child Adolesc Psychiatry. 2013;52(5):493-500. https://doi.org/10.1016/j.jaac.2012.05.010.
78. Glisson C. Assessing and changing organizational culture and climate for effective services. Res Soc Work Pract. 2007;17(6):736-47. https://doi.org/10.1177/1049731507301659.
79. Wang V, Maciejewski ML, Helfrich CD, Weiner BJ. Working smarter not harder: coupling implementation to de-implementation. Healthcare. 2018;6:104-7. https://doi.org/10.1016/j.hjdsi.2017.12.004.
80. McKay VR, Morshed AB, Brownson RC, Proctor EK, Prusaczyk B. Letting go: conceptualizing intervention de-implementation in public health and social service settings. Am J Community Psychol. 2018;62:189-202. https://doi.org/10.1002/ajcp.12258.
81. Podsakoff N. Common method biases in behavioral research: a critical review of the literature and recommended remedies. J Appl Psychol. 2003;88(5):879-903. https://doi.org/10.1037/0021-9010.88.5.879.
82. Bellg AJ, Borrelli B, Resnick B, Hecht J, Minicucci DS, Ory M, et al. Enhancing treatment fidelity in health behavior change studies: best practices and recommendations from the NIH behavior change consortium. Health Psychol. 2004;23(5):443-51. https://doi.org/10.1037/0278-6133.23.5.443.
83. Hogue A, Dauber S, Lichvar E, Bobek M, Henderson CE. Validity of therapist self-report ratings of fidelity to evidence-based practices for adolescent behavior problems: correspondence between therapists and observers. Admin Pol Ment Health. 2015;42(2):229-43. https://doi.org/10.1007/s10488-014-0548-2.
84. Hurlburt MS, Garland AF, Nguyen K, Brookman-Frazee L. Child and family therapy process: concordance of therapist and observational perspectives. Admin Pol Ment Health. 2010;37(3):230-44. https://doi.org/10.1007/s10488-009-0251-x.
85. McHugh RK, Barlow DH. The dissemination and implementation of evidence-based psychological treatments: a review of current efforts. Am Psychol. 2010;65(2):73-84. https://doi.org/10.1037/a0018121.
86. Aarons GA, Ehrhart MG, Farahnak LR, Hurlburt MS. Leadership and organizational change for implementation (LOCI): a randomized mixed method pilot study of a leadership and organization development intervention for evidence-based practice implementation. Implement Sci. 2015;10(1):11. https://doi.org/10.1186/s13012-014-0192-y.