A systematic review of empirical studies examining mechanisms of implementation in health

Implementation Science

Volumn 15, Issue 1, 2020, Pages

  Lewis, Cara C ^a,b,c   Boyd, Meredith R ^d   Walsh Bailey, Callie ^a,e   Lyon, Aaron R ^c   Beidas, Rinad ^f   Mittman, Brian ^a   Aarons, Gregory A ^g   Weiner, Bryan J ^h   Chambers, David A ⁱ  

^a KAISER PERMANENTE   (United States)

^b INDIANA UNIVERSITY   (United States)

^c UNIVERSITY OF WASHINGTON SCHOOL OF MEDICINE   (United States)

^d UNIVERSITY OF CALIFORNIA   (United States)

^e WASHINGTON UNIVERSITY IN ST LOUIS   (United States)

^f UNIVERSITY OF PENNSYLVANIA   (United States)

^g UNIVERSITY OF CALIFORNIA   (United States)

^h UNIVERSITY OF WASHINGTON   (United States)

ⁱ NATIONAL CANCER INSTITUTE   (United States)

Author keywords

Causal model; Determinant; Implementation; Mechanism; Mediator; Moderator; Theory

Indexed keywords

CAUSAL MODEL; CINAHL; DATA EXTRACTION; EMPIRICISM; HUMAN; IMPLEMENTATION SCIENCE; MEDLINE; QUALITATIVE RESEARCH; QUANTITATIVE ANALYSIS; REVIEW; SYSTEMATIC REVIEW; EVIDENCE BASED PRACTICE; HEALTH CARE DELIVERY; INFORMATION DISSEMINATION; METHODOLOGY; ORGANIZATION AND MANAGEMENT; TRANSLATIONAL RESEARCH;

DELIVERY OF HEALTH CARE; EVIDENCE-BASED PRACTICE; HUMANS; IMPLEMENTATION SCIENCE; INFORMATION DISSEMINATION; RESEARCH DESIGN; TRANSLATIONAL MEDICAL RESEARCH;

EID: 85083631852     PISSN: None     EISSN: 17485908     Source Type: Journal    
DOI: 10.1186/s13012-020-00983-3     Document Type: Review

References (138)

1. Eccles MP, Mittman BS. Welcome to implementation science. Implement Sci. 2006;1(1):1.
2. Powell BJ, McMillen JC, Proctor EK, Carpenter CR, Griffey RT, Bunger AC, et al. A compilation of strategies for implementing clinical innovations in health and mental health. Med Care Res Rev. 2012;69(2):123-57.
3. Powell BJ, Waltz TJ, Chinman MJ, Damschroder LJ, Smith JL, Matthieu MM, et al. A refined compilation of implementation strategies: Results from the Expert Recommendations for Implementing Change (ERIC) project. Implement Sci. 2015;10:21.
4. Kazdin AE. Mediators and mechanisms of change in psychotherapy research. Annu Rev Clin Psychol. 2007;3(1):1-27.
5. Kraemer HC, Wilson GT, Fairburn CG, Agras WS. Mediators and moderators of treatment effects in randomized clinical trials. Arch Gen Psychiatry. 2002;59(10):877-83.
6. Gerring J. Social science methodology: A criterial framework. Cambridge: Cambridge University Press; 2001.
7. Frazier PA, Tix AP, Barron KE. Testing moderator and mediator effects in counseling psychology research. US: American Psychological Association; 2004. p. 115-34.
8. Hill AB. The Environment and Disease: Association or Causation? Proc R Soc Med. 1965;58:295-300.
9. Bosch M, van der Weijden T, Wensing M, Grol R. Tailoring quality improvement interventions to identified barriers: A multiple case analysis. J Eval Clin Pract. 2007;13(2):161-8.
10. Claridge JA, Fabian TC. History and development of evidence-based medicine. World J Surg. 2005;29(5):547-53.
11. Cook SC, Schwartz AC, Kaslow NJ. Evidence-Based psychotherapy: Advantages and challenges. Neurotherapeutics. 2017;14(3):537-45.
12. Dissemination and Implementation Research in Health (R01 Clinical Trial Optional). National Institutes of Health (NIH); 2019. https://grants.nih.gov/grants/guide/pa-files/PAR-19-274.html.
13. Thomas J, Brunton J, Graziosi S. EPPI-Reviewer 4: Software for research synthesis. EPPI-Centre Software. London: Social Science Research Unit, UCL Institute of Education; 2010.
14. Williams NJ. Multilevel mechanisms of implementation strategies in mental health: Integrating theory, research, and practice. Adm Policy Ment Health. 2016;43(5):783-98.
15. Hong QN, Pluye P, Fabregues S, Bartlett G, Boardman F, Cargo M, et al. Mixed Methods Appraisal Tool (MMAT) Montreal, Canada: McGill University; 2018 [Available from: Http://mixedmethodsappraisaltoolpublic.pbworks.com/w/file/fetch/127916259/MMAT_2018_criteria-manual_2018-08-01_ENG.pdf.
16. Bardosh KL, Murray M, Khaemba AM, Smillie K, Lester R. Operationalizing mHealth to improve patient care: A qualitative implementation science evaluation of the WelTel texting intervention in Canada and Kenya. Global Health. 2017;13(1):87.
17. Brewster AL, Curry LA, Cherlin EJ, Talbert-Slagle K, Horwitz LI, Bradley EH. Integrating new practices: A qualitative study of how hospital innovations become routine. Implement Sci. 2015;10:168.
18. Carrera PM, Lambooij MS. Implementation of out-of-office blood pressure monitoring in the netherlands: From clinical guidelines to patients' adoption of innovation. Medicine. 2015;94(43):e1813.
19. Frykman M, Hasson H, Muntlin Athlin Å, von Thiele Schwarz U. Functions of behavior change interventions when implementing multi-professional teamwork at an emergency department: A comparative case study. BMC Health Serv Res. 2014;14:218.
20. Wiener-Ogilvie S, Huby G, Pinnock H, Gillies J, Sheikh A. Practice organisational characteristics can impact on compliance with the BTS/SIGN asthma guideline: Qualitative comparative case study in primary care. BMC Fam Pract. 2008;9:32.
21. Atkins MS, Frazier SL, Leathers SJ, Graczyk PA, Talbott E, Jakobsons L, et al. Teacher key opinion leaders and mental health consultation in low-income urban schools. J Consult Clin Psychol. 2008;76(5):905-8.
22. Baer JS, Wells EA, Rosengren DB, Hartzler B, Beadnell B, Dunn C. Agency context and tailored training in technology transfer: A pilot evaluation of motivational interviewing training for community counselors. J Subst Abuse Treat. 2009;37(2):191-202.
23. Bonetti D, Eccles M, Johnston M, Steen N, Grimshaw J, Baker R, et al. Guiding the design and selection of interventions to influence the implementation of evidence-based practice: An experimental simulation of a complex intervention trial. Soc Sci Med. 2005;60(9):2135-47.
24. Garner BR, Godley SH, Bair CML. The impact of pay-for-performance on therapists' intentions to deliver high quality treatment. J Subst Abuse Treat. 2011;41(1):97-103.
25. Glisson C, Schoenwald SK, Hemmelgarn A, Green P, Dukes D, Armstrong KS, et al. Randomized trial of MST and ARC in a two-level evidence-based treatment implementation strategy. J Consult Clin Psychol. 2010;78(4):537-50.
26. Holth P, Torsheim T, Sheidow AJ, Ogden T, Henggeler SW. Intensive quality assurance of therapist adherence to behavioral interventions for adolescent substance use problems. J Child Adolesc Subst Abuse. 2011;20(4):289-313.
27. Lee H, Hall A, Nathan N, Reilly KL, Seward K, Williams CM, et al. Mechanisms of implementing public health interventions: A pooled causal mediation analysis of randomised trials. Implement Sci. 2018;13(1):42.
28. Lochman JE, Boxmeyer C, Powell N, Qu L, Wells K, Windle M. Dissemination of the coping power program: Importance of intensity of counselor training. J Consult Clin Psychol. 2009;77(3):397-409.
29. Rapkin BD, Weiss E, Lounsbury D, Michel T, Gordon A, Erb-Downward J, et al. Reducing Disparities in cancer screening and prevention through community-based participatory research partnerships with local libraries: A comprehensive dynamic trial. Am J Community Psychol. 2017;60(1-2):145-59.
30. Rohrbach LA, Graham JW, Hansen WB. Diffusion of a school-based substance abuse prevention program: Predictors of program implementation. Prev Med. 1993;22(2):237-60.
31. Seys D, Bruyneel L, Sermeus W, Lodewijckx C, Decramer M, Deneckere S, et al. Teamwork and adherence to recommendations explain the effect of a care pathway on reduced 30-day readmission for patients with a COPD exacerbation. COPD. 2018;15(2):157-64.
32. Williams NJG, C. The role of organizational culture and climate in the dissemination and implementation of empirically-supported treatments for youth. Dissemination and implementation of evidence based practices in child and adolescent mental health. New York: Oxford University Press; 2014. p. 61-81.
33. Williams NJ, Glisson C, Hemmelgarn A, Green P. Mechanisms of change in the ARC Organizational strategy: Increasing mental health clinicians' EBP adoption through improved organizational culture and capacity. Adm Policy Ment Health. 2017;44(2):269-83.
34. Aarons GA, Sommerfeld DH, Walrath-Greene CM. Evidence-based practice implementation: The impact of public versus private sector organization type on organizational support, provider attitudes, and adoption of evidence-based practice. Implement Sci. 2009;4:83.
35. Becker SJ, Squires DD, Strong DR, Barnett NP, Monti PM, Petry NM. Training opioid addiction treatment providers to adopt contingency management: A prospective pilot trial of a comprehensive implementation science approach. Subst Abus. 2016;37(1):134-40.
36. Beenstock J, Sniehotta FF, White M, Bell R, Milne EMG, Araujo-Soares V. What helps and hinders midwives in engaging with pregnant women about stopping smoking? A cross-sectional survey of perceived implementation difficulties among midwives in the North East of England. Implement Sci. 2012;7:36.
37. Beets MW, Flay BR, Vuchinich S, Acock AC, Li KK, Allred C. School climate and teachers' beliefs and attitudes associated with implementation of the positive action program: A diffusion of innovations model. Prev Sci. 2008;9(4):264-75.
38. Bonetti D, Johnston M, Clarkson J, Turner S. Applying multiple models to predict clinicians' behavioural intention and objective behaviour when managing children's teeth. Psychol Health. 2009;24(7):843-60.
39. Chou AF, Vaughn TE, McCoy KD, Doebbeling BN. Implementation of evidence-based practices: Applying a goal commitment framework. Health Care Manage Rev. 2011;36(1):4-17.
40. Chambers D, Simpson L, Neta G, UvT S, Percy-Laurry A, Aarons GA, et al. Proceedings from the 9th annual conference on the science of dissemination and implementation. Implementation Sci. 2017;12(1):48.
41. David P, Schiff M. Self-efficacy as a mediator in bottom-up dissemination of a Research-supported intervention for young, traumatized children and their families. J Evid Inf Soc Work. 2017;14(2):53-69.
42. Edmunds JM, Read KL, Ringle VA, Brodman DM, Kendall PC, Beidas RS. Sustaining clinician penetration, attitudes and knowledge in cognitive-behavioral therapy for youth anxiety. Implement Sci. 2014;9.
43. Gnich W, Sherriff A, Bonetti D, Conway DI, Macpherson LMD. The effect of introducing a financial incentive to promote application of fluoride varnish in dental practice in Scotland: A natural experiment. Implement Sci. 2018;13(1):95.
44. Guerrero EG, Frimpong J, Kong Y, Fenwick K. Aarons GA. Health Care Manage Rev: Advancing theory on the multilevel role of leadership in the implementation of evidence-based health care practices; 2018.
45. Huis A, Holleman G, van Achterberg T, Grol R, Schoonhoven L, Hulscher M. Explaining the effects of two different strategies for promoting hand hygiene in hospital nurses: A process evaluation alongside a cluster randomised controlled trial. Implement Sci. 2013;8:41.
46. Little MA, Pokhrel P, Sussman S, Rohrbach LA. The process of adoption of evidence-based tobacco use prevention programs in California schools. Prev Sci. 2015;16(1):80-9.
47. Llasus L, Angosta AD, Clark M. Graduating baccalaureate students' evidence-based practice knowledge, readiness, and implementation. J Nurs Educ. 2014;53(Suppl 9):S82-9.
48. Nelson TD, Steele RG. Predictors of practitioner self-reported use of evidence-based practices: Practitioner training, clinical setting, and attitudes toward research. Adm Policy Ment Health. 2007;34(4):319-30.
49. Potthoff S, Presseau J, Sniehotta FF, Johnston M, Elovainio M, Avery L. Planning to be routine: Habit as a mediator of the planning-behaviour relationship in healthcare professionals. Implement Sci. 2017;12(1):24.
50. Presseau J, Grimshaw JM, Tetroe JM, Eccles MP, Francis JJ, Godin G, et al. A theory-based process evaluation alongside a randomised controlled trial of printed educational messages to increase primary care physicians' prescription of thiazide diuretics for hypertension [ISRCTN72772651]. Implement Sci. 2016;11(1):121.
51. Simmonds MJ, Derghazarian T, Vlaeyen JW. Physiotherapists' knowledge, attitudes, and intolerance of uncertainty influence decision making in low back pain. Clin J Pain. 2012;28(6):467-74.
52. Stockdale SE, Rose D, Darling JE, Meredith LS, Helfrich CD, Dresselhaus TR, et al. Communication among team members within the patient-centered medical home and patient satisfaction with providers: The mediating role of patient-provider communication. Med Care. 2018;56(6):491-6.
53. Wanless SB, Rimm-Kaufman SE, Abry T, Larsen RA, Patton CL. Engagement in training as a mechanism to understanding fidelity of implementation of the responsive classroom approach. Prev Sci. 2015;16(8):1107-16.
54. Armson H, Roder S, Elmslie T, Khan S, Straus SE. How do clinicians use implementation tools to apply breast cancer screening guidelines to practice? Implement Sci. 2018;13(1):79.
55. Birken SA, Lee S-YD, Weiner BJ, Chin MH, Chiu M, Schaefer CT. From strategy to action: How top managers' support increases middle managers' commitment to innovation implementation in healthcare organizations. Health Care Manage Rev. 2015;40(2):159-68.
56. Kauth MR, Sullivan G, Blevins D, Cully JA, Landes RD, Said Q, et al. Employing external facilitation to implement cognitive behavioral therapy in VA clinics: A pilot study. Implement Sci. 2010;5(1):75.
57. Lukas CV, Mohr DC, Meterko M. Team effectiveness and organizational context in the implementation of a clinical innovation. Qual Manag Health Care. 2009;18(1):25-39.
58. Panzano PC, Sweeney HA, Seffrin B, Massatti R, Knudsen KJ. The assimilation of evidence-based healthcare innovations: A management-based perspective. J Behav Health Serv Res. 2012;39(4):397-416.
59. Rangachari P, Madaio M, Rethemeyer RK, Wagner P, Hall L, Roy S, et al. The evolution of knowledge exchanges enabling successful practice change in two intensive care units. Health Care Manage Rev. 2015;40(1):65-78.
60. Shrubsole K, Worrall L, Power E, O'Connor DA. The acute aphasia implementation study (AAIMS): A pilot cluster randomized controlled trial. Int J Lang Commun Disord. 2018;53(5):1021-56.
61. Scott SD, Albrecht L, O'Leary K, Ball GD, Hartling L, Hofmeyer A, et al. Systematic review of knowledge translation strategies in the allied health professions. Implement Sci. 2012;7:70.
62. Yamada J, Squires JE, Estabrooks CA, Victor C, Stevens B, Pain CTiCs. The role of organizational context in moderating the effect of research use on pain outcomes in hospitalized children: A cross sectional study. BMC Health Serv Res. 2017;17(1):68.
63. Rogers E. Diffusion of innovations. 4th ed. New York: Free Press; 1995.
64. Rogers E. Diffusion of Innovations. 3rd ed. New York: Free Press; 1983.
65. Ajzen I. The theory of planned behavior. Organ Behav Hum Decis Process. 1991;50(2):179-211.
66. Michie S, Johnston M, Abraham C, Lawton R, Parker D, Walker A, et al. Making psychological theory useful for implementing evidence based practice: A consensus approach. Qual Saf Health Care. 2005;14(1):26-33.
67. Kitson A, Harvey G, McCormack B. Enabling the implementation of evidence based practice: A conceptual framework. Qual Health Care. 1998;7(3):149-58.
68. Damschroder LJ, Aron DC, Keith RE, Kirsh SR, Alexander JA, Lowery JC. Fostering implementation of health services research findings into practice: A consolidated framework for advancing implementation science. Implement Sci. 2009;4:50.
69. Klein KJ, Sorra JS. The challenge of innovation implementation. Acad Manage Rev. 1996;21(4):1055-80.
70. Davis FD. Perceived usefulness, perceived ease of use, and user acceptance of information technology. MIS Quarterly. 1989;13(3):319-40.
71. Thompson RS, Higgins CA, Howell JM. Personal computing: Toward a conceptual model of utilization. MIS Quarterly. 1991;15(1):125-43.
72. Braksick LW. Unlock behavior, unleash profits: Developing leadership behavior that drives profitability in your organization. New York, NY: McGraw-Hill; 2007.
73. Johnson J, Dakens L, Edwards P, Morse N. SwitchPoints: Culture change on the fast track to business success. Hoboken, NJ: John Wiley & Sons; 2008.
74. Hedeker D, Gibbons RD. Longitudinal data analysis. New York, NY: Wiley; 2006.
75. Krull JL, MacKinnon DP. Multilevel modeling of individual and group level mediated effects. Multivariate Behav Res. 2001;36(2):249-77.
76. Bandura A. Self-efficacy: The exercise of control. New York: Macmillan; 1997.
77. Bandura A. Exercise of human agency through collective efficacy. Curr Dir Psychol Sci. 2000;9(3):75-8.
78. Baron RM, Kenny DA. The moderator-mediator variable distinction in social psychological research: Conceptual, strategic, and statistical considerations. J Pers Soc Psychol. 1986;51(6):1173-82.
79. Sobel ME. Asymptotic confidence intervals for indirect effects in structural equation models. In: Leinhart S, editor. Sociological Methodology. San Francisco: Jossey-Bass; 1982.
80. Raudenbush SW, Bryk AS, Cheong YF, Congdon RT. HLM7: Hierarchical linear and nonlinear modeling. Chicago: Scientific Software International; 2004.
81. Hosmer DW, Lemeshow S. Applied logistic regression. New York, NY: John Wiley & Sons; 1989.
82. Raudenbush SW, Bryk A, Congdon RT. HLM 6. Scientific Software International: Lincolnwood, IL; 2005.
83. Singer JD, Willet JB. Applied longitudinal data analysis: Modeling change and event occurrence. New York, NY: Oxford University Press; 2003.
84. Cane J, O'Connor D, Michie S. Validation of the theoretical domains framework for use in behaviour change and implementation research. Implement Sci. 2012;7:37.
85. Imai K, Keele L, Tingley D. A general approach to causal mediation analysis. Psychol Methods. 2010;15(4):309-34.
86. van Buuren SG-O, K. Mice: Multivariate imputation by chained equations in R. J Stat Softw. 2010:1-68.
87. Rogers EM. Diffusion of innovations. 5th ed. New York, NY: Free Press; 2003.
88. Raudenbush SW, Liu X. Statistical power and optimal design for multisite randomized trials. Psychol Methods. 2000;5(2):199-213.
89. Allison PD. Event history analysis. Thousand Oaks, CA: SAGE Publications; 1984.
90. Yuk Fai C, Randall PF, Stephen WR. Efficiency and robustness of alternative estimators for two-A nd three-level models: The case of NAEP. J Educ Behav Stat. 2001;26(4):411-29.
91. Hox JJ, Maas CJM. The accuracy of multilevel structural equation modeling with pseudobalanced groups and small samples. Struct Equ Model. 2001;8(2):157-74.
92. Zhang Z, Zyphur MJ, Preacher KJ. Testing multilevel mediation using hierarchical linear models: Problems and solutions. Organizational Research Methods. 2009;12(4):695-719.
93. Scott WR. Institutions and Organizations. Thousand Oaks, CA: Sage; 2001.
94. Eisenberger R, Huntington R, Hutchison S, Sowa D. Perceived organizational support. Journal of Applied Psychology. 1986;71:500-7.
95. Preacher KJ, Hayes AF. SPSS and SAS procedures for estimating indirect effects in simple mediation models. Behav Res Methods Instrum Comput. 2004;36(4):717-31.
96. Chen HT. Theory-driven evaluations. In: Reynolds HJ, Walber HJ, editors. Advances in educational productivity: Evaluation research for educational productivity. 7th ed. Bingley, UK: Emerald Group Publishing Limited; 1998.
97. Marsh HW, Hau KT, Balla JR, Grayson D. Is More Ever Too Much? The Number of indicators per factor in confirmatory factor analysis. Multivariate Behav Res. 1998;33(2):181-220.
98. Bandalos DL, Finney SJ. Item parceling issues in structural equation modeling. In: Marcoulides GA, editor. New developments and techniques in structural equation modeling. Mahwah, NJ: Erlbaum; 2001. p. 269-96.
99. Bandura A. Health promotion from the perspective of social cognitive theory. Psychol Health. 1998;13(4):623-49.
100. Blackman D. Operant conditioning: An experimental analysis of behaviour. London, UK: Methuen; 1974.
101. Gollwitzer PM. Implementation intentions: Strong effects of simple plans. Am Psychol. 1999;54:493-503.
102. Leventhal H, Nerenz D, Steele DJ. Illness representations and coping with health threats. In: Baum A, Taylor SE, Singer JE, editors. Handbook of psychology and health, volume 4: Social psychological aspects of health. Hillsdale, NJ: Lawrence Erlbaum; 1984. p. 219-51.
103. Weinstein N. The precaution adoption process. Health Psychol. 1988;7:355-86.
104. Prochaska JO, DiClemente CC. Stages and processes of self-change of smoking: Toward an integrative model of change. J Consult Clin Psychol. 1983;51(3):390-5.
105. Landy FJ, Becker W. Motivation theory reconsidered. In: Cumming LL, Staw BM, editors. Research in organizational behavior. Greenwich, CT: JAI Press; 1987.
106. Locke EA, Latham GP. Building a practically useful theory of goal setting and task motivation: A 35-year odyssey. Am Psychol. 2002;57(9):705-17.
107. Kennedy P. A guide to econometrics. Cambridge, MA: MIT Press; 2003.
108. Joreskog KGS, D. LISRELR 8: User's reference guide. Lincolnwood, IL: Scientific Software International; 1996.
109. Valente TW. Social network thresholds in the diffusion of innovations. Social Networks. 1996;18:69-89.
110. Hayes AF. Beyond Baron and Kenny: Statistical mediation analysis in the new millennium. Communication Monographs. 2009;76:408-20.
111. Aarons GA, Hurlburt M, Horwitz SM. Advancing a conceptual model of evidence-based practice implementation in public service sectors. Adm Policy Ment Health. 2011;38(1):4-23.
112. Raudenbush SW, Bryk AS. Hierarchical linear models. Thousand Oaks: Sage; 2002.
113. Bryk AS, Raudenbush SW. Hierarchical linear models. Newbury Park, CA: Sage; 1992.
114. Muthén LK, Muthén BO. Mplus user's guide Los Angeles, CA: Muthén & Muthén 2012 [Seventh Edition:[Available from: Https://www.statmodel.com/download/usersguide/Mplus%20user%20guide%20Ver_7_r3_web.pdf.
115. Bentler PM. On tests and indices for evaluating structural models. Personal Individ Differ. 2007;42(5):825-9.
116. MacKinnon DP, Fairchild AJ, Fritz MS. Mediation analysis. Annu Rev Psychol. 2007;58:593-614.
117. Graham I, Logan J, Harrison M, Straus S, Tetroe J, Caswell W, et al. Lost in knowledge translation: Time for a map? J Contin Educ Health Prof. 2006;26.
118. Epstein S. Cognitive-experiential self-theory. In: Pervin LA, editor. Handbook of personality: Theory and research. New York: Guilford; 1990. p. 165-92.
119. Karlson KB, Holm A, Breen R. Comparing Regression coefficients between same-sample Nested models using logit and probit: A new method. Sociological Methodology. 2012;42(1):274-301.
120. Rycroft-Malone J, Kitson A, Harvey G, McCormack B, Seers K, Titchen A, et al. Ingredients for change: Revisiting a conceptual framework. BMJ Qual Saf. 2002;11(2):174-80.
121. Yukl G, Gordon A, Taber T. A hierarchical taxonomy of leadership behavior: Integrating a half century of behavior research. J Leadersh Organ Stud. 2002;9(1):15-32.
122. Shrout PE, Bolger N. Mediation in experimental and nonexperimental studies: New procedures and recommendations. Psychol Methods. 2002;7(4):422-45.
123. Fixsen DL, Naoom SF, Blase KA, Friedman RM. Implementation research: A synthesis of the literature; 2005.
124. Frambach R. An integrated model of organizational adoption and diffusion of innovations. Eur J Mark. 1993;27(5):22-41.
125. Institute of Medicine (IOM). Crossing the quality chasm: A new health system for the 21st century. Washington, DC: Institute of Medicine, National Academy Press; 2001.
126. Preacher KJ, Hayes AF. Asymptotic and resampling strategies for assessing and comparing indirect effects in multiple mediator models. Behav Res Methods. 2008;40(3):879-91.
127. Stahmer AC, Suhrheinrich J, Schetter PL, McGee HE. Exploring multi-level system factors facilitating educator training and implementation of evidence-based practices (EBP): A study protocol. Implement Sci. 2018;13(1):3.
128. Lewis CC, Klasnja P, Powell BJ, Lyon AR, Tuzzio L, Jones S, et al. From classification to causality: Advancing understanding of mechanisms of change in implementation science. Front Public Health. 2018;6:136.
129. Proctor E, Silmere H, Raghavan R, Hovmand P, Aarons G, Bunger A, et al. Outcomes for implementation research: Conceptual distinctions, measurement challenges, and research agenda. Adm Policy Ment Health. 2011;38(2):65-76.
130. Weiner BJ, Lewis MA, Clauser SB, Stitzenberg KB. In search of synergy: Strategies for combining interventions at multiple levels. J Natl Cancer Inst Monogr. 2012;2012(44):34-41.
131. Lewis CC, Weiner BJ, Stanick C, Fischer SM. Advancing implementation science through measure development and evaluation: A study protocol. Implement Sci. 2015;10:102.
132. Powell BJ, Stanick CF, Halko HM, Dorsey CN, Weiner BJ, Barwick MA, et al. Toward criteria for pragmatic measurement in implementation research and practice: A stakeholder-driven approach using concept mapping. Implement Sci. 2017;12(1):118.
133. Wu AD, Zumbo BD. Understanding and using mediators and moderators. Soc Indic Res. 2007;87(3):367.
134. MacKinnon DP, Lockwood CM, Hoffman JM, West SG, Sheets V. A comparison of methods to test mediation and other intervening variable effects. Psychol Methods. 2002;7(1):83.
135. Pituch KA, Murphy DL, Tate RL. Three-level models for indirect effects in school-A nd class-randomized experiments in education. J Exp Educ. 2009;78(1):60-95.
136. Moore GF, Audrey S, Barker M, Bond L, Bonell C, Hardeman W, et al. Process evaluation of complex interventions: Medical Research Council guidance. BMJ: British Medical Journal. 2015;350:h1258.
137. Pawson R, Manzano-Santaella A. A realist diagnostic workshop. Evaluation. 2012;18(2):176-91.
138. Pawson R, Greenhalgh T, Harvey G, Walshe K. Realist synthesis: An introduction. Manchester, UK: ESRC Research Methods Programme, University of Manchester; 2004.