Evaluation approaches for dissemination and implementation research

Dissemination and Implementation Research in Health: Translating Science to Practice, Second Edition

Volumn , Issue , 2017, Pages 317-334

  Gaglio, Bridget ^a   Glasgow, Russell E ^b  

^a Patient Centered Outcomes Research Institute (PCORI)   (United States)

^b UNIVERSITY OF COLORADO SCHOOL OF MEDICINE   (United States)

Author keywords

Design; Evaluability assessment; Evaluation; Pragmatic models; Qualitative; Quantitative; Threats to external validity

Indexed keywords

EID: 85046879707     PISSN: None     EISSN: None     Source Type: Book    
DOI: 10.1093/oso/9780190683214.003.0019     Document Type: Chapter

References (89)

1. Patton MQ. Utilization-focused evaluation. 4th ed. London: Sage Publications; 2008.
2. Trochim WM, Rubio DM, Thomas VG, Evaluation Key Function Committee of the CC. Evaluation guidelines for the Clinical and Translational Science Awards (CTSAs). Clin TranslSci. 2013;6(4):303-309.
3. Wholey JS, PHatry HP, Newcomer KE, eds. Handbook of practical program evaluation. 3rd ed. San Francisco: Jossey-Bass; 2010.
4. Yarbrough DB, Shulha LM, Hopson RK, Caruthers FA, eds. The program evaluation standards: A guide for evaluators and evaluation users. 3rd ed. Thousand Oaks, CA: Sage Publications; 2011.
5. American Evaluation Association. American Evaluation Association Guiding Principles for Evaluators. July 2004; http://www.eval.org/p/cm/ld/fid=51. Accessed December 20, 2016.
6. Ogrinc G, Davies L, Goodman D, Batalden P, Davidoff F, Stevens D. SQUIRE 2.0 (Standards for QUality Improvement Reporting Excellence): revised publication guidelines from a detailed consensus process. BMJ Qual Safety. 2016;25(12):986-992.
7. Curran GM, Bauer M, Mittman B, Pyne JM, Stetler C. Effectiveness-implementation hybrid designs: combining elements of clinical effectiveness and implementation research to enhance public health impact. Med Care. 2012;50(3):217-226.
8. Shadish WR, Cook TD, Campbell DT Experimental and quasi-experimental designs for generalized causal inference. Boston: Houghton Mifflin Company; 2002.
9. Leviton LC, Khan LK, Rog D, Dawkins N, Cotton D. Evaluability assessment to improve public health policies, programs, and practices. Annu Rev Public Health. 2010;31:213-233.
10. Proctor EK, Silmere H, Raghavan R, et al. Outcomes for implementation research: conceptual distinctions, measurement challenges, and research agenda. Adm Policy Ment Health. 2011;38(2):65-76.
11. Gaglio B, Shoup JA, Glasgow RE. The RE-AIM framework: a systematic review of use over time. Am J Public Health. 2013;103(6):e38-e46.
12. Kessler RS, Purcell EP, Glasgow RE, Klesges LM, Benkeser RM, Peek CJ. What does it mean to "employ" the RE-AIM model? Eval Health Prof. 2013;36(1):44-66.
13. Glasgow RE. What does it mean to be pragmatic? Pragmatic methods, measures, and models to facilitate research translation. Health Educ Behav. 2013;40(3):257-265.
14. Paul GL. Behavior modification research: Design and tactics. In: Franks CM, ed. Behavior therapy: Appraisal and status. New York: McGraw- Hill; 1969:26-62.
15. Trevisan MS, Walser TM. Evaluability assessment: Improving evaluation quality and use. Thousand Oaks, CA: Sage Publications; 2014.
16. Calhoun A, Mainor A, Moreland-Russell S, Maier RC, Brossart L, Luke DA. Using the Program Sustainability Assessment Tool to assess and plan for sustainability. Prev Chronic Dis. 2014;11:130185.
17. Rogers EM. Diffusion of innovations. 5th ed. New York: Free Press; 2003.
18. Green LW, Kreuter MW. Health program planning: An educational and ecological approach. 4th ed. New York: McGraw-Hill; 2005.
19. Glasgow RE, Vogt TM, Boles SM. Evaluating the public health impact of health promotion interventions: the RE-AIM framework. Am J Public Health. 1999;89(9):1322-1327.
20. Hovmand PS. Community-based system dynamics. New York: Springer; 2014.
21. Klesges LM, Estabrooks PA, Dzewaltowski DA, Bull SS, Glasgow RE. Beginning with the application in mind: designing and planning health behavior change interventions to enhance dissemination. Ann Behav Med. 2005;29 Suppl:66-75.
22. RE-AIM website. RE-AIM Self-rating Quiz Tool. http://re-aim.org/resources-and-tools/self-rating- quiz/. Accessed December 21, 2016.
23. Institute for Healthcare Improvement. Plan-Do- Study-Act (PDSA) Worksheet. http://www.ihi.org/resources/Pages/Tools/PlanDoStudyActWorksheet. aspx. Accessed December 21, 2016.
24. Langley GL, Moen R, Nolan KM, Nolan TW, Norman CL, Provost LP The improvement guide: A practical approach to enhancing organizational performance. 2nd ed. San Francisco: Jossey-Bass; 2009.
25. JaKa MM, Haapala JL, Trapl ES, et al. Reporting of treatment fidelity in behavioural paediatric obesity intervention trials: a systematic review. Obes Rev. 2016;17(12):1287-1300.
26. Breitenstein SM, Gross D, Garvey CA, Hill C, Fogg L, Resnick B. Implementation fidelity in community-based interventions. Res Nur Health. 2010;33(2):164-173.
27. Stirman SW, Miller CJ, Toder K, Calloway A. Development of a framework and coding system for modifications and adaptations of evidence- based interventions. Implement Sci: 2013;8:65.
28. Chambers DA, Norton WE. The adaptome: advancing the science of intervention adaptation. Am J Prevent Med. 2016;51(4 Suppl 2):S124-S131.
29. Glasgow RE, Emmons KM. How can we increase translation of research into practice? Types of evidence needed. Annu Rev Public Health. 2007;28:413-433.
30. Castro FG, Barrera M, Jr., Holleran Steiker LK. Issues and challenges in the design of culturally adapted evidence-based interventions. Annual review of clinical psychology. 2010;6:213-239.
31. Chambers DA, Glasgow RE, Stange KC. The dynamic sustainability framework: addressing the paradox of sustainment amid ongoing change. Implementation science: IS. 2013;8:117.
32. Gaglio B, Phillips SM, Heurtin-Roberts S, Sanchez MA, Glasgow RE. How pragmatic is it? Lessons learned using PRECIS and RE-AIM for determining pragmatic characteristics of research. Implementation science: IS. 2014;9:96.
33. Pawson R, Tilley N. Realistic evaluation. Thousand Oaks, CA: Sage Publications; 1997.
34. Bennett GG, Glasgow RE. The delivery of public health interventions via the Internet: actualizing their potential. Annu Rev Public Health. 2009;30:273-292.
35. Chalkidou K, Tunis S, Whicher D, Fowler R, Zwarenstein M. The role for pragmatic randomized controlled trials (pRCTs) in comparative effectiveness research. Clinical trials. 2012;9(4):436-446.
36. Kairalla JA, Coffey CS, Thomann MA, Shorr RI, Muller KE. Adaptive designs for comparative effectiveness research trials. Clinical research and regulatory affairs. 2015;32(1):36-44.
37. Glasgow RE, Steiner JF. Comparative effectiveness research to accelerate translation: Recommendatiosn for an emerging field of science. In: Brownson RC, Colditz G, Proctor EK, eds. Dissemination and implementation research in health. New York, NY: Oxford University Press; 2012:72-93.
38. Glasgow RE, Vinson C, Chambers D, Khoury MJ, Kaplan RM, Hunter C. National Institutes of Health approaches to dissemination and implementation science: current and future directions. Am J Public Health. 2012;102(7):1274-1281.
39. Loudon K, Treweek S, Sullivan F, Donnan P, Thorpe KE, Zwarenstein M. The PRECIS-2 tool: designing trials that are fit for purpose. BMJ. 2015;350:h2147.
40. Thorpe KE, Zwarenstein M, Oxman AD, et al. A pragmatic-explanatory continuum indicator summary (PRECIS): a tool to help trial designers. J Clin Epidemiol. 2009;62(5):464-475.
41. Pinnock H, Barwick M, Carpenter CR, et al. Standards for Reporting Implementation Studies (StaRI) Statement. BMJ. 2017;356: 6795.
42. Green LW, Glasgow RE. Evaluating the relevance, generalization, and applicability of research: issues in external validation and translation methodology. Eval Health Prof. 2006;29(1): 126-153.
43. Glasgow RE, Riley WT. Pragmatic measures: what they are and why we need them. Am J Prevent Med. 2013;45(2):237-243.
44. Rabin BA, Purcell EP, Glasgow RE. Harmonizing measures for implementation science using crowd-sourcing. Clin Med Res. 2013;11(3):158.
45. Toobert DJ, Hampson SE, Glasgow RE. The summary of diabetes self-care activities measure: results from 7 studies and a revised scale. Diabetes Care. 2000;23(7):943-950.
46. Glasgow RE, Lichtenstein E, Marcus AC. Why don't we see more translation of health promotion research to practice? Rethinking the efficacy- to-effectiveness transition. Am J Public Health. 2003;93(8):1261-1267.
47. Moore GF, Audrey S, Barker M, et al. Process evaluation of complex interventions: medical Research Council guidance. BMJ. 2015;350:h1258.
48. Craig P, Dieppe P, Macintyre S, Michie S, Nazareth I, Petticrew M. Developing and evaluating complex interventions: New guidance. London: Medical Research Council; 2006.
49. McLeroy KR, Bibeau D, Steckler A, Glanz K. An ecological perspective on health promotion programs. Health Educ Q. 1988;15(4):351-377.
50. Purtle J, Brownson RC, Proctor EK. Infusing science into politics and policy: the importance of legislators as an audience in mental health policy dissemination research. Admin Policy Ment Health. 2017;44(2):160-163.
51. Baquero B, Linnan L, Laraia BA, Ayala GX. Process evaluation of a food marketing and environmental change intervention in Tiendas that serve Latino immigrants in North Carolina. Health Promot Pract. 2014;15(6):839-848.
52. Bamberger M, Rao V, Woolcock M. Using mixed methods in monitoring and evaluation: Experiences from international development evaluation. In: Tashakkori A, Teddlie C, eds. Sage handbook of mixed methods in social and behavioral research. 2nd ed. Thousand Oaks, CA: Sage Publications; 2010:613-642.
53. Tabak RG, Khoong EC, Chambers DA, Brownson RC. Bridging research and practice: models for dissemination and implementation research. Am J Prevent Med. 2012;43(3):337-350.
54. Nilsen P. Making sense of implementation theories, models and frameworks. Implement Sci. 2015;10:53.
55. Damschroder LJ, Aron DC, Keith RE, Kirsh SR, Alexander JA, Lowery JC. Fostering implementation of health services research findings into practice: a consolidated framework for advancing implementation science. Implement Sci. 2009;4:50.
56. Harvey G, Kitson A. PARIHS revisited: from heuristic to integrated framework for the successful implementation of knowledge into practice. Implement Sci. 2016;11:33.
57. Kilbourne AM, Neumann MS, Pincus HA, Bauer MS, Stall R. Implementing evidence-based interventions in health care: application of the replicating effective programs framework. Implement Sci. 2007;2:42.
58. Bartholomew-Eldredge LK, Markham CM, Ruiter RAC, Fernandez ME, Kok G, Parcel GS. Planning health promotion programs: An intervention mapping approach. 4th ed. San Francisco: Jossey-Bass; 2016.
59. Gielen AC, McDonald EM, Gary TL, Bone LR. Using the PRECEDE-PROCEED model to apply health behavior theories. In: Glanz K, Rimer BK, Viswanath K, eds. Health behavior and health education: Theory, research, and practice. San Francisco: Jossey-Bass; 2008:407-433.
60. Weir C, McLeskey N, Brunker C, Brooks D, Supiano MA. The role of information technology in translating educational interventions into practice: an analysis using the PRECEDE/PROCEED model. J Am Med Inform Assoc. 2011;18(6):827-834.
61. Tramm R, McCarthy A, Yates P. Using the Precede-Proceed Model of Health Program Planning in breast cancer nursing research. J Adv Nurs. 2012;68(8):1870-1880.
62. Pocetta G, Votino A, Biribanti A, Rossi A. Recording non communicable chronic diseases at risk behaviours in general practice. a qualitative study using the PRECEDE-PROCEED Model. Ann Ig. 2015;27(3):554-561.
63. Kessler R, Glasgow RE. A proposal to speed translation of healthcare research into practice: dramatic change is needed. Am J Prevent Med. 2011;40(6):637-644.
64. Harden SM, Gaglio B, Shoup JA, et al. Fidelity to and comparative results across behavioral interventions evaluated through the RE-AIM framework: a systematic review. Syst Rev. 2015;4:155.
65. RE- AIM website. www.re-aim.org. Accessed December 21, 2016.
66. Feldstein AC, Glasgow RE. A practical, robust implementation and sustainability model (PRISM) for integrating research findings into practice. Jt Comm J Qual Patient Safe. 2008;34(4):228-243.
67. Liles EG, Schneider JL, Feldstein AC, et al. Implementation challenges and successes of a population-based colorectal cancer screening program: a qualitative study of stakeholder perspectives. Implementat Sci. 2015;10:41.
68. Glasgow RE, Gaglio B, Bennett G, et al. Applying the PRECIS criteria to describe three effectiveness trials of weight loss in obese patients with comorbid conditions. Health Serv Res. 2012;47(3 Pt 1):1051-1067.
69. Johnson KE, Tachibana C, Coronado GD, et al. A guide to research partnerships for pragmatic clinical trials. BMJ. 2014;349:g6826.
70. Sanchez MA, Rabin BA, Gaglio B, et al. A systematic review of eHealth cancer prevention and control interventions: new technology, same methods and designs? Transl Behav Med. 2013;3(4):392-401.
71. Koppenaal T, Linmans J, Knottnerus JA, Spigt M. Pragmatic vs. explanatory: an adaptation of the PRECIS tool helps to judge the applicability of systematic reviews for daily practice. J Clin Epidemiol. 2011;64(10):1095-1101.
72. Craig P, Dieppe P, Macintyre S, et al. Developing and evaluating complex interventions: the new Medical Research Council guidance. BMJ. 2008;337:a1655.
73. Neta G, Glasgow RE, Carpenter CR, et al. A framework for enhancing the value of research for dissemination and implementation. Am J Public Health. 2015;105(1):49-57.
74. Rabin BA, Lewis CC, Norton WE, et al. Measurement resources for dissemination and implementation research in health. Implement Sci. 2016;11:42.
75. Harden SM, McEwan D, Sylvester BD, et al. Understanding for whom, under what conditions, and how group-based physical activity interventions are successful: a realist review. BMC Public Health. 2015;15:958.
76. Brimhall KC, Fenwick K, Farahnak LR, Hurlburt MS, Roesch SC, Aarons GA. Leadership, organizational climate, and perceived burden of evidence-based practice in mental health services. AdminPolicy Ment Health. 2016;43(5):629-639.
77. Ritzwoller DP, Sukhanova A, Gaglio B, Glasgow RE. Costing behavioral interventions: a practical guide to enhance translation. Ann Behav Med. 2009;37(2):218-227.
78. Allicock M, Johnson LS, Leone L, et al. Promoting fruit and vegetable consumption among members of black churches, Michigan and North Carolina, 2008-2010. Prev Chronic Dis. 2013;10:E33.
79. Patton MQ. Qualitative research & evaluation methods: Integrating theory and practice. 4th ed. Thousand Oaks, CA: Sage; 2015.
80. Webster LA, Ekers D, Chew-Graham CA. Feasibility of training practice nurses to deliver a psychosocial intervention within a collaborative care framework for people with depression and long-term conditions. BMC Nurs. 2016;15:71.
81. Huntink E, Wensing M, Timmers IM, van Lieshout J. Process evaluation of a tailored intervention programme of cardiovascular risk management in general practices. Implement Sci. 2016;11(1):164.
82. Proctor EK, Landsverk J, Aarons G, Chambers D, Glisson C, Mittman B. Implementation research in mental health services: an emerging science with conceptual, methodological, and training challenges. Admin Policy Ment Health. 2009;36(1):24-34.
83. Creswell JW, Clark VLP. Designing and conducting mixed methods research. 2nd ed. Thousand Oaks, CA: Sage; 2011.
84. Teddlie C, Tashakkori A. Foundations of mixed methods research: Integrating quantitative and qualitative approaches in the social and behavioral sciences. Thousand Oaks, CA: Sage; 2009.
85. Ritzwoller DP, Toobert D, Sukhanova A, Glasgow RE. Economic analysis of the Mediterranean Lifestyle Program for postmenopausal women with diabetes. Diabetes Educ. 2006;32(5):761-769.
86. Krist AH, Cifuentes M, Dodoo MS, Green LA. Measuring primary care expenses. J Am Board Fam Med. 2010;23(3):376-383.
87. Neumann PJ, Sanders GD, Russell LB, Siegel JE, Ganiats TG, eds. Cost-effectiveness in health and medicine. 2nd ed. Oxford: Oxford University Press; 2017.
88. Khoury MJ, Clauser SB, Freedman AN, et al. Population sciences, translational research, and the opportunities and challenges for genomics to reduce the burden of cancer in the 21st century. Cancer Epidemiol Biomarkers Prev. 2011;20(10):2105-2114.
89. Collins FS, Varmus H. A new initiative on precision medicine. N Engl J Med. 2015;372(9):793-795