Quantitative approaches for the evaluation of implementation research studies

Psychiatry Research

Volumn 283, Issue , 2020, Pages

  Smith, Justin D ^a   Hasan, Mohamed ^a  

^a NORTHWESTERN UNIVERSITY   (United States)

Author keywords

Implementation measurement; Implementation research; Summative evaluation

Indexed keywords

ADOPTION; ARTICLE; CONTROLLED STUDY; HUMAN; IMPLEMENTATION SCIENCE; OUTCOME ASSESSMENT; PRIMARY HEALTH CARE; QUANTITATIVE ANALYSIS; RANDOMIZED CONTROLLED TRIAL; TAXONOMY; DEPRESSION; EVALUATION STUDY; HEALTH CARE DELIVERY; INTERSECTORAL COLLABORATION; MEDICAL RESEARCH; METHODOLOGY; PROCEDURES; PROGRAM EVALUATION; PSYCHOLOGY;

BIOMEDICAL RESEARCH; DELIVERY OF HEALTH CARE; DEPRESSION; EVALUATION STUDIES AS TOPIC; HUMANS; IMPLEMENTATION SCIENCE; INTERSECTORAL COLLABORATION; PRIMARY HEALTH CARE; PROGRAM EVALUATION; RESEARCH DESIGN;

EID: 85071358005     PISSN: 01651781     EISSN: 18727123     Source Type: Journal    
DOI: 10.1016/j.psychres.2019.112521     Document Type: Article

References (57)

1. Aarons, G.A., Mental health provider attitudes toward adoption of evidence-based practice: the Evidence-Based Practice Attitude Scale (EBPAS). Ment. Health Serv. Res. 6:2 (2004), 61–74.
2. Aarons, G.A., Ehrhart, M.G., Farahnak, L.R., The implementation leadership scale (ILS): development of a brief measure of unit level implementation leadership. Implement. Sci., 9, 2014, 10.1186/1748-5908-9-45.
3. Archer, J., Bower, P., Gilbody, S., Lovell, K., Richards, D., Gask, L., Coventry, P., Collaborative care for depression and anxiety problems. Cochrane Database Syst. Rev.(10), 2012, 10.1002/14651858.CD006525.pub2.
4. Armenakis, A.A., Bernerth, J.B., Pitts, J.P., Walker, H.J., Organizational change recipients beliefs scale: development of an assessment instrument. J. Appl. Behav. Sci. 42 (2007), 481–505, 10.1177/0021886307303654.
5. Bauer, M.S., Damschroder, L., Hagedorn, H., Smith, J., Kilbourne, A.M., An introduction to implementation science for the non-specialist. BMC Psychology, 3(1), 2015, 32, 10.1186/s40359-015-0089-9.
6. Bertrand, J.T., Holtgrave, D.R., Gregowski, A., Evaluating HIV/AIDS programs in the US and developing countries. Mayer, K.H., Pizer, H.F., (eds.) HIV Prevention, 2009, Academic Press, San Diego, 571–590.
7. Brown, C.H., Curran, G., Palinkas, L.A., Aarons, G.A., Wells, K.B., Jones, L., Cruden, G., An overview of research and evaluation designs for dissemination and implementation. Annu. Rev. Public Health, 38(1), 2017, 10.1146/annurev-publhealth-031816-044215 null.
8. Brown, C.H., Mohr, D.C., Gallo, C.G., Mader, C., Palinkas, L., Wingood, G., Poduska, J., A computational future for preventing HIV in minority communities: how advanced technology can improve implementation of effective programs. J. Acquir. Immune Defic. Syndr., 63, 2013, 10.1097/QAI.0b013e31829372bd.
9. Brown, C.H., PoVey, C., Hjorth, A., Gallo, C.G., Wilensky, U., Villamar, J., Computational and technical approaches to improve the implementation of prevention programs. Implement. Sci., 10(Suppl 1), 2015, A28, 10.1186/1748-5908-10-S1-A28.
10. Cheung, K., Duan, N., Design of implementation studies for quality improvement programs: an effectiveness–cost-effectiveness framework. Am. J. Public Health 104:1 (2014), e23–e30, 10.2105/ajph.2013.301579.
11. Colditz, G.A., Emmons, K.M., The promise and challenges of dissemination and implementation research. Brownson, R.C., Colditz, G.A., Proctor, E.K., (eds.) Dissemination and Implementation Research in Health: Translating Science to Practice, 2017, Oxford University Press, New York, NY.
12. Curran, G.M., Bauer, M., Mittman, B., Pyne, J.M., Stetler, C., Effectiveness-implementation hybrid designs: combining elements of clinical effectiveness and implementation research to enhance public health impact. Med. Care 50:3 (2012), 217–226, 10.1097/MLR.0b013e3182408812.
13. Department of Health and Human Services. PAR-19-274: Dissemination and Implementation Research in Health (R01 Clinical Trial Optional). 2019 Retrieved from https://grants.nih.gov/grants/guide/pa-files/PAR-19-274.html.
14. Eisman, A.B., Kilbourne, A.M., Dopp, A.R., Saldana, L., Eisenberg, D., Economic evaluation in implementation science: making the business case for implementation strategies. Psychiat. Res., 2019, 10.1016/j.psychres.2019.06.008.
15. Emmons, K.M., Weiner, B., Fernandez, M.E., Tu, S., Systems antecedents for dissemination and implementation: a review and analysis of measures. Health Educ. Behav., 39, 2012, 10.1177/1090198111409748.
16. Gaglio, B., Glasgow, R.E., Evaluation approaches for dissemination and implementation research. Brownson, R., Colditz, G., Proctor, E., (eds.) Dissemination and Implementation Research in Health: Translating Science Into Practice, second ed., 2017, Oxford University Press, New York, 317–334.
17. Gaglio, B., Shoup, J.A., Glasgow, R.E., The re-aim framework: a systematic review of use over time. Am. J. Public Health 103:6 (2013), e38–e46, 10.2105/ajph.2013.301299.
18. Glasgow, R.E., Emmons, K.M., How can we increase translation of research into practice? Types of evidence needed. Annu. Rev. Public Health 28 (2007), 413–433.
19. Glasgow, R.E., Harden, S.M., Gaglio, B., Rabin, B., Smith, M.L., Porter, G.C., Estabrooks, P.A., RE-AIM planning and evaluation framework: adapting to new science and practice with a 20-year review. Front. Public Health, 7(64), 2019, 10.3389/fpubh.2019.00064.
20. Glasgow, R.E., Klesges, L.M., Dzewaltowski, D.A., Estabrooks, P.A., Vogt, T.M., Evaluating the impact of health promotion programs: using the re-aim framework to form summary measures for decision making involving complex issues. Health Educ. Res. 21:5 (2006), 688–694, 10.1093/her/cyl081.
21. Glasgow, R.E., Riley, W.T., Pragmatic measures: what they are and why we need them. Am. J. Prev. Med., 45, 2013, 10.1016/j.amepre.2013.03.010.
22. Glasgow, R.E., Vogt, T.M., Boles, S.M., Evaluating the public health impact of health promotion interventions: the RE-AIM framework. Am. J. Public Health 89:9 (1999), 1322–1327, 10.2105/AJPH.89.9.1322.
23. Grimshaw, J., Campbell, M., Eccles, M., Steen, N., Experimental and quasi-experimental designs for evaluating guideline implementation strategies. Fam. Pract. 17:Suppl 1 (2000), S11–S16, 10.1093/fampra/17.suppl_1.s11.
24. Hamilton, A.B., Finley, E.P., Qualitative methods in implementation research: an introduction. Psychiat. Res., 2019, 112516, 10.1016/j.psychres.2019.112516.
25. Imel, Z.E., Pace, B.T., Soma, C.S., Tanana, M., Hirsch, T., Gibson, J., Atkins, D.C., Design feasibility of an automated, machine-learning based feedback system for motivational interviewing. Psychotherapy 56:2 (2019), 318–328, 10.1037/pst0000221.
26. Institute of Medicine Committee on Crossing the Quality Chasm. Adaption to Mental Health and Addictive Disorder: Improving the Quality of Health Care for Mental and Substanceuse Conditions. 2006 Retrieved from Washington, D.C.
27. Jacob, V., Chattopadhyay, S.K., Sipe, T.A., Thota, A.B., Byard, G.J., Chapman, D.P., Economics of collaborative care for management of depressive disorders: a community guide systematic review. Am. J. Prev. Med. 42:5 (2012), 539–549, 10.1016/j.amepre.2012.01.011.
28. Jordan, N., Graham, A.K., Berkel, C., Smith, J.D., Budget impact analysis of preparing to implement the family check-up 4 health in primary care to reduce pediatric obesity. Prev. Sci. 20:5 (2019), 655–664, 10.1007/s11121-018-0970-x.
29. Kirchner, J.E., Smith, J.L., Powell, B.J., Waltz, T.J., Proctor, E.K., Getting a clinical innovation into practice: an introduction to implementation strategies. Psychiat. Res., 2019, 10.1016/j.psychres.2019.06.042.
30. Kroenke, K., Spitzer, R., Williams, J.W., The PHQ-9. J. Gen. Intern. Med. 16:9 (2001), 606–613, 10.1046/j.1525-1497.2001.016009606.x.
31. Landsverk, J., Brown, C.H., Smith, J.D., Chamberlain, P., Palinkas, L.A., Ogihara, M., Horwitz, S.M., Design and analysis in dissemination and implementation research. Brownson, R.C., Colditz, G.A., Proctor, E.K., (eds.) Dissemination and Implementation Research in Health: Translating Research to Practice, second ed., 2017, Oxford University Press, New York, 201–227.
32. Lewis, C.C., Fischer, S., Weiner, B.J., Stanick, C., Kim, M., Martinez, R.G., Outcomes for implementation science: an enhanced systematic review of instruments using evidence-based rating criteria. Implement. Sci., 10(1), 2015, 155, 10.1186/s13012-015-0342-x.
33. Lewis, C.C., Proctor, E.K., Brownson, R.C., Measurement issues in dissemination and implementation research. Brownson, R.C., Colditz, G.A., Proctor, E.K., (eds.) Dissemination and Implementation Research in Health: Translating Research to Practice, second ed., 2017, Oxford University Press, New York, 229–244.
34. Lewis, C.C., Stanick, C.F., Martinez, R.G., Weiner, B.J., Kim, M., Barwick, M., Comtois, K.A., The society for implementation research collaboration instrument review project: a methodology to promote rigorous evaluation. Implement. Sci., 10(1), 2015, 2, 10.1186/s13012-014-0193-x.
35. Luke, D.A., Calhoun, A., Robichaux, C.B., Elliott, M.B., Moreland-Russell, S., The Program Sustainability Assessment Tool: a new instrument for public health programs. Prev. Chronic Dis., 11, 2014, E12, 10.5888/pcd11.130184.
36. Luke, D.A., Malone, S., Prewitt, K., Hackett, R., Lin, J., The Clinical Sustainability Assessment Tool (CSAT): assessing sustainability in clinical medicine settings. Conference on the Science of Dissemination and Implementation in Health, Washington, DC, 2018.
37. Martinez, R.G., Lewis, C.C., Weiner, B.J., Instrumentation issues in implementation science. Implement. Sci., 9, 2014, 10.1186/s13012-014-0118-8.
38. Powell, B.J., Stanick, C.F., Halko, H.M., Dorsey, C.N., Weiner, B.J., Barwick, M.A., Lewis, C.C., Toward criteria for pragmatic measurement in implementation research and practice: a stakeholder-driven approach using concept mapping. Implement. Sci., 12(1), 2017, 118, 10.1186/s13012-017-0649-x.
39. Powell, B.J., Waltz, T.J., Chinman, M.J., Damschroder, L.J., Smith, J.L., Matthieu, M.M., Kirchner, J.E., A refined compilation of implementation strategies: results from the expert recommendations for implementing change (ERIC) project. Implement. Sci., 10, 2015, 10.1186/s13012-015-0209-1.
40. Proctor, E., Silmere, H., Raghavan, R., Hovmand, P., Aarons, G., Bunger, A., Hensley, M., Outcomes for implementation research: conceptual distinctions, measurement challenges, and research agenda. Adm. Policy Ment. Health Ment. Health Serv. Res., 38, 2011, 10.1007/s10488-010-0319-7.
41. Rabin, B.A., Lewis, C.C., Norton, W.E., Neta, G., Chambers, D., Tobin, J.N., Glasgow, R.E., Measurement resources for dissemination and implementation research in health. Implement. Sci., 11(1), 2016, 42, 10.1186/s13012-016-0401-y.
42. Raghavan, R., The role of economic evaluation in dissemination and implementation research. Brownson, R.C., Colditz, G.A., Proctor, E.K., (eds.) Dissemination and Implementation Research in Health: Translating Science to Practice, second ed., 2017, Oxford University Press, New York.
43. Ritzwoller, D.P., Sukhanova, A., Gaglio, B., Glasgow, R.E., Costing behavioral interventions: a practical guide to enhance translation. Ann. Behav. Med. 37:2 (2009), 218–227.
44. Saldana, L., Schaper, H., Campbell, M., Chapman, J., Standardized measurement of implementation: the universal SIC. Implement. Sci., 10(1), 2015, A73, 10.1186/1748-5908-10-s1-a73.
45. Schoenwald, S., Garland, A., Chapman, J., Frazier, S., Sheidow, A., Southam-Gerow, M., Toward the effective and efficient measurement of implementation fidelity. Admin. Policy Ment. Health Ment. Health Serv. Res., 38, 2011, 10.1007/s10488-010-0321-0.
46. Smith, J.D., An implementation research logic model: a step toward improving scientific rigor, transparency, reproducibility, and specification. Implement. Sci., 14(Supp 1), 2018, S39.
47. Smith, J.D., Berkel, C., Jordan, N., Atkins, D.C., Narayanan, S.S., Gallo, C., Bruening, M.M., An individually tailored family-centered intervention for pediatric obesity in primary care: study protocol of a randomized type II hybrid implementation-effectiveness trial (Raising healthy children study). Implement. Sci. 13:11 (2018), 1–15, 10.1186/s13012-017-0697-2.
48. Smith, J.D., Li, D.H., Hirschhorn, L.R., Gallo, C., McNulty, M., Phillips, G.I., …Benbow, N.D. (2019). Landscape of HIV implementation research funded by the national institutes of health: a mapping review of project abstracts(submitted for publication).
49. Smith, J.D., Rafferty, M.R., Heinemann, A.W., Meachum, M.K., Villamar, J.A., Lieber, R.L., & Brown, C.H. (under review). Evaluation of the factor structure of implementation research measures adapted for a novel context and multiple professional roles.
50. Stetler, C.B., Legro, M.W., Wallace, C.M., Bowman, C., Guihan, M., Hagedorn, H., Smith, J.L., The role of formative evaluation in implementation research and the Queri experience. J. Gen. Intern. Med., 21(2), 2006, S1, 10.1007/s11606-006-0267-9.
51. Tabak, R.G., Khoong, E.C., Chambers, D.A., Brownson, R.C., Bridging research and practice: models for dissemination and implementation research. Am. J. Prev. Med. 43:3 (2012), 337–350.
52. Foundation, The Health, Evaluation: What to Consider. Commonly Asked Questions About How to Approach Evaluation of Quality Improvement in Health Care. 2015 Retrieved from London, England https://www.health.org.uk/sites/default/files/EvaluationWhatToConsider.pdf.
53. Unützer, J., Katon, W., Callahan, C.M., Williams, J., John, W., Hunkeler, E., Harpole, L., IMPACT Investigators. Improving Mood-Promoting Access to Collaborative Treatment. Collaborative care management of late-life depression in the primary care setting: a randomized controlled trial. JAMA 288:22 (2002), 2836–2845, 10.1001/jama.288.22.2836.
54. Upton, D., Upton, P., Development of an evidence‐based practice questionnaire for nurses. J. Adv. Nurs. 53:4 (2006), 454–458.
55. Wang, D., Ogihara, M., Gallo, C., Villamar, J.A., Smith, J.D., Vermeer, W., Brown, C.H., Automatic classification of communication logs into implementation stages via text analysis. Implement. Sci. 11:1 (2016), 1–14, 10.1186/s13012-016-0483-6.
56. Wyman, P.A., Henry, D., Knoblauch, S., Brown, C.H., Designs for testing group-based interventions with limited numbers of social units: the dynamic wait-listed and regression point displacement designs. Prev. Sci. 16:7 (2015), 956–966, 10.1007/s11121-014-0535-6.
57. Xiao, B., Imel, Z.E., Georgiou, P.G., Atkins, D.C., Narayanan, S.S., "Rate my therapist": automated detection of empathy in drug and alcohol counseling via speech and language processing. PLoS One, 10(12), 2015, e0143055, 10.1371/journal.pone.0143055.