Experimental and quasi-experimental designs in implementation research

Psychiatry Research

Volumn 283, Issue , 2020, Pages

  Miller, Christopher J ^a,b   Smith, Shawna N ^c,d   Pugatch, Marianne ^a  

^a VA BOSTON HEALTHCARE SYSTEM   (United States)

^b HARVARD MEDICAL SCHOOL   (United States)

^c UNIVERSITY OF MICHIGAN MEDICAL SCHOOL   (United States)

^d UNIVERSITY OF MICHIGAN   (United States)

Author keywords

Implementation; Interrupted time series; Pre post with non equivalent control group; Quasi experimental; SMART design; Stepped wedge

Indexed keywords

ADOPTION; ADULT; ARTICLE; CLINICAL PRACTICE; CONTROLLED STUDY; DRUG EFFICACY; EXPERIMENTAL DESIGN; HUMAN; IMPLEMENTATION SCIENCE; RANDOMIZATION; RANDOMIZED CONTROLLED TRIAL (TOPIC); SCIENTIST; TIME SERIES ANALYSIS; CONTROL GROUP; MEDICAL RESEARCH; METHODOLOGY; PROCEDURES;

BIOMEDICAL RESEARCH; CONTROL GROUPS; HUMANS; IMPLEMENTATION SCIENCE; RANDOMIZED CONTROLLED TRIALS AS TOPIC; RESEARCH DESIGN;

EID: 85067958738     PISSN: 01651781     EISSN: 18727123     Source Type: Journal    
DOI: 10.1016/j.psychres.2019.06.027     Document Type: Article

References (65)

1. Almirall, D., Compton, S.N., Gunlicks-Stoessel, M., Duan, N., Murphy, S.A., Designing a pilot sequential multiple assignment randomized trial for developing an adaptive treatment strategy. Stat. Med. 31:17 (2012), 1887–1902.
2. Bauer, M.S., McBride, L., Williford, W.O., Glick, H., Kinosian, B., Altshuler, L., Beresford, T., Kilbourne, A.M., Sajatovic, M., Cooperative Studies Program 430 Study, T. Collaborative care for bipolar disorder: part I. Impact on clinical outcome, function, and costs. Psychiatr. Serv. 57:7 (2006), 937–945.
3. Bauer, M.S., Miller, C., Kim, B., Lew, R., Weaver, K., Coldwell, C., Henderson, K., Holmes, S., Seibert, M.N., Stolzmann, K., Elwy, A.R., Kirchner, J., Partnering with health system operations leadership to develop a controlled implementation trial. Implement Sci., 11(1), 2016, 22.
4. Bauer, M.S., Miller, C.J., Kim, B., Lew, R., Stolzmann, K., Sullivan, J., Riendeau, R., Pitcock, J., Williamson, A., Connolly, S., Elwy, A.R., Weaver, K., Effectiveness of implementing a collaborative chronic care model for clinician teams on patient outcomes and health status in mental Health: a randomized clinical trial. JAMA Netw. Open, 2(3), 2019, e190230.
5. Bernal, J.L., Cummins, S., Gasparrini, A., Interrupted time series regression for the evaluation of public health interventions: a tutorial. Int. J. Epidemiol. 46:1 (2017), 348–355.
6. Betran, A.P., Bergel, E., Griffin, S., Melo, A., Nguyen, M.H., Carbonell, A., Mondlane, S., Merialdi, M., Temmerman, M., Gulmezoglu, A.M., Provision of medical supply kits to improve quality of antenatal care in Mozambique: a stepped-wedge cluster randomised trial. Lancet Glob. Health 6:1 (2018), e57–e65.
7. Bhatt, D.L., Mehta, C., Adaptive designs for clinical trials. N. Engl. J. Med. 375:1 (2016), 65–74.
8. Bodenheimer, T., Wagner, E.H., Grumbach, K., Improving primary care for patients with chronic illness. JAMA 288:14 (2002), 1775–1779.
9. Bodenheimer, T., Wagner, E.H., Grumbach, K., Improving primary care for patients with chronic illness: the chronic care model, Part 2. JAMA 288:15 (2002), 1909–1914.
10. Brown, C.A., Lilford, R.J., The stepped wedge trial design: a systematic review. BMC Med. Res. Methodol., 6(1), 2006, 54.
11. Byiers, B.J., Reichle, J., Symons, F.J., Single-subject experimental design for evidence-based practice. Am. J. Speech Lang. Pathol. 21:4 (2012), 397–414.
12. Cheung, Y.K., Chakraborty, B., Davidson, K.W., Sequential multiple assignment randomized trial (SMART) with adaptive randomization for quality improvement in depression treatment program. Biometrics 71:2 (2015), 450–459.
13. Collins, L.M., Dziak, J.J., Kugler, K.C., Trail, J.B., Factorial experiments: efficient tools for evaluation of intervention components. Am. J. Prev. Med. 47:4 (2014), 498–504.
14. Collins, L.M., Dziak, J.J., Li, R., Design of experiments with multiple independent variables: a resource management perspective on complete and reduced factorial designs. Psychol. Methods 14:3 (2009), 202–224.
15. Collins, L.M., Murphy, S.A., Nair, V.N., Strecher, V.J., A strategy for optimizing and evaluating behavioral interventions. Ann. Behav. Med. 30:1 (2005), 65–73.
16. Collins, L.M., Murphy, S.A., Strecher, V., The multiphase optimization strategy (MOST) and the sequential multiple assignment randomized trial (SMART): new methods for more potent eHealth interventions. Am. J. Prev. Med. 32:5 Suppl (2007), S112–S118.
17. Collins, L.M., Nahum-Shani, I., Almirall, D., Optimization of behavioral dynamic treatment regimens based on the sequential, multiple assignment, randomized trial (SMART). Clin. Trials 11:4 (2014), 426–434.
18. Coulton, S., Perryman, K., Bland, M., Cassidy, P., Crawford, M., Deluca, P., Drummond, C., Gilvarry, E., Godfrey, C., Heather, N., Kaner, E., Myles, J., Newbury-Birch, D., Oyefeso, A., Parrott, S., Phillips, T., Shenker, D., Shepherd, J., Screening and brief interventions for hazardous alcohol use in accident and emergency departments: a randomised controlled trial protocol. BMC Health Serv. Res., 9(1), 2009, 114.
19. Cousins, K., Connor, J.L., Kypri, K., Effects of the campus watch intervention on alcohol consumption and related harm in a university population. Drug Alcohol Depend. 143 (2014), 120–126.
20. Curran, G.M., Bauer, M., Mittman, B., Pyne, J.M., Stetler, C., Effectiveness-implementation hybrid designs: combining elements of clinical effectiveness and implementation research to enhance public health impact. Med. Care 50:3 (2012), 217–226.
21. Davey, C., Hargreaves, J., Thompson, J.A., Copas, A.J., Beard, E., Lewis, J.J., Fielding, K.L., Analysis and reporting of stepped wedge randomised controlled trials: synthesis and critical appraisal of published studies, 2010 to 2014. Trials, 16(1), 2015, 358.
22. Dimick, J.B., Ryan, A.M., Methods for evaluating changes in health care policy: the difference-in-differences approach. JAMA 312:22 (2014), 2401–2402.
23. Eccles, M., Grimshaw, J., Campbell, M., Ramsay, C., Research designs for studies evaluating the effectiveness of change and improvement strategies. Qual. Saf. Health Care 12:1 (2003), 47–52.
24. Fisher, R.A., Theory of statistical estimation. Mathematical Proceedings of the Cambridge Philosophical Society, 22, 1925, July, Cambridge University Press, 700–725.
25. Fisher, R.A., The Design of Experiments. 1935, Oliver and Boyd, Edinburgh.
26. Fretheim, A., Soumerai, S.B., Zhang, F., Oxman, A.D., Ross-Degnan, D., Interrupted time-series analysis yielded an effect estimate concordant with the cluster-randomized controlled trial result. J. Clin. Epidemiol. 66:8 (2013), 883–887.
27. Fretheim, A., Zhang, F., Ross-Degnan, D., Oxman, A.D., Cheyne, H., Foy, R., Goodacre, S., Herrin, J., Kerse, N., McKinlay, R.J., Wright, A., Soumerai, S.B., A reanalysis of cluster randomized trials showed interrupted time-series studies were valuable in health system evaluation. J. Clin. Epidemiol. 68:3 (2015), 324–333.
28. Gaglio, B., Shoup, J.A., Glasgow, R.E., The RE-AIM framework: a systematic review of use over time. Am. J. Public Health 103:6 (2013), e38–e46.
29. Gebski, V., Ellingson, K., Edwards, J., Jernigan, J., Kleinbaum, D., Modelling interrupted time series to evaluate prevention and control of infection in healthcare. Epidemiol. Infect. 140:12 (2012), 2131–2141.
30. Glasgow, R.E., Vogt, T.M., Boles, S.M., Evaluating the public health impact of health promotion interventions: the RE-AIM framework. Am. J. Public Health 89:9 (1999), 1322–1327.
31. Hanani, H., The existence and construction of balanced incomplete block designs. Ann. Math. Stat. 32:2 (1961), 361–386.
32. Hanbury, A., Farley, K., Thompson, C., Wilson, P.M., Chambers, D., Holmes, H., Immediate versus sustained effects: interrupted time series analysis of a tailored intervention. Implement Sci, 8(1), 2013, 130.
33. Handley, M.A., Lyles, C.R., McCulloch, C., Cattamanchi, A., Selecting and improving quasi-experimental designs in effectiveness and implementation research. Annu. Rev. Public Health 39:1 (2018), 5–25.
34. Hussey, M.A., Hughes, J.P., Design and analysis of stepped wedge cluster randomized trials. Contemp. Clin. Trials 28:2 (2007), 182–191.
35. Kilbourne, A.M., Almirall, D., Eisenberg, D., Waxmonsky, J., Goodrich, D.E., Fortney, J.C., Kirchner, J.E., Solberg, L.I., Main, D., Bauer, M.S., Kyle, J., Murphy, S.A., Nord, K.M., Thomas, M.R., Protocol: adaptive implementation of effective programs trial (ADEPT): cluster randomized SMART trial comparing a standard versus enhanced implementation strategy to improve outcomes of a mood disorders program. Implement Sci., 9(1), 2014, 132.
36. Kilbourne, A.M., Almirall, D., Goodrich, D.E., Lai, Z., Abraham, K.M., Nord, K.M., Bowersox, N.W., Enhancing outreach for persons with serious mental illness: 12-month results from a cluster randomized trial of an adaptive implementation strategy. Implement Sci., 9(1), 2014, 163.
37. Kilbourne, A.M., Bramlet, M., Barbaresso, M.M., Nord, K.M., Goodrich, D.E., Lai, Z., Post, E.P., Almirall, D., Verchinina, L., Duffy, S.A., Bauer, M.S., SMI life goals: description of a randomized trial of a collaborative care model to improve outcomes for persons with serious mental illness. Contemp. Clin. Trials 39:1 (2014), 74–85.
38. Kilbourne, A.M., Goodrich, D.E., Lai, Z., Clogston, J., Waxmonsky, J., Bauer, M.S., Life goals collaborative care for patients with bipolar disorder and cardiovascular disease risk. Psychiatr. Serv. 63:12 (2012), 1234–1238.
39. Kilbourne, A.M., Goodrich, D.E., Nord, K.M., Van Poppelen, C., Kyle, J., Bauer, M.S., Waxmonsky, J.A., Lai, Z., Kim, H.M., Eisenberg, D., Thomas, M.R., Long-term clinical outcomes from a randomized controlled trial of two implementation strategies to promote collaborative care attendance in community practices. Adm. Policy Ment. Health 42:5 (2015), 642–653.
40. Kilbourne, A.M., Neumann, M.S., Pincus, H.A., Bauer, M.S., Stall, R., Implementing evidence-based interventions in health care: application of the replicating effective programs framework. Implement Sci., 2(1), 2007, 42.
41. Kilbourne, A.M., Neumann, M.S., Waxmonsky, J., Bauer, M.S., Kim, H.M., Pincus, H.A., Thomas, M., Public-academic partnerships: evidence-based implementation: the role of sustained community-based practice and research partnerships. Psychiatr. Serv. 63:3 (2012), 205–207.
42. Kilbourne, A.M., Post, E.P., Nossek, A., Drill, L., Cooley, S., Bauer, M.S., Improving medical and psychiatric outcomes among individuals with bipolar disorder: a randomized controlled trial. Psychiatr. Serv. 59:7 (2008), 760–768.
43. Kirchner, J.E., Ritchie, M.J., Pitcock, J.A., Parker, L.E., Curran, G.M., Fortney, J.C., Outcomes of a partnered facilitation strategy to implement primary care-mental health. J. Gen. Intern. Med. 29:Suppl 4 (2014), 904–912.
44. Lei, H., Nahum-Shani, I., Lynch, K., Oslin, D., Murphy, S.A., A "SMART" design for building individualized treatment sequences. Annu. Rev. Clin. Psychol. 8 (2012), 21–48.
45. Lew, R.A., Miller, C.J., Kim, B., Wu, H., Stolzmann, K., Bauer, M.S., A robust method to reduce imbalance for site-level randomized controlled implementation trial designs. Implement. Sci., 14, 2019, 46.
46. Morgan, C.J., Reducing bias using propensity score matching. J. Nucl. Cardiol. 25:2 (2018), 404–406.
47. Morton, V., Torgerson, D.J., Effect of regression to the mean on decision making in health care. BMJ 326:7398 (2003), 1083–1084.
48. Nahum-Shani, I., Qian, M., Almirall, D., Pelham, W.E., Gnagy, B., Fabiano, G.A., Waxmonsky, J.G., Yu, J., Murphy, S.A., Experimental design and primary data analysis methods for comparing adaptive interventions. Psychol. Methods 17:4 (2012), 457–477.
49. NeCamp, T., Kilbourne, A., Almirall, D., Comparing cluster-level dynamic treatment regimens using sequential, multiple assignment, randomized trials: regression estimation and sample size considerations. Stat. Methods Med. Res. 26:4 (2017), 1572–1589.
50. Neumann, M.S., Sogolow, E.D., Replicating effective programs: HIV/AIDS prevention technology transfer. AIDS Educ. Prev. 12:5 Suppl (2000), 35–48.
51. Pallmann, P., Bedding, A.W., Choodari-Oskooei, B., Dimairo, M., Flight, L., Hampson, L.V., Holmes, J., Mander, A.P., Odondi, L.o., Sydes, M.R., Villar, S.S., Wason, J.M.S., Weir, C.J., Wheeler, G.M., Yap, C., Jaki, T., Adaptive designs in clinical trials: why use them, and how to run and report them. BMC Med., 16(1), 2018, 29.
52. Pape, U.J., Millett, C., Lee, J.T., Car, J., Majeed, A., Disentangling secular trends and policy impacts in health studies: use of interrupted time series analysis. J. R. Soc. Med. 106:4 (2013), 124–129.
53. Pellegrini, C.A., Hoffman, S.A., Collins, L.M., Spring, B., Optimization of remotely delivered intensive lifestyle treatment for obesity using the multiphase optimization Strategy: opt-IN study protocol. Contemp. Clin. Trials 38:2 (2014), 251–259.
54. Penfold, R.B., Zhang, F., Use of interrupted time series analysis in evaluating health care quality improvements. Acad. Pediatr. 13:6, Supplement (2013), S38–S44.
55. Pridemore, W.A., Snowden, A.J., Reduction in suicide mortality following a new national alcohol policy in Slovenia: an interrupted time-series analysis. Am. J. Public Health 99:5 (2009), 915–920.
56. Proctor, E., Silmere, H., Raghavan, R., Hovmand, P., Aarons, G., Bunger, A., Griffey, R., Hensley, M., Outcomes for implementation research: conceptual distinctions, measurement challenges, and research agenda. Adm. Policy Ment. Health 38:2 (2011), 65–76.
57. Robson, D., Spaducci, G., McNeill, A., Stewart, D., Craig, T.J.K., Yates, M., Szatkowski, L., Effect of implementation of a smoke-free policy on physical violence in a psychiatric inpatient setting: an interrupted time series analysis. Lancet Psychiatry 4:7 (2017), 540–546.
58. Schildcrout, J.S., Schisterman, E.F., Mercaldo, N.D., Rathouz, P.J., Heagerty, P.J., Extending the case-control design to longitudinal data: stratified sampling based on repeated binary outcomes. Epidemiology 29:1 (2018), 67–75.
59. Shadish, W.R., Cook, T.D., Campbell, D.T., Experimental and Quasi-Experimental Designs For Generalized Causal Inference. 2002, Houghton Miffflin Company, Boston, MA.
60. Simon, G.E., Ludman, E.J., Bauer, M.S., Unutzer, J., Operskalski, B., Long-term effectiveness and cost of a systematic care program for bipolar disorder. Arch. Gen. Psychiatry 63:5 (2006), 500–508.
61. Stetler, C.B., Legro, M.W., Rycroft-Malone, J., Bowman, C., Curran, G., Guihan, M., Hagedorn, H., Pineros, S., Wallace, C.M., Role of "external facilitation" in implementation of research findings: a qualitative evaluation of facilitation experiences in the veterans health administration. Implement. Sci., 1, 2006, 23.
62. Taljaard, M., McKenzie, J.E., Ramsay, C.R., Grimshaw, J.M., The use of segmented regression in analysing interrupted time series studies: an example in pre-hospital ambulance care. Implement. Sci., 9(1), 2014, 77.
63. Wagner, A.K., Soumerai, S.B., Zhang, F., Ross-Degnan, D., Segmented regression analysis of interrupted time series studies in medication use research. J. Clin. Pharm Ther. 27:4 (2002), 299–309.
64. Wagner, E.H., Austin, B.T., Von Korff, M., Organizing care for patients with chronic illness. Milbank Q. 74:4 (1996), 511–544.
65. Wyrick, D.L., Rulison, K.L., Fearnow-Kenney, M., Milroy, J.J., Collins, L.M., Moving beyond the treatment package approach to developing behavioral interventions: addressing questions that arose during an application of the multiphase optimization strategy (MOST). Transl. Behav. Med. 4:3 (2014), 252–259.