How to study improvement interventions: A brief overview of possible study types

Postgraduate Medical Journal

Volumn 91, Issue 1076, 2015, Pages 343-354

  Portela, Margareth Crisóstomo ^a,b   Pronovost, Peter J ^c   Woodcock, Thomas ^d   Carter, Pam ^a   Dixon Woods, Mary ^a  

^a UNIVERSITY OF LEICESTER   (United Kingdom)

^b INSTITUTO OSWALDO CRUZ   (Brazil)

^c JOHNS HOPKINS BLOOMBERG SCHOOL OF PUBLIC HEALTH   (United States)

^d CHELSEA AND WESTMINSTER HOSPITAL   (United Kingdom)

Author keywords

[No Author keywords available]

Indexed keywords

ARTICLE; ECONOMIC EVALUATION; HEALTH CARE ORGANIZATION; HEALTH SERVICE; HUMAN; INTERVENTION STUDY; PROGRAM EVALUATION; QUALITATIVE RESEARCH; QUASI-EXPERIMENTAL DESIGN; RANDOMIZED CONTROLLED TRIAL (TOPIC); STUDY DESIGN; SYSTEMATIC REVIEW (TOPIC); TOTAL QUALITY MANAGEMENT;

EID: 84930749606     PISSN: 00325473     EISSN: 14690756     Source Type: Journal    
DOI: 10.1136/postgradmed-2014-003620rep     Document Type: Article

References (114)

1. Djulbegovic B. A framework to bridge the gaps between evidence-basedMedicine, Health Outcomes, and Improvement and Implementation Science. J Oncol Pract 2014;10:200-2.
2. Margolis P, Provost LP, Schoettker PJ, et al. Quality improvement, Clinical Research, and Quality Improvement Research: opportunities for integration. Pediatr Clin N Am 2009;56:831-41.
3. Bryman A. Getting started: reviewing the literature. Social research methods. 4th edn. Oxford University Press 2012:97-128.
4. Berwick DM. The science of improvement. JAMA 2008;299:1182-4.
5. Batalden P, Davidoff F, Marshall M, et al. So what? Now what? Exploring, understanding and using the epistemologies that inform the improvement of healthcare. BMJ Qual Saf 2011;20(Suppl 1):i99-105.
6. Dixon-Woods M, Amalberti R, Goodman S, et al. Problems and promises of innovation: why healthcare needs to rethink it love/hate relationship with the new. BMJ Qual Saf 2011;20 (Suppl 1):i47-51.
7. Øvretveit J, Leviton L, Parry G. Increasing the generalizability of improvement research with an improvement replication programme. BMJ Qual Saf 2011;20(Suppl 1):i87-91.
8. Perla RJ, Parry GJ. The epistemology of quality improvement: it’s all Greek. BMJ Qual Saf 2011;20(Suppl 1):i24-7.
9. Campbell NC, Murray E, Darbyshire J, et al. Designing and evaluating complex interventions to improve health care. BMJ 2007;334:455-9.
10. Craig P, Dieppe P, Macintyre S, et al. Developing and evaluating complex interventions: the new Medical Research Council guidance. BMJ 2008;337:979-83.
11. Michie S, Abraham C, Eccles MP, et al. Strengthening evaluation and implementation by specifying components of behaviour change interventions: a study protocol. Implement Sci 2011;6:10.
12. Damschroder LJ, Aron DC, Keith RE, et al. Fostering implementation of health services research findings into practice: a consolidated framework for advancing implementation science. Implement Sci 2009;4:50.
13. Øvretveit J. Understanding the conditions for improvement: research to discover which influences affect improvement success. BMJ Qual Saf 2011;20(Suppl 1):i18-23.
14. Kaplan HC, Provost LP, Froehle CM, et al. The model for understanding success in quality (MUSIQ): building a theory of context in healthcare quality improvement. BMJ Qual Saf 2012;21:13-20.
15. Eccles MP, Grimshaw JM, MacLennan G, et al. Explaining clinical behaviors using multiple theoretical models. Implement Sci 2012;7:99.
16. Glanz K, Bishop DB. The role of behavioral science theory in development and implementation of public health interventions. Annu Rev Public Health 2010;31:399-418.
17. Kaplan HC, Brady PW, Dritz MC, et al. The influence of context on quality improvement success in health care: a systematic review of the literature. Milbank Q 2010;88:500-59.
18. Novotná G, Dobbins M, Henderson J. Institutionalization of evidence-informed practices in healthcare settings. Implement Sci 2012;7:112.
19. Dearing JW. Applying diffusion of innovation theory to intervention development. Res Soc Work Pract 2009;19:503-18.
20. May C. Towards a general theory of implementation. Implement Sci 2013;8:18.
21. Kitson A, Harvey G, McCormack B. Enabling the implementation of evidence based practice: a conceptual framework. Qual Health Care 1998;7:149-58.
22. Harvey G, Loftus-Hills A, Rycroft-Malone J, et al. Getting evidence into practice: the role and function of facilitation. J Adv Nurs 2002;37:577-88.
23. McCormack B, Kitson A, Harvey G, et al. Getting evidence into practice: the meaning of ‘context’. J Adv Nurs 2002;38:94-104.
24. Rycroft-Malone J, Kitson A, Harvey G, et al. Ingredients for change: revisiting a conceptual framework. Qual Saf Health Care 2002;11:174-80.
25. Rycroft-Malone J, Harvey G, Seers K, et al. An exploration of the factors that influence the implementation of evidence into practice. J Clin Nurs 2004;13:913-24.
26. Kitson AL, Rycroft-Malone J, Harvey G, et al. Evaluating the successful implementation of evidence into practice using the PARiHS framework: theoretical and practical challenges. Implement Sci 2008;3:1.
27. Helfrich CD, Damschroder LJ, Hagedom HJ, et al. A critical synthesis of literature on the promoting action on research implementation in health services (PARIHS) framework. Implement Sci 2010;5:82.
28. Stetler CB, Damschroder LJ, Helfrich CD, et al. A Guide for applying a revised version of the PARIHS framework for implementation. Implement Sci 2011;6:99.
29. Michie S, Johnston M, Abraham C, et al. Making psychological theory useful for implementing evidence based practice: a consensus approach. Qual Saf Health Care 2005;14:26-33.
30. Cane J, O’Connor D, Michie S. Validation of the theoretical domains framework for use in behaviour change and implementation research. Implement Sci 2012;7:37.
31. Francis JJ, O’Connor D, Curran J. Theories of behaviour change synthesised into a set of theoretical groupings: introducing a thematic series on the theoretical domains framework. Implement Sci 2012;7:35.
32. French SD, Green SE, O’Connor DA, et al. Developing theory-informed behaviour change interventions to implement evidence into practice: a systematic approach using the Theoretical Domains Framework. Implement Sci 2012;7:38.
33. Légaré F, Borduas F, Jacques A, et al. Developing a theory-based instrument to assess the impact of continuing professional development activities on clinical practice: a study protocol. Implement Sci 2011;6:17.
34. Gagnon MP, Labarthe J, Légaré F, et al. Measuring organizational readiness for knowledge translation in chronic care. Implement Sci 2011;6:72.
35. Marshall M, Pronovost P, Dixon-Woods M. Promotion of improvement as a science. Lancet 2013;381:419-21.
36. Deming WE. The new economics for Industry, Government, Education. 2nd edn. Boston, MA: MIT Press; 1994.
37. Perla RJ, Provost LP, Murray SK. The run chart: a simple analytical tool for learning from variation in healthcare processes. BMJ Qual Saf 2011;20:46-51.
38. Provost LP. Analytical studies: a framework for quality improvement design and analysis. BMJ Qual Saf 2011;20 (Suppl 1):i92-6.
39. Radnor ZJ, Holweg M, Waring J. Lean in healthcare: the unfilled promise? Soc Sci Med 2012;74:364-71.
40. Wears RL, Hunte GS. Seeing patient safety ‘Like a State’. Saf Sci 2014;67:50-7.
41. Fitzgerald L, McGivern G, Dopson S, et al. Making wicked problems governable: the case of managed networks in health care. Oxford University Press, 2013.
42. Bishop JP, Perry JE, Hine A. Efficient, compassionate, and fractured: contemporary care in the ICU. Hastings Cent Rep 2014;44:35-43.
43. Langley GJ, Moen R, Nolan KM, et al. The improvement guide: a practical approach to enhancing organizational performance. 2nd edn. San Francisco, CA: Jossey-Bass Publishers, 2009.
44. Perla RJ, Provost LP, Parry GJ. Seven propositions of the Science of Improvement: exploring foundations. Q Manage Health Care 2013;22:179-86.
45. Thor J, Lundberg J, Ask J, et al. Application of statistical process control in healthcare improvement: systematic review. Qual Saf Health Care 2007;16:387-99.
46. Shewhart WA. Economic control of quality of manufactured product. New York: D. Van Nostrand Company, Inc, 1931.
47. Benneyan JC, Lloyd RC, Plsek PE. Statistical process control as a tool for research and healthcare improvement. Qual Saf Health Care 2003;12:458-64.
48. Montgomery DC. Introduction to statistical quality control. John Wiley & Sons, 2007.
49. Provost LP, Murray SK. The health care data guide: Learning from data for improvement. San Francisco, CA: Jossey-Bass, 2011.
50. Mohammed MA. Using statistical process control to improve the quality of health care. Qual Saf Health Care 2004;13:243-5.
51. Mohammed MA, Worthington P, Woodall WH. Plotting basic control charts: tutorial notes for healthcare practitioners. Qual Saf Health Care 2008;17:137-45.
52. Pinto A, Burnett S, Benn J, et al. Improving reliability of clinical care practices for ventilated patients in the context of a patient safety improvement initiative. J Eval Clin Pract 2011;17:180-7.
53. Ogrinc G, Mooney SE, Estrada C, et al. The SQUIRE (Standards for Quality Improvement Reporting Excellence) guidelines for Quality Improvement reporting: explanation and elaboration. Qual Saf Health Care 2008;17(Suppl 1): i13-32.
54. Ernst MM, Wooldridge JL, Conway E, et al. Using quality improvement science to implement a multidisciplinary behavioural intervention targeting pediatric airway clearance. J Pediatr Psychol 2010;35:14-24.
55. Lynch-Jordan AM, Kashikar-Zuck S, Crosby LE, et al. Applying quality improvement methods to implement a measurement system for chronic pain-related disability. J Pediatr Psychol 2010;35:32-41.
56. Beckett DJ, Inglis M, Oswald S, et al. Reducing cardiac arrests in the acute admissions unit: a quality improvement journey. BMJ Qual Saf 2013;22:1025-31.
57. Green SA, Poots AJ, Marcano-Belisario J, et al. Mapping mental health service access: achieving equity through quality improvement. J Public Health (Oxf) 2013;35:286-92.
58. Poots AJ, Green SA, Honeybourne E, et al. Improving mental health outcomes: achieving equity through quality improvement. Int J Qual Health Care 2014;26:198-204.
59. Taylor MJ, McNicholas C, Nicolay C, et al. Systematic review of the application of the plan-do-study-act method to improve quality in healthcare. BMJ Qual Saf 2014;;23:290-8.
60. Benn J, Burnett S, Parand A, et al. Studying large-scale programmes to improve patient safety in whole care systems: challenges for research. Soc Sci Med 2009;69:1767-76.
61. Davidoff F, Dixon-Woods M, Leviton L, et al. Demystifying theory and its use in improvement. BMJ Qual Saf 2015;24:228-38.
62. Benning A, Ghaleb M, Suokas A, et al. Large scale organisational intervention to improve patient safety in four UK hospitals: mixed method evaluation. BMJ 2011;342:d195.
63. Auerbach AD, Landefeld CS, Shojania KG. The tension between needing to improve care and knowing how to do it. N Engl J Med 2007;357:608-13.
64. Eccles M, Grimshaw J, Campbell M, et al. Research designs for studies evaluating the effectiveness of change andimprovement strategies. Qual Saf Health Care 2003;12:47-52.
65. Grimshaw J, Campbell M, Eccles M, et al. Experimental and quasi-experimental designs for evaluating guideline implementation strategies. Fam Pract 2000;17(Suppl 1): S11-18.
66. Shojania KG, Grimshaw JM. Evidence-based quality improvement: the state of the science. Health Aff 2005;24:138-50.
67. Alexander JA, Hearld LR. What can we learn from quality improvement research? Med Care Res Rev 2009;66:235-71.
68. Ting HH, Shojania KG, Montori VM, et al. Quality improvement: science and action. Circulation 2009;119:1962-74.
69. Eccles M, Steen N, Grimshaw J, et al. Effect of audit and feedback, and reminder messages on primary-care radiology referrals: a randomised trial. Lancet 2001;357:1406-9.
70. Huis A, Holleman G, van Achterberg T, et al. Explaining the effects of two different strategies for promoting hand hygiene in hospital nurses: a process evaluation alongside a cluster randomised controlled trial. Implement Sci 2013;8:41.
71. French SD, McKenzie JE, O’Connor DA, et al. Evaluation of a theory-informed implementation intervention for the management of acute low back pain in general medical practice: the IMPLEMENTcluster randomised trial. PLoS ONE 2013;8:e65471.
72. Marsteller JA, Sexton JB, Hsu YJ, et al. A multicenter, phased, cluster-randomized controlled trial to reduce central line-associated bloodstream infections in intensive care units*. Crit Care Med 2012;40:2933-9.
73. van Breukelen GJ, Candel MJ. Calculating sample sizes for cluster randomized trials: we can keep it simple and efficient! J Clin Epidemiol 2012;65:1212-18.
74. Campbell MJ, Donner A, Klar N. Developments in cluster randomized trials and Statistics in Medicine. Stat Med 2007;26:2-19.
75. Brown CA, Lilford RJ. The stepped wedge trial design: a systematic review. BMC Med Res Methodol 2006;6:54.
76. Hemming K, Lilford R, Girling AJ. Stepped-wedge cluster randomised controlled trials: a generic framework including parallel and multiple-level designs. Stat Med 2015;34:181-96.
77. Ivers NM, Tricco AC, Taljaard M, et al. Quality improvement needed in quality improvement randomized trials: systematic review of interventions to improve care in diabetes. BMJ Open 2013;3:e002727.
78. Kirschner K, Braspenning J, Maassen I, et al. Improving access to primary care: the impact of a quality-improvement strategy. Qual Saf Health Care 2010;19:248-51.
79. Haynes AB, Weiser TG, Berry WR, et al. A surgical checklist to reduce morbidity and mortality in a global population. N Engl J Med 2009;360:491-9.
80. McAlister FA, Bakal JA, Majumdar SR, et al. Safely and effectively reducing inpatient length of stay: a controlled study of the General Internal Medicine Care Transformation Initiative. BMJ Qual Saf 2014;23:446-56.
81. Benning A, Dixon-Woods M, Nwulu U, et al. Multiple component patient safety intervention in English hospitals: controlled evaluation of second phase. BMJ 2011;342:d199.
82. Goetz MB, Hoang T, Knapp H, et al. Central implementation strategies outperform local ones in improving HIV testing in Veterans Healthcare Administration facilities. J Gen Intern Med 2013;28:1311-17.
83. Dodson JA, Lampert R, Wang Y, et al. Temporal trends in quality of care among ICD recipients: insights from the NCDR®. Circulation 2014;129:580-6.
84. Benn J, Burnett S, Parand A, et al. Factors predicting change in hospital safety climate and capability in a multi-site patient safety collaborative: a longitudinal survey study. BMJ Qual Saf 2012;21:559-68.
85. Avery AK, Del Toro M, Caron A. Increases in HIV screening in primary care clinics through an electronic reminder: an interrupted time series. BMJ Qual Saf 2014;23:250-6.
86. Pedrós C, Vallano A, Cereza G, et al. An intervention to improve spontaneous adverse drug reaction reporting by hospital physicians: a time series analysis in Spain. Drug Saf 2009;32:77-83.
87. Olomu AB, Stommel M, Holmes-Rovner MM, et al. Is quality improvement sustainable? Findings of the American college of cardiology’s guidelines applied in practice. Int J Qual Health Care 2014;26:215-22.
88. Tricco AC, Ivers NM, Grimshaw JM, et al. Effectiveness of quality improvement strategies on the management of diabetes: a systematic review and meta-analysis. Lancet 2012;379:2252-61.
89. Scott A, Sivey P, Ait Ouakrim D, et al. The effect of financial incentives on the quality of health care provided by primary care physicians (Review). Cochrane Database Syst Rev 2011(9): CD008451.
90. Ivers N, Jamtvedt G, Flottorp S, et al. Audit and feedback: effects on professional practice and healthcare outcomes. Cochrane Database Syst Rev 2012(6):CD000259.
91. Arditi C, Rège-Walther M, Wyatt JC, et al. Computer generated reminders delivered on paper to healthcare professionals; effects on professional practice and health care outcomes. Cochrane Database Syst Rev 2012(12): CD001175.
92. Weaver SJ, Lubomksi LH, Wilson RF, et al. Promoting a culture of safety as a patient safety strategy. Ann Intern Med 2013;158:369-74.
93. Dixon-Woods M, Cavers D, Agarwal S, et al. Conducting a critical interpretive synthesis of the literature on access to healthcare by vulnerable groups. BMC Med Res Methodol 2006;6:35.
94. Dixon-Woods M, Agarwal S, Jones D, et al. Synthesising qualitative and quantitative evidence: a review of possible methods. J Health Serv Res Policy 2005;10:45-53.
95. Alkin MC. Evaluation roots: a wider perspective of theorists’ views and influences. Los Angeles: SAGE Publications, 2013.
96. Shadish WR, Cook TD, Leviton LC. Foundations of program evaluation: theories of practice. Sage, 1991.
97. Weiss CH. Evaluation research. Methods for assessing program effectiveness. Englewood Cliffs, NJ: Prentice Hall, Inc.; 1972.
98. Weiss CH. Theory-based evaluation: past, present, and future. New Dir Eval 1997;1997:41-55.
99. Weiss CH. Methods for studying programs and policies. Upper Saddle River: Prentice Hall, 1998.
100. Aveling EL, McCulloch P, Dixon-Woods M. A qualitative study comparing experiences of the surgical safety checklist in hospitals in high-income and low-income countries. BMJ Open 2013;3:e003039.
101. Dixon-Woods M, Bosk CL, Aveling EL, et al. Explaining Michigan: developing an ex post theory of a quality improvement program. Milbank Q 2011;89:167-205.
102. Drummond MF, Sculpher MJ, Torrance GW, et al. Methods for the economic evaluation of health care programmes. 3rd edn. Oxford: Oxford University Press, 2005.
103. Taylor CB, Stevenson M, Jan S, et al. A systematic review of the costs and benefits of helicopter emergency medical services. Injury 2010;41:10-20.
104. Barasa EW, Ayieko P, Cleary S, et al. A multifaceted intervention to improve the quality of care of children in district hospitals in Kenya: a cost-effectiveness analysis. PLoS Med 2012;9:e1001238.
105. Rubio-Valera M, Bosmans J, Fernández A, et al. Cost-effectiveness of a community pharmacist intervention in patients with depression: a randomized controlled trial (PRODEFAR Study). PLoS One 2013;8:e70588.
106. Salisbury C, Foster NE, Hopper C, et al. A pragmatic randomised controlled trial of the effectiveness and cost-effectiveness of ‘PhysioDirect’ telephone assessment and advice services for physiotherapy. Health Technol Assess 2013;17:1-157.
107. Rauh SS, Wadsworth EB, Weeks WB, et al. The savings illusion--why clinical quality improvement fails to deliver bottom-line results. N Engl J Med 2011;365:e48.
108. Etchells E, Koo M, Daneman N, et al. Comparative economic analyses of patient safety improvement strategies in acute care: a systematic review. BMJ Qual Saf 2012;21:448-56.
109. Eccles MP, Armstrong D, Baker R, et al. An implementation research agenda. . Implement Sci 2009;4:18.
110. Meltzer D. Economic analysis in patient safety: a neglected necessity. BMJ Qual Saf 2012;;21:443-5.
111. Dixon-Woods M. The problem of context in quality improvement. London: Health Foundation, 2014.
112. Burnett S, Benn J, Pinto A, et al. Organisational readiness: exploring the preconditions for success in organisation-wide patient safety improvement programmes. Qual Saf Health Care 2010;19:313-17.
113. Sinkowitz-Cochran RL, Garcia-Williams A, Hackbarth AD, et al. Evaluation of organizational culture among different levels of healthcare staff participating in the Institute for Healthcare Improvement’s 100,Lives Campaign. Infect Control Hosp Epidemiol 2012;33:135-43.
114. Schierhout G, Hains J, Si D, et al. Evaluating the effectiveness of a multifaceted multilevel continuous quality improvement program in primary health care: developing a realist theory of change. Implement Sci 2013;8:119.