Cluster-randomized controlled trials evaluating complex interventions in general practices are mostly ineffective: a systematic review

Journal of Clinical Epidemiology

Volumn 94, Issue , 2018, Pages 85-96

  Siebenhofer, Andrea ^a,b   Paulitsch, Michael A ^a   Pregartner, Gudrun ^b   Berghold, Andrea ^b   Jeitler, Klaus ^b   Muth, Christiane ^a   Engler, Jennifer ^a  

^a GOETHE UNIVERSITY   (Germany)

^b MEDICAL UNIVERSITY OF GRAZ   (Austria)

Author keywords

Cluster randomized controlled trial; Complex intervention; Effectiveness; General practice; Shortcomings; Systematic review

Indexed keywords

CLINICAL EFFECTIVENESS; GENERAL PRACTICE; HUMAN; OUTCOME ASSESSMENT; PATIENT CARE; PRIMARY MEDICAL CARE; PRIORITY JOURNAL; RANDOMIZED CONTROLLED TRIAL (TOPIC); REVIEW; SYSTEMATIC REVIEW; PROCEDURES; SAMPLE SIZE; TREATMENT OUTCOME;

GENERAL PRACTICE; HUMANS; RANDOMIZED CONTROLLED TRIALS AS TOPIC; SAMPLE SIZE; TREATMENT OUTCOME;

EID: 85041601177     PISSN: 08954356     EISSN: 18785921     Source Type: Journal    
DOI: 10.1016/j.jclinepi.2017.10.010     Document Type: Review

References (63)

1. Campbell, M.K., Elbourne, D.R., Altman, D.G., CONSORT statement: extension to cluster randomised trials. BMJ 328 (2004), 702–708.
2. Campbell, M.K., Piaggio, G., Elbourne, D.R., Altman, D.G., Consort 2010 statement: extension to cluster randomised trials. BMJ, 345, 2012, e5661.
3. Ivers, N.M., Taljaard, M., Dixon, S., Bennett, C., McRae, A., Taleban, J., et al. Impact of CONSORT extension for cluster randomised trials on quality of reporting and study methodology: review of random sample of 300 trials, 2000-8. BMJ, 343, 2011, d5886.
4. Taljaard, M., Weijer, C., Grimshaw, J.M., Eccles, M.P., The Ottawa Statement on the ethical design and conduct of cluster randomised trials: precis for researchers and research ethics committees. BMJ, 346, 2013, f2838.
5. Möhler, R., Bartoszek, G., Kopke, S., Meyer, G., Proposed criteria for reporting the development and evaluation of complex interventions in healthcare (CReDECI): guideline development. Int J Nurs Stud 49:1 (2012), 40–46.
6. Möhler, R., Kopke, S., Meyer, G., Criteria for reporting the development and evaluation of complex interventions in healthcare: revised guideline (CReDECI 2). Trials, 16, 2015, 204.
7. Siebenhofer, A., Erckenbrecht, S., Pregartner, G., Berghold, A., Muth, C., How often are interventions in cluster-randomised controlled trials of complex interventions in general practices effective and reasons for potential shortcomings? Protocol and results of a feasibility project for a systematic review. BMJ Open, 6(2), 2016, e009414.
8. Djulbegovic, B., Kumar, A., Glasziou, P., Miladinovic, B., Chalmers, I., Medical research: trial unpredictability yields predictable therapy gains. Nature 500 (2013), 395–396.
9. Erler, A., Beyer, M., Petersen, J.J., Saal, K., Rath, T., Rochon, J., et al. How to improve drug dosing for patients with renal impairment in primary care - a cluster-randomized controlled trial. BMJ Fam Pract, 13(1), 2012, 91.
10. Hoffmann, B., Müller, V., Rochon, J., Müller, B., Albay, Z., Weppler, K., et al. Effects of a team-based assessment and intervention on patient safety culture in general practice: an open randomised controlled trial. BMJ Qual Saf 23:1 (2014), 35–46.
11. Siebenhofer A, Muth C, Fullerton B, Glasziou P, Beller E, Thorning S et al. The shortcomings of cluster-randomized controlled trials for the assessment of complex interventions in general practices: a systematic review. Centre Rev Dissemination - CRD. PROSPERO 2014:CRD42014009234. Available at. http://www.crd.york.ac.uk/PROSPERO/display_record.asp?ID=CRD42014009234. Accessed December 7, 2017.
12. Craig, P., Dieppe, P., Macintyre, S., Michie, S., Nazareth, I., Petticrew, M., Developing and evaluating complex interventions: the new Medical Research Council guidance. BMJ, 337, 2008, a1655.
13. Djulbegovic, B., Kumar, A., Glasziou, P.P., Perera, R., Reljic, T., Dent, L., et al. New treatments compared to established treatments in randomized trials. Cochrane Database Syst Rev, 10, 2012, MR000024.
14. Institute for Quality and Efficiency in Health Care. General Methods: Version 4.2 of 22 April 2015. Available at https://www.iqwig.de/download/IQWiG_General_Methods_Version_%204-2_no_longer_valid.pdf. Accessed December 7, 2017.
15. Taljaard, M., McGowan, J., Grimshaw, J.M., Brehaut, J.C., McRae, A., Eccles, M.P., et al. Electronic search strategies to identify reports of cluster randomized trials in MEDLINE: low precision will improve with adherence to reporting standards. BMC Med Res Methodol, 10, 2010, 15.
16. Bland, J.M., Cluster randomised trials in the medical literature: two bibliometric surveys. BMC Med Res Methodol, 4, 2004, 21.
17. Siebenhofer A, Muth C, Fullerton B, Glasziou P, Beller E, Thorning S et al. Our specific SRDR project database to be published after acceptance of our paper.
18. Systematic Review Data Repository [Software]. Available at http://srdr.ahrq.gov/. Accessed December 7, 2017.
19. Higgins, J.P.T., Green, S., Cochrane Handbook for Systematic Reviews of Interventions Version 5.1.0 [updated March 2011]. 2016 Available at www.handbook.cochrane.org. Accessed December 7, 2017.
20. Froud, R., Eldridge, S., Diaz Ordaz, K., Marinho, V.C.C., Donner, A., Quality of cluster randomized controlled trials in oral health: a systematic review of reports published between 2005 and 2009. Community Dent Oral Epidemiol 40:Suppl 1 (2012), 3–14.
21. Eldridge, S., Ashby, D., Bennett, C., Wakelin, M., Feder, G., Internal and external validity of cluster randomised trials: systematic review of recent trials. BMJ 336 (2008), 876–880.
22. Moher, D., Liberati, A., Tetzlaff, J., Altman, D.G., Preferred reporting items for systematic reviews and meta-analyses: the PRISMA statement. PLoS Med, 6(7), 2009, e1000097.
23. Rutterford, C., Taljaard, M., Dixon, S., Copas, A., Eldridge, S., Reporting and methodological quality of sample size calculations in cluster randomized trials could be improved: a review. J Clin Epidemiol 68 (2015), 716–723.
24. Bennewith, O., Stocks, N., Gunnell, D., Peters, T.J., Evans, M.O., Sharp, D.J., General practice based intervention to prevent repeat episodes of deliberate self harm: cluster randomised controlled trial. BMJ 324 (2002), 1254–1257.
25. Black, D.A., Taggart, J., Jayasinghe, U.W., Proudfoot, J., Crookes, P., Beilby, J., et al. The Teamwork Study: enhancing the role of non-GP staff in chronic disease management in general practice. Aust J Prim Health 19:3 (2013), 184–189.
26. Byng, R., Jones, R., Leese, M., Hamilton, B., McCrone, P., Craig, T., Exploratory cluster randomised controlled trial of shared care development for long-term mental illness. Br J Gen Pract 54:501 (2004), 259–266.
27. Coleman, E.A., Grothaus, L.C., Sandhu, N., Wagner, E.H., Chronic care clinics: a randomized controlled trial of a new model of primary care for frail older adults. J Am Geriatr Soc 47:7 (1999), 775–783.
28. Elley, C.R., Kerse, N., Arroll, B., Robinson, E., Effectiveness of counselling patients on physical activity in general practice: cluster randomised controlled trial. BMJ, 326, 2003, 793.
29. Gask, L., Dowrick, C., Dixon, C., Sutton, C., Perry, R., Torgerson, D., et al. A pragmatic cluster randomized controlled trial of an educational intervention for GPs in the assessment and management of depression. Psychol Med 34 (2004), 63–72.
30. Gensichen, J., von Korff, M., Peitz, M., Muth, C., Beyer, M., Guthlin, C., et al. Case management for depression by health care assistants in small primary care practices: a cluster randomized trial. Ann Intern Med 151 (2009), 369–378.
31. Griffin, S.J., Borch-Johnsen, K., Davies, M.J., Khunti, K., Rutten, G.E., Sandbæk, A., et al. Effect of early intensive multifactorial therapy on 5-year cardiovascular outcomes in individuals with type 2 diabetes detected by screening (ADDITION-Europe): a cluster-randomised trial. Lancet 378 (2011), 156–167.
32. Holton, C.H., Beilby, J.J., Harris, M.F., Harper, C.E., Proudfoot, J.G., Ramsay, E.N., et al. Systematic care for asthma in Australian general practice: a randomised controlled trial. Med J Aust 193 (2010), 332–337.
33. Jellema, P., van der Windt, D.A., van der Horst, H.E., Twisk, J.W., Stalman, W.A., Bouter, L.M., Should treatment of (sub)acute low back pain be aimed at psychosocial prognostic factors? Cluster randomised clinical trial in general practice. BMJ, 331, 2005, 84.
34. Juul, L., Maindal, H.T., Zoffmann, V., Frydenberg, M., Sandbaek, A., Effectiveness of a training course for general practice nurses in motivation support in type 2 diabetes care: a cluster-randomised trial. PLoS One, 9, 2014, e96683.
35. Kendrick, D., Marsh, P., Fielding, K., Miller, P., Preventing injuries in children: cluster randomised controlled trial in primary care. BMJ 318 (1999), 980–983.
36. Kennedy, A., Bower, P., Reeves, D., Blakeman, T., Bowen, R., Chew-Graham, C., et al. Implementation of self management support for long term conditions in routine primary care settings: cluster randomised controlled trial. BMJ, 346, 2013, f2882.
37. Kerse, N.M., Flicker, L., Jolley, D., Arroll, B., Young, D., Improving the health behaviours of elderly people: randomised controlled trial of a general practice education programme. BMJ 319 (1999), 683–687.
38. Kruis, A.L., Boland, M.R.S., Assendelft, W.J.J., Gussekloo, J., Tsiachristas, A., Stijnen, T., et al. Effectiveness of integrated disease management for primary care chronic obstructive pulmonary disease patients: results of cluster randomised trial. BMJ, 349, 2014, g5392.
39. Lobo, C.M., Frijling, B.D., Hulscher, M.E.J.L., Bernsen, R.M.D., Grol, R.P.T.M., Prins, A., et al. Effect of a comprehensive intervention program targeting general practice staff on quality of life in patients at high cardiovascular risk: a randomized controlled trial. Qual Life Res 13 (2004), 73–80.
40. Lord, J., Victor, C., Littlejohns, P., Ross, F.M., Axford, J.S., Economic evaluation of a primary care-based education programme for patients with osteoarthritis of the knee. Health Technol Assess 3 (1999), 1–55.
41. McGeoch, G.R.B., Willsman, K.J., Dowson, C.A., Town, G.I., Frampton, C.M., McCartin, F.J., et al. Self-management plans in the primary care of patients with chronic obstructive pulmonary disease. Respirology 11:5 (2006), 611–618.
42. Menn, P., Holle, R., Kunz, S., Donath, C., Lauterberg, J., Leidl, R., et al. Dementia care in the general practice setting: a cluster randomized trial on the effectiveness and cost impact of three management strategies. Value Health 15:6 (2012), 851–859.
43. Metzelthin, S.F., van Rossum, E., de, Witte L.P., Ambergen, A.W., Hobma, S.O., Sipers, W., et al. Effectiveness of interdisciplinary primary care approach to reduce disability in community dwelling frail older people: cluster randomised controlled trial. BMJ, 347, 2013, f5264.
44. Murphy, A.W., Cupples, M.E., Smith, S.M., Byrne, M., Byrne, M.C., Newell, J., Effect of tailored practice and patient care plans on secondary prevention of heart disease in general practice: cluster randomised controlled trial. BMJ, 339, 2009, b4220.
45. Olivarius, N.F., Beck-Nielsen, H., Andreasen, A.H., Horder, M., Pedersen, P.A., Randomised controlled trial of structured personal care of type 2 diabetes mellitus. BMJ, 323, 2001, 970.
46. Rea, H., McAuley, S., Stewart, A., Lamont, C., Roseman, P., Didsbury, P., A chronic disease management programme can reduce days in hospital for patients with chronic obstructive pulmonary disease. Intern Med J 34:11 (2004), 608–614.
47. Rosendal, M., Olesen, F., Fink, P., Toft, T., Sokolowski, I., Bro, F., A randomized controlled trial of brief training in the assessment and treatment of somatization in primary care: effects on patient outcome. Gen Hosp Psychiatry 29:4 (2007), 364–373.
48. Smith, J.R., Noble, M.J., Musgrave, S., Murdoch, J., Price, G.M., Barton, G.R., et al. The at-risk registers in severe asthma (ARRISA) study: a cluster-randomised controlled trial examining effectiveness and costs in primary care. Thorax 67:12 (2012), 1052–1060.
49. Thoonen, B.P.A., Schermer, T.R.J., van Den Boom, G., Molema, J., Folgering, H., Akkermans, R.P., et al. Self-management of asthma in general practice, asthma control and quality of life: a randomised controlled trial. Thorax 58:1 (2003), 30–36.
50. van Marwijk, H.W., Ader, H., de Haan, M., Beekman, A., Primary care management of major depression in patients aged or =55 years: outcome of a randomised clinical trial. Br J Gen Pract 58:555 (2008), 680–686 I–II; discussion 687.
51. Wagner, E.H., Grothaus, L.C., Sandhu, N., Galvin, M.S., McGregor, M., Artz, K., et al. Chronic care clinics for diabetes in primary care: a system-wide randomized trial. Diabetes Care 24 (2001), 695–700.
52. Zwarenstein, M., Bheekie, A., Lombard, C., Swingler, G., Ehrlich, R., Eccles, M., et al. Educational outreach to general practitioners reduces children's asthma symptoms: a cluster randomised controlled trial. Implement Sci, 2, 2007, 30.
53. Diaz-Ordaz, K., Slowther, A.M., Potter, R., Eldridge, S., Consent processes in cluster-randomised trials in residential facilities for older adults: a systematic review of reporting practices and proposed guidelines. BMJ Open, 3(7), 2013 e003057.
54. Pocock, S.J., Stone, G.W., The primary outcome is positive - is that good enough?. N Engl J Med 375 (2016), 971–979.
55. Pocock, S.J., Stone, G.W., The primary outcome fails - what next?. N Engl J Med 375 (2016), 861–870.
56. Cohen, J., Statistical power analysis for the behavioral sciences. 2nd ed., 1988, Erlbaum, Hillsdale, NJ.
57. Puffer, S., Torgerson, D., Watson, J., Evidence for risk of bias in cluster randomised trials: review of recent trials published in three general medical journals. BMJ 327 (2003), 785–789.
58. Hemkens, L.G., Contopoulos-Ioannidis, D.G., Ioannidis, J.P.A., Agreement of treatment effects for mortality from routinely collected data and subsequent randomized trials: meta-epidemiological survey. BMJ, 352, 2016, i493.
59. Hemkens, L.G., Contopoulos-Ioannidis, D.G., Ioannidis, J.P.A., Current use of routinely collected health data to complement randomized controlled trials: a meta-epidemiological survey. CMAJ Open 4:2 (2016), E132–E140.
60. Diaz-Ordaz, K., Kenward, M.G., Cohen, A., Coleman, C.L., Eldridge, S., Are missing data adequately handled in cluster randomised trials? A systematic review and guidelines. Clin Trials 11 (2014), 590–600.
61. Arnup, S.J., Forbes, A.B., Kahan, B.C., Morgan, K.E., McKenzie, J.E., Appropriate statistical methods were infrequently used in cluster-randomized crossover trials. J Clin Epidemiol 74 (2016), 40–50.
62. Brown, A.W., Li, P., Bohan Brown, M.M., Kaiser, K.A., Keith, S.W., Oakes, J.M., et al. Best (but oft-forgotten) practices: designing, analyzing, and reporting cluster randomized controlled trials. Am J Clin Nutr 102 (2015), 241–248.
63. Caille, A., Kerry, S., Tavernier, E., Leyrat, C., Eldridge, S., Giraudeau, B., Timeline cluster: a graphical tool to identify risk of bias in cluster randomised trials. BMJ, 354, 2016, i4291.