Cochrane Qualitative and Implementation Methods Group guidance series—paper 4: methods for assessing evidence on intervention implementation

Journal of Clinical Epidemiology

Volumn 97, Issue , 2018, Pages 59-69

  Cargo, Margaret ^a,b   Harris, Janet ^c   Pantoja, Tomas ^d   Booth, Andrew ^c   Harden, Angela ^e   Hannes, Karin ^f   Thomas, James ^g   Flemming, Kate ^h   Garside, Ruth ⁱ   Noyes, Jane ^j  

^a UNIVERSITY OF CANBERRA   (Australia)

^b UNIVERSITY OF SOUTH AUSTRALIA   (Australia)

^c UNIVERSITY OF SHEFFIELD   (United Kingdom)

^d PONTIFICIA UNIVERSIDAD CATÓLICA DE CHILE   (Chile)

^e UNIVERSITY OF EAST LONDON   (United Kingdom)

^f UNIVERSITY OF LEUVEN   (Belgium)

^g UNIVERSITY OF LONDON   (United Kingdom)

^h UNIVERSITY OF YORK   (United Kingdom)

ⁱ UNIVERSITY OF EXETER MEDICAL SCHOOL   (United Kingdom)

^j BANGOR UNIVERSITY   (United Kingdom)

more..

Author keywords

Cochrane; Implementation; Mixed method synthesis; Process evaluation; Qualitative evidence synthesis; Systematic reviews

Indexed keywords

COCHRANE LIBRARY; DATA EXTRACTION; DECISION MAKING; HUMAN; LOGIC; NOTE; PRIORITY JOURNAL; QUALITATIVE RESEARCH; STATISTICAL BIAS; SYSTEMATIC REVIEW (TOPIC); THEORY; UNCERTAINTY; EVIDENCE BASED MEDICINE; MEASUREMENT ACCURACY; MEDICAL RESEARCH;

BIOMEDICAL RESEARCH; DATA ACCURACY; DECISION MAKING; EVIDENCE-BASED MEDICINE; HUMANS; QUALITATIVE RESEARCH; SYSTEMATIC REVIEWS AS TOPIC;

EID: 85040552650     PISSN: 08954356     EISSN: 18785921     Source Type: Journal    
DOI: 10.1016/j.jclinepi.2017.11.028     Document Type: Note

References (54)

1. Pfadenhauer, L.M., Mozygemba, K., Gerhardus, A., Hofmann, B., Booth, A., Lysdahl, K.B., et al. Context and implementation: a concept analysis towards conceptual maturity. Z Evid Fortbild Qual Gesundhwes 109:2 (2015), 103–114.
2. Rossi, P.H., Lipsey, M.W., Freeman, H.E., Evaluation: a systematic approach. 7th ed., 2004, Sage Publications, Thousand Oaks.
3. Patton, M.Q., Qualitative research and evaluation methods: integrating theory and practice. 4th ed. 2015, Sage, Los Angeles.
4. Harris, J.L., Booth, A., Cargo, M., Hannes, K., Harden, A., Flemming, K., et al. Cochrane Qualitative and Implementation Methods Group guidance series—paper 2: methods for question formulation, searching and protocol development for qualitative evidence synthesis. J Clin Epidemiol 97 (2018), 39–48.
5. Noyes, J., Booth, A., Flemming, K., Garside, R., Harden, A., Lewin, S., et al. Cochrane Qualitative and Implementation Methods Group Guidance series—paper 3: methods for assessing methodological limitations, data extraction and synthesis, and confidence in synthesized qualitative findings. J Clin Epidemiol 97 (2018), 49–58.
6. Harden, A., Thomas, J., Cargo, M., Harris, J., Pantoja, T., Flemming, K., Booth, A., et al. Cochrane Qualitative and Implementation Methods Group guidance series—paper 5: methods for integrating qualitative and implementation evidence within intervention effectiveness reviews. J Clin Epidemiol 97 (2018), 70–78.
7. Higgins, J.P., Altman, D.G., Gotzsche, P.C., Juni, P., Moher, D., Oxman, A.D., et al. The Cochrane Collaboration's tool for assessing risk of bias in randomised trials. BMJ, 343, 2011, d5928.
8. Dane, A.V., Schneider, B.H., Program integrity in primary and early secondary prevention: are implementation effects out of control?. Clin Psychol Rev 18 (1998), 23–45.
9. Weiss, C.H., Evaluation research: methods of assessing program effectiveness. 1972, Prentice-Hall Inc, Englewood Cliffs, New Jersey.
10. Moore, G., Audrey, S., Barker, M., Lyndal, B., Bonell, C., Hardeman, W., et al. Process evaluation of complex interventions. 2015, Medical Research Council (MRC) Guidance, UK.
11. Milat, A., Newson, R., King, L., Rissel, C., Wolfenden, L., Bauman, A., et al. A guide to scaling up population health interventions. Public Health Res Pract, 26, 2016, e2611604.
12. Wilson, S.J., Lipsey, M.W., The effects of school-based social information processing interventions on aggressive behavior: part I: Universal programs. 2006, Campbell Systematic Reviews Available at https://www.campbellcollaboration.org/library/school-based-interventions-aggressive-behaviour-part1.html. Accessed June 2016.
13. Thorpe, K.E., Zwarenstein, M., Oxman, A.D., Treweek, S., Furberg, C.D., Altman, D.G., et al. A pragmatic-explanatory continuum indicator summary (PRECIS): a tool to help trial designers. J Clin Epidemiol 62:5 (2009), 464–475.
14. Schillinger, D., An introduction to effectiveness, dissemination and implementation research. A resource manual for communtiy-engaged research. 2010, University of California San Francisco Clinical and Translational Science Institute (CTSI) Community Engagement Program Available at http://ctsi.ucsf.edu/files/CE/edi_introguide.pdf. Accessed February 2016.
15. Noyes, J., Hendry, M., Lewin, S., Glenton, C., Chandler, J., Rashidian, A., Qualitative “trial-sibling” studies and “unrelated” qualitative studies contributed to complex intervention reviews. J Clin Epidemiol 74 (2016), 133–143.
16. Moore, G., Audrey, S., Barker, M., Bond, L., Bonell, C., Cooper, C., et al. Process evaluation in complex public health intervention studies: the need for guidance. J Epidemiol Community Health 68 (2013), 101–102.
17. Carroll, C., Patterson, M., Wood, S., Booth, A., Rick, J., Balain, S., A conceptual framework for implementation fidelity. Implementation Sci, 2, 2007, 40.
18. Steckler, A., Linnan, L., (eds.) Process evaluation for public health interventions and research, 2002, Jossey-Bass, San Francisco.
19. Armstrong, R., Waters, E., Moore, L., Riggs, E., Cuervo, L.G., Lumbiganon, P., et al. Improving the reporting of public health intervention research: advancing TREND and CONSORT. J Public Health 30:1 (2008), 103–109.
20. Creswell, J.W., Klassen, A.C., Plano Clark, V.L., Smith, K.C., Best practices for mixed methods research in the health sciences. 2011, National Institutes of Health, Office of Behavioral and Social Sciences Research Available at https://obssr.od.nih.gov/training/mixed-methods-research/. Accessed March 2016.
21. Chen, H.-T., Practical program evaluation. Assessing and improving planning, implementation and effectiveness. 2005, Sage Publications, Thousand Oaks, CA.
22. Farrington, D., Ttofi, M., School-based programs to reduce bullying and victimization. 2009, Campbell Systematic Reviews Available at https://www.campbellcollaboration.org/library/school-based-programmes-to-reduce-bullying-victimisation.html. Accessed April 2016.
23. Grant, M.J., Booth, A., A typology of reviews: an analysis of 14 review types and associated methodologies. Health Info Libr J 26 (2009), 91–108.
24. Funnell, S.C., Rogers, P.J., Purposeful program theory: effective use of theories of change and logic models. 2011, San Francisco, CA, John Wiley/Jossey-Bass.
25. Hoffmann, T., Glasziou, P., Boutron, I., Milne, R., Perera, R., Moher, D., Better reporting of interventions: template for intervention description and replication (TIDieR) checklist and guide. BMJ, 348, 2014, g1687.
26. Anderson, L.M., Petticrew, M., Rehfuess, E., Armstrong, R., Ueffing, E., Baker, P., et al. Using logic models to capture complexity in systematic reviews. Res Synth Methods 2 (2011), 33–42.
27. Barnett-Page, E., Thomas, J., Methods for the synthesis of qualitative research: a critical review. BMC Med Res Methodol, 9, 2009, 59.
28. Walker, L.O., Avant, K.C., Strategies for theory construction in nursing. 2005, Pearson Prentice Hall, Upper Saddle River, NJ.
29. Carroll, C., Booth, A., Leaviss, J., Rick, J., “Best fit” framework synthesis: refining the method. BMC Med Res Methodol, 13, 2013, 37.
30. Booth, A., Noyes, J., Flemming, K., Gerhardus, A., Wahlster, P., Van der Wilt, G.J., et al. Guidance on choosing qualitative evidence synthesis methods for use in health technology assessments of complex interventions. [Online], 2016 Available at http://www.integrate-hta.eu/downloads/ Accessed October 2016.
31. Noyes, J., Pearson, A., Hannes, K., Booth a on behalf of the Cochrane Qualitative Research Methods Group. Qualitative research and cochrane reviews. Higgins, J., Green, S., (eds.) Cochrane handbook for the systematic review of interventions: cochrane book series, 2008, John Wiley and Sons Ltd, UK, 571–592.
32. Oliver, K., Rees, R., Brady, L.M., Kavanagh, J., Oliver, S., Thomas, J., Broadening public participation in systematic reviews: a case example involving young people in two configurative reviews. Res Synth Methods 6 (2015), 206–217.
33. Jamal, F., Langford, R., Daniels, P., Thomas, J., Harden, A., Bonell, C., Consulting with young people to inform systematic reviews: an example from a review on the effects of schools on health. Health Expect 18 (2014), 3225–3235.
34. Lewin, S., Glenton, C., Oxman, A.D., Use of qualitative methods alongside randomised controlled trials of complex healthcare interventions: methodological study. BMJ, 339, 2009, b3496.
35. Adams, R.J., Huff, A.S., Shades of grey: guidelines for working with the grey literature in systematic reviews for management and organizational studies. Int J Management Rev 19:4 (2017), 432–454.
36. Harden, A., Oakley, A., Weston, R., A review of the effectiveness and appropriateness of peer-delivered health promotion for young people. 1999, London, UK, Institute of Education, University of London.
37. Lefebvre, C.M.E., Glanville, J., Chapter 6: searching for studies. (updated March 2011) Higgins, J., Green, S., (eds.) Cochrane handbook for systematic reviews of interventions version 510, 2011, The Cochrane Collaboration Available at www.cochrane-handbook.org Accessed January 2016.
38. Cooper Robbins, S.C., Ward, K., Skinner, S.R., School-based vaccination: a systematic review of process evaluations. Vaccine 29 (2011), 9588–9599.
39. Booth, A., Harris, J., Croot, E., Springett, J., Campbell, F., Wilkins, E., Towards a methodology for cluster searching to provide conceptual and contextual “richness” for systematic reviews of complex interventions: case study (CLUSTER). BMC Med Res Methodol, 13, 2013, 118.
40. Bonell, C., Farah, J., Harden, A., Wells, H., Parry, W., Fletcher, A., et al. Systematic review of the effects of schools and school environment interventions on health: evidence mapping and synthesis. Public Health Res, 1(1), 2013.
41. Lewin, S., Hendry, M., Chandler, J., Oxman, A.D., Michie, S., Shepperd, S., et al. Assessing the complexity of interventions within systematic reviews: development, content and use of a new tool (iCAT_SR). BMC Med Res Methodol, 17(1), 2017, 76.
42. Cargo, M., Stankov, I., Thomas, J., Saini, M., Rogers, P., Mayo-Wilson, E., et al. Development, inter-rater reliability and feasibility of a checklist to assess implementation (Ch-IMP) in systematic reviews: the case of provider-based prevention and treatment programs targeting children and youth. BMC Med Res Methodol, 15, 2015, 73.
43. Salmon, J., Ball, K., Crawford, D., Booth, M., Telford, A., Hume, C., et al. Reducing sedentary behaviour and increasing physical activity among 10-year-old children: overview and process evaluation of the 'Switch-Play' intervention. Health Promotion Int 20 (2005), 7–17.
44. Sutcliffe, K., Thomas, J., Stokes, G., Hinds, K., Bangpan, M., Intervention Component Analysis (ICA): a pragmatic approach for identifying the critical features of complex interventions. Syst Rev 4 (2015), 1–13.
45. Montgomery, P., Underhill, K., Gardner, F., Operario, D., Mayo-Wilson, E., The Oxford Implementation Index: a new tool for incorporating implementation data into systematic reviews and meta-analyses. J Clin Epidemiol 66 (2013), 874–882.
46. Armstrong, R.W.E., Jackson, N., Oliver, S., Popay, J., Shepherd, J., Petticrew, M., et al. Guidelines for systematic reviews of health promotion and public health interventions. Version 2. 2007, Melbourne University, Australia.
47. Cargo, M., Tools for critically appraising process and implementation in systematic reviews. Paper presented at The Joanna Briggs Institute Biennial International Convention Ripples to Revolution: From Bench to Bedside; Adelaide, Australia, 2009.
48. Rees, R., Oliver, K., Woodman, J., Thomas, J., Children's views about obesity, body size, shape and weight. A systematic review: Social Science Research Unit. 2009, London, UK, Institute of Education, University of London.
49. Shepherd, J., Kavanagh, J., Picot, J., Cooper, K., Harden, A., Barnett-Page, E., et al. The effectiveness and cost-effectiveness of behavioural interventions for the prevention of sexually transmitted infections in young people aged 13-19: a systematic review and economic evaluation. Health Technology Assess 14 (2010), 1–206 [iii-iv].
50. Health Care Practice Research and Development Unit. Evaluation tool for qualitative research. 2009 Available at http://usirsalfordacuk/12970/1/Evaluation_Tool_for_Qualitative_Studiespdf Accessed March 2016.
51. Hannes, K., Lockwood, C., Pearson, A., A comparative analysis of three online appraisal instruments' ability to assess validity in qualitative research. Qual Health Res 20 (2010), 1736–1743.
52. Lewin, S., Glenton, C., Munthe-Kaas, H., Carlsen, B., Colvin, C.J., Gulmezoglu, M., et al. Using qualitative evidence in decision making for health and social interventions: an approach to assess confidence in findings from qualitative evidence syntheses (GRADE-CERQual). PLoS Med, 12, 2015, e1001895.
53. Effective Public Health Practice Project Quality Assessment Tool. Quality assessment tool for quantitative studies. 2009 Available at http://www.ephpp.ca/PDF/Quality%20Assessment%20Tool_2010_2.pdf. Accessed January 2016.
54. Petrosino, A., Turpin-Petrosino, C., Hollis-Peel, M.E., Lavenberg, J.G., 'scared Straight’ and other juvenile awareness programs for preventing juvenile delinquency. Cochrane Database Syst Rev, 2013 CD002796.