The practice of 'doing' evaluation: Lessons learned from nine complex intervention trials in action

Implementation Science

Volumn 9, Issue 1, 2014, Pages

  Reynolds, Joanna ^a   DiLiberto, Deborah ^a   Mangham Jefferies, Lindsay ^a   Ansah, Evelyn K ^b   Lal, Sham ^a   Mbakilwa, Hilda ^c   Bruxvoort, Katia ^a   Webster, Jayne ^a   Vestergaard, Lasse S ^d,e   Yeung, Shunmay ^a   Leslie, Toby ^a   Hutchinson, Eleanor ^a   Reyburn, Hugh ^a   Lalloo, David G ^f   Schellenberg, David ^a   Cundill, Bonnie ^a   Staedke, Sarah G ^a   Wiseman, Virginia ^a,g   Goodman, Catherine ^a   Chandler, Clare I R ^a  

^a LONDON SCHOOL OF HYGIENE AND TROPICAL MEDICINE   (United Kingdom)

^b GHANA HEALTH SERVICE   (Ghana)

^c Joint Malaria Programme   (Tanzania)

^d UNIVERSITY OF COPENHAGEN   (Denmark)

^e Mental Health Services CPH   (Denmark)

^f LIVERPOOL SCHOOL OF TROPICAL MEDICINE   (United Kingdom)

^g UNIVERSITY OF NEW SOUTH WALES   (Australia)

Author keywords

Behavioural interventions; Complex interventions; Evaluation; Health service; Low income setting; Reflection; Trials

Indexed keywords

ARTICLE; COOPERATION; EDUCATION; HEALTH BEHAVIOR; HEALTH CARE PERSONNEL; HEALTH SERVICES RESEARCH; HUMAN; IN SERVICE TRAINING; INFORMATION PROCESSING; INTERPERSONAL COMMUNICATION; MALARIA; METHODOLOGY; ORGANIZATION AND MANAGEMENT; PERSONNEL; POVERTY; PROGRAM EVALUATION; PUBLIC RELATIONS; STANDARD;

COMMUNICATION; COOPERATIVE BEHAVIOR; DATA COLLECTION; HEALTH BEHAVIOR; HEALTH PERSONNEL; HEALTH SERVICES RESEARCH; HUMANS; INSERVICE TRAINING; INTERPROFESSIONAL RELATIONS; MALARIA; POVERTY; PROGRAM EVALUATION; RESEARCH DESIGN; RESEARCH PERSONNEL;

EID: 84902300501     PISSN: None     EISSN: 17485908     Source Type: Journal    
DOI: 10.1186/1748-5908-9-75     Document Type: Article

References (55)

1. Pawson R. Evidence-Based Policy. A Realist Perspective 2006, London: Sage.
2. Peterson S. Assessing the scale-up of child survival interventions. Lancet 2010, 375:530-531.
3. Medical Research Council A Framework For The Development And Evaluation Of RCTs For Complex Interventions To Improve Health 2000, London: Medical Research Council, Medical Research Council.
4. Medical Research Council Developing And Evaluating Complex Interventions: New Guidance 2008, London: Medical Research Council, Medical Research Council.
5. Campbell M, Fitzpatrick R, Haines A, Kinmonth AL, Sandercock P, Spiegelhalter D, Tyrer P. Framework for design and evaluation of complex interventions to improve health. BMJ 2000, 321:694-696.
6. Craig P, Dieppe P, Macintyre S, Michie S, Nazareth I, Petticrew M. Developing and evaluating complex interventions: the new medical research council guidance. BMJ 2008, 337:979-983.
7. Nazareth I, Freemantle N, Duggan C, Mason J, Haines A. Evaluation of a complex intervention for changing professional behaviour: the evidence based out reach (EBOR) trial. J Health Serv Res Policy 2002, 7:230-238.
8. Bonell CP, Hargreaves J, Cousens S, Ross D, Hayes R, Petticrew M, Kirkwood BR. Alternatives to randomisation in the evaluation of public health interventions: design challenges and solutions. J Epidemiol Community Health 2011, 65:582-587.
9. Cousens S, Hargreaves J, Bonell C, Armstrong B, Thomas J, Kirkwood BR, Hayes R. Alternatives to randomisation in the evaluation of public-health interventions: statistical analysis and causal inference. J Epidemiol Community Health 2011, 65:576-581.
10. Munro A, Bloor M. Process evaluation: the new miracle ingredient in public health research?. Qual Res 2010, 10:699-713.
11. Wells M, Williams B, Treweek S, Coyle J, Taylor J. Intervention description is not enough: evidence from an in-depth multiple case study on the untold role and impact of context in randomised controlled trials of seven complex interventions. Trials 2012, 13:95.
12. Saunders RP, Evans MH, Joshi P. Developing a process-evaluation plan for assessing health promotion program implementation: a how-to guide. Health Promot Pract 2005, 6:134-147.
13. Oakley A, Strange V, Bonell C, Allen E, Stephenson J, . RIPPLE Study Team Process evaluation in randomized controlled trials of complex interventions. BMJ 2006, 332:413-416. RIPPLE Study Team.
14. Grant A, Treweek S, Dreischulte T, Foy R, Guthrie B. Process evaluations for cluster-randomised trials of complex interventions: a proposed framework for design and reporting. Trials 2013, 14:1-10.
15. Moore G, Audrey S, Barker M, Bond L, Bonell C, Cooper C, Hardeman W, Moore L, O'Cathain A, Tinati T, Wight D, Baird J. Process evaluation in complex public health intervention studies: the need for guidance. J Epidemiol Commun Health 2014, 68:101-102.
16. Donovan J, Little P, Mills N, Smith M, Brindle L, Jacoby A, Peters T, Frankel S, Neal D, Hamdy F. Improving design and conduct of randomised trials by embedding them in qualitative research: protecT (prostate testing for cancer and treatment) study. BMJ 2002, 325:766-769.
17. Murtagh M, Thomson R, May C, Rapley T, Heaven B, Graham R, Kaner E, Stobbart L, Eccles M. Qualitative methods in a randomised controlled trial: the role of an integrated qualitative process evaluation in providing evidence to discontinue the intervention in one arm of a trial of a decision support tool. Qual Saf Health Care 2007, 16:224-229.
18. Koivisto J. What evidence base? Steps towards the relational evaluation of social interventions. Evid Pol 2007, 3:527-537.
19. National Institute for Health Research Clinical Trials Tool Kit [http://www.ct-toolkit.ac.uk/], National Institute for Health Research.
20. Petryna A. Ethical variability: drug development and globalizing clinical trials. Am Ethnol 2005, 32:183-197.
21. Kelly AH, Ameh D, Majambere S, Lindsay S, Pinder M. 'Like sugar and honey': the embedded ethics of a larval control project in the Gambia. Soc Sci Med 2010, 70:1912-1919.
22. Switula D. Principles of good clinical practice (GCP) in clinical research. Sci Eng Ethics 2000, 6:71-77.
23. European Medicines Agency ICH Harmonised Tripartite Guideline E6: Note for Guidance on Good Clinical Practice (CPMP/ICH/135/95) 2002, London: European Medicines Agency, [http://ichgcp.net/pdf/ich-gcp-en.pdf], European Medicines Agency.
24. Lang T, Chilengi R, Noor RA, Ogutu B, Todd JE, Kilama WL, Targett GA. Data safety and monitoring boards for African clinical trials. Trans R Soc Trop Med Hyg 2008, 102:1189-1194.
25. Lawton J, Jenkins N, Darbyshire J, Farmer A, Holman R, Hallowell N. Understanding the outcomes of multi-centre clinical trials: a qualitative study of health professional experiences and views. Soc Sci Med 2012, 74:574-581.
26. Chandler C, Jones C, Boniface G, Juma K, Reyburn H, Whitty C. Guidelines and mindlines: why do clinical staff over-diagnose malaria in Tanzania? A qualitative study. Malar J 2008, 7:53.
27. Whitty CJ, Chandler C, Ansah EK, Leslie T, Staedke S. Deployment of ACT antimalarials for treatment of malaria: challenges and opportunities. Malar J 2008, 7(Suppl 1):S7.
28. Mangham L, Cundill B, Ezeoke O, Nwala E, Uzochukwu B, Wiseman V, Onwujekwe O. Treatment of uncomplicated malaria at public health facilities and medicine retailers in south-eastern Nigeria. Malar J 2011, 10:155.
29. Godwin M, Ruhland L, Casson I, MacDonald S, Delva D, Birtwhistle R, Lam M, Seguin R. Pragmatic controlled clinical trials in primary care: the struggle between external and internal validity. BMC Med Res Methodol 2003, 3:28.
30. Stake R. The Art of Case Study Research 1995, London: Sage Publications Ltd.
31. Crowe S, Cresswell K, Robertson A, Huby G, Avery A, Sheikh A. The case study approach. BMC Med Res Methodol 2011, 11:100.
32. Pope C, Ziebland S, Mays N. Qualitative research in health care. Analysing qualitative data. BMJ 2000, 320:114-116.
33. Biruk C. Seeing like a research project: producing 'high-quality data' in AIDS research in Malawi. Med Anthropol 2012, 31:347-366.
34. Helgesson C-F. From Dirty Data To Credible Scientific Evidence: Some Practices Used To Clean Data In Large Randomised Clinical Trials. Medical Proofs, Social Experiments: Clinical Trials in Shifting Contexts 2010, 49-64. Farnham, UK: Ashgate, Will C, Moreira T.
35. Molyneux S, Kamuya D, Madiega PA, Chantler T, Angwenyi V, Geissler PW. Field workers at the interface. Dev World Bioeth 2013, 13:ii-iv.
36. Blaise P, Kegels G. A realistic approach to the evaluation of the quality management movement in health care systems: a comparison between European and African contexts based on Mintzberg's organizational models. Int J Health Plann Manage 2004, 19:337-364.
37. Baer AR, Zon R, Devine S, Lyss AP. The clinical research team. J Oncol Pract 2011, 7:188-192.
38. Barry C, Britten N, Barber N, Bradley C, Stevenson F. Using reflexivity to optimize teamwork in qualitative research. Qual Health Res 1999, 9:26-44.
39. Lawton J, Jenkins N, Darbyshire J, Holman R, Farmer A, Hallowell N. Challenges of maintaining research protocol fidelity in a clinical care setting: a qualitative study of the experiences and views of patients and staff participating in a randomized controlled trial. Trials 2011, 12:108.
40. Audrey S, Holliday J, Parry-Langdon N, Campbell R. Meeting the challenges of implementing process evaluation within randomized controlled trials: the example of ASSIST (a stop smoking in schools trial). Health Educ Res 2006, 21:366-377.
41. Clarke D, Hawkins R, Sadler E, Harding G, Forster A, McKevitt C, Godfrey M, Monaghan J, Farrin A. Interdisciplinary health research: perspectives from a process evaluation research team. Qual Prim Care 2012, 20:179-189.
42. Lang TA, White NJ, Hien TT, Farrar JJ, Day NPJ, Fitzpatrick R, Angus BJ, Denis E, Merson L, Cheah PY, Chilengi R, Kimutai R, Marsh K. Clinical research in resource-limited settings: enhancing research capacity and working together to make trials less complicated. PLoS Negl Trop Dis 2010, 4:e619.
43. Speed C, Heaven B, Adamson A, Bond J, Corbett S, Lake AA, May C, Vanoli A, McMeekin P, Moynihan P, Rubin G, Steen IN, McColl E. LIFELAX - diet and LIFEstyle versus LAXatives in the management of chronic constipation in older people: randomised controlled trial. Health Technol Assess 2010, 14:1-251.
44. Krishnankutty B, Bellary S, Kumar NB, Moodahadu LS. Data management in clinical research: an overview. Indian J Pharmacol 2012, 44:168-172.
45. Brandt CA, Argraves S, Money R, Ananth G, Trocky NM, Nadkarni PM. Informatics tools to improve clinical research study implementation. Contemp Clin Trials 2006, 27:112-122.
46. Fieldnotes: the making of anthropology 1990, Ithaca, New York: Cornell University Press, Sanjek R.
47. Finlay L. " Outing" the researcher: the provenance, process, and practice of reflexivity. Qual Health Res 2002, 12:531-545.
48. Barnes BR. The Hawthorne effect in community trials in developing countries. Int J Soc Res Methodol 2010, 13:357-370.
49. Will CM. The alchemy of clinical trials. Biosocieties 2007, 2:85-99.
50. Hesse-Biber S. Weaving a multimethodology and mixed methods praxis into randomized control trials to enhance credibility. Qual Inq 2012, 18:876-889.
51. Hertz R. Introduction: Reflexivity And Voice. Reflexivity And Voice 1997, vii-xviii. Thousand Oaks, C.A: SAGE Publications Inc, Hertz R.
52. Gough B. Deconstructing Reflexivity. Reflexivity: A Practical Guide for Researchers in Health and Social Sciences 2003, 21-36. Oxford: Blackwell Publishing Company, Finlay L, Gough B.
53. Moreira T, Will C. Conclusion: So What?. Medical Proofs, Social Experiments: Clinical Trials in Shifting Contexts 2010, 153-160. Farnham, UK: Ashgate, Will C, Moreira T.
54. Hawe P. The truth, but not the whole truth? Call for an amnesty on unreported results of public health interventions. J Epidemiol Community Health 2012, 66:285.
55. Marchal B, Dedzo M, Kegels G. A realist evaluation of the management of a well-performing regional hospital in Ghana. BMC Health Serv Res 2010, 10:24.