Evaluating complex interventions: Perspectives and issues for health behaviour change interventions

Psychology and Health

Volumn 30, Issue 1, 2015, Pages 35-51

  Tarquinio, Cyril ^a,b   Kivits, Joëlle ^a,b   Minary, Laetitia ^b,c,d   Coste, Joël ^b,e   Alla, François ^a,b,c,d  

^a UNIVERSITÉ DE LORRAINE   (France)

^b UNIVERSITÉ DE LORRAINE   (France)

^c UNIVERSITY HOSPITAL OF NANCY   (France)

^d INSERM   (France)

^e INSERM CIC EC CIE6 ^*  (France)

Author keywords

behaviour; bias; complex interventions; evaluation studies; health education; health promotion; psychology; public health; randomised controlled trial

Indexed keywords

HEALTH BEHAVIOR; HEALTH PROMOTION; HUMAN; PROCEDURES; PROGRAM EVALUATION; RANDOMIZED CONTROLLED TRIAL (TOPIC);

HEALTH BEHAVIOR; HEALTH PROMOTION; HUMANS; PROGRAM EVALUATION; RANDOMIZED CONTROLLED TRIALS AS TOPIC;

EID: 84918559715     PISSN: 08870446     EISSN: 14768321     Source Type: Journal    
DOI: 10.1080/08870446.2014.953530     Document Type: Article

References (81)

1. Ablon, J. S., & Jones, E. E. (2002). Validity of controlled clinical trials of psychotherapy: Findings from the NIMH treatment of depression collaborative research program. American Journal of Psychiatry, 159, 775–783.10.1176/appi.ajp.159.5.775
2. Arksey, H., & O’Malley, L. (2005). Scoping studies: Towards a methodological framework. International Journal of Social Research Methodology, 8, 19–32.10.1080/1364557032000119616
3. Barber, J. P., Connolly, M. B., Crits-Christoph, P., Gladis, L., & Siqueland, L. (2000). Alliance predicts patients’ outcome beyond in-treatment change in symptoms. Journal of Consulting and Clinical Psychology, 68, 1027–1032.10.1037/0022-006X.68.6.1027
4. Blaise, P., Marchal, B., Lefèvre, P., & Kegels, G. (2010). Au-delà des méthodes expérimentales: l'approche réaliste en évaluation. Réduire les inégalités sociales en santé [Beyond the experimental methods: The realistic evaluation approach]. In L. Potvin, M. J. Moquet, & C. Jones (Eds.), Reducing social inequalities in health. Paris: INPES.
5. Blatt, S. J., Sanislow, C. A., III, Zuroff, D. C., & Pilkonis, P. A. (1996). Characteristics of effective therapists: Further analyses of data from the National Institute of Mental Health Treatment of Depression Collaborative Research Program. Journal of Consulting and Clinical Psychology, 64, 1276–1284.10.1037/0022-006X.64.6.1276
6. Bonell, C., Fletcher, A., Morton, M., Lorenc, T., & Moore, L. (2012). Realist randomised controlled trials: A new approach to evaluating complex public health interventions. Social Science and Medicine, 75, 2299–2306.10.1016/j.socscimed.2012.08.032
7. Bonsergent, E., Agrinier, N., Thilly, N., Tessier, S., Legrand, K., Lecomte, E.,.. Briancon, S. (2013). Overweight and obesity prevention for adolescents: A cluster randomized controlled trial in a school setting. American Journal of Preventive Medicine, 44, 30–39.10.1016/j.amepre.2012.09.055
8. Brewin, C. R., & Bradley, C. (1989). Patient preferences and randomised clinical trials. British Medical Journal, 299, 313–315.10.1136/bmj.299.6694.313
9. Bugge, C., Williams, B., Hagen, S., Logan, J., Glazener, C., Pringle, S., & Sinclair, L. (2013). A process for decision-making after pilot and feasibility trials (ADePT): Development following a feasibility study of a complex intervention for pelvic organ prolapse. Trials, 14, 353–366.10.1186/1745-6215-14-353
10. Cambon, L., Minary, L., Ridde, V., & Alla, F. (2012). Transferability of interventions in health education: A review. BMC Public Health, 12, 497–510.10.1186/1471-2458-12-497
11. Campbell, D., & Stanley, J. (1963). Experimental and quasi-experimental designs for research. Hopewell, NJ: Houghton Mifflin Company.
12. Campbell, M., Fitzpatrick, R., Haines, A., Kinmonth, A. L., Sandercock, P., Spiegelhalter, D., & Tyrer, P. (2000). Framework for design and evaluation of complex interventions to improve health. British Medical Journal, 321, 694–696.10.1136/bmj.321.7262.694
13. Chambless, D. L., & Ollendick, T. H. (2001). Empirically supported psychological interventions: Controversies and evidence. Annual Review of Psychology, 52, 685–716.10.1146/annurev.psych.52.1.685
14. Colsher, P. L., & Wallace, R. B. (1989). Data quality and age: Health and psychobehavioral correlates of item nonresponse and inconsistent responses. Journal of Gerontology, 44, 45–52.
15. Craig, P., & Petticrew, M. (2013). Developing and evaluating complex interventions: Reflections on the 2008 MRC guidance. International Journal of Nursing Studies, 50, 585–587.10.1016/j.ijnurstu.2012.09.009
16. Department of Health and Human Services. (2009). Federal coordinating council for comparative effectiveness research. Report to the President and Congress. Washington, DC: Author.
17. Donner, A., & Klar, N. (2000). Design and analysis of cluster randomization trials. London: Arnold.
18. Dunn, G. (2002). The challenge of patient choice and nonadherence to treatment in randomized controlled trials of counseling or psychotherapy. Understanding Statistics, 1(1), 19–29.
19. Elvins, R., & Green, J. (2008). The conceptualization and measurement of therapeutic aliance: An empirical review. Clinical Psychology Review, 28, 1167–1187.10.1016/j.cpr.2008.04.002
20. Fernald, D. H., Coombs, L., DeAlleaume, L., West, D., & Parnes, B. (2012). An assessment of the Hawthorne Effect in practice-based research. Journal of the American Board of Family Medicine, 25, 83–86.
21. Fuller, D., & Potvin, L. (2012). Context by treatment interactions as the primary object of study in cluster randomized controlled trials of population health interventions. International Journal of Public Health, 57, 633–636.10.1007/s00038-012-0357-x
22. Green, R. J., & Herget, M. (1991). Outcomes of systemic/strategic team consultation: III. The importance of therapist warmth and active structuring. Family Process, 30, 321–336.10.1111/famp.1991.30.issue-3
23. Guevel, M. R., Pommier, J., & Jourdan, D. (2013). A health promotion intervention in French primary schools based on teacher training and support. Global Health Promotion, 20, 13–19.10.1177/1757975913483340
24. Guthrie, E. (2000). Psychotherapy for patients with complex disorders and chronic symptoms. The need for a new research paradigm. The British Journal of Psychiatry, 177, 131–137.10.1192/bjp.177.2.131
25. Hawe, P., Shiell, A., & Riley, T. (2004). Complex interventions: how “out of control” can a randomised controlled trial be? British Medical Journal, 328, 1561–1563.
26. Hawe, P., Shiell, A., & Riley, T. (2008). In response to Spillane V., Byrne M. C., Byrne M., Leathem C. S., O'Malley M. & Cupples M. E. (2007). Monitoring treatment fidelity in a randomized trial of a complex intervention. Journal of Advanced Nursing 60, 343–352. Important considerations for standardizing complex interventions. Journal of Advanced Nursing, 62, 267.
27. Heiman, G. W. (2002). Research methods in psychology (3rd ed.). Boston, MA: Houghton Mifflin Company.
28. Henry, W. P., Schacht, T. E., Strupp, H. H., Butler, S. F., & Binder, J. L. (1993). Effects of training in time-limited dynamic psychotherapy: Mediators of therapists' responses to training. Journal of Consulting and Clinical Psychology, 61, 441–447.10.1037/0022-006X.61.3.441
29. Higgins, J. P., & Altman, D. G. (2008). Assessing risk of bias in included studies. In J. P. T. Higgins, D. G. Altman, & J. A. C. Sterne (Eds.), Cochrane handbook for systematic reviews of interventions. Version 5.1.0 [updated March 2011]. Retrieved from www.cochrane-handbook.org.10.1002/9780470712184
30. Horvath, A. (2005). The therapeutic relationship: Research and theory. Psychotherapy Research, 15, 3–7.10.1080/10503300512331339143
31. Howard, I., Turner, R., Olkin, R., & Mohr, D. C. (2006). Therapeutic alliance mediates the relationship between interpersonal problems and depression outcome in a cohort of multiple sclerosis patients. Journal of Clinical Psychology, 62, 1197–1204.10.1002/(ISSN)1097-4679
32. Huisman, S., Maes, S., De Gucht, V. J., Chatrou, M., & Haak, H. R. (2009). Self-regulation and weight reduction in patients with type 2 diabetes: A pilot intervention study. Patient Education and Counseling, 75, 84–90.10.1016/j.pec.2008.09.027
33. Jack, S. M. (2006). Utility of qualitative research findings in evidence-based public health practice. Public Health Nursing, 23, 277–283.
34. Johansson, H., & Eklund, M. (2003). Patients’ opinion on what constitutes good psychiatric care. Scandinavian Journal of Caring Sciences, 17, 339–346.10.1046/j.0283-9318.2003.00233.x
35. Kannus, P., Parkkari, J., Niemi, S., Pasanen, M., Palvanen, M., Jarvinen, M., & Vuori, I. (2000). Prevention of hip fracture in elderly people with use of a hip protector. New England Journal of Medicine, 343, 1506–1513.
36. Kazdin, A. E. (2007). Mediators and mechanisms of change in psychotherapy research. Annual Review of Clinical Psychology, 3, 1–27.10.1146/annurev.clinpsy.3.022806.091432
37. Kim, D. M., Wampold, B. E., & Bolt, D. M. (2006). Therapist effects in psychotherapy: A random-effects modeling of the National Institute of Mental Health Treatment of Depression Collaborative Research Program data. Psychotherapy Research, 16, 161–172.
38. Klein, D. N., Schwartz, J. E., Santiago, N. J., Vivian, D., Vocisano, C., Castonguay, L. G., … Keller, M. B. (2003). Therapeutic alliance in depression treatment: Controlling for prior change and patient characteristics. Journal of Consulting and Clinical Psychology, 71, 997–1006.
39. Krupnick, J. L., Sotsky, S. M., Simmens, S., Moyer, J., Elkin, I., Watkins, J., & Pilkonis, P. A. (1996). The role of the therapeutic alliance in psychotherapy and pharmacotherapy outcome: Findings in the National Institute of Mental Health Treatment of Depression Collaborative Research Program. Journal of Consulting and Clinical Psychology, 64, 532–539.10.1037/0022-006X.64.3.532
40. Legrand, K., Bonsergent, E., Latarche, C., Empereur, F., Collin, J. F., Lecomte, E.,.. Briançon, S. (2012). Intervention dose estimation in health promotion programmes: A framework and a tool. Application to the diet and physical activity promotion PRALIMAP trial. BMC Medical Research Methodology, 12, 146–160.10.1186/1471-2288-12-146
41. Marchal, B., Westhorp, G., Wong, G., Van, B. S., Greenhalgh, T., Kegels, G., & Pawson, R. (2013). Realist RCTs of complex interventions – An oxymoron. Social Science and Medicine, 94, 124–128.
42. Martin, D. J., Garske, J. P., & Davis, M. K. (2000). Relation of the therapeutic alliance with outcome and other variables: A meta-analytic review. Journal of Consulting and Clinical Psychology, 68, 438–450.10.1037/0022-006X.68.3.438
43. Minary, L., Cambon, L., Martini, H., Wirth, N., Acouetey, D. S., Thouvenot, F., … Alla, F. (2013). Efficacy of a smoking cessation program in a population of adolescent smokers in vocational schools: A public health evaluative controlled study. BMC Public Health, 13, 149–159.
44. Minary, L., Martini, H., Wirth, N., Thouvenot, F., Acouetey, D. S., Martinet, Y., … Alla, F. (2009). TABADO: “Evaluation of a smoking cessation program among adolescents in vocational training centers”: Study protocol. BMC Public Health, 9, 411–417.
45. Mayo, E. (1933). The human problems of an industrial civilisation. New York, NY: Macmillan.
46. Mays, N., Roberts, E., & Popay, J. (2001). Synthesising research evidence. In N. Fulop, P. Allen, A. Clarke, & N. Black (Eds.), Studying the organization and delivery of health services: Research methods (pp. 188–220). London: Routledge.
47. Meyer, J. (2000). Qualitative research in health care. Using qualitative methods in health related action research. British Medical Journal, 320, 178–181.
48. Muller, L., & Spitz, E. (2003). Multidimensional assessment of coping: Validation of the brief COPE among French population. Encephale, 29, 507–518.
49. Musick, M., Wilson, J., & Bynum Jr., W. B. (2000). Race and formal volunteering: The differential effects of class and religion. Social Forces, 78, 1539–1571.
50. Mynors-Wallis, L. M., Gath, D. H., Day, A., & Baker, F. (2000). Randomised controlled trial of problem solving treatment, antidepressant medication, and combined treatment for major depression in primary care. British Medical Journal, 320, 26–30.10.1136/bmj.320.7226.26
51. Oakley, A., Strange, V., Bonell, C., Allen, E., & Stephenson, J. (2006). Process evaluation in randomised controlled trials of complex interventions. British Medical Journal, 332, 413–416.10.1136/bmj.332.7538.413
52. Oswald, P. A. (2003). Does the interpersonal reactivity index perspective-taking scale predict who will volunteer time to counsel adults entering college? Perceptual and Motor Skills, 97, 1184–1186.
53. Petticrew, M., Cummins, S., Ferrell, C., Findlay, A., Higgins, C., Hoy, C., … Sparks, L. (2005). Natural experiments: An underused tool for public health? Public Health, 119, 751–757.
54. Porta, M. (2008). A dictionary of epidemiology (5th ed.). New York, NY: Oxford University Press.
55. Power, R., Langhaug, L. F., Nyamurera, T., Wilson, D., Bassett, M. T., & Cowan, F. M. (2004). Developing complex interventions for rigorous evaluation–A case study from rural Zimbabwe. Health Education Research, 19, 570–575.10.1093/her/cyg073
56. Ridde, V., & Haddad, S. (2013). Pragmatism and realism for public health intervention evaluation. Rev Epidemiol Sante Publique, 61, S95–S106.
57. Ridde, V., & Mak, G. (2009). La place des données qualitatives dans les évaluations en santé publique [Realist approach to the test of actual program evaluation]. Actualité et dossier en santé publique, 69, 32–34.
58. Ridde, V., Robert, E., Guichard, A., Blaise, P., & Van Olmen, J. (2012). L'approche realist à l'épreuve du réel de l'évaluation des programmes [Realist approach to the test of actual program evaluation]. Canadian Journal Programme Evaluation, 26, 37–59.
59. Roland, M., & Torgerson, D. J. (1998). What are pragmatic trials? British Medical Journal, 316, 285.10.1136/bmj.316.7127.285
60. Rychetnik, L., Frommer, M., Hawe, P., & Shiell, A. (2002). Criteria for evaluating evidence on public health interventions. Journal of Epidemiology and Community Health, 56, 119–127.10.1136/jech.56.2.119
61. Schulz, K. F., & Grimes, D. A. (2002). Blinding in randomised trials: Hiding who got what. Lancet, 359, 696–700.10.1016/S0140-6736(02)07816-9
62. Schwartz, D., & Lellouch, J. (1967). Explanatory and pragmatic attitudes in therapeutical trials. Journal of Chronic Diseases, 20, 637–648.10.1016/0021-9681(67)90041-0
63. Shirk, S. R., & Karver, M. (2003). Prediction of treatment outcome from relationship variables in child and adolescent therapy: A meta-analytic review. Journal of Consulting and Clinical Psychology, 71, 452–464.
64. Stiles, W. B. (1999). Signs and voices in psychotherapy. Psychotherapy Research, 9, 1–21.10.1080/10503309912331332561
65. Strupp, H. H., & Hadley, S. W. (1979). Specific vs nonspecific factors in psychotherapy. A controlled study of outcome. Archives of General Psychiatry, 36, 1125–1136.10.1001/archpsyc.1979.01780100095009
66. Tessier, S. (2008). Effets bénéfiques de l’activité physique sur la qualité de vie liée à la santé et la corpulence: une approche épidémiologique et évaluative [Beneficial effects of the physical activity on health-related quality of life and the corpulence: Epidemiological and evaluative approaches] (Thèse). Université de Lorraine, p. 308.
67. Tessier, S., Vuillemin, A., & Briançon, S. (2006). Faut-il respecter la clause d’ambivalence dans les essais avec randomisation collective? [Should we respect the clause of ambivalence in trials with randomization group?]. Revue d'Epidémiologie et de Santé Publique 54 (Hors Série II), 89.
68. Thabane, L., Ma, J., Chu, R., Cheng, J., Ismaila, A., Rios, L. P.,.. Goldsmith, C. H. (2010). A tutorial on pilot studies: The what, why and how. BMC Medical Research Methodology, 10, 1–10.
69. The Cochrane Collaboration. (2004). Cochrane reviewers’ handbook 4.2.2 (1st ed.). Chichester: John Wiley & Sons.
70. Torgerson, D., & Torgerson, C. (2008). Designing and running randomised trials in health, education and the social sciences. Basingstoke: Palgrave Macmillan.10.1057/9780230583993
71. Treweek, S., & Zwarenstein, M. (2009). Making trials matter: Pragmatic and explanatory trials and the problem of applicability. Trials, 10, 37–46.10.1186/1745-6215-10-37
72. Trompette, J., Kivits, J., Minary, L., Cambon, L., & Alla, F. (2013). Stakeholders' perceptions of transferability criteria for health promotion interventions: A case study. Unpublished work.
73. Valbak, K. (2004). Suitability for psychoanalytic psychotherapy: A review. Acta Psychiatrica Scandinavica, 109, 164–178.10.1046/j.1600-0447.2003.00248.x
74. Victora, C. G., Habicht, J. P., & Bryce, J. (2004). Evidence-based public health: Moving beyond randomized trials. American Journal of Public Health, 94, 400–405.10.2105/AJPH.94.3.400
75. Wampold, B. E., & Brown, G. S. (2005). Estimating variability in outcomes attributable to therapists: A naturalistic study of outcomes in managed care. Journal of Consulting and Clinical Psychology, 73, 914–923.10.1037/0022-006X.73.5.914
76. Wang, S., Moss, J. R., & Hiller, J. E. (2006). Applicability and transferability of interventions in evidence-based public health. Health Promotion International, 21, 76–83.
77. Weisberg, H. I., Hayden, V. C., & Pontes, V. P. (2009). Selection criteria and generalizability within the counterfactual framework: Explaining the paradox of antidepressant-induced suicidality? Clinical Trials, 6, 109–118.10.1177/1740774509102563
78. Westen, D., & Weinberger, J. (2004). When clinical description becomes statistical prediction. American Psychologist, 59, 595–613.10.1037/0003-066X.59.7.595
79. WHO. (2001). Evaluation in health promotion: Principles and perspectives (Vol. 3). Copenhagen: Author.
80. Wood, L., Egger, M., Gluud, L. L., Schulz, K. F., Juni, P., Altman, D. G., … Sterne, J. A. (2008). Empirical evidence of bias in treatment effect estimates in controlled trials with different interventions and outcomes: Meta-epidemiological study. British Medical Journal, 336, 601–605.10.1136/bmj.39465.451748.AD
81. Zelen, M. (1979). A new design for randomized clinical trials. New England Journal of Medicine, 300, 1242–1245.10.1056/NEJM197905313002203