Examining implementer fidelity: Conceptualising and measuring adherence and competence

Journal of Children's Services

Volumn 6, Issue 1, 2011, Pages 18-33

  Cross, Wendi F ^a   West, Jennifer C ^a  

^a UNIVERSITY OF ROCHESTER MEDICAL CENTER   (United States)

Author keywords

Adherence; Competence; Fidelity; Implementer skill; Measurement; Observational methods

Indexed keywords

EID: 79955104912     PISSN: 17466660     EISSN: None     Source Type: Journal    
DOI: 10.5042/jcs.2011.0123     Document Type: Article

References (62)

1. Backer TE (2000) The failure of success: challenges of disseminating effective substance abuse prevention programs. Journal of Community Psychology 28 (3) 363-373. doi:10.1002/(SICI)1520-6629(200005)28:3<363::AID-JCOP10>3.0.CO;2-T.
2. Barber JP, Crits-Cristoph P & Luborsky L (1996) Effects of therapist adherence and competence on patient outcome in brief dynamic therapy. Journal of Consulting and Clinical Psychology 64 (3) 619-622. doi:10.1037/0022-006X.64.3.6198698958.
3. Brekke JS & Wolkon GH (1988) Monitoring program implementation in community mental health settings. Evaluation and the Health Professions 11 (4) 425-440. doi:10.1177/016327878801100402.
4. Bumbarger BK & Perkins DF (2008) After randomised trials: issues related to dissemination of evidence-based interventions. Journal of Children's Services 3 (2) 53-61.
5. Carroll KM, Nich C & Rounsaville BJ (1998) Use of observer and therapist ratings to monitor delivery of coping skills treatment for cocaine abusers: utility of therapist session checklists. Psychotherapy Research 8 307-320. doi:10.1093/ptr/8.3.307; doi:10.1080/10503309812331332407.
6. Carroll KM, Nich C, Sifry RL, Nuro KF, Frankforter TL, Ball SA, Fenton L & Rounsaville BJ (2000) A general system for evaluating therapist adherence and competence in psychotherapy research in the addictions. Drug and Alcohol Dependence 57 (3) 225-238. doi:10.1016/S0376-8716(99)00049-6.
7. Castonguay LG, Goldfried MF, Wiser S, Raue PJ & Hayes AM (1996) Predicting the effect of cognitive therapy for depression: a study of unique and common factors. Journal of Consulting and Clinical Psychology 64 (3) 497- 504. doi:10.1037/0022-006X.64.3.497.
8. Creed TA & Kendall PC (2005) Therapist alliancebuilding behavior within a cognitive-behavioral treatment for anxiety in youth. Journal of Consulting and Clinical Psychology 73 (3) 498-505. doi:10.1037/0022-006X.73.3.498.
9. Cross W (2009) A model of training and transfer of training. Paper presented to the Second Annual Conference on the Science of Implementation and Dissemination: Building Research Capacity to Bridge the Gap from Science to Service. NIMH. Bethesda: Maryland.
10. Cross W, West J, Wyman PA & Schmeelk-Cone KH (2009) Disaggregating and measuring implementer adherence and competence. Paper presented to the Second Annual Conference on the Science of Implementation and Dissemination: Building Research Capacity to Bridge the Gap from Science to Service. NIMH Bethesda: Maryland.
11. Cross W & Wyman PA (2005) Curriculum for the Rochester Child Resilience Project. Unpublished manuscript. Rochester, NY: University of Rochester.
12. Dane AV & Schneider BH (1998) Program integrity in primary and early secondary prevention: are implementation effects out of control? Clinical Psychology Review 18 (1) 23-45. doi:10.1016/S0272-7358(97)00043-3.
13. Dariotis JK, Bumbarger BK, Duncan LG & Greenberg MT (2008) How do implementation efforts relate to program adherence? Examining the role of organizational, implementer, and program factors. Journal of Community Psychology 36 (6) 744-760. doi:10.1002/ jcop.20255.
14. Davidson K, Scott J, Schmidt U, Tata P, Thornton S & Tyrer P (2004) Therapist competence and clinical outcome in the prevention of parasuicide by manual assisted behaviour therapy trial: the POPMACT study. Psychological Medicine 34 (5) 855-863. doi:10.1017/ S0033291703001855.
15. Dobson LD & Cook TJ (1980) Avoiding type III error in program evaluation: results from a field experiment. Evaluation and Program Planning 3 (4) 269-276. doi:10.1016/0149-7189(80)90042-7.
16. Domitrovich CE, Cortes RC & Greenberg MT (2007) Improving young children's social and emotional competence: a randomized trial of the preschool 'PATHS' curriculum. Journal of Primary Prevention 28 (2) 67-91. doi:10.1007/s10935-007-0081-0.
17. Domitrovich CE & Greenberg MT (2000) The study of implementation: current finding from effective programs for school-aged children. Journal of Educational and Psychological Consultation 11 (2) 193-221. doi:10.1207/ S1532768XJEPC1102_04.
18. Durlak JA (1997) School-based Prevention Programs for Children and Adolescents. New York: Plenum.
19. Durlak JAD & DuPre E (2008) Implementation matters: a review of research on the influence of implementation on program outcomes and the factors affecting implementation. American Journal of Community Psychology 41(3-4) 327-350.
20. Dusenbury L, Brannigan R, Falco M & Hansen WB (2003) A review of research on fidelity of implementation: implications for drug abuse prevention in school settings. Health Education Research 18 (2) 237- 256. doi:10.1093/her/18.2.237.
21. Fixsen DL, Naoom SF, Blase KA, Freidman RM & Wallace F (2005) Implementation Research: A Synthesis of the Literature. Tampa, FL: University of South Florida, Louis de la Parte Florida Mental Health Institute, The National Implementation Research Network.
22. Flannery-Schroeder E (2005) Treatment integrity: implications for training. Clinical Psychology: Science and Practice 12 (4) 388-390. doi:10.1093/clipsy/bpi047.
23. Forgatch MS, Patterson GR & DeGarmo DS (2005) Evaluating fidelity: predictive validity for a measure of competent adherence to the Oregon model of parent management training. Behavior Therapy 36 (1) 3-13. doi:10.1016/S0005-7894(05)80049-8.
24. Forgatch MS, Patterson GR, DeGarmo DS & Beldavs ZG (2009) Testing the Oregon delinquency model with nine-year follow-up of the Oregon Divorce Study. Development and Psychopathology 21 637-660. doi:10.1017/S0954579409000340.
25. Frank JC, Coviak CP, Healy TC, Belsa B & Casado BL (2008) Addressing fidelity in evidencebased health promotion programs for older adults. Journal of Applied Gerontology 27 (1) 4-33. doi:10.1177/0733464807308621.
26. Glasgow RE, Lichtenstein E & Marcus AC (2003) Why don't we see more translation of health promotion research to practice? Rethinking the efficacy-toeffectiveness transition. American Journal of Public Health 93 (8) 1261-7. doi:10.2105/AJPH.93.8.1261.
27. Greenberg MT, Domitrovich C & Bumbarger B (2001) The prevention of mental disorders in school-aged children: current state of the field. Prevention and Treatment 4 (1) 1-63. doi:10.1037/1522-3736.4.1.41a.
28. Greenberg MT, Kusche CA, Cook ET & Quamma JP (1995) Promoting emotional competence in schoolaged children: the effects of the PATHS curriculum. Development & Psychopathology 7 (1) 117-136. doi:10.1017/S0954579400006374.
29. Hansen WB, Graham JW, Wolkenstein BH & Rohrbach LA (1991) Program integrity as a moderator of prevention program effectiveness: results for fifth grade students in the adolescent alcohol prevention trial. Journal of Studies on Alcohol 52 (6) 568-579. PMid:1758184.
30. Harachi TW, Abbott RD, Catalano RF, Haggerty KP & Fleming CB (1999) Opening the black box: using process evaluation measures to assess implementation and theory building. American Journal of Community Psychology 27 (5) 711-731. doi:10.1023/A:1022194005511.
31. Henggeler SW, Melton GB, Brondino MJ, Schere DG & Hanley JH (1997) Multisystemic therapy with violent and chronic juvenile offenders and their families: the role of treatment fidelity in successful dissemination. Journal of Consulting and Clinical Psychology 65 821-833. doi:10.1037/0022-006X.65.5.821.
32. Henry WP, Schacht TE, Strupp HH, Butler SF & Binder JL (1993) Effects of training in time-limited dynamic psychotherapy: changes in therapist behavior. Journal of Consulting & Clinical Psychology 61 (3) 434-440. doi:10.1037/0022-006X.61.3.434.
33. Hill LG, Maucione K & Hood BK (2007) A focused approach to assessing program fidelity. Prevention Science 8 (1) 25-34. doi:10.1007/s11121-006-0051-4.
34. Hogue A, Henderson CE, Dauber S, Barajas PC, Fried A & Liddle HA (2008) Treatment adherence, competence, and outcome in individual and family therapy for adolescent behavior problems. Journal of Consulting and Clinical Psychology 76 (4) 544-555. doi:10.1037/0022-006X.76.4.544.
35. Huey SJ, Henggeler SW, Brondino MJ & Pickrel SG (2000) Mechanisms of change in multisystemic therapy: reducing delinquent behavior through therapist adherence and improved family and peer functioning. Journal of Consulting and Clinical Psychology 68 (3) 451- 467. doi:10.1037/0022-006X.68.3.451.
36. Hutchings J, Bywater T, Eames C & Martin P (2008) Implementing child mental health interventions in service settings: lessons from three pragmatic randomised controlled trials in Wales. Journal of Children's Services 3 (2) 17-27.
37. James IA, Blackburn IM, Milne DL & Reichfelt FK (2001) Moderators of trainee therapists' competence in cognitive therapy. British Journal of Clinical Psychology 40 (2) 131-141. doi:10.1348/014466501163580.
38. Knutson NM, Forgatch MS & Rains LA (2003) Fidelity of Implementation Rating System (FIMP): the training manual for PMTO. Eugene: Oregon Social Learning Center.
39. Lillehoj CJG, Griffin KW & Spoth R (2004) Program provider and observer ratings of school-based preventive intervention implementation: agreement and relation to youth outcomes. Health Education and Behavior 31 (2) 242-257. doi:10.1177/1090198103260514.
40. Lochman J, Boxmeyer C, Powell N, Qu L, Wells K & Windle M (2009) Dissemination of the Coping Power Program: importance of intensity of counselor training. Journal of Consulting and Clinical Psychology 77 (3) 397-409. doi:10.1037/a0014514.
41. McHugo J, Drake RE, Whitley R, Bond GR, Campbell K, Rapp CA, Goldman HH, Lutz WJ & Finnerty MT (2007) Fidelity outcomes in the National Implementing Evidence-Based Practices Project. Psychiatric Services 58 (10) 1279-1284. doi:10.1176/appi.ps.58.10.1279.
42. Mihalic SF & Irwin K (2003) Blueprints for violence prevention: from research to real-world settings - factors influencing the successful replication of model programs. Youth Violence and Juvenile Justice 1 (14) 307-329. doi:10.1177/1541204003255841.
43. Miller SJ & Binder JL (2002) The effects of manual-based training on treatment fidelity and outcome: a review of the literature on adult individual psychotherapy. Psychotherapy: Theory/Research/Practice/Training 39 (2) 184-198. doi:10.1037/0033-3204.39.2.184.
44. Miller WR & Mount KA (2001) A small study of training in motivational interviewing: does one workshop change clinician and client behavior? Behavioral and Cognitive Psychotherapy 29 457-471. doi:10.1017/ S1352465801004064.
45. Moore JE, Beck TC, Sylvertsen A & Domitrovich C (2009) Making sense of implementation: multiple dimensions, multiple sources, multiple methods. Paper presented to the Seventeenth Annual Meeting of the Society for Prevention Research. Washington, DC: USA.
46. Ogden T, Forgatch MS, Askeland E, Patterson GR & Bullock BM (2005) Implementation of parent management training at the national level: the case of Norway. Journal of Social Work Practice 19 (3) 317-329. doi:10.1080/02650530500291518.
47. Perepletchikova F & Kazdin AE (2005) Treatment integrity and therapeutic change: issues and research recommendations. Clinical Psychology: Science and Practice 12 (4) 365-383. doi:10.1093/clipsy/bpi045.
48. Poduska J, Kellam S, Brown CH, Ford C, Windham A, Keegan N & Wang W (2009) Study protocol for a group randomized controlled trial of a classroom-based intervention aimed at preventing early risk factors for drug abuse: integrating effectiveness and implementation research. Implementation Science 4 (56) 1-11.
49. Ringwalt CL, Ennett S, Johnson R, Rohrbach LA, Simons-Rudloph A, Vincus A & Thorne J (2003) Factors associated with fidelity to substance abuse prevention curriculum guides in the nation's middle schools. Health Education and Behavior 30 (3) 375-391. doi:10.1177/1090198103030003010.
50. Ringwalt CL, Pankratz MM, Jackson-Newsom J, Gottfredson NC, Hansen WB, Giles SM & Dusenbury L (2010) Three-year trajectory of teachers' fidelity to a drug prevention curriculum. Prevention Science 11 67-76. doi:10.1007/s11121-009-0150-0.
51. Rohrbach LA, Dent CW, Skara S, Sun P & Sussman S (2007) Fidelity of implementation in Project Towards No Drug Abuse (TND): a comparison of classroom teachers and program specialists. Prevention Science 8 (2) 125- 132. doi:10.1007/s11121-006-0056-z.
52. Rohrbach LA, Grana R, Sussman S & Valente T (2006) Type II translation: transporting preventive interventions from research to real-world settings. Evaluation and the Health Professions 29 (3) 302-333. doi:10.1177/0163278706290408.
53. Rohrbach LA, Gunning M, Sun P & Sussman S (2010) The Project Towards No Drug Abuse (TND) dissemination trial: implementation fidelity and immediate outcomes. Prevention Science 11 (1) 77-88. PMid:19002583; PMCid:2806652.
54. Shaw BF, Elkin I, Yamaguchi J, Olmstead M, Vallis TM, Dobson KS, Lowery A, Sotsky SM, Watkins JT & Imber SD (1999) Therapist competence ratings in relation to clinical outcome in cognitive therapy of depression. Journal of Consulting and Clinical Psychology 67 (6) 837- 846. doi:10.1037/0022-006X.67.6.837.
55. Snyder J, Reid J, Stoolmiller M, Howe G, Brown CH, Dagne G & Cross W (2006) Measurement systems for randomized intervention trials: the role of behavioral observation in studying treatment mediators. Prevention Science 7 (1) 43-56. doi:10.1007/s11121-005-0020-3.
56. Spoth R, Clair S, Greenberg M, Redmond C & Shin C (2007) Toward dissemination of evidence-based family interventions: maintenance of communitybased partnership recruitment results and associated factors. Journal of Family Psychology 21 (2) 137-146. doi:10.1037/0893-3200.21.2.137.
57. Stiles WB, Honos-Webb L & Surko M (1998) Responsiveness in psychotherapy. Clinical Psychology: Science and Practice 5 (4) 438-458. doi:10.1111/j.1468-2850.1998.tb00166.x.
58. Trepka C, Rees A, Shapiro DA, Hardy GE & Barkham M (2004) Therapist competence and outcome of cognitive therapy for depression. Cognitive Therapy and Research 28 (2) 143-157s. doi:10.1023/B:COTR.0000021536.39173.66.
59. Waltz J, Addis ME, Koerner K & Jacobson NS (1993) Testing the integrity of a psychotherapy protocol: assessment of adherence and competence. Journal of Consulting and Clinical Psychology 61 (4) 620-630. doi:10.1037/0022-006X.61.4.620.
60. Warren HK, Domitrovich CE & Greenberg MT (2009) Implementation quality in school-based research: roles for the prevention researcher. In: LM Dinella (Ed) Conducting Science-Based Psychology Research In Schools (pp129-151). Washington DC: American Psychological Association. doi:10.1037/11881-006.
61. Wyman PA, Cross W, Brown CH, Tu X & Yu Q (2010) Intervention to strengthen emotional self-regulation in children with emerging mental health problems: proximal impact on school behavior. Journal of Abnormal Child Psychology 38 (5) 707-720. doi: 10.1007/s10802-010-9398-X.
62. Zvoch K (2009) Treatment fidelity in multisite evaluation: A multilevel longitudinal examination of provider adherence status and change. American Journal of Evaluation 30 (1) 44-61. doi:10.1177/1098214008329523.