Fidelity criteria: Development, measurement, and validation

American Journal of Evaluation

Volumn 24, Issue 3, 2003, Pages 315-340

  Mowbray, Carol T ^a   Holter, Mark C ^a   Teague, Gregory B ^a   Bybee, Deborah ^a  

^a UNIVERSITY OF MICHIGAN   (United States)

Author keywords

[No Author keywords available]

Indexed keywords

EID: 0142056128     PISSN: 10982140     EISSN: None     Source Type: Journal    
DOI: 10.1016/S1098-2140(03)00057-2     Document Type: Article

References (76)

1. Allness, D. J., & Knoedler, W. H. (1998). The PACT model of community-based treatment for persons with severe and persistent mental illnesses: A manual for PACT start-up. Arlington, VA: National Alliance for the Mentally Ill (NAMI).
2. Anthony W.A. Cohen M. Farkas M. A psychiatric rehabilitation treatment program: Can I recognize one if I see one? Community Mental Health Journal 18 1982 83-96
3. Babor, T. F., Steinberg, K. L., McRee, B., Vendetti, J., & Carroll, K. M. (2001). Treating marijuana dependence in adults: A multi-site, randomized clinical trial. In J. M. Herrell & R. B. Straw (Eds.), Conducting multiple site evaluations in real-world settings. New Directions for Evaluation, no. 94 (pp. 17-30). San Francisco, CA: Jossey-Bass.
4. Bachrach L.L. The chronic patient: On exporting and importing model programs Hospital and Community Psychiatry 39 1988 1257-1258
5. Banks, S., McHugo, G. J., Williams, V., Drake, R. E., & Shinn, M. (2001). A prospective meta-analytic approach in a multi-site study of homelessness prevention. In J. M. Herrell & R. B. Straw (Eds.), Conducting multiple site evaluations in real-world settings. New Directions for Evaluation, no. 94 (pp. 45-59). San Francisco, CA: Jossey-Bass.
6. Becker D.R. Smith J. Tanzman B. Drake R.E. Tremblay T. Fidelity of supported employment programs and employment outcomes Psychiatric Services 52 2001 834-836
7. Berman, P. (1981). Educational change: An implementation paradigm. In R. Lehming & M. Kane (Eds.), Improving schools: Using what we know (pp. 253-286). Thousand Oaks, CA: Sage.
8. Bilsker D. Goldner E.M. Routine outcome measurement by mental health-care providers: Is it worth doing? The Lancet 360 2002 1689-1690
9. Blakely C.H. Mayer J.P. Gottschalk R.G. et al.The fidelity-adaptation debate: Implications for the implementation of public sector social programs American Journal of Community Psychology 15 1987 253-268
10. Bond G.R. Becker D.R. Drake R.E. et al. A fidelity scale for the individual placement and support model of supported employment Rehabilitation Counseling Bulletin 40 1997 265-284
11. Bond G.R. Evans L. Salyers M.P. Williams J. Kim H.W. Measurement of fidelity in psychiatric rehabilitation Mental Health Services Research 22 2000 75-87
12. Bond, G. R., Williams, J., Evans, L., et al. (2000). PN-44-psychiatric rehabilitation fidelity toolkit. Cambridge, MA: Human Services Research Institute.
13. Brekke J.S. What do we really know about community support programs? Strategies for better monitoring Hospital and Community Psychiatry 39 1988 946-952
14. Brekke J.S. Test M.A. A model for measuring the implementation of community support programs: Results from three sites Community Mental Health Journal 28 1992 227-247
15. Centre for Evidence-Based Medicine. (2002). Levels of evidence and grades of recommendation. Oxford, England (http://minerva.minervation.com/cebm/).
16. Chambless D.L. Ollendick T.H. Empirically supported psychological interventions: Controversies and evidence Annual Review of Psychology 52 2001 685-716
17. Chen, H. (1990). Theory-driven evaluations. Thousand Oaks, CA: Sage.
18. Clarke G. Intervention fidelity in the psychosocial prevention and treatment of adolescent depression Journal of Prevention and Intervention in the Community 17 1998 19-33
19. Donabedian, A. (1982). The criteria and standards of quality. Ann Arbor, MI: Health Administration Press.
20. Drake R. Goldman H. Leff H. Lehman A. Dixon L. Mueser K. Torrey W. Implementing evidence-based practices in routine mental health service settings Psychiatric Services 52 2001 179-182
21. Drake R.E. McHugo G.J. Becker D.R. The New Hampshire study of supported employment for people with severe mental illness Journal of Consulting and Clinical Psychology 64 1996 391-399
22. Fairweather, G. W., & Tornatzsky, L. G. (1971). Experimental methods for social policy research. New York, NY: Pergamon Press.
23. Frese F.J. Stanley J. Kress K. Vogel-Scibilia S. Integrating evidence-based practices and the recovery model Psychiatric Services 52 2001 1462-1468
24. Friesen, B. J., Green, B. L., Kruzich, J. M., Simpson, J., et al. (2002). Guidance for program design: Addressing the mental health needs of young children and their families in early childhood education settings. Retrieved May 23, 2002, from Portland State University, Reseach & Training Center on Family Support and Children's Mental Health Web site: http://www.rtc.pdx.edu/pgProjGuidance.php.
25. Giesler L.J. Hodge M. Case management in behavioral health care International Journal of Mental Health 27 1998 26-40
26. Glisson C. Hemmelgarn A. The effects of organizational climate and interorganizational coordination on the quality and outcomes of children's service systems Child Abuse and Neglect 22 1998 401-421
27. Hedeker D. McMahon S.D. Jason L.A. Salina D. Analysis of clustered data in community psychology: With an example from a worksite smoking cessation project American Journal of Community Psychology 22 1994 595-615
28. Henggeler S.W. Melton G.B. Brondino M.J. Scherer D.G. Hanley J.H. Multisystemic therapy with violent and chronic juvenile offenders and their families: The role of treatment fidelity in successful dissemination Journal of Consulting and Clinical Psychology 65 1997 821-833
29. Henggeler, S. W., & Schoenwald, S. K. (1998). The MST supervisory manual: Promoting quality assurance at the clinical level. Charleston, SC: MST Institute.
30. Henggeler S.W. Schoenwald S.K. Liao J.G. Letourneau E.J. Edwards D.L. Transporting efficacious treatments to field settings: The link between supervisory practices and therapist fidelity in MST programs Journal of Clinical Child and Adolescent Psychology 31 2002 155-167
31. Henry W.P. Butler S.F. Strupp H.H. Schacht T.E. Effects of training in time-limited dynamic psychotherapy: Changes in therapist behavior Journal of Consulting and Clinical Psychology 61 1993 434-440
32. Hermann R.C. Finnerty M. Provost S. Palmer R.H. Chan J. Lagodmos G. Teller T. Myrhol B.J. Process measures for the assessment and improvement of quality of care for schizophrenia Schizophrenia Bulletin 28 2002 95-104
33. Hernandez M. Gomez A. Lipien L. Greenbaum P.E. et al. Use of the system-of-care practice review in the national evaluation: Evaluating the fidelity of practice to system-of-care principles Journal of Emotional and Behavioral Disorders 9 2001 43-52
34. Hohmann A.A. Shear M.K. Community-based intervention research: Coping with the "noise" of real life in study design American Journal of Psychiatry 159 2002 201-207
35. Holden E.W. O'Connell S.R. Connor T. Branna A.M. Foster E.M. Blau G. Panciera H. Evaluation of the Connecticut Title IV-E Waiver Program: Assessing the effectiveness, implementation fidelity, and cost/benefits of a continuum of care Children and Youth Services Review 24 2002 409-430
36. Holter, M. C., Mowbray, C. T., Bellamy, C., MacFarlane, P., & Dukarski, J. (in press). "Critical ingredients" of consumer run services: Results of a national survey. Community Mental Health Journal.
37. Institute of Medicine. (2001). Improving the quality of long-term care. Washington, DC: National Academy Press.
38. Johnsen M. Samberg L. Calsyn R. Blasinsky M. et al. Case management models for persons who are homeless and mentally ill: The ACCESS demonstration project Community Mental Health Journal 35 1999 325-346
39. Kelly J.A. Heckman T.G. Stevenson L.Y. Williams P.N. Transfer of research-based HIV prevention interventions to community service providers: Fidelity and adaptation AIDS Education and Prevention 12 2000 87-98
40. Lebow J. Research assessing consumer satisfaction with mental health treatment: A review of findings Evaluation and Program Planning 6 1983 211-236
41. Leff, H. S., & Mulkern, V. (2002). Lessons learned about science and participation from multi-site evaluations. In J. M. Herrell & R. B. Straw (Eds.), Conducting multiple site evaluations in real-world settings. New Directions for Evaluation, no. 94 (pp. 89-100). San Francisco, CA: Jossey-Bass.
42. Luborsky L. McLellan A.T. Diguer L. Woody G. Seligman D.A. The psychotherapist matters: Comparison of outcomes across twenty-two therapists and seven patient samples Clinical Psychology: Science & Practice 4 1997 53-65
43. Lucca A.M. A Clubhouse fidelity index: Preliminary reliability and validity results Mental Health Services Research 2 2000 89-94
44. Macias C. Propst R. Rodican C. Boyd J. Strategic planning for ICCD clubhouse implementation: Development of the Clubhouse Research and Evaluation Screening Survey (CRESS) Mental Health Services Research 3 2001 155-167
45. Malysiak, R., Duchnowski, A., Black, M., & Greeson, M. (1996). Establishing wrap around fidelity through participatory evaluation. In Proceedings of the Ninth Annual Research Conference. A System of Care for Children's Mental Health: Expanding the Research Base.
46. McGrew J.H. Bond G.R. Dietzen L. Salyers M. Measuring the fidelity of implementation of a mental health program model Journal of Consulting and Clinical Psychology 62 1994 670-678
47. McGrew J.H. Pescosolido B. Wright E. Case managers' perspectives on critical ingredients of assertive community treatment and on its implementation Psychiatric Services 54 2003 370-376
48. McHugo G.J. Drake R.E. Teague G.B. Xie H. The relationship between model fidelity and client outcomes in the New Hampshire Dual Disorders Study Psychiatric Services 50 1999 818-824
49. Mills S.C. Ragan T.J. A tool for analyzing implementation fidelity of an integrated learning system (ILS) Educational Technology Research and Development 48 2000 21-41
50. Moncher F.J. Prinz R.J. Treatment fidelity in outcome studies Clinical Psychology Review 11 1991 247-266
51. Mueser K.T. Bond G.R. Drake R.E. et al. Models of community care for severe mental illness: A review of research on case management Schizophrenia Bulletin 24 1998 37-74
52. National MH Advisory Council. (1999). Bridging science and service: A report by the National Advisory Mental Health Council's Clinical Treatment and Services Research Workgroup.
53. Nguyen T.D. Attkisson C.C. Stegner B.L. Assessment of patient satisfaction: Development and refinement of a service questionnaire Evaluation and Program Planning 6 1983 299-314
54. NH-Dartmouth Psychiatric Research Center. (2002, January). Implementing Evidence-Based Practices Project National Meeting. In T. Singer & P. W. Singer (Eds.), Implementing Evidence-Based Practices Project Newsletter, 1.
55. Orwin R.G. Assessing program fidelity in substance abuse health services research Addiction 95Suppl. 3 2000 S309-S327
56. Paulson R.I. Post R.L. Herinckx H.A. Risser P. Beyond components: Using fidelity scales to measure and assure choice in program implementation and quality assurance Community Mental Health Journal 38 2002 119-128
57. Price, R. H., Friedland, D. S., Choi, J., & Caplan, R. D. (1998). Job-loss and work transitions in a time of global economic change. In X. Arriaga & S. Oskamp (Eds.), Addressing community problems: Psychological research and interventions (pp. 195-222). Thousand Oaks, CA: Sage.
58. Rapp C.A. The active ingredients of effective case management: A research synthesis Community Mental Health Journal 34 1998 363-380
59. Raudenbush, S., & Bryk, A. (2001). Hierarchical linear models: Applications and data analysis methods (2nd ed.). Thousand Oaks, CA: Sage.
60. Rog, D. J., & Randolph, F. L. (2002). A multi-site evaluation of supported housing: Lessons learned from cross-site collaboration. In J. M. Herrell & R. B. Straw (Eds.), Conducting multiple site evaluations in real-world settings. New Directions for Evaluation, no. 94 (pp. 61-72). San Francisco, CA: Jossey-Bass.
61. Salyers, M. P., Bond, G. R., Teague, G. B., Cox, J. F., Smith, M. E., Hicks, M. L., & Koop, J. I. (in press). Is it ACT yet? Real-world examples of evaluating the degree of implementation for assertive community treatment. Journal of Behavioral Health Services and Research.
62. Sechrest, L., West, S. G., Phillips, M. A., Redner, R., & Yeaton, W. (1979). Some neglected problems in evaluation research: Strength and integrity of treatments. In L. Sechrest, S. G. West, M. A. Phillips, R. Redner, & W. Yeaton (Eds.), Evaluation studies review annual (Vol. 4, pp. 15-35). Thousand Oaks, CA: Sage.
63. Snijders, T., & Bosker, R. (1999). Multilevel analysis: An introduction to basic and advanced multilevel modeling. Thousand Oaks, CA: Sage.
64. Stein L.I. Test M.A. An alternative to mental health treatment. I: Conceptual model, treatment program, and clinical evaluation Archives of General Psychiatry 37 1980 392-397
65. Szulanski G. Winter S. Getting it right the second time Harvard Business Review 80 2002 62-69
66. Teague, G. B. (2000). Patient perceptions of care measures. In A. J. Rush, H. A. Pincus, et al. (Eds.), Handbook of psychiatric measures (pp. 169-194). Washington, DC: American Psychiatric Association.
67. Teague G.B. Bond G.R. Drake R.E. Program fidelity and Assertive Community Treatment: Development and use of a measure American Journal of Orthopsychiatry 68 1998 216-232
68. Teague G.B. Drake R.E. Ackerson T.H. Evaluating use of continuous treatment teams for persons with mental illness and substance abuse Psychiatric Services 46 1995 689-695
69. Teague G.B. Ganju V. Hornik J.A. Johnson J.R. McKinney J. The MHSIP mental health report card: A consumer-oriented approach to monitoring the quality of mental health plans Evaluation Review 213 1997 330-341
70. Unrau Y.A. Using client exit interviews to illuminate outcomes in program logic models: A case example Evaluation and Program Planning 24 2001 353-361
71. Vincent M.L. Paine-Andrews A. Fisher J. Devereaux R.S. et al. Replication of a community-based multicomponent teen pregnancy prevention model: Realities and challenges Family and Community Health 23 2000 28-45
72. Weisman A. Nuechterlein K.H. Goldstein M.J. Snyder K.S. Controllability perceptions and reactions to symptoms of schizophrenia: A within-family comparison of relatives with high and low expressed emotion Journal of Abnormal Psychology 109 2000 167-171
73. Weisman A. Tompson M.C. Okazaki S. Gregory J. et al. Clinicians' fidelity to a manual-based family treatment as a predictor of the one-year course of bipolar disorder Family Process 41 2002 123-131
74. Yates B.T. Toward the incorporation of costs, cost-effectiveness analysis, and cost-benefit analysis into clinical research Journal of Consulting and Clinical Psychology 62 1994 729-736
75. Yates B.T. Cost-effectiveness analysis, cost-benefit analysis, and beyond: Evolving models for the scientist-manager-practitioner Clinical Psychology: Science & Practice 2 1995 385-398
76. Yates, B. T. (1996). Analyzing cost, procedures, processes, and outcomes in human services. Thousand Oaks, CA: Sage.