Beyond efficacy and effectiveness: A multifaceted approach to treatment evaluation

Professional Psychology: Research and Practice

Volumn 37, Issue 4, 2006, Pages 389-397

  Nelson, Timothy D ^a,b   Steele, Ric G ^a  

^a UNIVERSITY OF KANSAS   (United States)

^b 2006 Dole Human Development Center ^*  (United States)

Author keywords

Consumer satisfaction; Economic evaluation; Evidence based practice; Provider evaluation; Treatment evaluation

Indexed keywords

EID: 33746912301     PISSN: 07357028     EISSN: None     Source Type: Journal    
DOI: 10.1037/0735-7028.37.4.389     Document Type: Article

References (66)

1. Aarons, G. A. (2005). Measuring provider attitudes toward evidence-based practice: Considerations of organizational context and individual differences. Child and Adolescent Psychiatric Clinics of North America, 14, 255-271.
2. American Psychological Association (2005). Policy statement on evidencebased practice in psychology. Retrieved September 9. 2005. from http://www.apa.org/practice/ebpstatement.pdf.
3. Attkisson, C., & Greenfield, T. K. (20041. The UCSF Client Satisfaction Scales: I. The client satisfaction questionnaire-8. In M. E. Maruish (Ed.), The use of psychological testing for treatment planning and outcomes assessment: Vol. 3: Instruments for adults (3rd ed, pp. 799-811). Mahwah, NJ: Erlbaum.
4. Barkley, R. A., Edwards, G., Laneri, M., Fletcher, K., & Metevia, L. (2001). The efficacy of problem-solving communication training alone, behavior management training alone, and their combination for parentadolescent conflict in teenagers with ADHD and ODD. Journal of Consulting and Clinical Psychology, 69, 926-941.
5. Beuller, L. E., Williams, R. E., Wakefield, P. J., & Entwistle, S. R. (1995). Bridging scientist and practitioner perspectives in clinical psychology. American Psychologist, 50, 984-994.
6. Chambless, D. L., & Hollon, S. D. (1998). Defining empirically supported therapies. Journal of Consulting and Clinical Psychology, 66, 7-18.
7. Chambless, D. L., & Ollendick, T. H. (2001). Empirically supported psychological interventions: Controversies and evidence. Annual Review of Psychology, 52, 685-716.
8. Chambless, D. L., Sanderson, W. C., Shoham, V., Bennett-Johnson, S. B., Pope, K. S., Crits-Christoph, P., et al. (1996). An update on empirically validated therapies. Clinical Psychologist, 49, 5-18.
9. Chorpita, B. F. (2003). The frontier of evidence-based practice. In A. E. Kazdin & J. R. Weisz (Eds.), Evidence-based psychotherapies for children and adolescents (pp. 42-59). New York: Guilford.
10. Chorpita, B. F., & Donkervoet, C. (2005). Implementation of the Felix Consent Decree in Hawaii: The impact of policy and practice development efforts on service delivery. In R. G. Steele & M. C. Roberts (Eds.), Handbook of mental health services for children, adolescents, and families (pp. 317-332). New York: Kluwer Academic/Plenum Publishers.
11. Evans, E., & User, Y. I. (2004). Pilot-testing a statewide outcome monitoring system: Overview of the California Treatment Outcome Project (CALTOP). Journal of Psychoactive Drugs, Suppl. 2, 109-114.
12. Fals-Stewart, W., Yates, B. T., Klostermann, K. (2005). Assessing the costs, benefits, cost-benefit ratio, and cost-effectiveness of marital and family treatments: Why we should and how we can. Journal, of Family Psychology, 19, 28-39.
13. Foster, E. M., Dodge, K. A., & Jones, D. (2003). Issues in the economic evaluation of prevention programs. Applied Developmental Science, 7, 76-86.
14. Glisson, C. (2002). The organizational context of children's mental health services. Clinical Child and Family Psychology Review, 5, 233-253.
15. Gotham, H. J. (2004). Diffusion of mental health and substance abuse treatments: Development, dissemination, and implementation. Clinical Psychology: Science and Practice, 11, 160-176.
16. Greenfield, T. K., & Attkisson, C. (2004). The UCSF Client Satisfaction Scales: I. The Service Satisfaction Scale-30. In M. E. Maruish (Ed.), The use of psychological testing for treatment planning and outcomes assessment: Vol. 3: Instruments for adults (3rd ed, pp. 813-837). Mahwah, NJ: Erlbaum.
17. Hawley, K. M., & Weisz, J. R. (2005). Youth versus parent working alliance in usual clinical care: Distinctive associations with retention, satisfaction, and treatment outcome. Journal of Clinical Child and Adolescent Psychology, 34, 117-128.
18. Henggeler, S. W. (1999). Multisystemic therapy: An overview of clinical procedures, outcomes, and policy implications. Child Psychology and Psychiatry Review, 4, 2-10.
19. Henggeler, S. W., & Lee, T. (2003). Multisystemic treatment of serious clinical problems. In A. E. Kazdin & J. R. Weisz (Eds.), Evidence-based psychotherapies for children and adolescents (pp. 301-322). New York: Guilford.
20. Henggeler, S. W., Melton, G. B., Brondino, M. J., Scherer, D. G., & Hanley, J. H. (1997). Multisystemic therapy with violent and chronic juvenile offenders and their families: The role of treatment fidelity in successful dissemination. Journal of Consulting and Clinical Psychology, 65, 821-833.
21. Henggeler, S. W., Rowland, M. D., Randall, J., Ward, D. M., Pickrel, S. G., Cunningham, P. B., et al. (1999). Home-based multisystemic therapy as an alternative to the hospitalization of youths in psychiatric crisis: Clinical outcomes. Journal of the American Academy of Child and Adolescent Psychiatry, 38, 1331-1339.
22. Hoagwood, K., Jensen, P. S., Petti, T., Burns, B. J. (1996). Outcomes of mental health care for children and adolescents: I. A comprehensive conceptual model. Journal of the American Academy of Child and Adolescent Psychiatry, 35, 1055-1063.
23. Hoagwood, K., & Olin, S. (2002). The NIMH blueprint for change report: Research priorities in child and adolescent mental health. Journal of the American Academy of Child and Adolescent Psychiatry, 41, 760-767.
24. Hudak, P. L., MoKeever, P. D., & Wright, J. G. (2004). Understanding the meaning of satisfaction with treatment outcome. Medical Care, 42, 718-725.
25. Ingram, R. E., Hayes, A., & Scott, W. (2000). Empirically supported treatments: A critical analysis. In C. R. Snyder & R. E. Ingram (Eds.), Handbook of psychological change (pp. 40-60). New York: Wiley.
26. Institute of Medicine (2001). Crossing the quality chasm: A new health system for the 21st century. Washington, DC: National Academy Press.
27. Kazdin, A. E. (1997). A model for developing effective treatments: Progression and interplay of theory, research, and practice. Journal of Clinical Child Psychology, 26, 114-129.
28. Kazdin, A. E. (2000). Psychotherapy for children and adolescents: Directions for research and practice. New York: Oxford University Press.
29. Kazdin, A. E. (2006). Arbitrary metrics: Implications for identifying evidence-based treatments. American Psychologist, 61, 42-49.
30. Kazdin, A. E., & Nock, M. K. (2003). Delineating mechanisms of change in child and adolescent therapy. Journal of Child Psychology and Psychiatry, 44, 1116-1129.
31. Kazdin, A. E., & Weisz, J. R. (1998). Identifying and developing empirically supported child and adolescent treatments. Journal of Consulting and Clinical Psychology, 66, 19-36.
32. Lieberman, J. A., Stroup, T. S., McEvoy, J. P., Swartz, M. S., Rosenheck, R. A., Perkins, D. O., et al. (2005). Effectiveness of antipsychotic drugs in patients with chronic schizophrenia. New England Journal of Medicine, 353, 1209-1223.
33. Lorenz, K. A., Ryan, G. W., Morton, S. C., Chan, K. S., Wang, S., & Shekelle, P. G. (2005). A qualitative examination of primary care providers' and physician managers' uses and views of research evidence. International Journal for Quality in Health Care, 17, 1-6.
34. Martin, G. W., Herie, M. A., Turner, B. J., & Cunningham, J. A. (1998). A social marketing model for disseminating research-based treatments to addictions treatment providers. Addiction, 93, 1703-1715.
35. Martino, S., Carroll, K. M., O'Malley, S. S., & Rounsaville, B. J. (2000). Motivational interviewing with psychiatrically ill substance abusing patients. American Journal on Addictions, 9, 88-91.
36. Najavits, L. M. (2002). Clinicians' views on treating posttraumatic stress disorder and substance use disorder. Journal of Substance Abuse Treatment, 22, 79-85.
37. Najavits, L. M., Ghinassi, F., Van Horn, A., Weiss, R. D., Siqueland, L., Frank, A., et al. (2004). Therapist satisfaction with four manual-based treatments on a national multisite trial: An exploratory study. Psychotherapy: Theory, Research, Practice, Training, 41, 26-37.
38. Najavits, L. M., Weiss, R. D., Shaw, S. R., & Dierbeger, A. E. (2000). Psychotherapists' views of treatment manuals. Professional Psychology: Research and Practice, 31, 404-408.
39. National Advisory Mental Health Council. (2001). Bridging science and service. Washington, DC: National Institute of Mental Health.
40. Nelson, T. D., Steele, R. G., & Mize, J. A. (in press). Practitioner attitudes toward evidence-based practice: Themes and challenges. Administrative Policy in Mental Health and Mental Health Services Research.
41. Newhouse, R., Dearholt, S., Poe, S., Pugh, L. C., & White, K. M. (2005). Evidence-based practice: A practical approach to implementation. Journal of Nursing Administration, 35, 35-40.
42. Ogden, T., & Halliday Boykins, C. A. (2004). Multisystemic treatment of antisocial adolescents in Norway: Replication of clinical outcomes outside of the US. Child and Adolescent Mental Health, 9, 77-83.
43. Paramonczyk, A. (2005). Barriers to implementing research in clinical practice. Canadian Nurse, 101, 12-15.
44. Paul, G. L. (1967). Outcome research in psychotherapy. Journal of Consulting Psychology, 31, 109-118.
45. Quaglini, S., Ciccarese, P., Micieli, G., & Cavallini, A. (2004). Non-compliance with guidelines: Motivations and consequences in a case study. Studies in Health Technology and Informatics, 101, 75-87.
46. Roberts, M. C., & Steele, R. G. (2005). Program evaluation approaches to service delivery in child and family mental health. In R. G. Steele & M. C. Roberts (Eds.), Handbook of mental health services for children, adolescents, and families (pp. 351-369). New York: Kluwer Academic/Plenum Publishers.
47. Roberts, M. C., Vernberg, E. M., & Jackson, Y. (2000). Psychotherapy with children and families. In C. R. Snyder & R. E. Ingram (Eds.), Handbook of psychological change: Psychotherapy processes and practices for the 21st century (pp. 500-519). New York: Wiley.
48. Saxe, L., Cross, T., & Silverman, N. (1988). Children's mental health: The gap between what we know and what we do. American Psychologist, 43, 800-807.
49. Schwartz, S. M., Trask, P. C., Shanmugham, K., & Townsend, C. O. (2004). Conducting psychological research in medical settings: Challenges, limitations, and recommendations for effectiveness research. Professional Psychology: Research and Practice, 35, 500-508.
50. Seligman, M. E. (1995). The effectiveness of psychotherapy: The Consumer Reports study. American Psychologist, 50, 965-974.
51. Shapiro, J. P., Welker, C. J., & Jacobson, B. J. (1997). The youth client satisfaction questionnaire: Development, construct validation, and factor structure. Journal of Clinical Child Psychology, 26, 87-98.
52. Simons, A. D., & Wildes (2003). Therapy and interventions research with adults. In M. C. Roberts & S. S. Illardi (Eds.). Handbook of research methods in clinical psychology (pp. 329-351). Oxford: Blackwell Publishing.
53. Simpson, D. D. (2002). A conceptual framework for transferring research to practice. Journal of Substance Abuse Treatment, 22, 171-182.
54. Singh, N. N., & Oswald, D. P. (2004). Evidence-based practice. Part I: General methodology. Journal of Child and Family Studies, 13, 129-142.
55. Stirman, S. W., Crits-Christoph, P., & DeRubeis, R. J. (2004). Achieving successful dissemination of empirically supported psychotherapies: A synthesis of dissemination theory. Clinical Psychology: Science and Practice, 11, 343-359.
56. Task Force on Promotion and Dissemination of Psychological Procedures. (1995). Training in and dissemination of empirically-validated psychological treatments: Report and recommendations. Clinical Psychologist, 48, 3-23.
57. Trivedi, M. H., Rush, A. J., Wisniewski, S. R., Nierenberg, A. A., Warden, D., Ritz, L., et al. (2006). Evaluation of outcomes with Citalopram for depression using measurement-based care in STAR*D: Implications for clinical practice. American Journal of Psychiatry, 163, 28-40.
58. Vernberg, E. M., & Jacobs, A. K. (2001). Method of research in developmental psychopathology and treatment effectiveness. In C. E. Walker & M. C. Roberts (Eds.). Handbook of clinical child psychology (3rd ed., pp. 34-47). New York: Wiley.
59. Wang, P. S., Berglund, P., Olfson, M., Pincus, H. A., Wells, K. B., & Kessler, R. C. (2005). Failure and delay in initial treatment contact after first onset of mental disorders in the National Comorbidity Survey Replication. Archives of General Psychiatry, 62, 603-613.
60. Weersing, V. R., & Weisz, J. R. (2002). Community clinic treatment of depressed youth: Benchmarking usual care against CBT clinical trials. Journal of Consulting and Clinical Psychology, 70, 299-310.
61. Weisz, J. R., Donenberg, G. R., Han, S. S., & Weiss, B. (1995). Bridging the gap between laboratory and clinic in child and adolescent psychotherapy. Journal of Consulting and Clinical Psychology, 63, 688-701.
62. Weisz, J. R., Hawley, K. M., Pilkonis, P. A., Woody, S. R., & Follette. W. C. (2000). Stressing the (other) three Rs in the search for empirically supported treatments: Review procedures, research quality, relevance to practice and the public interest. Clinical Psychology: Science and Practice, 7, 243-258.
63. Weisz, J. R., Jensen, A. L., & McLeod, B. D. (2005). Development and dissemination of child and adolescent psychotherapie,: Milestones, methods, and a new deployment-focused model. In P. S. Jensen & E. D. Hibbs (Eds.), Psychosocial treatments for child and adolescent disorders: Empirically based strategies for clinical practice (2nd ed., pp. 9-39). Washington, DC: American Psychological Association.
64. Weisz, J. R., & Kazdin, A. E. (2003). Concluding thoughts: Present and future of evidence-based psychotherapies for children and adolescents. In A. E. Kazdin & J. R. Weisz (Eds.), Evidence-based psychotherapies for children and adolescents (pp. 439-451). New York: Guilford.
65. Wells, K. B. (1999). Treatment research at the crossroads: The scientific-interface of clinical trials and effectiveness research. American Journal of Psychiatry, 156, 5-10.
66. Yates, B. T. (2003). Toward the incorporation of costs, cost-effectiveness analysis, and cost-benefit analysis into clinical research. In A. E. Kazdin (Ed.), Methodological issues and strategies in clinical research (3rd ed., pp. 711-727). Washington, DC: American Psychological Association.