COMPARATIVE OUTCOME EVALUATIONS OF PSYCHOTHERAPIES: GUIDELINES FOR ADDRESSING EIGHT LIMITATIONS OF THE GOLD STANDARD OF CAUSAL INFERENCE

Psychotherapy

Volumn 44, Issue 2, 2007, Pages 161-174

  Staines, Graham L ^a  

^a Highland Park ^*  (United States)

Author keywords

causal inference; program evaluation; psychotherapeutic outcomes; randomized field experiment; treatment efficacy

Indexed keywords

ARTICLE; COMPARATIVE STUDY; FIELD EXPERIMENT; HEALTH PROGRAM; HUMAN; MENTAL HEALTH; OUTCOME ASSESSMENT; PRACTICE GUIDELINE; PSYCHOTHERAPIST; PSYCHOTHERAPY; TREATMENT DURATION; VALIDITY;

EID: 34548855342     PISSN: 00333204     EISSN: None     Source Type: Journal    
DOI: 10.1037/0033-3204.44.2.161     Document Type: Article

References (67)

1. Abelson R.P. Statistics as principled argument (1995), Erlbaum, Hillsdale, NJ
2. Ahn H., and Wampold B.E. Where oh where are the specific ingredients? A meta-analysis of component studies in counseling and psychotherapy. Journal of Counseling Psychology 48 (2001) 251-257
3. Baskin T.W., Tierney S.C., Minami T., and Wampold B.E. Establishing specificity in psychotherapy: A meta-analysis of structural equivalence of placebo controls. Journal of Consulting and Clinical Psychology 71 (2003) 973-979
4. Berk R.A. Regression analysis: A constructive critique (2005), Sage, Thousand Oaks, CA
5. Berk R.A. Randomized experiments as the bronze standard. Journal of Experimental Criminology 1 (2005) 417-433
6. Blankertz L., Magura S., Staines G.L., Madison E., Spinelli M., Horowitz E., et al. A new work placement model for unemployed methadone maintenance patients. Substance Use and Misuse 39 (2005) 2239-2260
7. Boruch R.F. Randomized controlled experiments for evaluation and planning. In: Bickman L., and Rog D.J. (Eds). Handbook of applied social research methods (1988), Sage, Thousand Oaks, CA 161-192
8. Boruch R.F. Randomized experiments for planning and evaluation (1997), Sage, Thousand Oaks, CA
9. Brady, H. E. (2002, July). Models of causal inference: Going beyond the Neyman-Rubin-Holland theory. Paper presented at the annual meetings of the Political Methodology Group, University of Washington, Seattle.
10. Brewer M. Research design and issues of validity. In: Reis H.T., and Judd C.M. (Eds). Handbook of research methods in social and personality psychology (2000), Cambridge University Press, New York 3-16
11. Cook T.D., and Campbell D. Quasi-experimentation: Design and analysis issues for field settings (1979), Houghton Mifflin, Boston
12. Cook T.D., and Shadish W.R. Social experiments: Some developments over the past fifteen years. Annual Review of Psychology 45 (1994) 545-580
13. Crits-Christoph P., Baranackie K., Kurcias J., Beck A.T., Carroll K., Perry K., et al. Meta-analysis of therapist effects in psychotherapy outcome studies. Psychotherapy Research 1 (1991) 81-91
14. Crits-Christoph P., Beebe K.L., and Connolly M.B. Therapist effects in the treatment of drug dependence: Implications for conducting comparative treatment studies. NIDA Research Monograph 104 (1990) 39-49
15. Crits-Cristoph P., and Gallop R. Therapist effects in the National Institute of Mental Health Treatment of Depression Collaborative Research Program and other psychotherapy studies. Psychotherapy Research 16 (2006) 178-181
16. Crits-Cristoph P., and Mintz J. Implications of therapist effects for the design and analysis of comparative studies of psychotherapies. Journal of Consulting and Clinical Psychology 59 (1991) 20-26
17. De Leon G., Staines G.L., Sacks S., Perlis T.E., McKendrick K., Hilton R., et al. Therapeutic community methods in methadone maintenance (Passages): An empirical evaluation. In: Tims F.M., Inciardi J.A., Fletcher B.W., and Horton A.M. (Eds). The effectiveness of innovative approaches in the treatment of drug abuse (1997), Greenwood Press, Westport, CT 147-164
18. Dennis M.L. Ethical and practical randomization field experiments. In: Wholey J.S., Hatry H.P., and Newcomer K.E. (Eds). Handbook of practical program evaluation (1994), Jossey-Bass, San Francisco 155-197
19. Elkin I. A major dilemma in psychotherapy outcome research: Disentangling therapists from therapies. Clinical Psychology: Science and Practice 6 (1999) 10-32
20. Elkin I., Falconnier L., Martinovich S., and Mahoney C. Therapist effects in the National Institute of Mental Health Treatment of Depression Collaborative Research Program. Psychotherapy Research 16 (2006) 144-160
21. Frank J.D., and Frank J.B. Persuasion and healing: A comparative study of psychotherapy. 3rd ed. (1991), Johns Hopkins University Press, Baltimore
22. Goodman R.M. Evaluation of community-based health programs: An alternate perspective. In: Schneiderman N., Speers M.A., Silva J.M., Tomes H., and Gentry J.J. (Eds). Integrating behavioral and social sciences with public health (2001), American Psychological Association, Washington, DC 293-304
23. Gordis L. Epidemiology. 2nd ed. (2000), W. B. Saunders, Philadelphia
24. Greenwald, A. G., Gonzalez, R., Harris, R. J., & Guthrie, D. (1993). Using p values and effect sizes to evaluate novel findings: Significance versus replicability and demonstrability. Unpublished manuscript, University of Washington.
25. Janevic M.R., Janz N.K., Dodge J.A., Lin X., Pan W., Sinco B.R., and Clark N.M. The role of choice in health education intervention trials: A review and case study. Social Science and Medicine 56 (2003) 1581-1594
26. Kaufman J.S., Kaufman S., and Poole C. Causal inference from randomized trials in social epidemiology. Social Science and Medicine 57 (2003) 2397-2409
27. Kim D., Wampold B.E., and Bolt D.M. Therapist effects in psychotherapy: A random-effects modeling of the National Institute of Mental Health Treatment of Depression Collaborative Research Program data. Psychotherapy Research 16 (2006) 161-172
28. Kraemer H.C., Kazdin A.E., Offord D.R., Kessler R.C., Jenson P.S., and Kupfer D.J. Coming to terms with the terms of risk. Archives of General Psychology 54 (1997) 337-343
29. Lipsey M.W., and Cordray D.S. Evaluation methods for social intervention. Annual Review of Psychology 51 (2000) 345-375
30. Lipsey M.W., and Wilson D.B. The efficacy of psychological, educational, and behavioral treatment: Confirmation from meta-analysis. American Psychologist 48 (1993) 1181-1209
31. Luborsky L., Crits-Christoph P., McLellan A., Woody G., Piper W., Liberman B., et al. Do therapists vary much in their success? Findings from four outcome studies. American Journal of Orthopsychiatry 56 (1986) 501-512
32. Luborsky L., Rosenthal R., Diguer L., Andrusyna T.P., Berman J.S., Levitt J.T., et al. The dodo bird verdict is alive and well-Mostly. Clinical Psychology: Science & Practice 9 (2002) 2-12
33. Luborsky L., Singer B., and Luborsky L. Comparative studies of psychotherapies: Is it true that "everybody has won and all must have prizes"?. Archives of General Psychiatry 32 (1975) 995-1008
34. Martindale C. The therapist-as-fixed-effect fallacy in psychotherapy research. Journal of Consulting and Clinical Psychology 46 (1978) 1526-1530
35. Okiishi J., Lambert M.J., Nielsen S.L., and Ogles B.M. Waiting for supershrink: An empirical analysis of therapist effects. Clinical Psychology & Psychotherapy 10 (2003) 361-373
36. Orr L.L. Social experiments: Evaluating public programs with experimental methods (1999), Sage, Thousand Oaks, CA
37. Pearl J. Causality: Models, reasoning and inference (2000), Cambridge University Press, Cambridge, England
38. Perry, K., & Howard, K. I. (1989, June). Therapist effects: In search of the therapists' contribution to psychotherapy outcome. Paper presented at the annual meeting of the Society for Psychotherapy Research, Toronto, Ontario, Canada.
39. Raudenbush S.W. Statistical analysis and optimal design for cluster randomized trials. Psychological Methods 2 (1997) 173-185
40. Raudenbush S.W., and Bryk A.S. Hierarchical linear models: Applications and data analysis methods. 2nd ed. (2002), Sage, Newbury Park, CA
41. Raudenbush S.W., and Liu X. Statistical power and optimal design for multisite randomized trials. Psychological Methods 5 (2000) 199-213
42. Riecken H.W., and Boruch R.F. Social experiments. Annual Review of Sociology 4 (1978) 511-532
43. Rosenzweig S. Some implicit common factors in diverse methods of psychotherapy. American Journal of Orthopsychiatry 6 (1936) 412-415
44. Rossi P.H., Freeman H.E., and Lipsey M.W. Evaluation: A systematic approach. 6th ed. (1999), Sage, Thousand Oaks, CA
45. Rossi P.H., and Wright J.D. Evaluation research: An assessment. Annual Review of Sociology 10 (1984) 331-352
46. Salovey P., and Williams-Piehota P. Field experiments in social psychology: Message framing and the promotion of health protective behaviors. American Behavioral Scientist 47 (2004) 488-505
47. Shadish W.R., Cook T.D., and Campbell D.T. Experimental and quasi-experimental design for generalized causal inference (2002), Houghton Mifflin, Boston
48. Shadish W.R., Montgomery P.W., Wilson M.R., Bright I., and Okwumabua T. Effects of family and marital psychotherapies: A meta-analysis. Journal of Consulting and Clinical Psychology 61 (1993) 992-1002
49. Shapiro D.A., Barkham M., Hardy G.E., and Morrison L.A. The second Sheffield Psychotherapy Project: Rationale, design, and preliminary outcome data. British Journal of Medical Psychology 63 (1990) 97-108
50. Shapiro D.A., Barkham M., Rees A., Hardy G.E., Reynolds S., and Startup M. Effects of treatment duration and severity of depression on the effectiveness of cognitive-behavioral and psychodynamic-interpersonal psychotherapy. Journal of Consulting and Clinical Psychology 62 (1994) 522-534
51. Shapiro D.A., and Firth J.A. Prescriptive vs. exploratory psychotherapy: Outcomes of the Sheffield Psychotherapy Project. British Journal of Psychiatry 151 (1987) 790-799
52. Shapiro D.A., and Shapiro D. Meta-analysis of comparative therapy outcome studies: A replication and refinement. Psychological Bulletin 92 (1982) 581-604
53. Smith E.S. Research design. In: Reis H.T., and Judd C.M. (Eds). Handbook of research methods in social and personality psychology (2000), Cambridge University Press, New York 17-39
54. Smith H.L. Some thoughts on causation as it relates to demography and population studies. Population and Development Review 29 (2003) 459-469
55. Smith M.L., and Glass G.V. Meta-analysis of psychotherapy outcome studies. American Psychologist 32 (1977) 752-760
56. Smith M.L., Glass G.V., and Miller T.I. The benefits of psychotherapy (1980), Johns Hopkins University Press, Baltimore
57. Soldz S. Models and meanings: Therapist effects and the stories we tell. Psychotherapy Research 16 (2006) 173-177
58. Staines G.L., Blankertz L., Magura S., Bali P., Madison E.M., Spinelli M., et al. Efficacy of the Customized Employment Supports (CES) model of vocational rehabilitation for unemployed methadone patients: Preliminary results. Substance Use and Misuse 39 (2005) 2261-2286
59. Staines G.L., Blankertz L., Magura S., Cleland C.M., and Bali P. Evaluating vocational rehabilitation programs for substance users: Issues in designing and implementing randomized studies. Evaluation and Program Planning 28 (2005) 61-68
60. Staines G.L., Cleland C.M., and Blankertz L. Counselor confounds in evaluations of vocational rehabilitation methods in substance dependency treatment. Evaluation Review 30 (2006) 139-170
61. Stiles W.B., Shapiro D.A., and Elliott R. Are all psychotherapies equivalent?. American Psychologist 41 (1986) 165-180
62. Wampold B.E. The great psychotherapy debate: Models, methods, and findings (2001), Erlbaum, Mahwah, NJ
63. Wampold B.E., and Bolt D.M. Therapist effects: Clever ways to make them (and everything else) disappear. Psychotherapy Research 16 (2006) 184-187
64. Wampold B.E., Mondin G.W., Moody M., Stich F., Benson K., and Ahn H. A meta-analysis of outcome studies comparing bona fide psychotherapies: Empirically, "All must have prizes.". Psychological Bulletin 122 (1997) 203-215
65. Wampold B.E., and Serlin R.C. The consequence of ignoring a nested factor on measures of effect size in analysis of variance. Psychological Methods 5 (2000) 425-433
66. West S.G., Biesanz J.C., and Pitts S.C. Causal inference and generalization in field settings: Experimental and quasi-experimental designs. In: Reis H.T., and Judd C.M. (Eds). Handbook of research methods in social and personality psychology (2000), Cambridge University Press, New York 40-88
67. Wilson D.B., and Lipsey M.W. The role of method in treatment effectiveness: Evidence from a meta-analysis. Psychological Methods 6 (2001) 413-429