Research on psychotherapy efficacy and effectiveness: Between scylla and charybdis?

Psychological Bulletin

Volumn 126, Issue 6, 2000, Pages 964-981

  Nathan, Peter E ^a   Stuart, Scott P ^b   Dolan, Sara L ^b  

^a E11 SSH   (United States)

^b NONE

Author keywords

[No Author keywords available]

Indexed keywords

CLINICAL TRIAL; CONTROLLED CLINICAL TRIAL; FORECASTING; HEALTH SERVICES RESEARCH; HUMAN; OUTCOME ASSESSMENT; PSYCHOTHERAPY; RANDOMIZED CONTROLLED TRIAL; REVIEW; TREATMENT OUTCOME; UNITED STATES;

FORECASTING; HEALTH SERVICES RESEARCH; HUMANS; OUTCOME ASSESSMENT (HEALTH CARE); PSYCHOTHERAPY; RANDOMIZED CONTROLLED TRIALS; TREATMENT OUTCOME; UNITED STATES;

EID: 0034328208     PISSN: 00332909     EISSN: None     Source Type: Journal    
DOI: 10.1037/0033-2909.126.6.964     Document Type: Article

References (138)

1. American Psychiatric Association. (1952). Diagnostic and statistical manual of mental disorders. Washington, DC: Author.
2. American Psychiatric Association. (1993). Practice guidelines for the treatment of major depressive disorder in adults. American Journal of Psychiatry, 150(Suppl. 4), 1-26.
3. American Psychiatric Association. (1994a). Diagnostic and statistical manual of mental disorders (4th ed.). Washington, DC: Author.
4. American Psychiatric Association. (1994b). Practice guideline for the treatment of patients with bipolar disorder. American Journal of Psychiatry, 151(Suppl. 12), 1-36.
5. American Psychiatric Association. (1995). Practice guideline for the treatment of patients with substance use disorders: Alcohol, cocaine, opioids. American Journal of Psychiatry, 152(Suppl. 11), 1-59.
6. American Psychiatric Association. (1996). Practice guideline for the treatment of patients with nicotine dependence. American Journal of Psychiatry, 153(Suppl. 10), 1-31.
7. American Psychiatric Association. (1997). Practice guideline for the treatment of patients with schizophrenia. American Journal of Psychiatry, 754(Suppl. 4), 1-63.
8. American Psychological Association Task Force. (1995). Template for developing guidelines: Interventions for mental disorders and psychosocial aspects of physical disorders. Washington, DC: American Psychological Association.
9. Anderson, E., & Lambert, M. (1995). Short-term dynamically oriented psychotherapy: a review and meta-analysis. Clinical Psychology Review, 15, 503-514.
10. Barlow, D. H. (1981). On the relation of clinical research to clinical practice: Current issues. Journal of Consulting and Clinical Psychology, 49, 147-155.
11. Barlow, D. H. (1996). Health care policy, psychotherapy research, and the future of psychotherapy. American Psychologist, 51, 1050-1058.
12. Beck, A. T., Rush, A. J., Shaw, B. F., & Emery, G. (1979). Cognitive therapy of depression. New York: Guilford Press.
13. Bergin, A. E. (1966). Some implications of psychotherapy research for therapeutic practice. Journal of Abnormal Psychology, 71, 235-246.
14. Bergin, A. E., & Suinn, R. M. (1975). Individual psychotherapy and behavior therapy. In M. R. Rosenzweig & L. W. Porter (Eds.), Annual review of psychology (Vol. 26, pp. 509-556). Palo Alto, CA: Annual Reviews.
15. Berman, J. S., & Norton, N. C. (1985). Does professional training make a therapist more effective? Psychological Bulletin, 98, 401-407.
16. Beutler, L. E. (1998). Identifying empirically supported treatments: What if we didn't? Journal of Consulting and Clinical Psychology, 66, 113-120.
17. Beutler, L. E., Machado, P. P. P., & Neufeldt, S. A. (1994). Therapist variables. In A. E. Bergin & S. L. Garfield (Eds.), Handbook of psychotherapy and behavior change (4th ed., pp. 229-269). New York: Wiley.
18. Butler, S. F., & Strupp, H. H. (1986). Specific and nonspecific factors in psychotherapy: A problematic paradigm for psychotherapy research. Psychotherapy, 23, 30-40.
19. Carroll, L. (1962). Alice's adventures in Wonderland. Harmondsworth, Middlesex, England: Penguin Books. (Original work published in 1865)
20. Chambless, D. L., Baker, M. J., Baucom, D. H., Beutler, L. E., Calhoun, K. S., Crits-Christoph, P., Daivto, A., DeRobeis, R., Detweiler, J., Haaga, D. A. F., Sanderson, W. C., Shoham, V., Stickle, T., Williams, D. A., & Woody, S. R. (1998). Update on empirically validated therapies, II. Clinical Psychologist, 51, 3-16.
21. Chambless, D. L., Sanderson, W. C., Shoham, V., Johnson, S. B., Pope, K. S., Crits-Christoph, P., Baker, M., Johnson, B., Woody, S. R., Sue, S., Beutler, L., Williams, D. A., & McCurry, S. (1996). An update on empirically validated therapies. Clinical Psychologist, 49, 5-18.
22. Cohen, J. (1965). Some statistical issues in psychological research. In B. Wolman (Ed.), Handbook of clinical psychology (pp. 95-121). New York: McGraw-Hill.
23. Crits-Christoph, P. (1996). The dissemination of efficacious psychological treatments. Clinical Psychology: Science and Practice, 3, 260-263.
24. Crits-Christoph, P. (1997). Limitations of the dodo bird verdict and the role of clinical trials in psychotherapy research: Comment on Wampold et al. (1997). Psychological Bulletin, 122, 216-220.
25. Crits-Christoph, P., Baranacki, K., Kurcias, J. S., Beck, A. T., Carroll, K., Perry, K., Luborsky, L., McLellan, A. T., Woody, G. E., Thompson, L., Gallagher, D., & Zitrin, C. (1991). Meta-analysis of therapist effects in psychotherapy outcome studies. Psychotherapy Research, 1, 81-91.
26. Cronbach, L. J. (1957). Two disciplines of scientific psychology. American Psychologist, 12, 671-684.
27. Eaton, T. T., Abeles, N., & Gutfreund, M. J. (1988). Therapeutic alliance and outcome: Impact of treatment length and pretreatment symptomatology. Psychotherapy, 25, 536-542.
28. Edelson, M. (1994). Can psychotherapy research answer this psychotherapist's questions? In P. F. Talley, H. H. Strupp, & S. F. Butler (Eds.), Psychotherapy research and practice: Bridging the gap (pp. 60-87). New York: Basic Books.
29. Eifert, G. H., Schulte, D., Zvolensky, M. J., Lejuez, C. W., & Lau, A. W. (1997). Manualized behavior therapy: Merits and challenges. Behavior Therapy, 28, 499-509.
30. Elkin, I. (1994). The NIMH Treatment of Depression Collaborative Research Program: Where we began and where we are. In A. E. Bergin & S. L. Garfield (Eds.), Handbook of psychotherapy and behavior change (pp. 114-139). New York: Wiley.
31. Elkin, I. (1995). Further differentiation of common factors. Clinical Psychology: Science and Practice, 2, 75-78.
32. Elkin, I., Parloff, M. B., Hadley, S. W., & Autry, J. H. (1985). NIMH Treatment of Depression Collaborative Research Program: Background and research plan. Archives of General Psychiatry, 42, 305-316.
33. Elkin, I., Pilkonis, P. A., Docherty, J. P., & Sotsky, S. M. (1988). Conceptual and methodological issues in the comparative studies of psychotherapy and pharmacotherapy: II. Nature and timing of treatment effects. American Journal of Psychiatry, 145, 1070-1076.
34. Elkin, I., Shea, M. T., Watkins, J. T., Imber, S. D., Sotsky, S. M., Collins, J. F., Glass, D. R., Pilkonis, P. A., Leber, W. R., Docherty, J. P., Fiester, S. J., & Parloff, M. B. (1989). National Institute of Mental Health Treatment of Depression Collaborative Research Program: General effectiveness of treatments. Archives of General Psychiatry, 46, 971-982.
35. Eysenck, H. J. (1952). The effects of psychotherapy: An evaluation. Journal of Consulting Psychology, 16, 319-324.
36. Eysenck, H. J. (1960). Behavior therapy and the neuroses. Oxford, England: Pergamon Press.
37. Fensterheim, H., & Raw, S. D. (1996). Empirically validated treatments, psychotherapy integration, and the politics of psychotherapy. Journal of Psychotherapy Integration, 6, 207-215.
38. Frank, E., Kupfer, D. J., & Perel, J. M. (1990). Three-year outcomes for maintenance therapies in recurrent depression. Archives of General Psychiatry, 47, 1093-1099.
39. Frank, E., Kupfer, D. J., Wagner, E. F., McEachran, A. B., & Cornes, C. (1992). Efficacy of interpersonal psychotherapy as a maintenance treatment of recurrent depression: Contributing factors. Archives of General Psychiatry, 48, 1053-1059.
40. Frank, J. D. (1971). Therapeutic factors in psychotherapy. American Journal of Psychotherapy, 25, 350-361.
41. Frank, J. D. (1973). Persuasion and healing: A comparative study of psychotherapy (rev. ed.). Baltimore: Johns Hopkins University Press.
42. Garfield, S. L. (1996). Some problems associated with "validated" forms of psychotherapy. Clinical Psychology: Science and Practice, 3, 218-229.
43. Gendlin, E. T., & Rychlak, J. F. (1970). Psychotherapeutic processes. In P. H. Mussen & M. R. Rosenzweig (Eds.), Annual review of psychology (Vol. 21, pp. 148-190). Palo Alto, CA: Annual Reviews.
44. Glass, G. V. (1980). Summarizing effect sizes. In R. Rosenthal (Ed.), New directions for methodology of social and behavioral science: Quantitative assessment of research domains (pp. 13-31). San Francisco: Jossey-Bass.
45. Goldfried, M. R., Greenberg, L., & Marmar, C. (1990). Individual psychotherapy: Process and outcome. In M. R. Rosenzweig & L. W. Porter (Eds.), Annual review of psychology (Vol. 41, pp. 659-688). Palo Alto, CA: Annual Reviews.
46. Goldfried, M. R., & Wolfe, B. E. (1998). Toward a more clinically valid approach to therapy research. Journal of Consulting and Clinical Psychology, 66, 143-150.
47. Gomes-Schwartz, B., Hadley, S. W., & Strupp, H. H. (1978). Individual psychotherapy and behavior therapy. In M. R. Rosenzweig & L. W. Porter (Eds.), Annual review of psychology (Vol. 29, pp. 435-472). Palo Alto, CA: Annual Reviews.
48. Gottman, J. M., & Rushe, R. H. (1993). The analysis of change: Issues, fallacies, and new ideas. Journal of Consulting and Clinical Psychology, 61, 907-910.
49. Greenberg, J. (1994). Psychotherapy research: A clinician's view. In P. F. Talley, H. H. Strupp, & S. F. Butler (Eds.), Psychotherapy research and practice: Bridging the gap (pp. 1-18). New York: Basic Books.
50. Hays, W. L. (1963). Statistics. New York: Holt, Rinehart & Winston.
51. Hollon, S. D. (1996). The efficacy and effectiveness of psychotherapy relative to medications. American Psychologist, 51, 1025-1030.
52. Hollon, S. D. (1999). Allegiance effects in treatment research: A commentary. Clinical Psychology: Science and Practice, 6, 107-112.
53. Horowitz, M. J., Marmar, C., Krupnick, K., Wilner, N., Kaltreider, N., & Wallerstein, R. (1984). Personality styles and brief psychotherapy. New York: Basic Books.
54. Horvath, A. O., & Symonds, D. B. (1991). Relationship between working alliance and outcome in psychotherapy: A meta-analysis. Journal of Counseling Psychology, 38, 139-149.
55. Horvath, P. (1988). Placebos and common factors in two decades of psychotherapy research. Psychological Bulletin, 104, 214-225.
56. Howard, K. I., Kopta, S. M., Krause, M. S., & Orlinsky, D. E. (1986). The dose-effect relationship in psychotherapy. American Psychologist, 41, 159-164.
57. Howard, K. I., Krause, M. S., Saunders, S. M., & Kopta, S. M. (1997). Trials and tribulations in the meta-analysis of treatment differences: Comments on Wampold et al. (1997). Psychological Bulletin, 122, 221-225.
58. Howard, K. I., Moras, K., Brill, P. L., Martinovich, Z., & Lutz, W. (1996). Evaluation of psychotherapy: Efficacy, effectiveness, and patient progress. American Psychologist, 51, 1059-1064.
59. Howard, K. I., & Orlinsky, D. E. (1972). Psychotherapeutic processes. In P. H. Mussen & M. R. Rosenzweig (Eds.), Annual review of psychology (Vol. 23, pp. 615-668). Palo Alto, CA: Annual Reviews.
60. Howard, K. I., Orlinsky, D. E., & Lueger, R. J. (1995). The design of clinically relevant outcome research: Some considerations and an example. In M. Aveline & D. A. Shapiro (Eds.), Research foundations for psychotherapy practice (pp. 3-47). Sussex, England: Wiley.
61. Jacobson, N. S., & Christensen, A. (1996). Studying the effectiveness of psychotherapy: How well can clinical trials do the job? American Psychologist, 51, 1031-1039.
62. Jacobson, N. S., Follette, W. C., & Revenstorf, D. (1984). Psychotherapy outcome research: Methods for reporting variability and evaluating clinical significance. Behavior Therapy, 15, 336-352.
63. Jacobson, N. S., Schmaling, K. B., Holtzworth-Munroe, A., Karr, J. L., Wood, L. R., & Follette, V. M. (1989). Research-structured vs. clinically flexible versions of social learning based marital therapy. Behaviour Research and Therapy, 27, 173-180.
64. Jacobson, N. S., & Truax, P. (1991). Clinical significance: A statistical approach to defining meaningful change in psychotherapy research. Journal of Consulting and Clinical Psychology, 59, 12-19.
65. Kazdin, A. E. (1986). Comparative outcome studies of psychotherapy: Methodological issues and strategies. Journal of Consulting and Clinical Psychology, 54, 95-105.
66. Kernberg, O., Selzer, M., Koenigsberg, H., Carr, A., & Appelbaum, A. (1989). Psychodynamic psychotherapy of borderline patients. New York: Basic Books.
67. Kiesler, D. J. (1994). Standardization of intervention: The tie that binds psychotherapy research and practice. In P. F. Talley, H. H. Strupp, & S. F. Butler (Eds.), Psychotherapy research and practice: Bridging the gap (pp. 143-153). New York: Basic Books.
68. Klein, D. F. (1999). Dealing with the effects of therapy allegiances. Clinical Psychology: Science and Practice, 6, 124-126.
69. Klein, D. F., & Smith, L. B. (1999). Organizational requirements for effective clinical effectiveness studies. Prevention & Treatment, 2, Article 0002a. Retrieved August 21, 2000, From the World Wide Web: http://journals.apa.org/prevention/volume2/pre0020002a.html
70. Klerman, G. L., Weissman, M. M., Rounsaville, B. J., & Chevron, E. S. (1984). Interpersonal psychotherapy of depression. New York: Basic Books.
71. Kopta, S. M., Lueger, R. J., Saunders, S. M., & Howard, K. I. (1999). Individual psychotherapy outcome and process research: Challenges leading to greater turmoil or a positive transition? In J. T. Spence, J. M. Darley, & D. J. Foss (Eds.), Annual review of psychology (Vol. 50, pp. 441-470). Palo Alto, CA: Annual Reviews.
72. Koss, M. P., Butcher, J. N., & Strupp, H. H. (1986). Brief psychotherapy methods in clinical research. Journal of Consulting and Clinical Psychology, 54, 60-67.
73. Krasner, L. (1971). Behavior therapy. In P. H. Mussen & M. R. Rosenzweig (Eds.), Annual review of psychology (Vol. 22, pp. 483-532). Palo Alto, CA: Annual Reviews.
74. Lambert, M. J. (1992). Psychotherapy outcome research: Implications for integrative and eclectic therapies. In J. C. Norcross & M. R. Goldfried (Eds.), Handbook of psychotherapy integration (pp. 94-129). New York: Basic Books.
75. Lambert, M. J. (1999). Are differential treatment effects inflated by researcher therapy allegiance? Could Clever Hans count? Clinical Psychology: Science and Practice, 6, 127-130.
76. Lambert, M. J., & Bergin, A. E. (1994). The effectiveness of psychotherapy. In S. L. Garfield & A. E. Bergin (Eds.), Handbook of psychotherapy and behavior change (4th ed., pp. 143-189). New York: Wiley.
77. Linehan, M. M. (1993). Skills training manual for treating borderline personality disorder. New York: Guilford Press.
78. Luborsky, L. (1984). Principles of psychoanalytic psychotherapy: A manual for supportive-expressive (SE) treatment. New York: Basic Books.
79. Luborsky, L., & Diguer, L. (1995, June). The psychotherapist as a neglected variable: The therapist's treatment effectiveness. Paper presented at the meeting of the Society for Psychotherapy Research, Vancouver, British Columbia, Canada.
80. Luborsky, L., Diguer, L., Seligman, D. A., Rosenthal, R., Krause, E. D., Johnson, S., Halperin, G., Bishop, M., Berman, J. S., & Schweizer, E. (1999). The researcher's own therapy allegiances: A "wild card" in comparisons of treatment efficacy. Clinical Psychology: Science and Practice, 6, 95-106.
81. Luborsky, L., Singer, B., & Luborsky, L. (1975). Comparative studies of psychotherapies: Is it true that "everybody has won and all must have prizes?" Archives of General Psychiatry, 32, 995-1008.
82. Maling, M. S., & Howard, K. I. (1994). From research to practice to research to . . . . In P. F. Talley, H. H. Strupp, & S. F. Butler (Eds.), Psychotherapy research and practice: Bridging the gap (pp. 246-253). New York: Basic Books.
83. Mental health: Does therapy help? (1995, November). Consumer Reports, pp. 734-739.
84. Mintz, J., Drake, R. E., & Crits-Cristoph, P. (1996). Efficacy and effectiveness of psychotherapy: Two paradigms, one science. American Psychologist, 51, 1084-1085.
85. Nathan, P. E. (1998). Practice guidelines: Not yet ideal. American Psychologist, 53, 290-299.
86. Nathan, P. E. (1999). And the dodo bird asked: "Well, what about studies of psychotherapy effectiveness?" Invited address at the 107th Annual Convention of the American Psychological Association, Boston, MA.
87. National Advisory Mental Health Council Workgroup. (1999). Bridging science and service. Washington, DC: National Institute of Mental Health.
88. Niederehe, G., Street, L. L., & Lebowitz, B. D. (1999). NIMH support for psychotherapy research: Opportunities and questions. Prevention & Treatment, 2, Article 0003a.
89. Norquist, G., Lebowitz, B., & Hyman, S. (1999). Expanding the frontier of treatment research. Prevention & Treatment, 2, Article 0001a. Retrieved August 21, 2000, from the National Wide Web: http://joumals.apa.org/ prevention/volume2/pre0020001a.html
90. Ogles, B. M., Lambert, M. J., & Sawyer, J. D. (1995). Clinical significance of the National Institute of Mental Health Treatment of Depression Collaborative Research Program data. Journal of Consulting and Clinical Psychology, 63, 321-326.
91. Orlinsky, D. E., & Howard, K. I. (1975). Varieties of psychotherapeutic experience. New York: Teachers College Press.
92. Orlinsky, D. E., & Howard, K. I. (1986). Process and outcome in psychotherapy. In S. L. Garfield & A. E. Bergin (Eds.), Handbook of psychotherapy and behavior change (3rd ed., pp. 311-381). New York: Wiley.
93. Parloff, M. B. (1984). Psychotherapy research and its incredible credibility crisis. Clinical Psychology Review, 4, 95-109.
94. Parloff, M. B. (1986). Placebo controls in psychotherapy research a sine qua non or a placebo for research problems? Journal of Consulting and Clinical Psychology, 54, 79-87.
95. Persons, J. B. (1991). Psychotherapy outcome studies do not accurately represent current models of psychotherapy: A proposed remedy. American Psychologist, 46, 99-106.
96. Powers, E., & Witmer, H. (1951). AH experiment in the prevention of delinquency: The Cambridge-Somerville youth study. New York: Columbia University Press.
97. Project MATCH Research Group. (1993). Project MATCH: Rationale and methods for a multisite clinical trial matching patients to alcoholism treatment. Alcoholism: Clinical and Experimental Research, 17, 1130-1145.
98. Project MATCH Research Group. (1997). Matching alcoholism treatment to client heterogeneity: Project MATCH post-treatment drinking outcomes. Journal of Studies on Alcohol, 58, 7-29.
99. Rachman, S., & Wilson, G. T. (1980). The effects of the psychological therapies. Oxford, England: Pergamon Press.
100. Robinson, L. A., Berman, J. S., & Neimeyer, R. A. (1990). Psychotherapy for the treatment of depression: A comprehensive review of controlled outcome research. Psychological Bulletin, 108, 30-49.
101. Rogers, C. R. (1957). The necessary and sufficient conditions of therapeutic personality change. Journal of Consulting Psychology, 21, 95-103.
102. Rogers, C. R. (1961). On becoming a person. Boston: Houghton Mifflin.
103. Rogers, C. R., & Dymond, R. (1954). Psychotherapy and personality change. Chicago: University of Chicago Press.
104. Rosenthal, R. (1983). Assessing the statistical and social importance of the effects of psychotherapy. Journal of Consulting and Clinical Psychology, 51, 4-13.
105. Rosenthal, R., & Rubin, D. B. (1979). A note on percent variance explained as a measure of the importance of effects. Journal of Applied Social Psychology, 9, 395-396.
106. Rosenthal, R., & Rubin, D. B. (1982). A simple, general purpose display of magnitude of experimental effect. Journal of Educational Psychology, 74, 166-169.
107. Rosenzweig, S. (1936). Some implicit common factors in diverse methods in psychotherapy. American Journal of Orthopsychiatry, 6, 412-415.
108. Sargent, M., & Cohen, L. H. (1983). Influence of psychotherapy research on clinical practice: An experimental survey. Journal of Consulting and Clinical Psychology, 51, 718-720.
109. Seligman, M. E. P. (1995). The effectiveness of psychotherapy: The Consumer Reports study. American Psychologist, 50, 965-974.
110. Seligman, M. E. P. (1996). Science as an ally of practice. American Psychologist, 51, 1072-1079.
111. Shadish, W. R., Matt, G. E., Navarro, A. M., & Phillips, G. (in press). The effects of psychological therapies in clinically representative conditions: A meta-analysis. Psychological Bulletin.
112. Shadish, W. R., Matt, G. E., Navarro, A. M., Siegle, G., Crits-Christoph, P., Hazelrigg, M. D., Jorm, A. F., Lyons, L. C., Nietzel, M. T., Prout, H. T. Robinson, L., Smith, M. L., Svartberg, M., and Weiss, B. (1997). Evidence that therapy works in clinically representative conditions. Journal of Consulting and Clinical Psychology, 65, 355-365.
113. Shadish, W. R., Montgomery, L. M., Wilson, D., Wilson, M. R., Bright, I., & Okwumabua, T. (1993). Effects of family and marital psychotherapies: A meta-analysis. Journal of Consulting and Clinical Psychology, 61, 992-1002.
114. Shadish, W. R., & Ragsdale, K. (1996). Random versus nonrandom assignment in controlled experiments: Do you get the same answer? Journal of Consulting and Clinical Psychology, 64, 1290-1305.
115. Shapiro, D. A., & Shapiro, D. (1982). Meta-analysis of comparative therapy outcome studies: A replication and refinement. Psychological Bulletin, 92, 581-604.
116. Shapiro, D. A., & Shapiro, D. (1983). Comparative therapy outcome research: Methodological implicatios of meta-analysis. Journal of Consulting and Clinical Psychology, 51, 42-53.
117. Shoham, V., & Rohrbaugh, M. J. (1999). Beyond allegiance to comparative outcome studies. Clinical Psychology: Science and Practice, 6, 120-123.
118. Smith, M. L. (1980). Integrating studies of psychotherapy outcomes. In R. Rosenthal (Ed.), New directions for methodology of social and behavioral science: Quantitative assessment of research domains (pp. 47-61). San Francisco: Jossey-Bass.
119. Smith, M. L., & Glass, G. V. (1977). Meta-analysis of psychotherapy outcome studies. American Psychologist, 32, 752-760.
120. Smith, M. L., Glass, G. V., & Miller, T. I. (1980). Benefits of psychotherapy. Baltimore: Johns Hopkins University Press.
121. Spence, D. P. (1994). The failure to ask the hard questions. In P. F. Talley, H. H. Strupp, & S. F. Butler (Eds.), Psychotherapy research and practice: Bridging the gap (pp. 19-38). New York: Basic Books.
122. Stiles, W. B., & Shapiro, D. A. (1989). Abuse of the drug metaphor in psychotherapy process-outcome research. Clinical Psychology Review, 9, 521-543.
123. Stiles, W. B., Shapiro, D. A., & Elliott, R. (1986). "Are all psychotherapies equivalent?" American Psychologist, 41, 165-180.
124. Stoler, N. (1963). Client likeability: A variable in the study of psychotherapy. Journal of Consulting Psychology, 27, 175-178.
125. Stricker, G. (1997). Are science and practice commensurable? American Psychologist, 52, 442-448.
126. Strupp, H. H. (1973). Psychotherapy: Clinical, research, and theoretical issues. New York: Jason Aronson.
127. Strupp, H. H. (1989). Psychotherapy: Can the practitioner learn from the researcher? American Psychologist, 44, 717-724.
128. Strupp, H. H., & Binder, J. L. (1984). Psychotherapy in a new key: A guide to time-limited psychotherapy. New York: Basic Books.
129. Svartberg, M., Seltzer, M. H., & Stiles, T. C. (1998). The effects of common and specific factors in short-term anxiety-provoking psychotherapy. Journal of Nervous and Mental Disease, 186, 691-696.
130. Task Force on Promotion and Dissemination of Psychological Procedures. (1995). Training in and dissemination of empirically-validated psychological treatments: Report and recommendations. Clinical Psychologist, 48, 3-23.
131. Teuber, N., & Powers, E. (1953). Evaluating therapy in a delinquency prevention program. Proceedings of the Association of Nervous and Mental Disease, 3, 138-147.
132. Thase, M. E. (1999a). How should efficacy be evaluated in randomized clinical trials of treatments for depression? Journal of Clinical Psychiatry, 60(Suppl. 4), 23-32.
133. Thase, M. E. (1999b). What is the investigator allegiance effect and what should we do about it? Clinical Psychology: Science and Practice, 6, 113-115.
134. Tingey, R. C., Lambert, M. J., Burlinhame, G. M., & Hansen, N. B. (1996). Assessing clinical significance: Proposed extensions to method. Psychotherapy Research, 6, 109-123.
135. Wampold, B. E. (1997). Methodological problems in identifying efficacious psychotherapies. Psychotherapy Research, 7, 21-43.
136. Wampold, B. E., Mondin, G. W., Moody, M., Stich, F., Benson, K., & Ahn, H.-N. (1997). A meta-analysis of outcome studies comparing bona fide psychotherapies: Empirically, "all must have prizes." Psychotherapy Bulletin, 122, 203-215.
137. Weiss, J., Sampson, H., & the Mount Zion Psychotherapy Research Group. (1986). The psychoanalytic process: Theory, clinical observations, and empirical research. New York: Guilford Press.
138. Zlotnick, C., Elkin, I., & Shea, M. T. (1998). Does the gender of a patient or the gender of a therapist affect the treatment of patients with major depression? Journal of Consulting and Clinical Psychology, 1998, 655-659.