Intraclass correlation associated with therapists: Estimates and applications in planning psychotherapy research

Cognitive Behaviour Therapy

Volumn 40, Issue 1, 2011, Pages 15-33

  Baldwin, Scott A ^a   Murray, David M ^b   Shadish, William R ^c   Pals, Sherri L ^d   Holland, Jason M ^e   Abramowitz, Jonathan S ^f   Andersson, Gerhard ^g   Atkins, David C ^h   Carlbring, Per ⁱ   Carroll, Kathleen M ^j   Christensen, Andrew ^k   Eddington, Kari M ^l   Ehlers, Anke ^m   Feaster, Daniel J ⁿ   Keijsers, Ger P J ^o   Koch, Ellen ^p   Kuyken, Willem ^q   Lange, Alfred ^r   Lincoln, Tania ^s   Stephens, Robert S ^t   Taylor, Steven ^u   Trepka, Chris ^v   Watson, Jeanne ^w   more..

^a BRIGHAM YOUNG UNIVERSITY   (United States)

^b OHIO STATE UNIVERSITY   (United States)

^c UNIVERSITY OF CALIFORNIA   (United States)

^d University of Memphis   (United States)

^e STANFORD UNIVERSITY SCHOOL OF MEDICINE   (United States)

^f University of North Carolina at Chapel Hill   (United States)

^g LINKÖPING UNIVERSITY   (Sweden)

^h UNIVERSITY OF WASHINGTON   (United States)

ⁱ UMEÅ UNIVERSITY   (Sweden)

^j YALE UNIVERSITY SCHOOL OF MEDICINE   (United States)

^k UNIVERSITY OF CALIFORNIA   (United States)

^l UNIVERSITY OF NORTH CAROLINA AT GREENSBORO   (United States)

^m KING'S COLLEGE LONDON   (United Kingdom)

ⁿ UNIVERSITY OF MIAMI MILLER SCHOOL OF MEDICINE   (United States)

^o RADBOUD UNIVERSITY NIJMEGEN   (Netherlands)

^p EASTERN MICHIGAN UNIVERSITY   (United States)

^q UNIVERSITY OF EXETER   (United Kingdom)

^r UNIVERSITY OF AMSTERDAM   (Netherlands)

^s PHILIPPS UNIVERSITY MARBURG   (Germany)

^t VIRGINIA POLYTECHNIC INSTITUTE AND STATE UNIVERSITY   (United States)

^u UNIVERSITY OF BRITISH COLUMBIA   (Canada)

^v BRADFORD DISTRICT CARE NHS FOUNDATION TRUST   (United Kingdom)

^w UNIVERSITY OF TORONTO   (Canada)

more..

Author keywords

Intraclass correlation; Power; Psychotherapy research; Statistical dependence; Therapist effects

Indexed keywords

ARTICLE; CLINICAL TRIAL (TOPIC); HUMAN; METHODOLOGY; OUTCOME ASSESSMENT; PSYCHOTHERAPY; REPRODUCIBILITY; TREATMENT OUTCOME;

CLINICAL TRIALS AS TOPIC; HUMANS; OUTCOME ASSESSMENT (HEALTH CARE); PSYCHOTHERAPY; REPRODUCIBILITY OF RESULTS; RESEARCH DESIGN; TREATMENT OUTCOME;

EID: 79951780040     PISSN: 16506073     EISSN: 16512316     Source Type: Journal    
DOI: 10.1080/16506073.2010.520731     Document Type: Article

References (65)

1. Abramowitz, J. S., Foa, E. B., & Franklin, M. E. (2003). Exposure and ritual prevention for obsessive-compulsive disorder: Effects of intensive versus twice-weekly sessions. Journal of Consulting and Clinical Psychology, 71, 394-398.
2. Baldwin, S. A., Murray, D. M., & Shadish, W. R. (2005). Empirically supported treatments or type I errors? Problems with the analysis of data from group-administered treatments. Journal of Consulting and Clinical Psychology, 73, 924-935.
3. Baldwin, S. A., Wampold, B. E., & Imel, Z. E. (2007). Untangling the alliance-outcome correlation: Exploring the relative importance of therapist and patient variability in the alliance. Journal of Consulting and Clinical Psychology, 75, 842-852.
4. Beck, A. T., Steer, R. A., & Brown, G. K. (1996). Manual for Beck Depression Inventory-II. San Antonio, TX: Psychological Corporation.
5. Beck, A. T., Ward, C. H., Mendelson, M., Mock, J., & Erbaugh, J. (1961). An inventory for measuring depression. Archives of General Psychiatry, 4, 561-571.
6. Bergin, A. E. (1966). Some implications of psychotherapy research for therapeutic practice. Journal of Abnormal Psychology, 71, 235-246.
7. Blitstein, J. L., Hannan, P. J., Murray, D. M., & Shadish, W. R. (2005). Increasing the degrees of freedom in existing group randomized trials: The df* approach. Evaluation Review, 29, 241-267.
8. Carlbring, P., Bohman, S., Brunt, S., Buhrman, M., Westling, B. E., Ekselius, L., & Andersson, G. (2006). Remote treatment of panic disorder: A randomized trial of Internet-based cognitive behavior therapy supplemented with telephone calls. American Journal of Psychiatry, 163, 2119-2125.
9. Carroll, K. M., Fenton, L. R., Ball, S. A., Nich, C., Frankforter, T. L., Shi, J., & Rounsaville, B. J. (2004). Efficacy of disulfiram and cognitive behavior therapy in cocaine-dependent outpatients: A randomized placebo-controlled trial. Archives of General Psychiatry, 61, 264-272.
10. Christensen, A., Atkins, D. C., Berns, S., Wheeler, J., Baucom, D. H., & Simpson, L. E. (2004). Traditional versus integrative behavioral couple therapy for significantly and chronically distressed married couples. Journal of Consulting and Clinical Psychology, 72, 176-191.
11. Cornfield, J. (1978). Randomization by group: A formal analysis. American Journal of Epidemiology, 108, 100-102.
12. Crits-Christoph, P., Baranackie, K., Kurcias, J. S., Beck, A. T., Carroll, K., Perry, K.,... Zitrin, C. (1991). Meta-analysis of therapist effects in psychotherapy outcome studies. Psychotherapy Research, 1, 81-91.
13. Crits-Christoph, P., & Mintz, J. (1991). Implications of therapist effects for the design and analysis of comparative studies of psychotherapies. Journal of Consulting and Clinical Psychology, 59, 20-26.
14. Crits-Christoph, P., Tu, X., & Gallop, R. (2003). Therapists as fixed versus random effects-Some statistical and conceptual issues: A comment on Siemer and Joormann (2003). Psychological Methods, 8, 518-523.
15. Dinger, U., Strack, M., Leichsenring, F., Wilmers, F., & Schauenburg, H. (2008). Therapist effects on outcome and alliance in inpatient psychotherapy. Journal of Clinical Psychology, 64, 344-354.
16. Donner, A. (1986). A review of inference procedures for the intraclass correlation coefficient in the one-way random effects model. International Statistics Review, 54, 67-82.
17. Donner, A., & Klar, N. (2000). Design and analysis of cluster randomization trials in health research. London: Arnold.
18. Ehlers, A., Clark, D. M., Hackmann, A., McManus, F., Fennell, M., Herbert, C., & Mayou, R. (2003). A randomized controlled trial of cognitive therapy, a self-help booklet, and repeated assessments as early interventions for posttraumatic stress disorder. Archives of General Psychiatry, 60, 1024-1032.
19. Elkin, I., Falconnier, L., Martinovich, Z., & Mahoney, C. (2006). Therapist effects in the National Institute of Mental Health Treatment of Depression Collaborative Research Program. Psychotherapy Research, 16, 144-160.
20. Fleiss, J. L., Levin, B., & Paik, M. C. (2003). Statistical methods for rates and proportions (3rd ed.). Hoboken, NJ: Wiley.
21. Gail, M. H., Mark, S. D., Carroll, R. J., Green, S. B., & Pee, D. (1996). On design considerations and randomization-based inference for community intervention trials. Statistics in Medicine, 15, 1069-1092.
22. Gulliford, M. C., Ukomunne, O. C., & Chinn, S. (1999). Components of variance and intraclass correlation for the design of community-based surveys and intervention studies. American Journal of Epidemiology, 149, 876-883.
23. Hedges, L. V., & Hedberg, E. C. (2007). Intraclass correlation values for planning group-randomized trials in education. Educational Evaluation and Policy Analysis, 29, 60-87.
24. Higgins, J. P. T., & Thompson, S. G. (2002). Quantifying heterogeneity in a meta-analysis. Statistics in Medicine, 21, 1539-1558.
25. Kenny, D. A., Kashy, D. A., & Bolger, N. (1998). Data analysis in social psychology. In D. T. Gilbert, S. T. Fiske, & G. Lindzey (Eds.), The handbook of social psychology, (Vol. 1, pp. 233-265). New York, NY: Oxford University Press.
26. Kenny, D. A., Mannetti, L., Peirro, A., Livi, S., & Kashy, D. A. (2002). The statistical analysis of data from small groups. Journal of Personality and Social Psychology, 83, 126-137.
27. Kim, D.-M., Wampold, B. E., & Bolt, D. M. (2006). Therapist effects in psychotherapy: A random-effects modeling of the National Institute of Mental Health Treatment of Depression Collaborative Research Program data. Psychotherapy Research, 16, 161-172.
28. Kish, L. (1965). Survey sampling. New York, NY: Wiley.
29. Koch, E. I., Spates, C. R., & Himle, J. A. (2004). Comparison of behavioral and cognitive-behavioral one-session exposure treatments for small animal phobias. Behaviour Research and Therapy, 42, 1483-1504.
30. Kuyken, W. (2004). Cognitive therapy outcome: The effects of hopelessness in a naturalistic outcome study. Behaviour Research and Therapy, 42, 631-646.
31. Lange, A., Rietdijk, D., Hudcovicova, M., van de Ven, J. P., Schrieken, B., & Emmelkamp, P. M. (2003). Interapy: A controlled randomized trial of the standardized treatment of posttraumatic stress through the Internet. Journal of Consulting and Clinical Psychology, 71, 901-909.
32. Lincoln, T. M., Rief, W., Hahlweg, K., Frank, M., von Witzleben, I., Schroeder, B., & Fiegenbaum, W. (2003). Effectiveness of an empirically supported treatment for social phobia in the field. Behaviour Research and Therapy, 41, 1251-1269.
33. Lutz, W., Leon, S. C., Martinovich, Z., Lyons, J. S., & Stiles, W. B. (2007). Therapist effects in outpatient psychotherapy: A three-level growth curve approach. Journal of Counseling Psychology, 54, 32-39.
34. Marijuana Treatment Project Research Group. (2004). Brief treatments for cannabis dependence: Findings from a randomized multisite trial. Journal of Consulting and Clinical Psychology, 72, 455-466.
35. Martindale, C. (1978). The therapist-as-fixed-effect fallacy in psychotherapy research. Journal of Consulting and Clinical Psychology, 46, 1526-1530.
36. Maxwell, S. E., & Delaney, H. D. (2004). Designing experiments and analyzing data: A model comparison approach (2nd ed.). Mahwah, NJ: Erlbaum.
37. Merrill, K. A., Tolbert, V. E., & Wade, W. A. (2003). Effectiveness of cognitive therapy for depression in a community mental health center: A benchmarking study. Journal of Consulting and Clinical Psychology, 71, 404-409.
38. Murray, D. M. (1998). Design and analysis of grouprandomized trials. New York, NY: Oxford University Press.
39. Murray, D. M., & Blitstein, J. L. (2003). Methods to reduce the impact of intraclass correlation in group-randomized trials. Evaluation Review, 27, 79-103.
40. Murray, D. M., Hannan, P. J., & Baker, W. L. (1996). A Monte Carlo study of alternative responses to intraclass correlation in community trials. Is it ever possible to avoid Cornfield's penalties? Evaluation Review, 20, 313-337.
41. Murray, D. M., Pals, S. P., Blitstein, J. L., Alfano, C. M., & Lehman, J. (2008). Design and analysis of group-randomized trials in cancer: A review of current practices. Journal of the National Cancer Institute, 100, 483-491.
42. Murray, D. M., Rooney, B. L., Hannan, P. J., Peterson, A. V., Ary, D. V., Biglan, A.,... Schinke, S.P. (1994). Intraclass correlation among common measures of adolescent smoking: Estimates, correlates, and applications in smoking prevention studies. American Journal of Epidemiology, 140, 1038-1050.
43. Murray, D. M., Varnell, S. P., & Blitstein, J. L. (2004). Design and analysis of group-randomized trials: A review of recent methodological developments. American Journal of Public Health, 94, 423-432.
44. Okiishi, J. C., Lambert, M. J., Nielsen, L., & Ogles, B. M. (2003). Waiting for supershrink: An empirical analysis of therapist effects. Clinical Psychology and Psychotherapy, 10, 361-373.
45. Pals, S. P., Murray, D. M., Alfano, C. M., Shadish, W. R., Hannan, P. J., & Baker, W. L. (2008). Individually randomized group treatment studies: Are the most frequently used analytic models misleading? American Journal of Public Health, 98, 1418-1424.
46. Pinheiro, J. C., & Bates, D. M. (2000). Mixedeffects models in S and S-Plus. New York, NY: Springer.
47. Roberts, C. (1999). The implications of variation in outcome between health professionals for the design and analysis of randomized controlled trials. Statistics in Medicine, 18, 2605-2615.
48. Roberts, C., & Roberts, S. A. (2005). Design and analysis of clinical trials with clustering effects due to treatment. Clinical Trials, 2, 152-162.
49. Satterthwaite, F. W. (1946). An approximate distribution of estimates of variance components. Biometrics Bulletin, 2, 110-140.
50. Schnurr, P. P., Friedman, M. J., Lavori, P. W., & Hsieh, F. Y. (2001). Design of Department of Veterans Affairs Cooperative Study No. 420: Group treatment of posttraumatic stress disorder. Controlled Clinical Trials, 22, 74-88.
51. Snedecor, G. W., & Cochran, W. G. (1989). Statistical methods (8th ed.). Ames, IA: Iowa State University Press.
52. StataCorp (2009). Stata Statistical Software (Version 11) [Computer software]. College Station, TX: Author.
53. Swallow, W. H., & Monahan, J. F. (1984). Monte Carlo comparison of ANOVA, MIVQUE, REML, and ML estimators of variance components. Technometrics, 26, 47-57.
54. Szapocznik, J., Feaster, D. J., Mitrani, V. B., Prado, G., Smith, L., Robinson-Batista, C.,... Robbins, M. S. (2004). Structural ecosystems therapy for HIV-seropositive African American women: Effects on psychological distress, family hassles, and family support. Journal of Consulting and Clinical Psychology, 72, 288-303.
55. Taylor, S., Thordarson, D. S., Maxfield, L., Fedoroff, I. C., Lovell, K., & Ogrodniczuk, J. (2003). Comparative efficacy, speed, and adverse effects of three PTSD treatments: Exposure therapy, EMDR, and relaxation training. Journal of Consulting and Clinical Psychology, 71, 330-338.
56. Trepka, C., Rees, A., Shapiro, D. A., Hardy, G. E., & Barkham, M. (2004). Therapist competence and outcome of cognitive therapy for depression. Cognitive Therapy and Research, 28, 143-157.
57. van Minnen, A., Hoogduin, K. A., Keijsers, G. P., Hellenbrand, I., & Hendriks, G. J. (2003). Treatment of trichotillomania with behavioral therapy or fluoxetine: A randomized, waitinglist controlled study. Archives of General Psychiatry, 60, 517-522.
58. Varnell, S., Murray, D. M., Janega, J. B., & Blitstein, J. L. (2004). Design and analysis of group-randomized trials: A review of recent practices. American Journal of Public Health, 94, 393-399.
59. Verma, V., & Le, T. (1996). An analysis of sampling errors for the demographic and health surveys. International Statistical Review, 64, 265-294.
60. Wampold, B. E. (2001). The great psychotherapy debate. Mahwah, NJ: Erlbaum.
61. Wampold, B. E., & Brown, G. S. (2005). Estimating variability in outcomes attributable to therapists: A naturalistic study of outcomes in managed care. Journal of Consulting and Clinical Psychology, 73, 914-923.
62. Wampold, B. E., & Serlin, R. C. (2000). The consequence of ignoring a nested factor on measures of effect size in analysis of variance. Psychological Methods, 5, 425-433.
63. Watson, J. C., Gordon, L. B., Stermac, L., Kalogerakos, F., & Steckley, P. (2003). Comparing the effectiveness of process-experiential with cognitive-behavioral psychotherapy in the treatment of depression. Journal of Consulting and Clinical Psychology, 71, 773-781.
64. Winer, B. J., Brown, D. R., & Michels, K. (1991). Statistical principles in experimental design. New York, NY: McGraw-Hill.
65. Zucker, D. M. (1990). An analysis of variance pitfall: The fixed effects analysis in a nested design. Educational and Psychological Measurement, 50, 731-738.