Empirically supported treatments or type I errors? Problems with the analysis of data from group-administered treatments

Journal of Consulting and Clinical Psychology

Volumn 73, Issue 5, 2005, Pages 924-935

  Baldwin, Scott A ^a,c,e   Murray, David M ^a,c,d   Shadish, William R ^b  

^a UNIVERSITY OF MEMPHIS   (United States)

^b UNIVERSITY OF CALIFORNIA   (United States)

^c University of Memphis   (United States)

^d OHIO STATE UNIVERSITY   (United States)

^e Ohio State University ^*  (United States)

Author keywords

Empirically supported treatments; Group administered treatments; Intraclass correlation; Nested designs

Indexed keywords

ANALYTICAL ERROR; ANXIETY; ARTICLE; CLINICAL TRIAL; CONTROLLED CLINICAL TRIAL; CONTROLLED STUDY; EMOTION; EMPIRICALLY SUPPORTED TREATMENT; FAMILY THERAPY; GROUP THERAPY; HUMAN; INTERVIEW; INTROVERSION; MAJOR CLINICAL STUDY; MARITAL THERAPY; MENTAL DISEASE; META ANALYSIS; PSYCHOTHERAPY; SCHIZOPHRENIA; STATISTICAL ANALYSIS; SYSTEMATIC REVIEW; TREATMENT OUTCOME;

ANALYSIS OF VARIANCE; BEHAVIOR THERAPY; BIAS (EPIDEMIOLOGY); DATA INTERPRETATION, STATISTICAL; EMPIRICAL RESEARCH; EVIDENCE-BASED MEDICINE; HUMANS; PSYCHOTHERAPY, GROUP; REPRODUCIBILITY OF RESULTS; RESEARCH DESIGN; TREATMENT OUTCOME;

EID: 27844572370     PISSN: 0022006X     EISSN: None     Source Type: Journal    
DOI: 10.1037/0022-006X.73.5.924     Document Type: Article

References (62)

1. Arean, P. A., Perri, M. G., Nezu, A. M., Schein, R. L., Christopher, F., & Joseph, T. X. (1993). Comparative effectiveness of social problem-solving therapy and reminiscence therapy as treatments for depression in older adults. Journal of Consulting and Clinical Psychology, 61, 1003-1010.
2. Arnow, B. A., Taylor, C. B., Agras, W. S., & Telch, M. J. (1985). Enhancing agoraphobia treatment outcome by changing couple communication patterns. Behavior Therapy, 16, 452-467.
3. Azrin, N. H. (1976). Improvements in the community-reinforcement approach to alcoholism. Behaviour Research and Therapy, 14, 339-348.
4. Azrin, N. H., Besalel, V. A., Bechtel, R., Michalicek, A., Mancera, M., Carroll, D., et al. (1980). Comparison of reciprocity and discussion-type counseling for marital procedures. American Journal of Family Therapy, 8, 21-28.
5. Blitstein, J. L., Hannan, P. J., Murray, D. M., & Shadish, W. R., (in press-a). Increasing the degrees of freedom in existing group randomized trials: The df* approach. Evaluation Review.
6. Blitstein, J. L., Hannan, P. J., Murray, D. M., & Shadish, W. R. (in press-b). Increasing the degrees of freedom in future group randomized trials: The df* approach. Evaluation Review.
7. Burlingame, G. M., Kircher, J. C., & Honts, C. R. (1994). Analysis of variance versus bootstrap procedures for analyzing dependent observations in small group research. Small Group Research, 25, 486-501.
8. Chambless, D. L., & Hollon, S. D. (1998). Defining empirically supported therapies. Journal of Consulting and Clinical Psychology, 66, 7-18.
9. Cinciripini, P. M., Lapitsky, L., Seay, S., Wallfisch, A., Kitchens, K., & Vunakis, H. V. (1995). The effects of smoking schedules on cessation outcome: Can we improve on common methods of gradual and abrupt nicotine withdrawal? Journal of Consulting and Clinical Psychology, 63, 388-399.
10. Cinciripini, P. M., Lapitsky, L., Wallfisch, A., Mace, R., Nezami, E., & Vunakis, H. V. (1994). An evaluation of a multicomponent treatment program involving scheduled smoking and relapse prevention procedures: Initial findings. Addictive Behaviors, 19, 13-22.
11. Craemer, M., Morris, P., Biddle, D., & Elliot, P. (1999). Treatment outcome in Australian veterans with combat-related posttraumatic stress disorder. A cause for cautious optimism? Journal of Traumatic Stress, 12, 545-558.
12. Dishion, T. J., McCord, J., & Poulin, F. (1999). When interventions harm: Peer groups and problem behavior. American Psychologist, 54, 755-764.
13. Donner, A., Birkett, N., & Buck, C., (1981). Randomization by cluster: Sample size requirements and analysis. American Journal of Epidemiology, 14, 322-326.
14. Donner, A., & Klar, N. (2000). Design and analysis of cluster randomization trials in health research. London: Arnold.
15. Drummond, D. C., & Glautier, S. (1994), A controlled trial of cue exposure treatment in alcohol dependence. Journal of Consulting and Clinical Psychology, 62, 809-817.
16. Eriksen, L., Bjornstad, S., & Gunnar, G. K. (1986). Social skills training in groups for alcoholics: One-year treatment outcome for groups and individuals. Addictive Behaviors, 11, 309-329.
17. Etringer, B. D., Gregory, V. R., & Lando, H. A. (1984). Influence of group cohesion on the behavioral treatment of smoking. Journal of Consulting and Clinical Psychology, 52, 1080-1086.
18. Fuchs, C. Z., & Rehm, L. P. (1977). A self-control behavior therapy program for depression. Journal of Consulting and Clinical Psychology, 45, 206-215.
19. Heimberg, R. G., Dodge, C. S., Hope, D. A., Kennedy, C. R., Zollo, L. J., & Baker, R. E. (1990). Cognitive behavioral group treatment for social phobia: Comparison with a credible placebo control. Cognitive Therapy and Research, 14, 1-23.
20. Herzog, T. A., Lazev, A. B., Irvin, J. E., Juliano, L. M., Greenbaum, P. E., & Brandon, T. H. (2002). Testing for group membership effects during and after treatment: The example of group therapy for smoking cessation. Behavior Therapy, 33, 29-43.
21. Higgins, S. T., Budney, A. J., Bickel, W. K., Hughes, J. R., Foerg, F., & Badger, G. (1993). Achieving cocaine abstinence with a behavioral approach. American Journal of Psychiatry, 150, 763-769.
22. Hurlbert, D. F., White, L. C., & Powell, R. D. (1993). Orgasm consistency training in the treatment of women reporting hypoactive sexual desire: An outcome comparison of women-only groups and couples-only groups. Journal of Behavior Therapy and Experimental Psychiatry, 24, 3-13.
23. Kazdin, A. E., & Bass, D. (1989). Power to detect differences between alternative treatments in comparative psychotherapy outcome research. Journal of Consulting and Clinical Psychology, 57, 138-147.
24. Keefe, F. J., Caldwell, D. S., Williams, D. A., Gil, K. M., Mitchell, D., Robertson, C., et al. (1990a). Pain coping skills training in the management of osteoarthritic knee pain: A comparative study. Behavior Therapy, 21, 49-62.
25. Keefe, F. J., Caldwell, D. S., Williams, D. A., Gil, K. M., Mitchell, D., Robertson, C., et al. (1990b). Pain coping skills training in the management of osteoarthritic knee pain-II: Follow-up results. Behavior Therapy, 21, 435-447.
26. Kenny, D. A., & Judd, C. M. (1986). Consequences of violating the independence assumption in analysis of variance. Psychological Bulletin, 99, 422-431.
27. Kenny, D. A., Mannetti, L., Pierro, A., Livi, S., & Kashy, D. A. (2002). The statistical analysis of data from small groups. Journal of Personality and Social Psychology, 83, 126-137.
28. Keppel, G. (1991). Design and analysis: A researcher's handbook (3rd ed.). Englewood Cliffs, NJ: Prentice Hall.
29. Linehan, M. M., Armstrong, H. E., Suarez, A., Allmon, D., & Hearld, H. L. (1991). Cognitive-behavioral treatment of chronically parasuicidal borderline patients. Archives of General Psychiatry, 48, 1060-1064.
30. Marder, S. R., Wirshing, W. C., Mintz, J., McKenzie, J., Johnston, K., Eckman, T. A., et al. (1996). Two-year outcome of social skills training and group psychotherapy for outpatients with schizophrenia. Archives of General Psychiatry, 153, 1585-1592.
31. Murray, D. M. (1998). Design and analysis of group-randomized trials. New York: Oxford University Press.
32. Murray, D. M., & Blitstein, J. L. (2003). Methods to reduce the impact of intraclass correlation in group-randomized trials. Evaluation Review, 27, 79-103.
33. Murray, D. M., Hannan, P. J., & Baker, W. L. (1996). A Monte Carlo study of alternative responses to intraclass correlation in community trials: Is it ever possible to avoid Cornfield's penalties? Evaluation Review, 20, 313-337.
34. Murray, D. M., Varnell, S. P., & Blitstein, J. L. (2004). Design and analysis of group-randomized trials: A review of recent methodological developments. American Journal of Public Health, 94, 423-432.
35. Newton-John, T. R. O., Spence, S. H., & Schotte, D. (1995). Cognitive-behavioural therapy versus EMG biofeedback in the treatment of chronic low back pain. Behaviour Research and Therapy, 33, 691-697.
36. Nezu, A. M. (1986). Efficacy of a social problem-solving therapy approach for unipolar depression. Journal of Consulting and Clinical Psychology, 54, 196-202.
37. Nezu, A. M. & Perri, M. G. (1989). Social problem-solving therapy for unipolar depression: An initial dismantling investigation. Journal of Consulting and Clinical Psychology, 57, 408-413.
38. Nicholas, M. K., Wilson, P. H., & Goyen, J. (1991). Operant-behavioural and cognitive-behavioural treatment for chronic low back pain. Behaviour Research and Therapy, 29, 225-238.
39. O'Farrell, T. J., Cutter, H. S. G., Choquette, K. A., Floyd, F. J., & Bayog, R. D. (1992). Behavioral marital therapy for male alcoholics: Marital and drinking adjustment during the two years after treatment. Behavior Therapy, 23, 529-549.
40. O'Farrell, T. J., Cutter, H. S. G., & Floyd, F. J. (1985). Evaluating behavioral marital therapy for male alcoholics: Effects on marital adjustment and communication from before to after treatment. Behavior Therapy, 16, 147-167.
41. Otto, M. W., Pollack, M. H., Sachs, G. S., Reiter, S. R., Meltzer-Broody, S., & Rosenbaum, J. F. (1993). Discontinuation of benzodiazepine treatment: Efficacy of cognitive-behavioral therapy for patients with panic disorder. American Journal of Psychiatry, 150, 1485-1490.
42. Parker, J. C., Frank, R. G., Beck, N. C., Smarr, K. L., Beuescher, K. L., Phillips, L. R., et al. (1988). Pain management in rheumatoid arthritis patients: A cognitive-behavioral approach. Arthritis and Rheumatism, 31, 593-601.
43. Paul, G. L., & Shannon, D. T. (1966). Treatment of anxiety through systematic desensitization in therapy groups. Journal of Abnormal Psychology, 71, 124-135.
44. Payne, A., & Blanchard, E. B. (1995). A controlled comparison of cognitive therapy and self-help support groups in the treatment of irritable bowel syndrome. Journal of Consulting and Clinical Psychology, 63, 779-786.
45. Rehm, L. P., Fuchs, C. Z., Roth, D. M., Kornblith, S. J., & Romano, J. M. (1979). A comparison of self-control and assertion skills treatments of depression. Behavior Therapy, 10, 429-442.
46. Rooney, B. L. (1992). A meta-analysis of smoking prevention programs after adjustment for study designs (Volumes I and II). Dissertation Abstracts International, 42(11-B), 5669.
47. Siemer, M., & Joorman, J. (2003b). Assumptions and consequences of treating providers in therapy studies as fixed versus random effects: Reply to Crits-Christoph, Tu, and Gallop (2003) and Serlin, Wampold, and Levin (2003). Psychological Methods, 8, 535-544.
48. Siemer, M., & Joorman, J. (2003a). Power and measures of effect size in analysis of variance with fixed versus random nested factors. Psychological Methods, 8, 497-517.
49. Stevens, V. J., & Hollis, J. F. (1989). Preventing smoking relapse, using an individually tailored skills-training technique. Journal of Consulting and Clinical Psychology, 57, 420-424.
50. Task Force on Promotion and Dissemination of Psychological Procedures. (1995). Training in and dissemination of empirically-validated psychological treatments: Report and recommendations. Clinical Psychologist, 48, 3-23.
51. Task Force on Promotion and Dissemination of Psychological Procedures. (1998). Update on empirically supported therapies II. Clinical Psychologist, 51, 3-16.
52. Telch, C. F., Agras, W. S., Rossiter, E. M., Wilfley, D., & Kenardy, J. (1990). Group cognitive-behavioral treatment for the nonpurging bulimic: An initial evaluation. Journal of Consulting and Clinical Psychology, 58, 629-635.
53. Turner, J. A., & Clancy, S. (1988). Comparison of operant behavioral and cognitive-behavioral group treatment for low back pain. Journal of Consulting and Clinical Psychology, 56, 261-266.
54. Turner, J. A., Clancy, S., McQuade, K. J., & Cardenas, D. D. (1990). Effectiveness of behavioral therapy for chronic low back pain: A component analysis. Journal of Consulting and Clinical Psychology, 58, 573-579.
55. Varnell, S. P., Murray, D. M., & Baker, W. L. (2001). An evaluation of analysis options for the one-group-per-condition design: Can any of the alternatives overcome the problems inherent in this design? Evaluation Review, 25, 440-453.
56. Varnell, S. P., Murray, D. M., Hannan, P. J., & Baker, W. L. (2001). Intraclass correlation at the level of the unit of intervention in a randomized clinical trial: Implications for analysis. Paper presented at the Annual Meeting of the American Evaluation Association, St. Louis, MO.
57. Varnell, S. P., Murray, D. M., Janega, J. B., & Blitstein, J. L. (2004). Design and analysis of group-randomized trials: A review of recent practices. American Journal of Public Health, 94, 393-399.
58. Walter, H. I., & Gilmore, S. K. (1973). Placebo versus social learning effects in parent raining procedures designed to alter the behavior of aggressive boys. Behavior Therapy, 4, 361-377.
59. Weisz, J. R., & Hawley, K. M. (2001). Procedural and coding manual for identification of evidence-based treatments. Washington, DC: American Psychological Association, Society for Clinical Psychology Division 12 Committee on Science and Practice.
60. Wellek, S. (2003). Testing statistical hypotheses of equivalence, Boca Raton FL: Chapman and Hall/CRC.
61. Wilfley, D., Agras, W. S., Telch, C. F., Rossiter, E. M., Schneider, J. A., Cole, A. G., et al. (1993). Group cognitive-behavioral therapy and group interpersonal psychotherapy for the nonpurging bulimic individual: A controlled comparison. Journal of Consulting and Clinical Psychology, 61, 296-305.
62. Zucker, D. M. (1990). An analysis of variance pitfall: The fixed effect analysis in a nested design. Educational and Psychological Measurement, 50, 731-738.