Assumptions and Consequences of Treating Providers in Therapy Studies as Fixed Versus Random Effects: Reply to Crits-Christoph, Tu, and Gallop (2003) and Serlin, Wampold, and Levin (2003)

Psychological Methods

Volumn 8, Issue 4, 2003, Pages 535-544

  Siemer, Matthias ^a   Joormann, Jutta ^b  

^a UNIVERSITY OF GREIFSWALD   (Germany)

^b RUHR UNIVERSITY BOCHUM   (Germany)

Author keywords

[No Author keywords available]

Indexed keywords

ANALYSIS OF VARIANCE; ARTICLE; BAYES THEOREM; CLINICAL TRIAL; CONTROLLED CLINICAL TRIAL; EPIDEMIOLOGY; HUMAN; MONTE CARLO METHOD; OUTCOME ASSESSMENT; PSYCHOLOGICAL ASPECT; PSYCHOMETRY; PSYCHOTHERAPY; RANDOMIZED CONTROLLED TRIAL; STATISTICS;

ANALYSIS OF VARIANCE; BAYES THEOREM; BIAS (EPIDEMIOLOGY); HUMANS; MONTE CARLO METHOD; OUTCOME ASSESSMENT (HEALTH CARE); PSYCHOMETRICS; PSYCHOTHERAPY; RANDOMIZED CONTROLLED TRIALS;

EID: 0347062733     PISSN: 1082989X     EISSN: None     Source Type: Journal    
DOI: 10.1037/1082-989X.8.4.535     Document Type: Article

References (36)

1. Beaton, A. E. (1978). Salvaging experiments: Interpreting least squares in non-random samples. In D. Hogben & D. W. Fife (Eds.), Computer science and statistics: Tenth annual symposium on the interface (pp. 137-145). Washington DC: U.S. Department of Commerce.
2. Box, G. E. P., Hunter, W. G., & Hunter, J. S. (1978). Statistics for experimenters: An introduction to design, data analysis and model building. New York: Wiley.
3. Cook, T. D., & Campbell, D. T. (1979). Quasi-experimentation. Design and analysis issues for field settings. Chicago: Rand McNally.
4. Crits-Christoph, P., & Mintz, J. (1991). Implications of therapist effects for the design and analysis of comparative studies of psychotherapies. Journal of Consulting and Clinical Psychology, 59, 20-26.
5. Crits-Christoph, P., Tu, X., & Gallop, R. (2003). Therapists as fixed versus random effects - Some statistical and conceptual issues: A comment on Siemer and Joormann (2003). Psychological Methods, 8, 518-523.
6. Dar, R., Serlin, R. C., & Orner, H. (1994). The misuse of statistical tests in three decades of psychotherapy research. Journal of Clinical and Counseling Psychology, 63, 75-82.
7. Davison, A. C., & Hinkley, D. V. (1997). Bootstrap methods and their application. Cambridge, England: Cambridge University Press.
8. Edgington, E. S. (1995). Randomization tests (3rd ed.). New York: Dekker.
9. Efron, B., & Tibshiriani, R. J. (1993). An introduction to the bootstrap. New York: Chapman & Hall.
10. Freedman, D., & Lane, D. (1983a). A nonstochastic interpretation of reported significance levels. Journal of Business and Economic Statistics, 1, 292-298.
11. Freedman, D., & Lane, D. (1983b). Significance testing in a nonstochastic setting. In K. A. Doksum & J. L. Hodges (Eds.), A Festschrift for Erich L. Lehmann: In honor of his 65th birthday (pp. 185-208). Belmont, CA: Wadsworth.
12. Hays, W. L. (1972). Statistics for the social sciences. New York: Holt, Rinehart & Winston.
13. Holland, P. W. (1986). Statistics and causal inference. Journal of the American Statistical Association, 81, 945-960.
14. Howard, G. S., Maxwell, S. E., & Fleming, K. J. (2000). The proof of the pudding: An illustration of the relative strengths of null hypothesis, meta-analysis, and Bayesian analysis. Psychological Methods, 5, 315-332.
15. Howson, C., & Urbach, P. (1989). Scientific reasoning: The Bayesian approach. La Salle, IL: Open Court.
16. Johnstone, D. (1989). On the necessity for random sampling. British Journal for the Philosophy of Science, 40, 443-457.
17. Keppel, G. (1991). Design and analysis: A researcher's handbook (3rd ed.). Upper Saddle River, NJ: Prentice Hall.
18. Lipsey, M. W. (2000). Statistical conclusion validity for intervention research. A significant (p < .05) problem. In L. Bickman (Ed.), Contributions to research design: Donald Campbell's legacy (pp. 101-120). Newbury Park, CA: Sage.
19. Lunneborg, C. E. (2000). Data analysis by resampling: Concepts and applications. Pacific Grove, CA: Duxbury.
20. Lunneborg, C. E. (2001). Random assignment of available cases: Bootstrap standard errors and confidence intervals. Psychological Methods, 6, 402-412.
21. Manly, B. F. J. (1997). Randomization, bootstrap and Monte Carlo methods in biology. London: Chapman & Hall.
22. Marascuilo, L. A., & Serlin, R. C. (1988). Statistical methods for the social and behavioral sciences. New York: Freeman.
23. Maxwell, S. E., & Delaney, H. D. (1990). Designing experiments and analyzing data. A model comparison perspective. Pacific Grove, CA: Brooks/Cole.
24. Neyman, J. (1990). On the application of probability theory to agricultural experiments. Essay on principles. Section 9. Statistical Science, 5, 465-480. (Original work published 1923)
25. Papineau, D. (1994). The virtues of randomization. British Journal for the Philosophy of Science, 45, 437-450.
26. Pitman, E. J. G. (1937). Significance tests which may be applied to samples from any population. Journal of the Royal Statistical Society, 4(Suppl.), 101-120.
27. Reichardt, C. S., & Gollob, H. F. (1999). Justifying the use and increasing the power of a t test for a randomized experiment with a convenience sample. Psychological Methods, 4, 117-124.
28. Rubin, D. B. (1990). Comment: Neyman (1923) and causal inference in experiments and observational studies. Statistical Science, 5, 472-480.
29. Schmidt, F. L., & Hunter, J. E. (1997). Eight common but false objections to the discontinuation of significance testing in the analysis of research data. In L. L. Harlow, S. A. Mulaik, & J. H. Steiger (Eds.), What if there were no significance tests? (pp. 37-64). Mahwah, NJ: Erlbaum.
30. Seaman, M. A., & Serlin, R. C. (1998). Equivalence confidence intervals for two-group comparisons of means. Psychological Methods, 3, 403-411.
31. Serlin, R. C., Wampold, B. E., & Levin, J. R. (2003). Should providers of treatment be regarded as a random factor? If it ain't broke, don't "fix" it: A comment on Siemer and Joormann (2003). Psychological Methods, 8, 524-534.
32. Shadish, W. R., Cook, T. D., & Campbell, D. T. (2002). Experimental and quasi-experimental designs for generalized causal inference. Boston: Houghton Mifflin.
33. Siemer, M., & Joormann, J. (2003). Power and measures of effect size in analysis of variance with fixed versus random nested factors. Psychological Methods, 8, 497-517.
34. Wampold, B. E. (2001). The great psychotherapy debate: Models, methods, and findings. Mahwah, NJ: Erlbaum.
35. Wampold, B. E., & Serlin, R. C. (2000). The consequence of ignoring a nested factor on measures of effect size in analysis of variance. Psychological Methods, 5, 425-433.
36. Wilkinson, L., & the Task Force on Statistical Inference. (1999). Statistical methods in psychological journals: Guidelines and explanations. American Psychologist, 54, 594-604.