Randomized controlled studies and alternative designs in outcome studies: Challenges and opportunities

Research on Social Work Practice

Volumn 21, Issue 6, 2011, Pages 636-643

  Shadish, William R ^a  

^a UNIVERSITY OF CALIFORNIA   (United States)

Author keywords

nonrandomized experiment; propensity score analysis; randomized experiment; regression discontinuity design

Indexed keywords

EID: 80054745690     PISSN: 10497315     EISSN: 15527581     Source Type: Journal    
DOI: 10.1177/1049731511403324     Document Type: Review

References (28)

1. Campbell D.T.,Stanley J.C.Experimental and quasi-experimental designs for research. Chicago, IL: Rand-McNally; 1966:.
2. Cook T.D.,Campbell D.T.Quasi-experimentation: Design and analysis issues for field settings. Chicago, IL: Rand-McNally; 1979:.
3. Cook T.D.,Shadish W.R.,Wong V.C.Three conditions under which experiments and observational studies produce comparable causal estimates: New findings from within-study comparisons.Journal of Policy Analysis and Management. 2008;27:724-750.
4. Fox J.Multiple and generalized nonparametric regression. Thousand Oaks, CA: SAGE; 2000a:.
5. Fox J.Nonparametric simple regression: Smoothing scatterplots. Thousand Oaks, CA: SAGE; 2000b:.
6. GlaserR. R. (1996). Differences between randomized and nonrandomized experiments: An attempted replication (Unpublished master's thesis). The University of Memphis, Memphis, TN.
7. Goldberger, A. S. (1972). Selection bias in evaluating treatment effects: The case of interaction. Discussion paper. Madison, WI: Institute for Research on Poverty, University of Wisconsin.
8. Goldberger, A. S. (2008). Selection bias in evaluating treatment effects: Some formal illustrations. In T. Fomby, R. C. Hill, D. L. Millimet, J. A. Smith & E. J. Vytlacil (Eds.), Modeling and evaluating treatment effects in economics (pp. 1-31). Amsterdam: JAI Press.
9. Heinsman D.T.,Shadish W.R.Assignment methods in experimentation: When do nonrandomized experiments approximate answers from randomized experiments?.Psychological Methods. 1996;1:154-169.
10. Kownacki R.J.,Shadish W.R.Does alcoholics anonymous work? The results from a meta-analysis of controlled experiments.Substance Use and Misuse. 1999;34:1897-1916.
11. Lipsey M.W.,Wilson D.B.The efficacy of psychological, educational, and behavioral treatment: Confirmation from meta-analysis.American Psychologist. 1993;48:1181-1209.
12. Luellen, J. (2007). A comparison of propensity score estimation and adjustment methods on simulated data (Unpublished doctoral dissertation). The University of Memphis, Memphis, TN.
13. Pohl S.,Steiner P.M.,Eisermann J.,Soellner R.,Cook T.D.Unbiased causal inference from an observational study: Results of a within-study comparison.Educational Evaluation and Policy Analysis. 2009;31:463-479.
14. Ragsdale K.Selection bias in nonrandomized experiments: An investigation using meta-analysis. (Unpublished doctoral dissertation). The University of Memphis, Memphis, TN.
15. Rubin D.B.Assignment to treatment group on the basis of a covariate.Journal of Educational Statistics. 1977;2:1-26.
16. Shadish, W.R. (2000). Theempirical programof quasi-experimentation. In L. Bickman (Ed.), Validity and social experimentation: Donald Campbell's legacy (pp. 13-35). Thousand Oaks, CA: SAGE.
17. Shadish W.R.,Clark M.H.,Steiner P.M.Can nonrandomized experiments yield accurate answers? A randomized experiment comparing random to nonrandom assignment.Journal of the American Statistical Association. 2008;103:1334-1343.
18. Shadish W.R.,Cook T.D.The renaissance of field experimentation in evaluating interventions.Annual Review of Psychology. 2009;60:607-629.
19. Shadish W.R.,Cook T.D.,Campbell D.T.Experimental and quasi-experimental designs for generalized causal inference. Boston, MA: Houghton-Mifflin; 2002:.
20. Shadish W.R.,Galindo R.G.,Steiner P.M.,Wong V.C.,Cook T.D.A randomized experiment comparing random to cutoff-based assignment.Psychological Methods. 2011;16:179-191.
21. Shadish W.R.,Matt G.E.,Navarro A.M.,Phillips G.The effects of psychological therapies under clinically representative conditions: A meta-analysis.Psychological Bulletin. 2000;126:512-529.
22. Shadish W.R.,Ragsdale K.Random versus nonrandom assignment in controlled experiments: Do you get the same answer?.Journal of Consulting and Clinical Psychology. 1996;64:1290-1305.
23. Shadish, W. R., Sullivan, K. J., Hedges, L. V., & Rindskopf, D. M. (2010, November 12). A d-estimator for single case designs. Invited presentation at the conference of a California Consortium for Quantitative Research and Education, Davis, CA.
24. Smith M.L.,Glass G.V.Meta-analysis of psychotherapy outcome studies.American Psychologist. 1977;32:752-760.
25. Steiner P.M.,Cook T.D.,Shadish W.R.On the importance of reliable covariate measurement in selection bias adjustments using propensity scores.Journal of Educational and Behavioral Statistics. in press;:.
26. Steiner P.M.,Cook T.D.,Shadish W.R.,Clark M.H.The importance of covariate selection in controlling for selection bias in observational studies.Psychological Methods. 2010;15:250-267.
27. Thistlewaite D.L.,Campbell D.T.Regression-discontinuity analysis: An alternative to the ex post facto experiment.Journal of Educational Psychology. 1960;51:309-317.
28. Wood S.N.Generalized additive models: An introduction with R. Boca Raton. FL: Chapman & Hall/CRC; 2006:.