Improving Generalizations From Experiments Using Propensity Score Subclassification: Assumptions, Properties, and Contexts

Journal of Educational and Behavioral Statistics

Volumn 38, Issue 3, 2013, Pages 239-266

  Tipton, Elizabeth ^a  

^a Och Spine at New York Presbyterian Hospitals   (United States)

Author keywords

causal inference; experiment; external validity; program evaluation; propensity scores

Indexed keywords

EID: 84877984187     PISSN: 10769986     EISSN: None     Source Type: Journal    
DOI: 10.3102/1076998612441947     Document Type: Article

References (39)

1. Bloom H. S.Learning more from social experiments: Evolving analytic approaches. New York, NY: Russell Sage Foundation; 2005:.
2. Bohrnstedt G. W.,Stecher B. MPalo Alto, CA: CSR Research Consortium, Year 1 Evaluation Report, American Institutes for Research; 1999:.
3. Cochran W. G.The effectiveness of adjustment by subclassification in removing bias in observational studies.Biometrics. 1968;24:295-313.
4. Cochran W. G.,Cox G. MExperimental designs. Wiley classics library. New York, NY: Wiley; 1992:.
5. Cole S. R.,Stuart E. A.Generalizing evidence from randomized clinical trials to target populations: The ACTG-320 trial.American Journal of Epidemiology. 2010;172:107-115.
6. Cook T. D.Understanding causes and generalizing about them: New directions for program evaluation. Sechrest L.Scott A. G., ed. San Francisco, CA: Jossey-Bass; 1993:39-82.
7. Cook T. D.,Shadish W. R.,Wong V. C.Three conditions under which experiments and observational studies produce comparable causal estimates: New findings from within-study comparisons.Journal of Policy Analysis and Management. 2008;27:724-750.
8. Cornfield J.,Tukey J. W.Average values of mean squares in factorials.The Annals of Mathematical Statistics. 1956;27:907-949.
9. Cronbach L. J.Designing evaluations of educational and social programs. San Francisco, CA: Jossey-Bass; 1982:.
10. Dehejia R. H.,Wahba S.Causal effects in nonexperimental studies: Reevaluating the evaluation of training programs.Journal of American Statistical Association. 1999;94:1053-1062.
11. Dehejia R. H.,Wahba S.Propensity score-matching methods for nonexperimental causal studies.The Review of Economics and Statistics. 2002;84:151-161.
12. Groves R. M.,Fowler F. J.,Couper M. P.,Lepkowski J. M.,Singer E.Survey methodology. New York, NY: John Wiley; 2009:.
13. Hedges L. V.Effect sizes in cluster-randomized designs.Journal of Educational and Behavioral Statistics. 2007;32:341-370.
14. Hedges L. V.,O'Muircheartaigh C. AImproving generalizations from designed experiments. Northwestern University. 2011:.
15. Hill J.Discussion of research using propensity-score matching: Comments on 'A critical appraisal of propensity-score matching in the medical literature between 1996 and 2003' by Peter Austin.Statistics in Medicine. 2008;27:2055-2061.
16. Ho D. E.,Imai K.,King G.,Stuart E. A.Matchit: Nonparametric preprocessing for parametric causal inference.Journal of Statistical Software. 2007;42:1-28.
17. Holland P. W.Statistics and causal inference.Journal of the American Statistical Association. 1986;81:945-960.
18. Hong G.,Raudenbush S. W.Evaluating kindergarten retention policy: A case study of causal inference for multi-level observational data.Journal of the American Statistical Association. 2006;101:901-910.
19. Imai K.,King G.,Stuart E. A.Misunderstandings between experimentalists and observationalists about causal inference.Journal of the Royal Statistical Society, Series A. 2008;171:481-502.
20. ImaiK.KingG.LauO. (2006). Zelig: Everyone's statistical software. R package version 3.5.3. Retrieved fromhttp://CRAN.R-project.org/package=Zelig.
21. Kang J. D. Y.,Schafer J. L.Demystifying double robustness: A comparison of alternative strategies for estimating a population mean from incomplete data.Statistical Science. 2007;22:523-539.
22. Kish L.Statistical design for research. Wiley series in probability and mathematical statistics. New York, NY: John Wiley; 1987:.
23. Kitagawa E. M.Standardized comparisons in population research.Demography. 1964;1:296-315.
24. Little R. J. A.Survey nonresponse adjustments.International Statistical Review. 1986;54:139-157.
25. Nadarajah S.,Kotz S.R programs for computing truncated distributions.Journal of Statistical Software. 2006;16:1-8.
26. Pearl J.,Bareinboim E.Proceedings of the 25th AAAI Conference on Artificial Intelligence. San Francisco, CA Menlo Park, CA: AAAI Press; 2011:7 247-11 254.
27. R Development Core Team. (2011). R: A language and environment for statistical computing. Vienna, Austria: R Foundation for Statistical Computing. Retrieved fromhttp://www.R-project.org/.
28. Roschelle J.,Roschelle J.,Schechtman N.,Tatar D.,Hegedus S.,Hopkins B.,Empson S.,Knudsen J.,Gallagher L.Integration of technology, curriculum, and professional development for advancing middle school mathematics: Three large-scale studies.American Educational Research Journal. 2010;47:833-878.
29. Rosenbaum P. R.,Rubin D. B.The central role of the propensity score in observational studies for causal effects.Biometrika. 1983;70:41-55.
30. Rosenbaum P. R.,Rubin D. B.Reducing bias in observational studies using subclassification on the propensity score.Journal of the American Statistical Association. 1984;79:516-524.
31. Rubin D. B.Estimating causal effects of treatments in randomized and nonrandomized studies.Journal of Educational Psychology. 1974;66:688-701.
32. Rubin D. B.Bayesian inference for causal effects: The role of randomization.The Annals of Statistics. 1978;6:34-58.
33. Rubin D. B.Comment: Neyman (1923) and causal inference in experiments and observational studies.Statistical Science. 1980;5:472-480.
34. Rubin D. B.Formal modes of statistical inference for causal effects.Journal of Statistical Planning and Inference. 1990;25:279-292.
35. Schneider B.,McDonald S. K.Scale-up in education: Ideas in principle. Lanham, MD: Rowman & Littlefield; 2007:.
36. Shadish W. R.Campbell and Rubin: A primer and comparison of their approaches to causal inference in field settings.Psychological methods. 2010;15:3-17.
37. Shadish W. R.,Cook T. D.,Campbell D. T.Experimental and quasi-experimental designs for generalized causal inference. Boston, MA: Houghton-Mifflin; 2002:.
38. Stuart E. A.,Cole S. R.,Bradshaw C. P.,Leaf P. J.The use of propensity scores to assess the generalizability of results from randomized trials.Journal of the Royal Statistical Society: Series A. 2011;174:369-386.
39. Zanutto E. L.A comparison of propensity score and linear regression analysis of complex survey data.Journal of Data Science. 2006;4:67-91.