Comparing propensity score methods in balancing covariates and recovering impact in small sample educational program evaluations

Practical Assessment, Research and Evaluation

Volumn 18, Issue 13, 2013, Pages

  Stone, Clement A ^a   Tang, Yun ^a  

^a UNIVERSITY OF PITTSBURGH   (United States)

Author keywords

[No Author keywords available]

Indexed keywords

EID: 84887584557     PISSN: None     EISSN: 15317714     Source Type: Journal    
DOI: None     Document Type: Article

References (34)

1. An, W. (2010). Bayesian propensity score estimators. Sociological Methodology, 40(1), 151-189.
2. Austin, P. (2007). A critical appraisal of propensity-score matching in the medical literature between 1996 and 2003. Statistics in Medicine, 27, 2037-2049.
3. Austin, P. (2010). The performance of different propensityscore methods for estimating differences in proportions (risk differences or absolute risk reduction) in observational studies. Statistics in Medicine, 29, 2137-2148.
4. Coca-Perraillon, M. (2007). Local and global optimal propensity score matching. SUGI Global Forum 2007. Paper 185-2007.
5. Cohen, J. (1988). Statistical power analysis for the behavioral sciences (2nd ed.). Hillsdale, NJ: Lawrence Erlbaum Associates.
6. Cook, T.D., & Campbell, D.T. (1979). Quasi-experimentation: Design and Analytic Issues for Field Settings. Chicago: Rand McNally.
7. D'Agostino, R.B. (1998). Tutorial in biostatistics: Propensity score methods for bias reduction in the comparison of a treatment to a non-randomized control group. Statistics in Medicine, 17, 2265-2281.
8. Freedman, D.A., & Berk, R.A. (2008). Weighting regressions by propensity scores. Evaluation Review, 32(4), 392-409.
9. Gu, X. S. & Rosenbaum, P. R. (1993). Comparison of multivariate matching methods: structures, distances, and algorithms. Journal of Computational and Graphical Statistics, 2(4), 405-420.
10. Guo, S. & Fraser, M.W. (2010). Propensity score analysis: Statistical methods and applications. Thousand Oaks, CA: Sage.
11. Harder, V.S., Stuart, E. A., & Anthony, J. C. (2010). Propensity score techniques and the assessment of measured covariate balance to test causal associations in psychological research. Psychological Methods, 15(3), 234 - 249.
12. Heckman, J.J. (2005). The scientific model of causality. Sociological Methodology, 35, 1-97.
13. Imbens, G.W. (2004). Nonparametric estimation of average treatment effects under exogeneity: A review. The Review of Economics and Statistics, 86, 4-29.
14. Kang, J. D. Y., & Schafer, J. L. (2007). Demystifying double robustness: A comparison of alternative strategies for estimating a population mean from incomplete data. Statistical Science, 22, 523-539.
15. Kaplan, D., & Chen, C. J. S. (2010, March). A Bayesian perspective on methodologies for drawing causal inferences in experimental and non-experimental settings. Paper presented at the 2010 annual research conference of the Society for Research on Educational Effectiveness, Washington, DC.
16. Luellen, J.K. (2007). A comparison of propensity score estimation and adjustment methods on simulated data. Unpublished Dissertation, University of Memphis.
17. Luellen, J.K., Shadish, W.R., & Clark, M.H. (2005). Propensity scores: An introduction and experimental test. Evaluation Review, 29(6), 530-558.
18. McCaffrey, D.F., Ridgeway, G., & Morral, A.R. (2004). Propensity score estimation with boosted regression for evaluating causal effects in observational studies. Psychological Methods, 9(4), 403-425.
19. McCandless, L.C., Gustafson, P., & Austin, P.C. (2009). Bayesian propensity score analysis for observational data. Statistics in Medicine, 28, 94-112.
20. Parsons, L.S. (2001). Reducing bias in a propensity score matched-pair sample using greedy matching techniques. Proceedings of the Twenty-Sixth Annual SAS Users Group International Conference, Cary, NC: SAS Institute Inc. Retrieved from: http://www2.sas.com/proceedings/sugi26/p214-26.pdf.
21. Ridgeway, G., McCaffrey, D.F., Morral, A.R., Burgette, L., & Griffin, B.A. (2012). Toolkit for weighting and analysis of nonequivalent groups: a tutorial for the twang package. Retrieved from: http://cran.rproject. org/web/packages/twang/vignettes/twang.pdf
22. Rosenbaum, P.R., & Rubin, D.B. (1983). The central role of the propensity score in observational studies. Biometrika, 70, 41-45.
23. Rosenbaum, P.R, & Rubin, D.B. (1984). Reducing bias in observational studies using subclassification on the propensity score. Journal of the American Statistical Association, 79, 516-524.
24. Rosenbaum, P. R., & Rubin, D. B. (1985). Constructing a control group using multivariate matched sampling that incorporates the propensity score. The American Statistician, 39, 33-38.
25. Rubin, D.B. (1997). Estimating causal effects from large data sets using propensity scores. Annals of Internal Medicine, 127, 757-763.
26. Rubin, D.B. (2001). Using propensity scores to help design observational studies: application to the tobacco litigation. Health Services and Outcomes Research Methodology, 2, 169-188.
27. Rudner, LM.., & Peyton, J. (2006). Consider Propensity Scores to Compare Treatments. Practical Assessment Research & Evaluation, 11(9). Available online: http://pareonline.net/getvn.asp?v=11&n=9
28. SAS Institute Inc. (2008). SAS (Version 9.2) [Computer software]. Cary, NC: Author.
29. Schafer, J.L. & Kang, J. (2008). Average causal effects from nonrandomized studies: A practical guide and simulated example. Psychological Methods, 13, 279-313.
30. Schneider, B., Carnoy, M., Kilpatrick, J., Schmidt, W.H., & Shavelson, R. J. 2007. Estimating causal effects using experimental and observational designs (report from the Governing Board of the American Educational Research Association Grants Program), Washington, DC: American Educational Research Association.
31. Shadish, W.R., Cook, T.D., & Campbell, D.T. (2002). Experimental and quasi-experimental designs for generalized causal inference. Belmont, CA: Wadsworth.
32. Steiner, P.M., Shadish, W.R., Cook, T.D., & Clark, M.H. (2010). The importance of covariate selection in controlling for selection bias in observational studies. Psychological Methods, 15(1), 250-267.
33. Wilde, E.T., & Hollister, R. (2007). How close is close enough? Evaluating propensity score matching using data from a class size reduction experiment. Journal of Policy Analysis and Management, 26(3), 455-477.
34. Stone, Clement A. & Tang, Yun (2013). Comparing Propensity Score Methods in Balancing Covariates and Recovering Impact in Small Sample Educational Program Evaluations. Practical Assessment, Research & Evaluation, 18(13). Available online: http://pareonline.net/getvn.asp?v=18&n=13