On the importance of reliable covariate measurement in selection bias adjustments using propensity scores

Journal of Educational and Behavioral Statistics

Volumn 36, Issue 2, 2011, Pages 213-236

  Steiner, Peter M ^a   Cook, Thomas D ^b   Shadish, William R ^c  

^a UNIVERSITY OF WISCONSIN MADISON   (United States)

^b NORTHWESTERN UNIVERSITY   (United States)

^c UNIVERSITY OF CALIFORNIA   (United States)

Author keywords

attenuation bias; measurement error; propensity score; selection bias; strong ignorability

Indexed keywords

EID: 79955062603     PISSN: 10769986     EISSN: None     Source Type: Journal    
DOI: 10.3102/1076998610375835     Document Type: Article

References (40)

1. Aiken, L.S., & West, S.G. (1991). Multiple regression: Testing and interpreting interactions. Newbury Park, CA: Sage.
2. Beck, A.T., & Beck, R.W. (1972). Screening depressed patients in family practice: A rapid technic. Postgraduate Medicine, December, 51, 81-85.
3. Bollen, K.A. (1989). Structural equations with latent variables. New York, NY: Wiley.
4. Campbell, D.T., & Erlebacher, A.E. (1970). How regression artifacts can mistakenly make compensatory education programs look harmful. In J. Hellmuth (Ed.), The disadvantaged child: Vol. 3, Compensatory education: A national debate (pp. 185-210). New York, NY: Brunner/Mazel.
5. Cicirelli, V.G., & &, Associates (1969). The impact of Head Start: An evaluation of the effects of Head Start on children's cognitive and affective development Vols. 1-2. Athens: Ohio University and Westinghouse Learning Corporation.
6. Cochran, W.G. (1968). The effectiveness of adjustment by subclassification in removing bias in observational studies. Biometrics, 24, 295-313.
7. Cohen, J., Cohen, P., West, S.G., & Aiken, L.S. (2003). Applied multiple regression/ correlation analysis for the behavioral sciences (3rd ed.). Mahwah, NJ: Erlbaum.
8. Cook, T.D. (2008). "Waiting for life to arrive": A history of the regression-discontinuity design in psychology statistics and economics. Journal of Econometrics, 142, 636-54.
9. Cronbach, L.J. (1982). Designing evaluations of educational and social programs. San Francisco, CA: Jossey-Bass.
10. Cronbach, L.J., Rogosa, D.R., Floden, R.E., & Price, G.G. (1976). Analysis of covariance: Angel of salvation, or temptress and deluder? Paper of the Stanford Evaluation Consortium, Stanford University. Retrieved from http://www.eric.ed.gov/ERICDocs/data/ericdocs2sql/content_storage_01/0000019b/80/36/51/b2.pdf.
11. Educational Testing Service. (1962). Vocabulary Test II (V-2). Kit of factor referenced cognitive tests. Princeton, NJ: Author.
12. Educational Testing Service. (1993). Arithmetic Aptitude Test (RG-1). Kit of factor referenced cognitive tests. Princeton, NJ: Author.
13. Faust, M.W., Ashcraft, M.H., & Fleck, D.E. (1996). Mathematics anxiety effects in simple and complex addition. Mathematical Cognition, 2, 25-62.
14. Gillespie, E.S. (1983). The influence of measurement errors in multiple linear regression. International Journal of Mathematical Education in Science and Technology, 14, 485-487.
15. Goldberg, L.R. (1999). A broad-bandwidth, public-domain, personality inventory measuring the lower-level facets of several five-factor models. In I. Mervielde, I. Deary, F. De Fruyt, & F. Ostendorf (Eds.), Personality psychology in Europe, Vol. 7 (pp. 7-28). Tilburg, The Netherlands: Tilburg University Press.
16. Heckman, J., & Navarro-Lozano, S. (2004). Using matching, instrumental variables, and control functions to estimate economic choice models. The Review of Economics and Statistics, 86, 30-57.
17. Judd, C.M., & Kenny, D.A. (1981). Estimating the effects of social interventions. Cambridge, England: Cambridge University Press.
18. Kaplan, D. (1999). An extension of the propensity score adjustment method for the analysis of group differences in MIMIC models. Multivariate Behavioral Research, 34, 467-492.
19. Liu, K. (1988). Measurement error and its impact on partial correlation and multiple regression analyses. American Journal of Epidemiology, 127, 864-874.
20. Lunceford, J.K., & Davidian, M. (2004). Stratification and weighting via propensity score in estimation of causal treatment effects: A comparative study. Statistical Medicine, 23, 2937-2960.
21. Magidson, J. (1977). Toward a causal model approach for adjusting for preexisting differences in the nonequivalent control group situation. Evaluation Quarterly, 1, 399-402.
22. Morgan, S.L., & Winship, C. (2007). Counterfactuals and causal inference: Methods and principles for social research. Cambridge, England: Cambridge University Press.
23. Pearl, J. (2000). Causality: Models, reasoning, and inference. Cambridge, England: Cambridge University Press.
24. R Development Core Team. (2007). R: A language and environment for statistical computing. R Foundation for Statistical Computing, Vienna, Austria. http://www.R-project.org.
25. Robins, J.M., & Rotnitzky, A. (1995). Semiparametric efficiency in multivariate regression models with missing data. Journal of the American Statistical Association, 90, 122-129.
26. Robins, J.M., & Rotnitzky, A. (2001). Comment on 'Inference for semiparametric models: Some questions and answers' by Bickel and Kwon. Statistica Sinica, 11, 920-936.
27. Rogosa, D. (1980). A critique of cross-lagged correlation. Psychological Bulletin, 88, 245-258.
28. Rosenbaum, P.R. (2002). Observational studies (2nd ed.). New York, NY: Springer-Verlag.
29. Rosenbaum, P.R., & Rubin, D.B. (1983). The central role of the propensity score in observational studies for causal effects. Biometrika, 70, 41-55.
30. Rosenbaum, P.R., & Rubin, D.B. (1984). Reducing bias in observational studies using subclassification on the propensity score. Journal of the American Statistical Association, 79, 516-524.
31. Rubin, D.B. (2001). Using propensity scores to help design observational studies: Application to the tobacco litigation. Health Services and Outcomes Research Methodology, 2, 169-188.
32. Rubin, D.B. (2006). Matched sampling for causal effects. Cambridge, England: Cambridge University Press.
33. Rubin, D.B. (2008). For objective causal inference, design trumps analysis. The Annals of Applied Statistics, 2, 808-840.
34. Shadish, W.R., Cook, T.D., & Campbell, D.T. (2002). Experimental and quasi-experimental designs for generalized causal inference. Boston, MA: Houghton-Mifflin.
35. Shadish, W.R., Clark, M.H., & Steiner, P.M. (2008). Can nonrandomized experiments yield accurate answers? A randomized experiment comparing random to nonrandom assignment. Journal of the American Statistical Association, 103, 1334-1343.
36. Stanley, J.C. (1971). Reliability. In R. L. Thorndike (Ed.), Educational measurement (2nd ed, pp. 356-442). Washington, DC: American Council on Education.
37. Steiner, P.M., Cook, T.D., Shadish, W.R., & Clark, M.H. (2010). The importance of covariate selection in controlling for selection bias in observational studies. Psychological Methods, 15(3), 250-267.
38. Steyer, R. (2005). Analyzing individual and average causal effects via structural equation models. Methodology: European Journal of Research Methods for the Behavioral and Social Sciences, 1, 39-54.
39. Thistlewaite, D.L., & Campbell, D.T. (1960). Regression-discontinuity analysis: An alternative to the ex post facto experiment. Journal of Educational Psychology, 51, 309-317.
40. Zinbarg, R.E., Suzuki, S., Uliaszek, A.A., & Lewis, A. R. (2010). Biased parameter estimates and inflated Type I error rates in analysis of covariance (and analysis of partial variance) arising from unreliability: Alternatives and remedial strategies. Journal of Abnormal Psychology, 119, 307-319.