Teaching statistical inference for causal effects in experiments and observational studies

Journal of Educational and Behavioral Statistics

Volumn 29, Issue 3, 2004, Pages 343-367

  Rubin, Donald B ^a,b  

^a HARVARD UNIVERSITY   (United States)

^b HARVARD UNIVERSITY   (United States)

Author keywords

Bayes; Causal inference; Fisher; Neyman; Noncompliance; Rubin's causal model

Indexed keywords

EID: 10044230409     PISSN: 10769986     EISSN: None     Source Type: Journal    
DOI: 10.3102/10769986029003343     Document Type: Article

References (108)

1. Angrist, J. D., Imbens, G. W., & Rubin, D. B. (1996). Identification of causal effects using instrumental variables. Journal of the American Statistical Association, 91, 434, [as Applications Invited Discussion Article with discussion and rejoinder, 444-472].
2. Ashenfelter, O., & Card, D. (1985). Using the longitudinal structure of earnings to estimate the effect of training programs. Review of Economics and Statistics, 67, 648-660.
3. Ashenfelter, O., & Krueger, A. (1993). Estimates of the economic return to schooling from a new sample of twins. American Economic Review, 84, 1157-1173.
4. Baker, S. G. (1998). Analysis of survival data from a randomized trial with all-or-none compliance: Estimating the cost-effectiveness of a cancer screening program. Journal of the American Statistical Association, 93, 929-934.
5. Baker, S. G., & Lindeman, K. S. (1994). The paired availability design: A proposal for evaluating epidural analgesia during labor. Statistics in Medicine, 13, 2269-2278.
6. Barnard, J., Frangakis, C., Hill, J., & Rubin, D. B. (2002). School choice in NY City: A Bayesian analysis of an imperfect randomized experiment. In C. Gatsonis, B. Carlin, & A. Carriquiry (Eds.), Case studies in Bayesian statistics: Vol. 5 [With discussion and rejoinder] (pp. 3-97). New York: Springer.
7. Bloom, H. S. (1984). Accounting for no-shows in experimental evaluation designs. Evaluation Review, 8, 225-246.
8. Cochran, W. G. (1963). Sampling techniques (2nd ed.). New York: John Wiley.
9. Cochran, W. G. (1968). The effectiveness of adjustment by subclassification in removing bias in observational studies. Biometrics, 24, 295-313.
10. Cochran, W. G., & Rubin, D. B. (1973). Controlling bias in observational studies: A review. Sankhya-A, 35, 417-446.
11. Cornfield, J., Haenszel, E., Hammond, C., Lilienfeld, A., Shimkin, M. D., & Wynder, E. L. (1959). Smoking and lung cancer; recent evidence and a discussion of some questions. Journal of the National Cancer Institute, 22, 173-203.
12. Cox, D. R. (1958). Planning of experiments. New York: John Wiley.
13. Cox, D. R. (1992). Causality: Some statistical aspects. Journal of the Royal Statistical Society, Series A, 155, 291-301.
14. D'Agostino, R., Jr., & Rubin, D. B. (1999). Estimation and use of propensity scores with incomplete data. Journal of the American Statistical Association, 95, 749-759.
15. Dawid, A. P. (2000). Causal inference without counterfactuals. Journal of the American Statistical Association, 95, 407-424 [With discussion].
16. Dehejia, R. H., & Wahba, S. (1999). Causal effects in non-experimental studies: Re-evaluating the evaluation of training programs. Journal of the American Statistical Association, 94, 1053-1062.
17. Dempster, A. P. (1990). Causality and statistics. Journal of Statistical Planning and Inference, 25, 261-278.
18. Dempster, A. P., Laird, N., & Rubin, D. B. (1977). Maximum likelihood from incomplete data via the EM algorithm. Journal of the Royal Statistical Society, Series B, 39, 1-38. [With discussion and reply].
19. Efron, B. (1971). Forcing a sequential experiment to be balanced. Biometrika, 58, 403-417.
20. Efron, B., & Feldman, D. (1991). Compliance as an explanatory variable in clinical trials. Journal of the American Statistical Association, 86, 9-17.
21. Ettner, S. L. (1996). The timing of preventive services for women and children: The effect of having a usual source of care. American Journal of Public Health, 86, 1748-1754.
22. Fisher, R. A. (1918). The causes of human variability. Eugenics Review, 10, 213-220.
23. Fisher, R. A. (1925). Statistical methods for research workers. London: Oliver & Boyd.
24. Frangakis, C., & Rubin, D. B. (1999). Addressing complications of intention-to-treat analysis in the combined presence of all-or-none treatment-noncompliance and subsequent missing outcomes. Biometrika, 86, 366-379.
25. Frangakis, C., & Rubin, D. B. (2002). Principal stratification in causal inference. Biometrics, 58, 21-29.
26. Frangakis, C., Rubin, D. B., & Zhou, X.-H. (2002). Clustered encouragement designs with individual noncompliance: Bayesian inference with randomization, and application to advance directive forms. [With discussion and rejoinder]. Biostatistics, 3, 147-177.
27. Gelman, A., & King, G. (1991). Estimating incumbency advantage without bias. American Journal of Political Science, 34, 1142-1164.
28. Gelman, A., Carlin, J. B., Stern, H. S. & Rubin, D. B. (2003). Bayesian data analysis (2nd ed.). New York: Chapman & Hall.
29. Goetghebeur, E., & Molenberghs, G. (1996). Causal inference in a placebo-controlled clinical trial with binary outcome and ordered compliance. Journal of the American Statistical Association, 91, 928-934.
30. Greenland, S., & Poole, C. (1988). Invariants and noninvariants in the concept of interdependent effects. Scandinavian Journal of Work and Environmental Health, 14, 125-129.
31. Greenland, S., Robins, J. M., & Pearl, J. (1999). Confounding and collapsibility in causal inference. Statistical Science, 14, 29-46.
32. Haavelmo, T. (1944). The probability approach in econometrics. Econometrica, 15, 413-419.
33. Heckman, J. J. (1979). Sample selection bias as a specification error. Econometrica, 47, 153-161.
34. Heckman, J. J. (1989). Causal inference and nonrandom samples. Journal of Educational Statistics, 14, 159-168.
35. Heitjan, D., & Rubin, D. B. (1991). Ignorability and coarse data. The Annals of Statistics, 19, 2244-2253.
36. Hirano, K., Imbens, G. W., Rubin, D. B., & Zhou, X.-H. (2000). Assessing the effect of an influenza vaccine in an encouragement design. Biostatistics, 1, 69-88.
37. Holland, P. (1986). Statistics and causal inference. Journal of the American Statistical Association, 81, 945-970.
38. Holland, P. W. (1988a). Causal inference, path analysis, and recursive structural equation models. Sociological Methodology, 449-484.
39. Holland, P. W. (1988b). Comment on "Employment discrimination and statistical science" by A. P. Dempster. Statistical Science, 3, 186-188.
40. Holland, P. W. (1989). It's very clear. Comment on "Choosing among alternative non-experimental methods for estimating the impact of social programs: The case of manpower training" by J. Heckman and V. Hotz. Journal of the American Statistical Association, 84, 875-877.
41. Holland, P. W., & Rubin, D. B. (1983). On Lord's paradox. In Wainer & Messick (Eds.), Principals of modern psychological measurement: A Festschrift for Frederic M. Lord (pp. 3-25). Hillsdale, NJ: Erlbaum.
42. Horowitz, J. L., & Manski, C. F. (2000). Nonparametric analysis of randomized experiments with missing covariates and outcome data. Journal of the American Statistical Association, 95, 77-84.
43. Horvitz, D. G., & Thompson, D. J. (1952). A generalization of sampling without replacement from a finite population. Journal of the American Statistical Association, 47, 663-685.
44. Imbens, G., & Rubin, D. B. (1997). Bayesian inference for causal effects in randomized experiments with noncompliance. The Annals of Statistics, 25, 305-327.
45. Imbens, G. B., Rubin, D. B., & Sacerdote, B. (1999). Estimating the effect of unearned income on labor supply, earnings, savings and consumption: Evidence from a survey of lottery players. Tilberg University, Center for Economic Research, Discussion Paper 9934. Also, National Bureau of Economic Research Working Paper 7001.
46. Imbens, G. B., Rubin, D. B., & Sacerdote, B. (1999). Estimating the effect of unearned income on labor supply, earnings, savings and consumption: Evidence from a survey of lottery players. Tilberg University, Center for Economic Research, Discussion Paper 9934. Also, National Bureau of Economic Research Working Paper 7001.
47. Kadane, J. B., & Seidenfeld, T. (1990). Randomization in a Bayesian perspective. Journal of Statistical Planning and Inference, 25, 329-346.
48. Kempthorne, O. (1976). Comments on "On rereading R. A. Fisher." Annals of Statistics, 4, 495-497.
49. LaLonde, R. (1986). Evaluating the econometric evaluations of training programs. American Economic Review, 76, 604-620.
50. Little, R. J. A., & Rubin, D. B. (2002). Statistical analysis with missing data (2nd ed.). New York: John Wiley.
51. Little, R. J., & Yau, L. (1998). Statistical techniques for analyzing data from prevention trials: Treatment of no-shows using Rubin's causal model. Psychological Methods, 3, 147-159.
52. Lord, F. M. (1967). A paradox in the interpretation of group comparisons. Psychological Bulletin, 68, 304-305.
53. Liu, C. H., & Rubin, D. B. (1994). The ECME algorithm: A simple extension of EM and ECM with faster monotone convergence. Biometrika, 81, 633-648.
54. Liu, C. H., Rubin, D. B., & Wu, Y. N. (1998). Parameter expansion for EM acceleration: The PX-EM algorithm. Biometrika, 85, 755-770.
55. Manski, C. F., Sandefur, G. D., McLanahan, S., & Powers, D. (1992). Alternative estimates of the effect of family structure during adolescence on high school graduation. Journal of the American Statistical Association, 87, 25-37.
56. Manski, C. F., & Nagin, D. S. (1998). Bounding disagreements about treatment effects: A study of sentencing and recidivism. Sociological Methodology, 28, 99-137.
57. Meng, X. L., & Rubin, D. B. (1993). Maximum likelihood estimation via the ECM algorithm. A general framework. Biometrika, 80, 267-278.
58. Meng, X. L., & van Dyk, D. A. (1997). The EM algorithm-An old folk song sung to a fast new tune [with discussion]. Journal of the Royal Statistical Society, Series B, 59, 511-567.
59. Neyman, J. (1923). On the application of probability theory to agricultural experiments: Essay on principles, section 9. Translated in Statistical Science, 5, 465-480, 1990.
60. Neyman, J. (1934). On the two different aspects of the representative method: The method of stratified sampling and the method of purposive selection. Journal of the Royal Statistical Society, Series A, 97, 558-606.
61. Pratt, J. W., & Schlaifer, R. (1984). On the nature and discovery of structure. Journal of the American Statistical Association, 79, 9-33.
62. Pratt, J. W., & Schlaifer, R. (1988). On the interpretation and observation of laws. Journal of Econometrics, 39, 23-52.
63. Reinisch, J., Sanders, S., & Mortensen, E. (1995). In utero exposure to phenobarbital and intelligence deficits in adult men. The Journal of the American Medical Association, 274, 1518-1525.
64. Robins, J. M. (1987). A graphical approach to the identification and estimation of causal parameters in mortality studies with sustained exposure periods. Journal of Chronic Diseases 40(Suppl. 2), 139S-161S.
65. Robins, J. M. (1989). The control of confounding by intermediate variables. Statistics in Medicine, 8, 679-701.
66. Robins, J. M. (1997). Causal inference from complex longitudinal data. In M. Berkane (Ed.), Latent variable modeling and applications to causality: Lecture notes in statistics, Vol. 120 (pp. 69-117). New York: Springer.
67. Rosenbaum, P. R. (1984). From association to causation in observational studies: The role of tests of strongly ignorable treatment assignment. Journal of the American Statistical Association, 79, 41-48.
68. Rosenbaum, P. R. (1987). The role of a second control group in an observational study [with Discussion]. Statistical Science, 2, 292-316.
69. Rosenbaum, P. R. (2002). Observational Studies (2nd ed.). New York: Springer.
70. Rosenbaum, P. R., & Rubin, D. B. (1983a). The central role of the propensity score in observational studies for causal effects. Biometrika, 70, 41-55.
71. Rosenbaum, P. R., & Rubin, D. B. (1983b). Assessing sensitivity to an unobserved binary covariate in an observational study with binary outcome. The Journal of the Royal Statistical Society, Series B, 45, 212-218.
72. Rosenbaum, P. R., & Rubin, D. B. (1984a). Estimating the effects caused by treatments. Discussion of "On the nature and discovery of structure" by Pratt and Schlaifer. Journal of the American Statistical Association, 79, 26-28.
73. Rosenbaum, P. R., & Rubin, D. B. (1984b). Reducing bias in observational studies using subclassification on the propensity score. Journal of the American Statistical Association, 79, 516-524.
74. Rosenbaum, P. R., & Rubin, D. B. (1985). Constructing a control group using multivariate matched sampling incorporating the propensity score. The American Statistician, 39, 33-38.
75. Rubin, D. B. (1973a). Matching to remove bias in observational studies. Biometrics, 29, 159-183. [Printer's correction note 30, 728.]
76. Rubin, D. B. (1973b). The use of matched sampling and regression adjustment to remove bias in observational studies. Biometrics, 29, 184-203.
77. Rubin, D. B. (1974). Estimating causal effects of treatments in randomized and nonrandomized studies. Journal of Educational Psychology, 66, 688-701.
78. Rubin, D. B. (1976a). Inference and missing data. Biometrika, 63, 581-592. [With discussion and reply.]
79. Rubin, D. B. (1976b). Multivariate matching methods that are equal percent bias reducing, I: Some examples. Biometrics, 32, 109-120. [Printer's correction note p. 955.]
80. Rubin, D. B. (1976c). Multivariate matching methods that are equal percent bias reducing, II: Maximums on bias reduction for fixed sample sizes. Biometrics, 32, 121-132. [Printer's correction note p. 955.]
81. Rubin, D. B. (1977). Assignment to treatment group on the basis of a covariate. Journal of Educational Statistics, 2, 1-26. [Printer's correction note 3, p. 384.]
82. Rubin, D. B. (1978). Bayesian inference for causal effects: The role of randomization. The Annals of Statistics, 7, 34-58.
83. Rubin, D. B. (1979). Using multivariate matched sampling and regression adjustment to control bias in observational studies. The Journal of the American Statistical Association, 74, 318-328.
84. Rubin, D. B. (1980). Discussion of "Randomization analysis of experimental data in the Fisher randomization test" by Basu. The Journal of the American Statistical Association, 75, 591-593.
85. Rubin, D. B. (1984). William G. Cochran's contributions to the design, analysis, and evaluation of observational studies. In Rao & Sedransk (Eds.), W. G. Cochran's impact on statistics (pp. 37-69). New York: John Wiley.
86. Rubin, D.B. (1985). The use of propensity scores in applied Bayesian inference. In Bernardo, DeGroot, Lindley, & Smith (Eds.), Bayesian statistics, Vol. 2 (pp. 463-472). North Holland.
87. Rubin, D. B. (1987). Multiple imputation for nonresponse in surveys. New York: John Wiley.
88. Rubin, D. B. (1990a). Neyman (1923) and causal inference in experiments and observational studies. Statistical Science, 5, 472-480.
89. Rubin, D. B. (1990b). Formal modes of statistical inference for causal effects. Journal of Statistical Planning and Inference, 25, 279-292.
90. Rubin, D. B. (1997). Estimating causal effects from large data sets using propensity scores. Annals of Internal Medicine, 127, 757-763.
91. Rubin, D. B. (2002). Using propensity scores to help design observational studies: Application to the tobacco litigation. Health Services & Outcomes Research Methodology, 2, 169-188.
92. Rubin, D. B., & Thomas, N. (1992a). Affinely invariant matching methods with ellipsoidal distributions. The Annals of Statistics, 20, 1079-1093.
93. Rubin, D. B., & Thomas, N. (1992b). Characterizing the effect of matching using linear propensity score methods with normal covariates. Biometrika, 79, 797-809.
94. Rubin, D. B., & Thomas, N. (1996). Matching using estimated propensity scores: Relating theory to practice. Biometrics, 52, 249-264.
95. Rubin, D. B., & Thomas, N. (2000). Combining propensity score matching with additional adjustments for prognostic covariates. Journal of the American Statistical Association, 95, 573-585.
96. Smith, H. L. (1997). Matching with multiple controls to estimate treatment effects in observational studies. Sociological Methodology, 27, 325-353.
97. Smith, T. M. F., & Sugden, R. A. (1988). Sampling and assignment mechanisms in experiments, surveys and observational studies. International Statistical Review, 56, 165-180.
98. Sobel, M. E. (1990). Effect analysis and causation in linear structural equation models. Psychometrika, 55, 495-515.
99. Sobel, M. E. (1995). Causal inference in the social and behavioral sciences. In Arminger, Clogg, & Sobel (Eds.), Handbook of statistical modeling far the social and behavioral sciences (pp. 1-38). New York: Plenum.
100. Sobel, M. E. (1996). An introduction to causal inference. Sociological Methods & Research, 24, 353-379.
101. Sommer, A., & Zeger, S. (1991). On estimating efficacy from clinical trials. Statistics in Medicine, 10, 45-52.
102. Sugden, R. A. (1988). The 2 × 2 table in observational studies. In Bernardo, DeGroot, Lindley, & Smith (Eds.), Bayesian statistics 3 (pp. 785-790). New York: Oxford University Press.
103. Tanner, M. A., & Wong, W. H. (1987). The calculation of posterior distributions by data augmentation. Journal of the American Statistical Association, 82, 543-546.
104. Tinbergen, J. (1930). Determination and interpretation of supply curves: An example. Zeitschrift für Nationalokonomie. Reprinted in: The foundations of economics, Henry & Morgan (Eds.).
105. Tinbergen, J. (1930). Determination and interpretation of supply curves: An example. Zeitschrift für Nationalokonomie. Reprinted in: The foundations of economics, Henry & Morgan (Eds.).
106. Ware, J. H. (1989). Investigating therapies of potentially great benefit: ECMO. Statistical Science, 4, 298-340. [With discussion and rejoinder.]
107. Wilkinson, L., & Task Force on Statistical Inference. (1999). Statistical methods in psychology journals: Guidelines and expectations. American Psychologist, 54, 594-604.
108. Zelen, M. (1979). A new design for randomized clinical trials. New England Journal of Medicine, 300, 1242-1245.