Can Nonexperimental Estimates Replicate Estimates Based on Random Assignment in Evaluations of School Choice? A Within-Study Comparison

Journal of Policy Analysis and Management

Volumn 31, Issue 3, 2012, Pages 729-751

  Couch, Kenneth A ^a   Bifulco, Robert ^b  

^a UNIVERSITY OF CONNECTICUT   (United States)

^b SYRACUSE UNIVERSITY   (United States)

Author keywords

[No Author keywords available]

Indexed keywords

EID: 84862579941     PISSN: 02768739     EISSN: 15206688     Source Type: Journal    
DOI: 10.1002/pam.20637     Document Type: Article

References (19)

1. Agodini, R., & Dynarski, M. (2004). Are experiments the only option? A look at dropout prevention programs. Review of Economics and Statistics, 86, 180-194.
2. Aiken, L. S., West, S. G., Schwalm, D. E., Carroll, J., & Hsuing, S. (1998). Comparison of a randomized and two quasi-experimental designs in a single outcome evaluation: Efficacy of a university-level remedial writing program. Evaluation Review, 22, 207-244.
3. Becker, S. O., & Ichino, A. (2002). Estimation of average achievement effects using propensity scores. The STATA Journal, 2, 358-377.
4. Betts, J., & Hill, P. T. (2006). Key issues in studying charter schools and achievement: A review and suggestions for national guidelines. Seattle, WA: National Charter School Research Project.
5. Betts, J., Rice, L., Zau, A., Tang, E., & Koedel, C. (2006). Does school choice work? Effects on student integration and academic achievement. San Francisco, CA: Public Policy Institute of California.
6. Bloom, H. S., Hill, C. J., Black, A. R., & Lipsey, M. W. (2008). Performance trajectories and performance gaps as achievement effect-size benchmarks for educational interventions. Journal of Research on Educational Effectiveness, 1, 289-328.
7. Cook, T., Shadish, W. R., & Wong, V. C. (2008). Three conditions under which experiments and observational studies produce comparable causal estimates: New findings from within-study comparisons. Journal of Policy Analysis and Management, 27, 724-750.
8. Cullen, J. B., Jacob, B. A., & Levitt, S. (2006). The effect of school choice on student outcomes: Evidence from randomized lotteries. Econometrica, 74, 1191-1230.
9. Glazerman, S., Levy, D. M., & Meyers, D. (2003). Nonexperimental versus experimental estimates of earnings impacts. Annals of the American Academy, 589, 63-93.
10. Heckman, J. J., Ichimura, H., & Todd, P. E. (1997). Matching as an econometric evaluation estimator: Evidence from evaluating a job training programme. The Review of Economic Studies, 64, 605-654.
11. Heckman, J. J., Ichimura, H., Smith, J. & Todd, P. E. (1998). "Characterizing selection bias using experimental data, "Econometrica, 66, 1017-1098.
12. Heckman, J. J., LaLonde, R. J., & Smith, J. A. (1999). The economics and econometrics of active labor market programs. In O. Ashenfelter & D. Card (Eds.), Handbook of labor economics, Vol. III (pp. 1865-2097). New York: Elsevier.
13. Hoxby, C. M., & Murarka, S. (2008). Methods of assessing the achievement of students in charter schools. In M. Berends, M. G. Springer, & H. J. Walberg (Eds.), Charter school outcomes (pp. 7-38). New York: Lawrence Erlbaum & Associates.
14. Hoxby, C. M., & Rockoff, J. (2004). The impact of charter schools on student achievement. Unpublished manuscript. Cambridge, MA: Harvard University.
15. Howell, W. G, Wolf, P. J., Campbell, D. E., & Peterson, P. E. (2002). School vouchers and academic performance: Results from three randomized field trials. Journal of Policy Analysis and Management, 21, 191-218.
16. Pirog, M. A., Buffardi, A. L., Chrisinger, C. K., Singh, P., & Briney, J. (2009). Are alternatives to randomized assignment nearly as good? Statistical corrections to nonrandomized evaluations. Journal of Policy Analysis and Management, 28, 169-172.
17. Rouse, C. E. (1998). Private school vouchers and student achievement: An evaluation of the Milwaukee parental choice program. The Quarterly Journal of Economics, 113, 553-602.
18. Shadish, W. R., Clark, M. H., & Steiner, P. M. (2008). Can nonrandomized experiments yield accurate answers? A randomized experiment comparing random to nonrandom assignment. Journal of American Statistical Association, 103, 1334-1344.
19. Wilde, E. T., & Hollister, R. (2007). How close is close enough? Evaluating propensity score matching using data from a class size reduction experiment. Journal of Policy Analysis and Management, 26, 455-477.