Can propensity-score methods match the findings from a random assignment evaluation of mandatory welfare-to-work programs?

Review of Economics and Statistics

Volumn 86, Issue 1, 2004, Pages 156-179

  Michalopoulos, Charles ^a   Bloom, Howard S ^a   Hill, Carolyn J ^b  

^a MDRC ^*

^b GEORGETOWN UNIVERSITY   (United States)

Author keywords

[No Author keywords available]

Indexed keywords

EID: 1842533914     PISSN: 00346535     EISSN: None     Source Type: Journal    
DOI: 10.1162/003465304323023732     Document Type: Conference Paper

References (35)

1. Ashenfelter, Orley C., "Estimating the Effect of Training Programs on Earnings," this REVIEW, 60:1 (1978), 47-57.
2. Ashenfelter, Orley, and David Card, "Using the Longitudinal Structure of Earnings to Estimate the Effect of Training Programs," this REVIEW, 67:4 (1985), 648-660.
3. Bell, Stephen H., Larry L. Orr, John D. Blomquist, and Glen G. Cain, Program Applicants as a Comparison Group in Evaluating Training Programs (Kalamazoo, MI: W. E. Upjohn Institute for Employment Research, 1995).
4. Bloom, Dan, James J. Kemple, Pamela Morris, Susan Scrivener, Nandita Verma, and Richard Hendra, The Family Transition Program: Final Report on Florida's Initial Time-Limited Welfare Program (New York: MDRC, 2000).
5. Bloom, Dan, Susan Scrivener, Charles Michalopoulos, Pamela Morris, Richard Hendra, Diana Adams-Ciardullo, and Johanna Walter, Final Report on Connecticut's Welfare Reform Initiative (New York: MDRC, 2002a).
6. Bloom, Howard S., and Maureen A. McLaughlin, "CETA Training Programs - Do They Work for Adults?" U.S. Congressional Budget Office and National Commission for Employment Policy (1982).
7. Bloom, Howard S., Charles Michalopoulos, Carolyn J. Hill, and Ying Lei, "Can Nonexperimental Comparison Group Methods Match the Findings from a Random Assignment Evaluation of Mandatory Welfare-to-Work Programs?" (New York: MDRC, 2002b).
8. Campbell, Donald T., and Julian C. Stanley, Experimental and Quasi-experimental Design for Research (Chicago: Rand McNally, 1963).
9. Cook, Thomas, and Donald Campbell, Quasi-experimentation: Design and Analysis Issues for Field Settings (Chicago: Rand-McNally, 1979).
10. Dehejia, Rajeev H., and Sadek Wahba, "Causal Effects in Nonexperimental Studies: Reevaluating the Evaluation of Training Programs, " Journal of the American Statistical Association 94:448 (1999), 1053-1062.
11. Economic Report of the President (Washington: U.S. Government Printing Office, 2000).
12. Fraker, Thomas, and Rebecca Maynard, "The Adequacy of Comparison Group Designs for Evaluations of Employment-Related Programs," Journal of Human Resources 22:2 (1987), 194-227.
13. Freedman, Stephen, Daniel Friedlander, Gayle Hamilton, JoAnn Rock, Marisa Mitchell, Jodi Nudelman, Amanda Schweder, and Laura Storto, National Evaluation of Welfare-to-Work Strategies: Evaluating Alternative Welfare-to-Work Approaches: Two Year Impacts for Eleven Programs (Washington DC: U.S. Department of Health and Human Services, Administration for Children and Families, Office of the Assistant Secretary for Planning and Evaluation and U.S. Department of Education, Office of the Under Secretary, Office of Vocational and Adult Education, June 2000).
14. Friedlander, Daniel, and Philip K. Robins, "Evaluating Program Evaluations: New Evidence on Commonly Used Nonexperimental Methods," American Economic Review 85:4 (1995), 923-937.
15. Hamilton, Gayle, and Thomas Brock, The JOBS Evaluation: Early Lessons from Seven Sites (Washington DC: U.S. Department of Health and Human Services, Administration for Children and Families, Office of the Assistant Secretary for Planning and Evaluation, and U.S. Department of Education, Office of the Under Secretary, Office of Vocational and Adult Education, 1994).
16. Hamilton, Gayle, Stephen Freedman, Lisa A. Gennetian, Charles Michalopoulos, Johanna Walter, Diana Adams-Ciardullo, Anna Gassman-Pines, Sharon McGroder, Martha Zaslow, J. Brooks, and S. Ahluwalia, How Effective Are Different Welfare-to-Work Approaches? Five-Year Adult and Child Impacts for Eleven Programs (Washington: U.S. Department of Health and Human Services, Office of the Assistant Secretary for Planning and Evaluation and Administration for Children and Families, and U.S. Department of Education, 2001).
17. Heckman, James J., and V. Joseph Hotz, "Choosing among Alternative Nonexperimental Methods for Estimating the Impact of Social Programs: The Case of Manpower Training," Journal of the American Statistical Association 84:408 (1989), 862-874.
18. Heckman, James J., Hidehiko Ichimura, Jeffrey Smith, and Petra Todd, "Characterizing Selection Bias Using Experimental Data," Econometrica 66:5 (1998), 1017-1098.
19. Heckman, James J., Hidehiko Ichimura, and Petra Todd, "Matching as an Econometric Evaluation Estimator: Evidence from Evaluating a Job Training Program," Review of Economic Studies 64:4 (1997), 605-654.
20. Heckman, James J., Hidehiko Ichimura, and Petra Todd, "Matching as an Econometric Evaluation Estimator," Review of Economic Studies 65:2 (1998), 261-294.
21. Heckman, James J., and Jeffrey Smith, "The Pre-programme Earnings Dip and the Determinants of Participation in a Social Programme: Implications for Simple Programme Evaluation Strategies," The Economic Journal 109 (July 1999), 313-348.
22. Hirano, Keisuke, Guido W. Imbens, and Geert Ridder, "Efficient Estimation of Average Treatment Effects Using the Estimated Propensity Score," NBER Working Paper no. T0251 (2000).
23. Hollister, Robinson G., and Jennifer Hill, "Problems in the Evaluation of Community-Wide Initiatives," in James P. Connell, Anne C. Kubisch, Lisbeth B. Schorr, and Carol H. Weiss (Eds.), New Approaches to Evaluating Community Initiatives: Concepts, Methods, and Contexts (Washington, DC: Aspen Institute, 1995).
24. Hsiao, Cheng, Analysis of Panel Data (Cambridge, UK: Cambridge University Press, 1990).
25. Imbens, Guido, "Semiparametric Estimation of Average Treatment Effects under Exogeneity: A Review," UC Berkeley and NBER, unpublished manuscript (2003).
26. LaLonde, Robert J., "Evaluating the Econometric Evaluations of Training Programs with Experimental Data," American Economic Review 76:4 (1986), 604-620.
27. Lee, Wang S., "Propensity Score Matching on Commonly Available Nonexperimental Comparison Groups," Abt Associates, unpublished manuscript (Bethesda, MD, 2001).
28. Manski, Charles F., and Irwin Garfinkel (Eds.), Evaluating Welfare and Training Programs (Cambridge, MA: Harvard University Press, 1992).
29. Miller, Cynthia, Virginia Knox, Lisa A. Gennetian, Martey Dodoo, JoAnna Hunter, and Cindy Redcross, Reforming Welfare and Rewarding Work: Final Report on the Minnesota Family Investment Program (New York: MDRC, 2000).
30. Riccio, James, Daniel Friedlander, and Stephen Freedman, GAIN: Benefits, Costs, and Three-Year Impacts of a Welfare-To-Work Program (New York: MDRC, 1994).
31. Rosenbaum, Paul R., and Donald B. Rubin, "The Central Role of the Propensity Score in Observational Studies for Causal Effects," Biometrika 70:1 (1983), 41-55.
32. Scrivener, Susan, Richard Hendra, Cindy Redcross, Dan Bloom, Charles Michalopoulos, and Johanna Walter, Final Report on Vermont's Welfare Restructuring Project (New York: MDRC, 2002).
33. Shadish, William R., Thomas D. Cook, and Donald T. Campbell, Experimental and Quasi-experimental Designs for Generalized Causal Inference (Boston: Houghton Mifflin, 2002).
34. Smith, Jeffrey, and Petra Todd, "Does Matching Overcome LaLonde's Critique of Nonexperimental Estimators?" Journal of Econometrics (forthcoming).
35. Zhao, Zhong, "Using Matching to Estimate Treatment Effects: Data Requirements, Matching Metrics, and Monte Carlo Evidence," this REVIEW, this issue (2004).