Chapter 61 Using Randomization in Development Economics Research: A Toolkit

Handbook of Development Economics

Volumn 4, Issue , 2007, Pages 3895-3962

  Duflo, Esther ^a   Glennerster, Rachel ^a   Kremer, Michael ^b  

^a MASSACHUSETTS INSTITUTE OF TECHNOLOGY   (United States)

^b HARVARD UNIVERSITY   (United States)

Author keywords

development; experiments; program evaluation; randomized evaluations

Indexed keywords

EID: 60349126881     PISSN: 15734471     EISSN: None     Source Type: Book Series    
DOI: 10.1016/S1573-4471(07)04061-2     Document Type: Review

References (116)

1. Angrist J.D. Lifetime earnings and the Vietnam era draft lottery: Evidence from social security administrative records. American Economic Review 80 3 (1990) 313-336
2. Angrist J.D. Estimating the labor market impact of voluntary military service using social security data on military applicants. Econometrica 66 2 (1998) 249-288
3. Angrist J.D., Bettinger E., and Kremer M. Long-term educational consequences of secondary school vouchers: Evidence from administrative records in Colombia. American Economic Review 96 3 (2006) 847-862
4. Angrist J.D., and Imbens G. Identification and estimation of local average treatment effects. Econometrica 62 2 (1994) 467-475
5. Angrist J.D., and Imbens G. Two-stage least squares estimation of average causal effects in models with variable treatment intensity. Journal of the American Statistical Association 90 430 (1995) 431-442
6. Angrist J.D., Imbens G.W., and Rubin D.B. Identification of causal effects using instrumental variables. Journal of the American Statistical Association 91 (1996) 444-455
7. Angrist J.D., Bettinger E., Bloom E., King E., and Kremer M. Vouchers for private schooling in Colombia: Evidence from a randomized natural experiment. American Economic Review 92 5 (2002) 1535-1558
8. Ashenfelter O., Harmon C.P., and Oosterbeek H. A review of estimates of the schooling/earnings relationship. Labour Economics 6 4 (1999) 453-470
9. Ashraf N., Karlan D.S., and Yin W. Tying Odysseus to the mast: Evidence from a commitment savings product in the Philippines. Quarterly Journal of Economics 121 2 (2006) 635-672
10. Attanasio, O., Meghir, C., Santiago, A. (2005). "Education choices in Mexico: Using a structural model and a randomised experiment to evaluate Progresa". Working paper EWP05/01. IFS
11. Banerjee A. New development economics and the challenge to theory. Economic and Political Weekly 40 (2006) 4340-4344
12. Banerjee A., and Duflo E. Addressing absence. Journal of Economic Perspectives 20 1 (2006) 117-132
13. Banerjee, A., Bardhan, P., Basu, K., Kanbur, R., Mookherjee, D. (2005). "New directions in development economics: Theory or empirics?". Working paper No. 106. BREAD, A Symposium in Economic and Political Weekly
14. Banerjee A., Duflo E., Cole S., and Linden L. Remedying education: Evidence from two randomized experiments in India. Quarterly Journal of Economics 122 3 (2007) 1235-1264
15. Bardhan, P. (2005). "Theory or empirics in development economics". Mimeo. University of California at Berkeley
16. Basu, K. (2005). "The new empirical development economics: Remarks on its philosophical foundations". Economic and Political Weekly
17. Bertrand M., Karlan D.S., Mullainathan S., Shafir E., and Zinman J. What's psychology worth? A field experiment in the consumer credit market. Working paper 918. Economic Growth Center, Yale University. Available at. http://ideas.repec.org/p/egc/wpaper/918.html
18. Bertrand, M., Djankov, S., Hanna, R., Mullainathan, S. (2006). "Does corruption produce unsafe drivers?" Working paper #12274. NBER
19. Bhushan, I., Keller, S., Schwartz, B. (2002). "Achieving the twin objectives of efficiency and equity: Contracting health services in Cambodia". ERD Policy Brief Series. Asian Development Bank, vol. 6
20. Bloom, E., Bhushan, I., Clingingsmith, D., Hung, R., King, E., Kremer, M., Loevinsohn, B., Schwartz, B. (2006). "Contracting for health: Evidence from Cambodia". Mimeo
21. Bloom H.S. Minimum detectable effects: A simple way to report the statistical power of experimental designs. Evaluation Review 19 (1995) 547-556
22. Bloom H.S. Learning more from social experiments. Randomizing Groups to Evaluate Place-Based Programs (2005), Russell Sage Foundation, NY 115-172
23. Bobonis, G.J., Miguel, E., Sharma, C.P. (2004). "Iron deficiency anemia and school participation". Paper No. 7. Poverty Action Lab
24. Buddlemeyer, H., Skofias, E. (2003). "An evaluation on the performance of regression discontinuity design on progresa". Discussion paper No. 827. Institute for Study of Labor
25. Campbell D.T. Reforms as experiments. American Psychologist 24 (1969) 407-429
26. Card D. The causal effect of education on earnings. In: Ashenfelter O., and Card D. (Eds). Handbook of Labor Economics vol. 3 (1999), North-Holland 1801-1863
27. Card D., and Krueger A.B. Myth and Measurement: The New Economics of the Minimum Wage (1995), Princeton Univ. Press, Princeton, NJ
28. Chattopadhyay R., and Duflo E. Women as policy makers: Evidence from a randomized policy experiment in India. Econometrica 72 5 (2004) 1409-1443
29. Cohen J. Statistical Power Analysis for the Behavioral Science. second ed (1988), Lawrence Erlbaum, Hillsdale, NJ
30. Cook, T.D., Shadish, W.R., Wong, V.C. (2006). "Within study comparisons of experiments and non-experiments: Can they help decide on evaluation policy?" Mimeo. Northwestern University
31. Cox D., and Reid N. Theory of the Design of Experiments (2000), Chapman and Hall, London
32. Das, J., Krishnan, P., Habyarimana, J., Dercon, S. (2004). "When can school inputs improve test scores?". Working paper No. WPS 3217. World Bank Policy Research
33. Deaton A. The Analysis of Household Surveys (1997), International Bank for Reconstruction and Development, World Bank
34. Dehejia R.H., and Wahba S. Causal effects in nonexperimental studies: Reevaluating the evaluation of training programs. Journal of the American Statistical Association 94 448 (1999) 1053-1062
35. DeLong J.B., and Lang K. Are all economic hypotheses false?. Journal of Political Economy 100 6 (1992) 1257-1272
36. Diaz J.J., and Handa S. An assessment of propensity score matching as a non-experimental impact estimator: Evidence from Mexico's Progresa program. Journal of Human Resources 41 2 (2006) 319-345
37. Dickson R., Awasthi S., Williamson P., Demelweek C., and Garner P. Effect of treatment for intestinal helminth infection on growth and cognitive performance in children: systematic review of randomized trials. British Medical Journal 320 (2000) 1697-1701
38. Donald, S., Lang, K. (2001). "Inference with differences-in-differences and other panel data". Discussion paper. Boston University Department of Economics
39. Duflo E. Accelerating development. Scaling Up and Evaluation (2004), Oxford Univ. Press and World Bank
40. Duflo, E. (2006). "Field experiments in development economics". Discussion paper
41. Duflo, E., Hanna, R. (2006). "Monitoring works: Getting teachers to come to school". Working paper No. 11880. NBER
42. Duflo E., and Kremer M. Use of randomization in the evaluation of development effectiveness. In: Feinstein O., Ingram G.K., and Pitman G.K. (Eds). Evaluating Development Effectiveness. Transaction Publishers vol. 7 (2005), New Brunswick, NJ and London, UK 205-232
43. Duflo, E., Kremer, M., Robinson, J. (2006). "Understanding technology adoption: Fertilizer in western Kenya, preliminary results from field experiments". Mimeo
44. Duflo E., Mullainathan S., and Bertrand M. How much should we trust difference-in-differences estimates?. Quarterly Journal of Economics 119 1 (2004) 249-275
45. Duflo E., and Saez E. The role of information and social interactions in retirement plan decisions: Evidence from a randomized experiment. Quarterly Journal of Economics 118 3 (2003) 815-842
46. Duflo, E., Dupas, P., Kremer, M., Sinei, S. (2006). "Education and HIV/AIDS prevention: Evidence from a randomized evaluation in western Kenya". Mimeo. MIT
47. Dupas P. (2006). "Relative risks and the market for sex: Teenagers, sugar daddies, and HIV in Kenya". Mimeo, Dartmouth College
48. Fisher R.A. The arrangement of field experiments. Journal of the Ministry of Agriculture 33 (1926) 503-513
49. Gertler, P.J., Boyce, S. (2001). "An experiment in incentive-based welfare: The impact of Progresa on health in Mexico". Mimeo, UC-Berkeley
50. Glazerman S., Levy D., and Myers D. Nonexperimental Replications of Social Experiments: A Systematic Review (2003), Mathematica Policy Research, Inc., Princeton, NJ
51. Glewwe, P., Kremer, M. (2003). "Teacher incentives". Working paper 9671. National Bureau of Economic Research
52. Glewwe P., and Kremer M. Schools, teachers, and education outcomes in developing countries. Chapter 16 in:. Handbook on the Economics of Education vol. 2 (2006), North-Holland 945-1018
53. Glewwe, P., Kremer, O., Moulin, S. (2004). "Textbooks and test scores: Evidence from a prospective evaluation in Kenya". Mimeo, Harvard University
54. Glewwe P., Kremer M., Moulin S., and Zitzewitz E. Retrospective vs. prospective analyses of school inputs: The case of flip charts in Kenya. Journal of Development Economics 74 (2004) 251-268
55. Grasdal A. The performance of sample selection estimators to control for attrition bias. Health Economics 10 (2001) 385-398
56. Harrison G., and List J.A. Field experiments. Journal of Economic Literature XLII (2004) 1013-1059
57. Hausman J.A., and Wise D.A. Attrition bias in experimental and panel data: The Gary income maintenance experiment. Econometrica 47 2 (1979) 455-473
58. Heckman J. Sample selection bias as a specification error. Econometrica 47 1 (1979) 153-161
59. Heckman J., Ichimura H., and Todd P. Matching as an econometric evaluation estimator: Evidence from evaluating a job training programme. Review of Economic Studies 64 4 (1997) 605-654
60. Heckman J., Ichimura H., and Todd P. Matching as an econometric evaluation estimator. Review of Economic Studies 65 2 (1998) 261-294
61. Heckman, J., Lochner, L., Taber, C. (1998). "General equilibrium treatment effects: A study of tuition policy". Working paper 6426. National Bureau of Economic Research
62. Heckman J., and Vytlacil O. Structural equations, treatment effects and econometric policy evaluation. Econometrica 73 3 (2005) 669-738
63. Heckman J., Ichimura H., Smith J., and Todd P. Characterizing selection bias using experimental data. Econometrica 66 5 (1998) 1017-1098
64. Hedges L. Modeling publication selection effects in meta-analysis. Statistical Science 7 (1992) 227-236
65. Hedges L.V., and Olkin I. Statistical Methods for Meta-Analysis (1985), Academic Press, San Diego, CA
66. Holland P.W. Causal inference, path analysis, and recursive structural equations models. Sociological Methodology 18 (1988) 449-484
67. Holm O. A simple sequentially rejective multiple test procedure. Scandinavian Journal of Statistics 6 (1979) 65-70
68. Hoxby C.M. The economics of school choice. School Choice and School Productivity (or, Could School Choice be a Rising Tide that Lifts All Boats?) (2003), Univ. of Chicago Press, Chicago
69. Hsieh, C.-T., Urquiola, M.S. (2003). "When schools compete, how do they compete? An assessment of Chile's nationwide school voucher program". Working paper No. W10008, NBER
70. Imbens G.W. Nonparametric estimation of average treatment effects under exogeneity: A review. The Review of Economics and Statistics 86 1 (2004) 4-29
71. Imbens, G., King, G., Ridder, G. (2006). "On the benefits of stratification in randomized experiments". Mimeo, Harvard
72. International Food Policy Research (2000). "Monitoring and evaluation system". Discussion paper. International Food Policy Research (IFPRI)
73. Karlan, D.S., Zinman, J. (2005a). "Elasticities of demand for consumer credit". Mimeo, Yale University
74. Karlan D.S., and Zinman J. Observing unobservables: Identifying information asymmetries with a consumer credit field experiment. Working paper 911. Economic Growth Center, Yale University. Available at. http://ideas.repec.org/p/egc/wpaper/911.html
75. Karlan D.S., and Zinman J. Observing unobservables: Identifying information asymmetries with a consumer credit field experiment. Discussion paper. Available at. http://ideas.repec.org/p/egc/wpaper/911.html
76. Karlan, D.S., Zinman, J. (2006a). "Expanding credit access: Using randomized supply decisions to estimate the impacts". Mimeo. Yale University
77. Karlan, D.S., Zinman, J. (2006b). "Observing unobservables: Identifying information asymmetries with a consumer credit field experiment". Mimeo
78. Keller, S., Schwartz, B. (2001). "Final evaluation report: contracting for health services pilot project". Report No. 1447-CAM. Asian Development Bank on Loan, Unpublished
79. Kling, J.R., Liebman, J.B. (2004). "Experimental analysis of neighborhood effects on youth". Working paper No. RWP04-034, KSG
80. Kling, J.R., Liebman, J.B., Katz, L.F., Sanbonmatsu, L. (2004). "Moving to opportunity and tranquility: Neighborhood effects on adult economic self-sufficiency and health from a randomized housing voucher experiment". Working paper No. 481. Princeton University, Department of Economics, Industrial Relations Section
81. Kremer M. Randomized evaluations of educational programs in developing countries: Some lessons. American Economic Review 93 2 (2003) 102-106
82. Kremer, M., Miguel, E., Thornton, R. (2004). "Incentives to learn". Mimeo, Harvard University
83. Krueger A., and Whitmore D. Would smaller classes help close the black-white achievement gap?. In: Chubb J., and Loveless T. (Eds). Bridging the Achievement Gap (2002), Brookings Institute Press, Washington, DC
84. La Londe R.J. Evaluating the econometric evaluations of training programs using experimental data. American Economic Review 76 4 (1986) 602-620
85. Lee, D.S. (2002). "Trimming for bounds on treatment effects with missing outcomes". Working paper 51
86. Liebman, J.B., Katz, L.F., Kling, J. (2004). "Beyond treatment effects: Estimating the relationship between neighborhood poverty and individual outcomes in the MTO experiment". Working paper 493. Princeton, IRS
87. Logan, and Tamhane. Accurate critical constants for the one-sided approximate likelihood ratio test of a normal mean vector when the covariance matrix is estimated. Biometrics 58 (2003) 650-656
88. Maluccio, J.A., Flores, R. (2005). "Impact evaluation of a conditional cash transfer program". Discussion paper. International Food Policy Research Institute, Research Report No. 141
89. Manski C.F. Schooling as experimentation: A reappraisal of the postsecondary dropout phenomenon. Economics of Education Review 8 4 (1989) 305-312
90. Manski C.F. Identification of exogenous social effects: The reflection problem. Review of Economic Studies 60 (1993) 531-542
91. Miguel, E., Kremer, M. (2003). "Networks, social learning, and technology adoption: The case of deworming drugs in Kenya". Working paper 61
92. Miguel E., and Kremer M. Worms: Identifying impacts on education and health in the presence of treatment externalities. Econometrica 72 1 (2004) 159-218
93. Mookherjee D. Is there too little theory in development economics?. Economic and Political Weekly 40 40 (2005)
94. Morduch, J. (1998). "Does microfinance really help the poor? New evidence from flagship programs in Bangladesh". Mimeo, Princeton University
95. Moulton B.R. An illustration of a pitfall in estimating the effects of aggregate variables on micro units. Review of Economics and Statistics 72 2 (1990) 334-338
96. O'Brien P.C. Procedures for comparing samples with multiple endpoints. Biometrics 40 (1984) 1079-1087
97. Olken, B. (2005). "Monitoring corruption: Evidence from a field experiment in Indonesia". Mimeo, Harvard University
98. Parker S., Rubalcava L., and Teruel G. Evaluating conditional schooling and health transfer programs. Handbook of Development Economics, vol. 4 (2008), Elsevier/North-Holland, Amsterdam (Chapter 62 in this book.)
99. Pitt, M., Khandker, S. (1998). "The impact of group-based credit programs on poor households in Bangladesh: Does the gender of participants matter?" Journal of Political Economy
100. Raudenbush S.W., Spybrook J., Liu X., and Congdon R. Optimal design for longitudinal and multilevel research: Documentation for the optimal design software. Retrieved April 15, 2005 from. http://sitemaker.umich.edu/group-based/optimal_design_software
101. Ravallion M. Evaluating anti-poverty programs. In: Evenson R.E., and Schultz T.P. (Eds). Handbook of Development Economics vol. 4 (2008), Elsevier/North-Holland, Amsterdam (Chapter 59 in this book.)
102. Rosenbaum P.R. Covariance adjustment in randomized experiments and observational studies (with discussion). Statistical Science 17 (2002) 286-327
103. Rosenbaum P.R., and Rubin D.B. The central role of the propensity score in observational studies for causal effects. Biometrika 70 (1983) 41-55
104. Rubin D.B. Estimating causal effects of treatments in randomized and non-randomized studies. Journal of Educational Psychology 66 (1974) 688-701
105. Savin N.E. Multiple hypothesis testing. Handbook of Econometrics vol. II (1984), Elsevier, Amsterdam 827-879
106. Schady, N.R., Araujo, M.C. (2006). "Cash, conditions, school enrollment and child work: Evidence from a randomized experiment in Ecuador". Unpublished manuscript
107. Schultz T.P. School subsidies for the poor: Eevaluating the Mexican Progresa poverty program. Journal of Development Economics 74 1 (2004) 199-250
108. Schwartz, B., Bhushan, I. (2004). "Reducing inequity in the provision of primary health care services: Contracting in Cambodia". Mimeo
109. Smith J., and Todd P. Does matching overcome Lalonde's critique of nonexperimental estimators?. Journal of Econometrics 125 1-2 (2005) 305-353
110. Thomas, D., Frankenberg, E., Friedman, J., Habicht, J.-P., Al E. (2003). "Iron deficiency and the well being of older adults: Early results from a randomized nutrition intervention". Mimeo, UCLA
111. Todd P. Evaluating social programs with endogenous program placement and selection of the treated. Handbook of Development Economics, vol. 4 (2008), Elsevier/North-Holland, Amsterdam (Chapter 60 in this book.)
112. Todd, P.E., Wolpin, K.I. (2006). "Ex ante evaluation of social programs". Mimeo, University of Pennsylvania
113. Vermeersch, C., Kremer, M. (2004). "School meals, educational achievement, and school competition: Evidence from a randomized evaluation". Working paper No. 3523. World Bank Policy Research
114. Westfall P., and Young S. Resampling-Based Multiple Testing: Examples and Methods for p-Value Adjustment (1993), Wiley, New York
115. Wooldridge J.M. Cluster-sample methods in applied econometrics. American Economic Review 93 2 (2004) 133-138
116. Wooldridge J.M. Inverse probability weighted M-estimators for sample selection, attrition, and stratification. Portuguese Economic Journal 1 (2002) 117-139