In pursuit of balance: Randomization in practice in development field experiments

American Economic Journal: Applied Economics

Volumn 1, Issue 4, 2009, Pages 200-232

  Bruhn, Miriam ^a   McKenzie, David ^a  

^a WORLD BANK   (United States)

Author keywords

[No Author keywords available]

Indexed keywords

EID: 78049298073     PISSN: 19457782     EISSN: 19457790     Source Type: Journal    
DOI: 10.1257/app.1.4.200     Document Type: Article

References (56)

1. Aickin, Mikel. 2001. "Randomization, Balance, and the Validity and Efficiency of Design-Adaptive Allocation Methods." Journal of Statistical Planning and Inference, 94(1): 97-119.
2. Altman, Douglas G. 1985. "Comparability of Randomized Groups." Statistician, 34: 125-36.
3. Andrabi, Tahir, Jishnu Das, Asim I. Khwaja, and Tristan Zajonc. 2008. "Do Value-Added Estimates Add Value? Accounting for Learning Dynamics." Harvard University Center for International Development Working Paper 158.
4. Ashraf, Nava, James Berry, and Jesse M. Shapiro. 2007. "Can Higher Prices Stimulate Product Use? Evidence from a Field Experiment in Zambia." National Bureau of Economic Research Working Paper 13247.
5. Ashraf, Nava, Dean Karlan, and Wesley Yin. 2006a. "Deposit Collectors." B. E. Journal of Economic Analysis and Policy: Advances in Economic Analysis and Policy, 6(2): 1-22.
6. Ashraf, Nava, Dean Karlan, and Wesley Yin. 2006b. "Tying Odysseus to the Mast: Evidence from a Commitment Savings Product in the Philippines." Quarterly Journal of Economics, 121(2): 635- 72.
7. Banerjee, Abhijit V., Shawn Cole, Esther Duflo, and Leigh Linden. 2007. "Remedying Education: Evidence from Two Randomized Experiments in India." Quarterly Journal of Economics, 122(3): 1235-64.
8. Banerjee, Abhijit Vinayak, with Alice H. Amsden, Robert H. Bates, Jagdish Bhagwati, Angus Deaton, and Nicholas Stern. 2007. Making Aid Work. Cambridge, MA: MIT Press.
9. Bertrand, Marianne, Simeon Djankov, Rema Hanna, and Sendhil Mullainathan. 2007. "Obtaining a Driver's License in India: An Experimental Approach to Studying Corruption." Quarterly Journal of Economics, 122(4): 1639-76.
10. Björkman, Martina, and Jakob Svensson. 2009. "Power to the People: Evidence from a Randomized Field Experiment on Community-Based Monitoring in Uganda." Quarterly Journal of Economics, 124(2): 735-69.
11. Bobonis, Gustavo J., Edward Miguel, and Charu Puri-Sharma. 2006. "Anemia and School Participation." Journal of Human Resources, 41(4): 692-721.
12. Box, George E. P., J. Stuart Hunter, and William G. Hunter. 2005. Statistics for Experimenters: Design, Innovation, and Discovery. 2nd ed. Hoboken, NJ: Wiley-Interscience.
13. Burtless, Gary. 1995. "The Case for Randomized Field Trials in Economic and Policy Research." Journal of Economic Perspectives, 9(2): 63-84.
14. Committee for Proprietary Medicinal Products (CPMP). 2003. "Points to Consider on Adjustment for Baseline Covariates." The European Agency for the Evaluation of Medicinal Products, Evaluation of Medicines for Human Use Report CPMP/EWP/2863/99. http://www.emea.europa.eu/pdfs/human/ewp/286399en.pdf (accessed February 6, 2008). London: EMEA.
15. Deaton, Angus. 2008. "Instruments of Development: Randomization in the Tropics, and the Search for the Elusive Keys to Economic Development." The Keynes Lecture in Economics, British Academy, London, October 9, 2008.
16. de Mel, Suresh, David McKenzie, and Christopher Woodruff. 2008. "Returns to Capital in Microenterprises: Evidence from a Field Experiment." Quarterly Journal of Economics, 123(4): 1329-72.
17. Duflo, Esther. 2005. "Evaluating the Impact of Development Aid Programmes: The Role of Randomised Evaluations." In Development Aid: Why and How? Toward Strategies for Effectiveness.
18. Proceedings of the AFD -EUD N Conference, 2004, 205-47. Paris: Agence Française de Développement. http://www.afd-libreville.org/jahia/webdav/site/afd/users/administrateur/public/publications/notesetdocuments/ND-22.pdf#page=208.
19. Duflo, Esther, Rachel Glennerster, and Michael Kremer. 2008. "Using Randomization in Development Economics Research: A Toolkit." In Handbook of Development Economics, Vol. 4, ed. T. Paul Schultz and John Strauss, 3895-3962. Amsterdam, NH: North Holland.
20. Duflo, Esther, Rema Hanna, and Stephen Ryan. 2007. "Monitoring Works: Getting Teachers to Come to School." Bureau of Research and Economic Analysis Development (BREAD) Working Paper 103.
21. Duflo, Esther, and Michael Kremer. 2004. "Use of Randomization in the Evaluation of Development Effectiveness." In Evaluating Development Effectiveness, Vol. 7, World Bank Series on Evaluation and Development, ed. George Keith Pitman, Osvaldo N. Feinstein, and Gregory K. Ingram, 205-32. Piscataway, NJ: Transaction Publishers.
22. Dupas, Pascaline. 2006. "Relative Risks and the Market for Sex: Teenagers, Sugar Daddies and HIV in Kenya." http://www.internationalpolicy.umich.edu/edts/pdfs/DupasRelativeRisks.pdf.
23. Ferraz, Claudio, and Frederico Finan. 2008. "Exposing Corrupt Politicians: The Effects of Brazil's Publicly Released Audits on Electoral Outcomes." Quarterly Journal of Economics, 123(2): 703-45.
24. Field, Erica, and Rohini Pande. 2008. "Repayment Frequency and Default in Microfinance: Evidence from India." Journal of the European Economic Association, 6(2-3): 501-09.
25. Fisher, Ronald A. 1935. The Design of Experiments. Edinburgh: Oliver and Boyd.
26. Freedman, David A. 2008. "On Regression Adjustments in Experiments with Several Treatments." Annals of Applied Statistics, 2(1): 176-96.
27. Glewwe, Paul, Michael Kremer, Sylvie Moulin, and Eric Zitzewitz. 2004. "Retrospective vs. Prospective Analyses of School Inputs: The Case of Flip Charts in Kenya." Journal of Development Economics, 74(1): 251-68.
28. Glewwe, Paul, Albert Park, and Meng Zhao. 2006. "The Impact of Eyeglasses on Academic Performance of Primary School Students: Evidence from a Randomized Trial in Rural China." University of Minnesota Center for International Food and Agricultural Products Conference Paper 6644.
29. Greevy, Robert, Bo Lu, Jeffrey H. Silber, and Paul Rosenbaum. 2004. "Optimal Multivariate Matching Before Randomization." Biostatistics, 5(2): 263-75.
30. He, Fang, Leigh L. Linden, and Margaret MacLeod. 2007. "Teaching What Teachers Don't Know: An Assessment of the Pratham English Language Program." http://www.columbia.edu/~mgm2115/PicTalk%20Working%20Paper%202007-03-26.pdf.
31. Imai, Kosuke, Gary King, and Clayton Nall. Forthcoming. "The Essential Role of Pair Matching in Cluster-Randomized Experiments, with Application to the Mexican Universal Health Insurance Evaluation." Statistical Science.
32. Imai, Kosuke, Gary King, and Elizabeth A. Stuart. 2008. "Misunderstandings between Experimentalists and Observationalists About Causal Inference." Journal of the Royal Statistical Society: Series A (Statistics in Society), 171(2): 481-502.
33. Imbens, Guido, Gary King, David McKenzie, and Geert Ridder. 2009. "On the Finite Sample Benefits of Stratification in Randomized Experiments." Unpublished.
34. Karlan, Dean, and Martin Valdivia. 2006. "Teaching Entrepreneurship: Impact of Business Training on Microfinance Clients and Institutions." http://aida.econ.yale.edu/karlan/papers/TeachingEntrepeneurship.pdf.
35. Kernan, Walter N., Catherine M. Viscoli, Robert W. Makuch, Lawrence M. Brass, and Ralph I. Horwitz. 1999. "Stratified Randomization for Clinical Trials." Journal of Clinical Epidemiology, 52(1): 19-26.
36. King, Gary, Emmanuela Gakidou, Nirmala Ravishankar, Ryan T. Moore, Jason Lakin, Manett Vargas, Martha María Téllez-Rojo, Juan Eugenio Hernández Ávila, Mauricio Hernández Ávila, and Héctor Hernández Llamas. 2007. "A 'Politically Robust' Experimental Design for Public Policy Evaluation, with Application to the Mexican Universal Health Insurance Program." Journal of Policy Analysis and Management, 26(3): 479-506.
37. Kremer, Michael. 2003. "Randomized Evaluations of Educational Programs in Developing Countries: Some Lessons." Amercan Economic Review, 93(2): 102-06.
38. Kremer, Michael, Jessica Leino, Edward Miguel, and Aliz Peterson Zwane. 2006. "Spring Cleaning: A Randomized Evaluation of Source Water Quality Improvement." http://www.sscnet.ucla.edu/polisci/wgape/papers/11_Miguel.pdf.
39. Levitt, Steven D., and John A. List. 2008. "Field Experiments in Economics: The Past, The Present, and The Future." National Bureau of Economic Research Working Paper 14356.
40. Martin, Donald C., Paula Diehr, Edward B. Perrin, and Thomas D. Koepsell. 1993. "The Effect of Matching on the Power of Randomized Community Intervention Studies." Statistics in Medicine, 12(3-4): 329-38.
41. Miguel, Edward, and Michael Kremer. 2004. "Worms: Identifying Impacts on Education and Health in the Presence of Treatment Externalities." Econometrica, 72(1): 159-217.
42. Olken, Benjamin A. 2007a. "Monitoring Corruption: Evidence from a Field Experiment in Indonesia." Journal of Political Economy, 115(2): 200-249.
43. Olken, Benjamin A. 2007b. "Political Institutions and Local Public Goods: Evidence from a Field Experiment in Indonesia." http://www.povertyactionlab.org/papers/51_Olken_Political_Institutions.pdf.
44. Permutt, Thomas. 1990. "Testing for Imbalance of Covariates in Controlled Experiments." Statistics in Medicine, 9(12): 1455-62.
45. Pocock, S. J., and R. Simon. 1975. "Sequential Treatment Assignment with Balancing for Prognostic Factors in the Controlled Clinical Trial." Biometrics, 3(1): 103-15.
46. Raab, Gillian M., Simon Day, and Jill Sales. 2000. "How to Select Covariates to Include in the Analysis of a Clinical Trial." Controlled Clinical Trials, 21(4): 330-42.
47. Rosenbaum, Paul R. 2002. Observational Studies. 2nd ed. New York: Springer.
48. Schulz, Kenneth F. 1996. "Randomised Trials, Human Nature, and Reporting Guidelines." Lancet, 348(9027): 596-98.
49. Schulz, Kenneth F., and David A. Grimes. 2002. "Allocation Concealment in Randomised Trials: Defending Against Deciphering." Lancet, 359(9306):614-18.
50. Scott, Neil W., Gladys C. McPherson, Craig R. Ramsay, and Marion K. Campbell. 2002. "The Method of Minimization for Allocation to Clinical Trials: A Review." Controlled Clinical Trials, 23(6): 662-74.
51. Senn, Stephen. 2004. "Added Values: Controversies Concerning Randomization and Additivity in Clinical Trials." Statistics in Medicine, 23(24): 3729-53.
52. Skoufias, Emmanuel. 2005. "PROGRESA and Its Impact on the Welfare of Rural Households in Mexico." International Food Policy Research Institute (IFPRI) Research Report 139.
53. Soares, Jose F., and C. F. Jeff Wu. 1983. "Some Restricted Randomization Rules in Sequential Designs." Communications in Statistics: Theory and Methods, 12(17): 2017-34.
54. Taves, D. R. 1974. "Minimization: A New Method of Assigning Patients to Treatment and Control Groups." Clinical Pharmacology Therapeutics, 15(5): 443-53.
55. Therneau, Terry M. 1993. "How Many Stratification Factors Are 'Too Many' to Use in a Randomization Plan?" Controlled Clinical Trials, 14(2): 98-108.
56. Treasure, Tom, and Kenneth D. MacRae. 1998. "Minimisation: The Platinum Standard for Trials?" British Medical Journal, 317(7155): 362-63.