Cautionary notes on the use of field experiments to address policy issues

Oxford Review of Economic Policy

Volumn 30, Issue 4, 2014, Pages 753-763

  Harrison, Glenn W ^a  

^a GEORGIA STATE UNIVERSITY   (United States)

Author keywords

Field experiments; Randomized evaluations; Welfare

Indexed keywords

ECONOMIC POLICY; EXPERIMENTAL STUDY; POLICY ANALYSIS; WELFARE ECONOMICS;

EID: 84929747767     PISSN: 0266903X     EISSN: 14602121     Source Type: Journal    
DOI: 10.1093/oxrep/gru037     Document Type: Article

References (58)

1. Andersen, S., Fountain, J., Harrison, G. W., and Rutström, E. E. (2014c), 'Estimating Subjective Probabilities', Journal of Risk and Uncertainty, 48, 207-29.
2. Andersen, S., Harrison, G. W., Lau, M. I., and Rutström, E. E. (2008), 'Eliciting Risk and Time Preferences', Econometrica, 76(3), 583-619.
3. Andersen, S., Harrison, G. W., Lau, M. I., and Rutström, E. E. (2014a), 'Discounting Behavior: A Reconsideration', European Economic Review, 71, 15-33.
4. Andersen, S., Harrison, G. W., Lau, M. I., and Rutström, E. E. (2014b), 'Dual Criteria Decisions', Journal of Economic Psychology, 41, 101-13.
5. Banerjee, A. V., and Duflo, E. (2009), 'The Experimental Approach to Development Economics', Annual Review of Economics, 1, 151-78.
6. Banerjee, A. V., and Duflo, E. (2011), Poor Economics: A Radical Rethinking of the Way to Fight Global Poverty, New York, Public Affairs.
7. Berry, D. A. (2005), 'Introduction to Bayesian Methods III: Use and Interpretation of Bayesian Tools in Design and Analysis', Clinical Trials, 2(4), 295-300.
8. Bohm, P. (2003), 'Experimental Evaluations of Policy Instruments', in K. G. Mäler and J. R. Vincent (eds), Handbook of Environmental Economics, Vol. 1, Amsterdam, Elsevier, 438-60.
9. Campbell, G. (2005), 'The Experience in the FDA's Center for Devices and Radiological Health with Bayesian Strategies', Clinical Trials, 2(4), 359-63.
10. Collins, D., Morduch, J., Rutherford, S., and Ruthven, O. (2009), Portfolios of the Poor: How the World's Poor Live on $2 a Day, Princeton, NJ, Princeton University Press.
11. Crawford, V. P., and Meng, J. (2011), 'New York City Cab Drivers' Labor Supply Revisited: Referencedependent Preferences with Rational-expectations Targets for Hours and Income', American Economic Review, 101, 1912-32.
12. Deaton, A. (2010), 'Instruments, Randomization, and Learning about Development', Journal of Economic Literature, 48(2), 424-55.
13. Deci, E. L. (1971), 'Effects of Externally Mediated Rewards on Intrinsic Motivation', Journal of Personality and Social Psychology, 18(1), 105-11.
14. Della Vigna, S., List, J. A., and Malmendier, U. (2012), 'Testing for Altruism and Social Pressure in Charitable Giving', Quarterly Journal of Economics, 127(1), 1-56.
15. Dixit, V., Harrison, G. W., and Rutström, E. E. (2014), 'Estimating the Subjective Risks of Driving Simulator Accidents', Accident Analysis and Prevention, 62, 63-78.
16. Easterly, W. (2007), The White Man's Burden: Why the West's Efforts to Aid the Rest Have Done So Much Ill and So Little Good, New York, Oxford University Press.
17. Evans, C. H., and Ildstad, S. T. (eds) (2001), Small Clinical Trials: Issues and Challenges, Washington, DC, National Academy Press.
18. Farber, H. S. (2008), 'Reference-dependent Preferences and Labor Supply: The Case of New York City Taxi Drivers', American Economic Review, 98(3), 1069-82.
19. Ferber, R., and Hirsch, W. Z. (1978), 'Social Experimentation and Economic Policy: A Survey', Journal of Economic Literature, 16(4), 1379-414.
20. Ferber, R., and Hirsch, W. Z. (1981), Social Experimentation and Economic Policy, New York, Cambridge University Press.
21. Fiore, S. M., Harrison, G, W., Hughes, C. E., and Rutström, E. E. (2009), 'Virtual Experiments and Environmental Policy', Journal of Environmental Economics and Management, 57(1), 65-86.
22. Food and Drug Administration (2010a), Adaptive Design Clinical Trials for Drugs and Biologics, Washington, DC, US Department of Health and Human Services, February.
23. Food and Drug Administration (2010b), Guidance for the Use of Bayesian Statistics in Medical Device Clinical Trials, Washington, DC, US Department of Health and Human Services, February.
24. Garfinkel, I., Manski, C., and Michalopoulos, C. (1992), 'Micro Experiments and Macro Effects', in C. Manski and I. Garfinkel (eds), Evaluating Welfare and Training Programs, Cambridge, MA, Harvard University Press.
25. Gill, D., and Prowse, V. (2012), 'A Structural Analysis of Disappointment Aversion in a Real Effort Competition', American Economic Review, 102(1), 469-503.
26. Goodman, S. N. (2005), 'Introduction to Bayesian Methods I: Measuring the Strength of Evidence', Clinical Trials, 2(4), 282-90.
27. Harrison, G. W. (2005), 'Field Experiments and Control', in J. Carpenter, G. W. Harrison, and J. A. List (eds), Field Experiments in Economics, Greenwich, CT, JAI Press, Research in Experimental Economics, Vol. 10, 17-50.
28. Harrison, G. W. (2011a), 'Experimental Methods and the Welfare Evaluation of Policy Lotteries', European Review of Agricultural Economics, 38(3), 335-60.
29. Harrison, G. W. (2011b), 'Randomisation and Its Discontents', Journal of African Economies, 20(4), 626-52.
30. Harrison, G. W. (2013), 'Field Experiments and Methodological Intolerance', Journal of Economic Methodology, 20(2), 103-17.
31. Harrison, G. W. (2014), 'Impact Evaluation and Welfare Evaluation', European Journal of Development Research, 26(1), 39-45.
32. Harrison, G. W., and Lau, M. I. (2014), 'Risk Attitudes, Sample Selection and Attrition in a Longitudinal Field Experiment', Working Paper 2014-04, Center for the Economic Analysis of Risk, Robinson College of Business, Georgia State University.
33. Harrison, G. W., List, J. A. (2004), 'Field Experiments', Journal of Economic Literature, 42(4), 1013-59.
34. Harrison, G. W., Haruvy, E., and Rutström, E. E. (2011), 'Remarks on Virtual World and Virtual Reality Experiments', Southern Economic Journal, 78(1), 87-94.
35. Harrison, G. W., Lau, M. I., and Rutström, E. E. (2009), 'Risk Attitudes, Randomization to Treatment, and Selfselection into Experiments', Journal of Economic Behavior and Organization, 70(3), 498-507.
36. Hausman, J. A., and Wise, D. A. (eds) (1985), Social Experimentation, Chicago, IL, University of Chicago Press.
37. Heckman, J. J. (1992), 'Randomization and Social Program Evaluation', in C. Manski and I. Garfinkel (eds), Evaluating Welfare and Training Programs, Cambridge, MA, Harvard University Press.
38. Heckman, J. J. (2010), 'Building Bridges between Structural and Program Evaluation Approaches to Evaluating Policy', Journal of Economic Literature, 48(2), 356-98.
39. Jennison, C., and Turnbull, B. W. (2011), Group Sequential and Adaptive Methods for Clinical Trials, Boca Raton, FL, Chapman & Hall.
40. Karlan, D., and Appel, J. (2011), More Than Good Intentions: How a New Economics is Helping to Solve Global Poverty, New York, Dutton.
41. Keane, M. P. (2010a), 'Structural vs Atheoretic Approaches to Econometrics', Journal of Econometrics, 156, 3-20.
42. Keane, M. P. (2010b), 'A Structural Perspective on the Experimentalist School', Journal of Economic Perspectives, 24(2), 47-58.
43. Leamer, E. E. (2010), 'Tantalus on the Road to Asymptopia', Journal of Economic Perspectives, 24(2), 31-46.
44. Louis, T. A. (2005), 'Introduction to Bayesian Methods II: Fundamental Concepts', Clinical Trials, 2(4), 291-4.
45. Manski, C., and Garfinkel, I. (eds) (1992), Evaluating Welfare and Training Programs, Cambridge, MA, Harvard University Press.
46. Morgan, S. L., and Winship, C. (2007), Counterfactual and Causal Inference: Methods and Principles for Social Research, New York, Cambridge University Press.
47. Munk, N. (2013), The Idealist: Jeffrey Sachs and the Quest to End Poverty, New York, Doubleday.
48. Rosenzweig, M. R., and Wolpin, K. I. (2000), 'Natural "Natural Experiments" in Economics', Journal of Economic Literature, 38, 827-74.
49. Ross, D. (2014), Philosophy of Economics, Basingstoke, Palgrave Macmillan.
50. Sunstein, C. (2013), Simpler: The Future of Government, New York, Simon & Schuster.
51. Thaler, R., and Sunstein, C. (2008), Nudge: Improving Decisions About Health, Wealth and Happiness, New Haven, CT, Yale University Press.
52. Toubia, O., Johnson, E., Evgeniou, T., and Delquié, P. (2013), 'Dynamic Experiments for Estimating Preferences: An Adaptive Method for Eliciting Time and Risk Parameters', Management Science, 59(3), 613-40.
53. Viceisza, A. C. G. (2012), Treating the Field as a Lab: A Basic Guide to Conducting Economics Experiments for Policymaking, Washington, DC, International Food Policy Research Institute.
54. Wakker, P. P. (2010), Prospect Theory for Risk and Ambiguity, New York, Cambridge University Press.
55. Wang, S., Filiba, M., and Camerer, C. F. (2010), 'Dynamically Optimized Sequential Experimentation (DOSE) for Estimating Economic Preference Parameters', Working Paper, Division of Humanities and Social Sciences, California Institute of Technology.
56. White, H. (2014), 'Current Challenges in Impact Evaluation', European Journal of Development Research, 26(1), 18-30.
57. Whitehead, J. (1997), The Design and Analysis of Sequential Clinical Trials, 2nd edn, New York, Wiley.
58. Worrall, J. (2007), 'Why There's No Cause to Randomize', British Journal of the Philosophy of Science, 58, 451-88.