Transparency, Replication, and Cumulative Learning: What Experiments Alone Cannot Achieve

Annual Review of Political Science

Volumn 19, Issue , 2016, Pages 541-563

  Dunning, Thad ^a  

^a UNIVERSITY OF CALIFORNIA   (United States)

Author keywords

Cumulation; Experiments; Replication; Transparency

Indexed keywords

EID: 84969791287     PISSN: 10942939     EISSN: None     Source Type: Book Series    
DOI: 10.1146/annurev-polisci-042114-015939     Document Type: Review

References (49)

1. Angrist JD, Pischke J-S. 2009. Mostly Harmless Econometrics: An Empiricist's Companion. Princeton, NJ: Princeton Univ. Press
2. Angrist JD, Pischke J-S. 2014. Mastering 'Metrics: The Path from Cause to Effect. Princeton,NJ: PrincetonUniv. Press
3. Angrist JD, Pischke J-S. 2015. Mastering metrics: teaching econometrics. VOX: CEPR's Policy Portal. May 21. http://www. voxeu. org/article/mastering-metrics-teaching-econometrics
4. Banerjee AV, Banerji R, Duflo E, Glennerster R, Khemani S. 2008. Pitfalls of participatory programs: evidence from a randomized evaluation in education in India. NBERWork. Pap. 14311
5. Banerjee AV, Duflo E. 2009. The experimental approach to development economics. Annu. Rev. Econ. 1:151-78
6. Benjamini Y, Hochsberg Y. 1995. Controlling the false discovery rate: a practical and powerful approach to multiple testing. J. R. Stat. Soc. Ser. B 57(1):289-300
7. Bj orkman M, Svensson J. 2009. Power to the people: evidence from a randomized field experiment on community-based monitoring in Uganda. Q. J. Econ. 124(2):735-69
8. Brady HE, Collier D, eds. 2010. Rethinking Social Inquiry: Diverse Tools, Shared Standards. Lanham, Maryland: Rowman &Littlefield. 2nd ed.
9. Broockman D, Kalla J, Aronow P. 2015. Irregularities in LaCour 2014. Work. pap. , Stanford Univ. http:// stanford. edu/~dbroock/broockman-kalla-aronow-lg-irregularities. pdf
10. Cartwright N, Hardie J. 2012. Evidence-Based Policy: A Practical Guide to Doing It Better. New York: Oxford Univ. Press
11. Clemens M. 2015. The meaning of failed replications: a review and proposal. Cent. Glob. Dev. Work. Pap. 399
12. Collier D, Brady HE, Dunning T. 2015. The set-theoretic comparative method (STCM): fundamental problems and better options. Work. pap. , Univ. Calif. , Berkeley
13. De Rooij EA, Green DP, Gerber AS. 2009. Field experiments on political behavior and collective action. Annu. Rev. Polit. Sci. 12:389-95
14. Dunning T. 2012. Natural Experiments in the Social Sciences: A Design-Based Approach. New York: Cambridge Univ. Press
15. Dunning T, Hyde S. 2014. Replicate it! A proposal to improve the study of political accountability. Monkey Cage blog. Washington Post, May 16. http://www. washingtonpost. com/blogs/monkey-cage/wp/ 2014/05/16/replicate-it-A-proposal-to-improve-The-study-of-political-accountability
16. Dunning T, Monestier F, Pi neiro R, Rosenblatt F, Tu nón G. 2015. Positive versus negative incentives for compliance: evaluating a randomized tax holiday in Uruguay. Work. pap. , Univ. Calif. , Berkeley
17. Fang A, Gordon G, Humphreys M. 2015. Does registration reduce publication bias? No evidence from medical sciences. Work. pap. , Columbia Univ.
18. Freedman DA. 2009a. Statistical Models: Theory and Practice. New York: Cambridge Univ. Press. 2nd ed.
19. Freedman DA. 2009b. Randomization does not justify logistic regression. In Statistical Models and Causal Inference: A Dialogue with the Social Sciences, ed. DA Freedman, D Collier, JS Sekhon, PB Stark, pp. 219-42. New York: Cambridge Univ. Press
20. Freedman DA, Pisani R, Purves R. 2007. Statistics. New York: W. W. Norton. 4th ed.
21. Gerber AS, Green DP, Nickerson DW. 2001. Testing for publication bias in political science. Polit. Anal. 9:385-92
22. Gerber AS, Malhotra N. 2008. Do statistical reporting standards affect what is published? Publication bias in two leading political science journals. Q. J. Polit. Sci. 3(3):313-26
23. Gherghina S, Katsanidou A. 2013. Data availability in political science journals. Eur. Polit. Sci. 12:333-49
24. Green D, Gerber A. 2012. Field Experiments: Design, Analysis, Interpretation. New York: W. W. Norton
25. Hamermesh DS. 2007. Replication in economics. NBERWork. Pap. No. 13026
26. Humphreys M, de la Sierra RS, van derWindt P. 2011. Social and economic impacts of TUUNGUANE 1: mock report. http://cu-csds. org/wp-content/uploads/2011/03/20110304-MOCK-REPORT-SHAREDFOR-REGISTRATION. pdf
27. Humphreys M, de la Sierra RS, van der Windt P. 2013. Fishing, commitment, and communication. Polit. Anal. 21(1):1-20
28. Humphreys M, Weinstein JM. 2009. Field experiments and the political economy of development. Annu. Rev. Polit. Sci. 12:367-78
29. Hutchings VL, Jardina AE. 2009. Experiments on racial priming in political campaigns. Annu. Rev. Polit. Sci. 12:397-402
30. Hyde S. 2015. Experiments in international relations: lab, survey, and field. Annu. Rev. Polit. Sci. 18:403-24
31. LaCour MJ, Green DP. 2014. When contact changes minds: an experiment on transmission of support for gay equality. Science 346(6215):1366-69
32. Laitin D. 2013. Fisheries management. Polit. Anal. 21(1):42-47
33. Lieberman ES, Posner DN, Tsai LL. 2013. Does information lead to more active citizenship? Evidence from an education intervention in rural Kenya. World Dev. 60:69-83
34. LinW. 2013. Agnostic notes on regression adjustment to experimental data: reexamining Freedman's critique. Ann. Appl. Stat. 7(1):295-318
35. Malhotra N. 2014. Publication bias in political science: using TESS experiments to unlock the file drawer. Presented at West Coast Exp. Conf. , Claremont Graduate Univ. , May 9
36. Mauldon J, Malvin J, Stiles J, Nicosia N, Seto E. 2000. Impact of California's Cal-Learn demonstration project: final report. UC Data, Univ. Calif. Berkeley
37. McDermott R. 2002. Experimental methods in political science. Annu. Rev. Polit. Sci. 5:31-61
38. McCullough BD, McGeary KA, Harrison TD. 2006. Lessons from the JMCB archive. J. Mon. Credit Banking 38(4):1093-107
39. Miguel E, Camerer C, Casey K, Cohen J, Esterling KM, et al. 2014. Promoting transparency in social science research. Science 343:30-31
40. Monogan JE III. 2013. A case for registering studies of political outcomes: an application in the 2010 House elections. Polit. Anal. 21(1):21-37
41. Monogan JEIII. 2015. Research preregistration in political science: the case, counterarguments, and a response to critiques. PS Polit. Sci. Polit. 48(3):425-29
42. Olken B. 2007. Monitoring corruption: evidence from a field experiment in Indonesia. J. Polit. Econ. 115(2):200-49
43. Palfrey TR. 2009. Laboratory experiments in political economy. Annu. Rev. Polit. Sci. 12:379-88
44. Ragin CC. 1987. The Comparative Method: Moving Beyond Qualitative and Quantitative Strategies. Berkeley: Univ. Calif. Press
45. Ragin CC. 2000. Fuzzy-Set Social Science. Chicago: Univ. Chicago Press
46. Rosenbaum PR. 2002. Observational Studies. Springer Ser. Stat. New York: Springer-Verlag. 2nd ed.
47. SimonsohnU,Neilson LD, Simmons JP. 2014. P-curve: a key to the file drawer. J. Exp. Psychol. 143(2):534-47
48. Singal J. 2015. Michael LaCour probably fabricated a document about research integrity. New York, June http://nymag. com/scienceofus/2015/06/lacour-probably-fabricated-an-integrity-document. html
49. Tanner S. 2014. QCA and causal inference: a poor match for public policy research. Qual. Multi-Method Res. 12(1):15-24