Transparency, reproducibility, and the credibility of economics research

Journal of Economic Literature

Volumn 56, Issue 3, 2018, Pages 920-980

  Christensen, Garret ^a   Miguel, Edward ^a,b  

^a UNIVERSITY OF CALIFORNIA   (United States)

^b NATIONAL BUREAU OF ECONOMIC RESEARCH   (United States)

Author keywords

[No Author keywords available]

Indexed keywords

EID: 85054134819     PISSN: 00220515     EISSN: None     Source Type: Journal    
DOI: 10.1257/jel.20171350     Document Type: Review

References (330)

1. Abreu, Maria, Henri L. F. de Groot, and Raymond J. G. M. Florax. 2005. “A Meta-analysis of β-Convergence: The Legendary 2%.” Journal of Economic Surveys 19 (3): 389–420.
2. Acemoglu, Daron, Simon Johnson, and James A. Robinson. 2001. “The Colonial Origins of Comparative Development: An Empirical Investigation.” American Economic Review 91 (5): 1369–401.
3. Acemoglu, Daron, Simon Johnson, and James A. Robinson. 2012. “The Colonial Origins of Comparative Development: An Empirical Investigation: Reply.” American Economic Review 102 (6): 3077–110.
4. Aiken, Alexander M., Calum Davey, James R. Hargreaves, and Richard J. Hayes. 2015. “Re-analysis of Health and Educational Impacts of a School-Based Deworming Programme in Western Kenya: A Pure Replication.” International Journal of Epidemiology 44 (5): 1572–80.
5. Alatas, Vivi, Abhijit Banerjee, Rema Hanna, Benjamin A. Olken, and Julia Tobias. 2012. “Targeting the Poor: Evidence from a Field Experiment in Indonesia.” American Economic Review 102 (4): 1206–40.
6. Albouy, David Y. 2012. “The Colonial Origins of Comparative Development: An Empirical Investigation: Comment.” American Economic Review 102 (6): 3059–76.
7. Allcott, Hunt, and Dmitry Taubinsky. 2015. “Evaluating Behaviorally Motivated Policy: Experimental Evidence from the Lightbulb Market.” American Economic Review 105 (8): 2501–38.
8. Anderson, Christopher J., et al. 2016. “Response to Comment on ‘Estimating the Reproducibility of Psychological Science’.” Science 351 (6277): 1037.
9. Anderson, Melissa S., Brian C. Martinson, and Raymond De Vries. 2007. “Normative Dissonance in Science: Results from a National Survey of U.S. Scientists.” Journal of Empirical Research on Human Research Ethics 2 (4): 3–14.
10. Anderson, Michael L. 2008. “Multiple Inference and Gender Differences in the Effects of Early Intervention: A Reevaluation of the Abecedarian, Perry Preschool, and Early Training Projects.” Journal of the American Statistical Association 103 (484): 1481–95.
11. Anderson, Richard G., and William G. Dewald. 1994. “Replication and Scientific Standards in Applied Economics a Decade after the Journal of Money, Credit and Banking Project.” Federal Reserve Bank of St. Louis Review 76 (6): 79–83.
12. Anderson, Richard G., William H. Greene, B. D. McCullough, and H. D. Vinod. 2008. “The Role of Data/Code Archives in the Future of Economic Research.” Journal of Economic Methodology 15 (1): 99–119.
13. Andreoli-Versbach, Patrick, and Frank Mueller-Langer. 2014. “Open Access to Data: An Ideal Professed but Not Practised.” Research Policy 43 (9): 1621–33.
14. Angrist, Joshua D., and Jörn-Steffen Pischke. 2010. “The Credibility Revolution in Empirical Economics: How Better Research Design Is Taking the Con Out of Econometrics.” Journal of Economic Perspectives 24 (2): 3–30.
15. Annas, George J. 2003. “HIPAA Regulations—A New Era of Medical-Record Privacy?” New England Journal of Medicine 348 (15): 1486–90.
16. Applied Economics Letters. 2012. “Statement of Retraction.” Applied Economics Letters 19 (16): 1649.
17. Ashenfelter, Orley, and Michael Greenstone. 2004. “Estimating the Value of a Statistical Life: The Importance of Omitted Variables and Publication Bias.” Princeton University Center for Economic Policy Studies Working Paper 97.
18. Ashenfelter, Orley, Colm Harmon, and Hessel Oosterbeek. 1999. “A Review of Estimates of the Schooling/ Earnings Relationship, with Tests for Publication Bias.” Princeton, Department of Economics—Industrial Relations Sections, Princeton, Department of Economics—Industrial Relations Sections.
19. Backhouse, Roger E., and Mary S. Morgan. 2000. “Introduction: Is Data Mining a Methodological Problem?” Journal of Economic Methodology 7 (2): 171–81.
20. Bai, Liang, Benjamin Handel, Edward Miguel, and Gautam Rao. 2015. “Self-Control and Chronic Illness: Evidence from Commitment Contracts for Doctor Visits.” Unpublished.
21. Baicker, Katherine, Amy Finkelstein, Jae Song, and Sarah Taubman. 2014. “The Impact of Medicaid on Labor Market Activity and Program Participation: Evidence from the Oregon Health Insurance Experiment.” American Economic Review 104 (5): 322–28.
22. Baicker, Katherine, et al. 2013. “The Oregon Experiment—Effects of Medicaid on Clinical Outcomes.” New England Journal of Medicine 368 (18): 1713–22.
23. Bakkensen, Laura A., and William Larson. 2014. “Population Matters When Modeling Hurricane Fatalities.” Proceedings of the National Academy of Sciences 111 (50): E5331–32.
24. Bateman, Ian, Daniel Kahneman, Alistair Munro, Chris Starmer, and Robert Sugden. 2005. “Testing Competing Models of Loss Aversion: An Adversarial Collaboration.” Journal of Public Economics 89 (8): 1561–80.
25. Begg, Colin, et al. 1996. “Improving the Quality of Reporting of Randomized Controlled Trials: The CONSORT Statement.” Journal of the American Medical Association 276 (8): 637–39.
26. Bellavance, François, Georges Dionne, and Martin Lebeau. 2009. “The Value of a Statistical Life: A Meta-analysis with a Mixed Effects Regression Model.” Journal of Health Economics 28 (2): 444–64.
27. Benjamini, Yoav, and Yosef Hochberg. 1995. “Controlling the False Discovery Rate: A Practical and Powerful Approach to Multiple Testing.” Journal of the Royal Statistical Society. Series B 57 (1): 289–300.
28. Benjamini, Yoav, Abba M. Krieger, and Daniel Yekutieli. 2006. “Adaptive Linear Step-Up Procedures that Control the False Discovery Rate.” Biometrika 93 (3): 491–507.
29. Benjamini, Yoav, and Daniel Yekutieli. 2001. “The Control of the False Discovery Rate in Multiple Testing under Dependency.” Annals of Statistics 29 (4): 1165–88.
30. Berge, Lars Ivar Oppedal, et al. 2015. “How Strong Are Ethnic Preferences?” National Bureau of Economic Research Working Paper 21715.
31. Berger, Ulrich. 2009. “The Convergence of Fictitious Play in Games with Strategic Complementarities: A Comment.” https://mpra.ub.uni-muenchen.de/20241/1/MPRA_paper_20241.pdf.
32. Bernanke, Ben S. 2004. “Editorial Statement.” American Economic Review 94 (1): 404.
33. Bertrand, Marianne, and Sendhil Mullainathan. 2004. “Are Emily and Greg More Employable than Lakisha and Jamal? A Field Experiment on Labor Market Discrimination.” American Economic Review 94 (4): 991–1013.
34. Bhattacharjee, Yudhijit. 2013. “Diederik Stapel’s Audacious Academic Fraud.” New York Times, April 26.
35. Bland, J. Martin, and Douglas G. Altman. 1995. “Multiple Significance Tests: The Bonferroni Method.” British Medical Journal 310 (6973): 170.
36. Bohannon, John. 2016. “About 40% of Economics Experiments Fail Replication Survey.” http://www.sciencemag.org/news/2016/03/about-40-econom-ics-experiments-fail-replication-survey.
37. Bonferroni, C. E. 1936. “Teoria statistica delle classi e calcolo delle probabilità.” Pubblicazioni del R Istituto Superiore di Scienze Economiche e Commerciali di Firenze 8: 3–62.
38. Bontemps, Christophe, and Grayham E. Mizon. 2008. “Encompassing: Concepts and Implementation.” Oxford Bulletin of Economics and Statistics 70 (S1): 721–50.
39. Boutron, Isabelle, David Moher, Douglas G. Altman, Kenneth F. Schulz, and Philippe Ravaud. 2008. “Extending the CONSORT Statement to Randomized Trials of Nonpharmacologic Treatment: Explanation and Elaboration.” Annals of Internal Medicine 148 (4): 295–309.
40. Broad, William, and Nicholas Wade. 1983. Betrayers of the Truth: Fraud and Deceit in the Halls of Science. New York: Simon and Schuster.
41. Brock, William A., Steven N. Durlauf, and Kenneth D. West. 2003. “Policy Evaluation in Uncertain Economic Environments.” Brookings Papers on Economic Activity 1: 235–301.
42. Brodeur, Abel, Mathias Lé, Marc Sangnier, and Yanos Zylberberg. 2016. “Star Wars: The Empirics Strike Back.” American Economic Journal: Applied Economics 8 (1): 1–32.
43. Broockman, David, Joshua Kalla, and Peter Aronow. 2015. “Irregularities in LaCour (2014).” https://stanford.edu/~dbroock/broockman_kalla_aronow_lg_ irregularities.pdf.
44. Buckheit, Jonathan B., and David L. Donoho. 1995. “Wave Lab and Reproducible Research.” In Wavelets and Statistics, edited by A. Antoniadis and G. Oppenheim, 55–81. New York: Springer.
45. Burnside, Craig, and David Dollar. 2000. “Aid, Policies, and Growth.” American Economic Review 90 (4): 847–68.
46. Burnside, Craig, and David Dollar. 2004. “Aid, Policies, and Growth: Reply.” American Economic Review 94 (3): 781–84.
47. Calvert, Melanie, Jane Blazeby, Douglas G. Altman, Dennis A. Revicki, David Moher, and Michael D. Brundage. 2013. “Reporting of Patient-Reported Outcomes in Randomized Trials: The CONSORT PRO Extension.” Journal of the American Medical Association 309 (8): 814–22.
48. Camerer, Colin F., et al. 2016. “Evaluating Replicability of Laboratory Experiments in Economics.” Science 351 (6280): 1433–36.
49. Camfield, Laura, and Richard Palmer-Jones. 2013. “Three ‘Rs’ of Econometrics: Repetition, Reproduction and Replication.” Journal of Development Studies 49 (12): 1607–14.
50. Campbell, Marion K., Diana R. Elbourne, and Douglas G. Altman. 2004. “CONSORT Statement: Extension to Cluster Randomised Trials.” British Medical Journal 328 (7441): 702–08.
51. Campbell, Marion K., Gilda Piaggio, Diana R. Elbourne, and Douglas G. Altman. 2012. “Consort 2010 Statement: Extension to Cluster Randomised Trials.” British Medical Journal 345: E5661.
52. Campolieti, Michele, Morley Gunderson, and Chris Riddell. 2006. “Minimum Wage Impacts from a Prespecified Research Design: Canada 1981–1997.” Industrial Relations 45 (2): 195–216.
53. Card, David. 1992a. “Do Minimum Wages Reduce Employment? A Case Study of California, 1987–89.” ILR Review 46 (1): 38–54.
54. Card, David. 1992b. “Using Regional Variation in Wages to Measure the Effects of the Federal Minimum Wage.” ILR Review 46 (1): 22–37.
55. Card, David, Raj Chetty, Martin Feldstein, and Emmanuel Saez. 2010. “Expanding Access to Administrative Data for Research in the United States.” https://www.nsf.gov/sbe/sbe_2020/2020_pdfs/Card_David_112. pdf.
56. Card, David, and Alan B. Krueger. 1994. “Minimum Wages and Employment: A Case Study of the Fast-Food Industry in New Jersey and Pennsylvania.” American Economic Review 84 (4): 772–93.
57. Card, David, and Alan B. Krueger. 1995. “Time-Series Minimum-Wage Studies: A Meta-analysis.” American Economic Review 85 (2): 238–43.
58. Card, David, and Alan B. Krueger. 2000. “Minimum Wages and Employment: A Case Study of the Fast-Food Industry in New Jersey and Pennsylvania: Reply.” American Economic Review 90 (5): 1397–420.
59. Carey, Benedict. 2011. “Noted Dutch Psychologist, Stapel, Accused of Research Fraud.” New York Times. https://www.nytimes.com/2011/11/03/health/research/noted-dutch-psychologist-stapel-ac-cused-of-research-fraud.html.
60. Casey, Katherine, Rachel Glennerster, and Edward Miguel. 2012. “Reshaping Institutions: Evidence on Aid Impacts Using a Preanalysis Plan.” Quarterly Journal of Economics 127 (4): 1755–812.
61. Casselman, Ben. 2012. “Economists Set Rules on Ethics.” Wall Street Journal. https://www.wsj.com/articles/SB100014240529702034369045771489404106 67970.
62. Chambers, Christopher D. 2013. “Registered Reports: A New Publishing Initiative at Cortex.” Cortex 49 (3): 609–10.
63. Chambers, Christopher D., Eva Feredoes, Suresh D. Muthukumaraswamy, and Peter J. Etchells. 2014. “Instead of ‘Playing the Game’ It Is Time to Change the Rules: Registered Reports at AIMS Neuroscience and Beyond.” AIMS Neuroscience 1 (1): 4–17.
64. Chang, Andrew C., and Phillip Li. 2015. “Is Economics Research Replicable? Sixty Published Papers from Thirteen Journals Say ‘Usually Not’.” Federal Reserve Board Finance and Economics Discussion Paper 2015-083.
65. Chenault, Larry A. 1984. “A Note on the Stability Limitations in ‘A Stable Price Adjustment Process’.” Quarterly Journal of Economics 99 (2): 385–86.
66. Christensen, Bjorn, and Soren Christensen. 2014. “Are Female Hurricanes Really Deadlier than Male Hurricanes?” Proceedings of the National Academy of Sciences 111 (34): E3497–98.
67. Ciccone, Antonio. 2011. “Economic Shocks and Civil Conflict: A Comment.” American Economic Journal: Applied Economics 3 (4): 215–27.
68. Clemens, Michael. 2015. “The Meaning of Failed Replications: A Review and Proposal.” Center for Global Development Working Paper 399.
69. Coffman, Lucas C., and Muriel Niederle. 2015. “Pre-analysis Plans Have Limited Upside, Especially Where Replications Are Feasible.” Journal of Economic Perspectives 29 (3): 81–98.
70. Cohen-Cole, Ethan, Steven Durlauf, Jeffrey Fagan, and Daniel Nagin. 2009. “Model Uncertainty and the Deterrent Effect of Capital Punishment.” American Law and Economics Review 11 (2): 335–69.
71. Cowen, Tyler, and Alex Tabarrok. 2016. “A Skeptical View of the National Science Foundation’s Role in Economic Research.” Journal of Economic Perspectives 30 (3): 235–48.
72. Dahl Rasmussen, Ole, Nikolaj Malchow-Møller, and Thomas Barnebeck Andersen. 2011. “Walking the Talk: The Need for a Trial Registry for Development Interventions.” Journal of Development Effectiveness 3 (4): 502–19.
73. Dal-Ré, Rafael, et al. 2014. “Making Prospective Registration of Observational Research a Reality.” Science Translational Medicine 6 (224).
74. Davis, Graham A. 2013. “Replicating Sachs and Warner’s Working Papers on the Resource Curse.” Journal of Development Studies 49 (12): 1615–30.
75. De Angelis, Catherine, et al. 2004. “Clinical Trial Registration: A Statement from the International Committee of Medical Journal Editors.” New England Journal of Medicine 351 (12): 1250–51.
76. Deere, Donald, Kevin M. Murphy, and Finis Welch. 1995. “Employment and the 1990–1991 Minimum Wage Hike.” American Economic Review 85 (2): 232–37.
77. Dekel, Eddie, David K. Levine, Costas Meghir, Whitney K. Newey, and Andrew Postlewaite. 2006. “Report of the Editors.” Econometrica 74 (1): 307–10.
78. DellaVigna, Stefano, and Devin Pope. 2016. “Predicting Experimental Results: Who Knows What?” National Bureau of Economic Research Working Paper 22566.
79. DeLong, J. Bradford, and Kevin Lang. 1992. “Are All Economic Hypotheses False?” Journal of Political Economy 100 (6): 1257–72.
80. Denton, Frank T. 1985. “Data Mining as an Industry.” Review of Economics and Statistics 67 (1): 124–27.
81. Dewald, William G., Jerry G. Thursby, and Richard G. Anderson. 1986. “Replication in Empirical Economics: The Journal of Money, Credit and Banking Project.” American Economic Review 76 (4): 587–603.
82. Dezhbakhsh, Hashem, Paul H. Rubin, and Joanna M. Shepherd. 2003. “Does Capital Punishment Have a Deterrent Effect? New Evidence from Postmora-torium Panel Data.” American Law and Economics Review 5 (2): 344–76.
83. Donohue, John J. III, and Steven D. Levitt. 2001. “The Impact of Legalized Abortion on Crime.” Quarterly Journal of Economics 116 (2): 379–420.
84. Donohue, John J. III, and Steven D. Levitt. 2008. “Measurement Error, Legalized Abortion, and the Decline in Crime: A Response to Foote and Goetz.” Quarterly Journal of Economics 123 (1): 425–40.
85. Donohue, John J., and Justin Wolfers. 2010. “Uses and Abuses of Empirical Evidence in the Death Penalty Debate.” Stanford Law Review 58 (3): 791–845.
86. Doucouliagos, Chris. 2005. “Publication Bias in the Economic Freedom and Economic Growth Literature.” Journal of Economic Surveys 19 (3): 367–87.
87. Doucouliagos, Chris, and Patrice Laroche. 2003. “What Do Unions Do to Productivity? A Meta-analysis.” Industrial Relations 42 (4): 650–91.
88. Doucouliagos, Chris, and T. D. Stanley. 2013. “Are All Economic Facts Greatly Exaggerated? Theory Competition and Selectivity.” Journal of Economic Surveys 27 (2): 316–39.
89. Doucouliagos, Chris, T. D. Stanley, and Margaret Giles. 2012. “Are Estimates of the Value of a Statistical Life Exaggerated?” Journal of Health Economics 31 (1): 197–206.
90. Doucouliagos, Hristos, and T. D. Stanley. 2009. “Publication Selection Bias in Minimum-Wage Research? A Meta-regression Analysis.” British Journal of Industrial Relations 47 (2): 406–28.
91. Doucouliagos, Hristos, T. D. Stanley, and W. Kip Viscusi. 2014. “Publication Selection and the Income Elasticity of the Value of a Statistical Life.” Journal of Health Economics 33: 67–75.
92. Drazen, Jeffrey M. 2016. “Data Sharing and the Journal.” New England Journal of Medicine 374 (19): E24.
93. Dreber, Anna, et al. 2015. “Using Prediction Markets to Estimate the Reproducibility of Scientific Research.” Proceedings of the National Academy of Sciences 112 (50): 15343–47.
94. Drukker, David M., and Vince Wiggins. 2004. “Verifying the Solution from a Nonlinear Solver: A Case Study: Comment.” American Economic Review 94 (1): 397–99.
95. Dube, Arindrajit, T. William Lester, and Michael Reich. 2010. “Minimum Wage Effects across State Borders: Estimates Using Contiguous Counties.” Review of Economics and Statistics 92 (4): 945–64.
96. Duflo, Esther, Rachel Glennerster, and Michael Kremer. 2007. “Using Randomization in Development Economics Research: A Toolkit.” In Handbook of Development Economics: Volume 4, edited by T. Paul Schultz and John Strauss, 3895–962. Amsterdam and Boston: Elsevier, North-Holland.
97. Duvendack, Maren, and Richard W. Palmer-Jones. 2013. “Replication of Quantitative Work in Development Studies: Experiences and Suggestions.” Progress in Development Studies 13 (4): 307–22.
98. Duvendack, Maren, Richard W. Palmer-Jones, and W. Robert Reed. 2015. “Replications in Economics: A Progress Report.” Econ Journal Watch 12 (2): 164–91.
99. Dwork, Cynthia, and Adam Smith. 2010. “Differential Privacy for Statistics: What We Know and What We Want to Learn.” Journal of Privacy and Confidentiality 1 (2).
100. Easterbrook, P. J., R. Gopalan, J. A. Berlin, and D. R. Matthews. 1991. “Publication Bias in Clinical Research.” The Lancet 337 (8746): 867–72.
101. Easterly, William, Ross Levine, and David Roodman. 2004. “Aid, Policies, and Growth: A Comment.” American Economic Review 94 (3): 774–80.
102. Eble, Alex, Peter Boone, and Diana Elbourne. 2014. “On Minimizing the Risk of Bias in Randomized Controlled Trials in Economics.” SSRN Scholarly Paper 2272141.
103. Eich, Eric. 2014. “Business Not as Usual.” Psychological Science 25 (1): 3–6.
104. Enders, Walter, and Gary A. Hoover. 2004. “Whose Line Is It? Plagiarism in Economics.” Journal of Economic Literature 42 (2): 487–93.
105. Enders, Walter, and Gary A. Hoover. 2006. “Plagiarism in the Economics Profession: A Survey.” Challenge 49 (5): 92–107.
106. Epidemiology. 2010. “The Registration of Observational Studies—When Metaphors Go Bad.” Epidemiology 21 (5): 607–09.
107. European Journal of Social Psychology. 2016. “Retraction: ‘Correction or Comparison? The Effects of Prime Awareness on Social Judgments’, by M. Maringer and D. Stapel.” European Journal of Social Psychology 46 (1): 132.
108. Fanelli, Daniele. 2009. “How Many Scientists Fabricate and Falsify Research? A Systematic Review and Meta-analysis of Survey Data.” PloS ONE 4 (5): E5738.
109. Fanelli, Daniele. 2013. “Why Growing Retractions Are (Mostly) a Good Sign.” PloS Med 10 (12).
110. Fang, Ferric C., and Arturo Casadevall. 2011. “Retracted Science and the Retraction Index.” Infection and Immunity 79 (10): 3855–59.
111. Feldstein, Martin S. 1974. “Social Security, Induced Retirement, and Aggregate Capital Accumulation.” Journal of Political Economy 82 (5): 905–26.
112. Feldstein, Martin S. 1982. “Social Security and Private Saving: Reply.” Journal of Political Economy 90 (3): 630–42.
113. Fernández, Carmen, Eduardo Ley, and Mark F. J. Steel. 2001. “Model Uncertainty in Cross-Country Growth Regressions.” Journal of Applied Econometrics 16 (5): 563–76.
114. Findley, Michael G., Nathan M. Jensen, Edmund J. Malesky, and Thomas B. Pepinsky. 2016. “Can Results-Free Review Reduce Publication Bias? The Results and Implications of a Pilot Study.” Comparative Political Studies 49 (13): 1667–703.
115. Fink, Gunther, Margaret McConnell, and Sebastian Vollmer. 2014. “Testing for Heterogeneous Treatment Effects in Experimental Data: False Discovery Risks and Correction Procedures.” Journal of Development Effectiveness 6 (1): 44–57.
116. Finkelstein, Amy, et al. 2012. “The Oregon Health Insurance Experiment: Evidence from the First Year.” Quarterly Journal of Economics 127 (3): 1057–106.
117. Fleischmann, Martin, and Stanley Pons. 1989. “Electrochemically Induced Nuclear Fusion of Deuterium.” Journal of Electroanalytical Chemistry and Interfacial Electrochemistry 261 (2 Part 1): 301–08.
118. Food and Drug Administration. 1998. “Guidance for Industry: E9 Statistical Principles for Clinical Trials.” https://www.fda.gov/downloads/drugs/guidancecomplianceregulatoryinformation/guidances/ ucm073137.pdf.
119. Foote, Christopher L., and Christopher F. Goetz. 2008. “The Impact of Legalized Abortion on Crime: Comment.” Quarterly Journal of Economics 123 (1): 407–23.
120. Foster, Andrew, Dean Karlan, and Edward Miguel. 2018. “Registered Reports: Piloting a Pre-Results Review Process at the Journal of Development Economics.” World Bank Development Impact Blog, March 9, 2018. https://blogs.worldbank.org/impactevaluations/registered-reports-piloting-pre-results-review-process-journal-development-economics.
121. Franco, Annie, Neil Malhotra, and Gabor Simonovits. 2014. “Publication Bias in the Social Sciences: Unlocking the File Drawer.” Science 345 (6203): 1502–05.
122. Frey, René L., Bruno S. Frey, and Reiner Eichenberger. 1999. “A Case of Plagiarism.” Kyklos 52 (3): 311.
123. Fuess, Scott M., Jr. 1996. “On Replication in Business and Economics Research: The QJBE Case.” Quarterly Journal of Business and Economics 35 (2): 3–13.
124. Galbraith, R. F. 1988. “A Note on Graphical Presentation of Estimated Odds Ratios from Several Clinical Trials.” Statistics in Medicine 7 (8): 889–94.
125. Ganimian, Alejandro. 2014. “Pre-analysis Plan Template.” https://osf.io/exyb8/.
126. Gelman, Andrew, and Eric Loken. 2013. “The Garden of Forking Paths: Why Multiple Comparisons Can Be a Problem, Even When There Is No ‘Fishing Expedition’ or ‘p-Hacking’ and the Research Hypothesis Was Posited Ahead of Time.” http://www.stat.columbia.edu/~gelman/research/unpublished/p_hacking. pdf.
127. Gelman, Andrew, Cristian Pasarica, and Rahul Dodhia. 2002. “Let’s Practice What We Preach: Turning Tables into Graphs.” The American Statistician 56 (2): 121–30.
128. Gerber, Alan S., et al. 2014. “Reporting Guidelines for Experimental Research: A Report from the Experimental Research Section Standards Committee.” Journal of Experimental Political Science 1 (1): 81–98.
129. Gerber, Alan S., Donald P. Green, and David Nickerson. 2001. “Testing for Publication Bias in Political Science.” Political Analysis 9 (4): 385–92.
130. Gerber, Alan S., and Neil Malhotra. 2008a. “Do Statistical Reporting Standards Affect What Is Published? Publication Bias in Two Leading Political Science Journals.” Quarterly Journal of Political Science 3 (3): 313–26.
131. Gerber, Alan S., and Neil Malhotra. 2008b. “Publication Bias in Empirical Sociological Research: Do Arbitrary Significance Levels Distort Published Results?” Sociological Methods and Research 37 (1): 3–30.
132. Gerking, Shelby, and William E. Morgan. 2007. “Effects of Environmental and Land Use Regulation in the Oil and Gas Industry Using the Wyoming Checkerboard as a Natural Experiment: Retraction.” American Economic Review 97 (3): 1032.
133. Gherghina, Sergiu, and Alexia Katsanidou. 2013. “Data Availability in Political Science Journals.” European Political Science 12 (3): 333–49.
134. Gilbert, Christopher. 1989. “LSE and the British Approach to Time Series Econometrics.” Oxford Economic Papers 41 (1): 108–28.
135. Gilbert, Daniel T., Gary King, Stephen Pettigrew, and Timothy D. Wilson. 2016. “Comment on ‘Estimating the Reproducibility of Psychological Science’.” Science 351 (6277): 1037.
136. Glandon, Phillip. 2010. “Report on the American Economic Review Data Availability Compliance Project.” http://digital.kenyon.edu/cgi/viewcontent.cgi?article=1011&context=economics_publications.
137. Glennerster, Rachel, and Kudzai Takavarasha. 2013. Running Randomized Evaluations: A Practical Guide. Princeton and Oxford: Princeton University Press.
138. Görg, Holger, and Eric Strobl. 2001. “Multinational Companies and Productivity Spillovers: A Meta-analysis.” Economic Journal 111 (475): F723–39.
139. Grant, Sean, Paul Montgomery, Sally Hopewell, Geraldine Macdonald, David Moher, and Evan Mayo-Wilson. 2013. “Developing a Reporting Guideline for Social and Psychological Intervention Trials.” Journal of Experimental Criminology 9 (3): 355–67.
140. Hamermesh, Daniel S. 2007. “Viewpoint: Replication in Economics.” Canadian Journal of Economics 40 (3): 715–33.
141. Hansen, Peter Reinhard. 2005. “A Test for Superior Predictive Ability.” Journal of Business and Economic Statistics 23 (4): 365–80.
142. Harvey, Campbell R., Yan Liu, and Heqing Zhu. 2016. “. . . and the Cross-Section of Expected Returns.” Review of Financial Studies 29 (1): 5–68.
143. Hawkins, Carlee Beth, Cailey E. Fitzgerald, and Brian A. Nosek. 2015. “In Search of an Association between Conception Risk and Prejudice.” Psychological Science 26 (2): 249–52.
144. Hedges, Larry V. 1992. “Modeling Publication Selection Effects in Meta-analysis.” Statistical Science 7 (2): 246–55.
145. Hedges, Larry V., and Jack L. Vevea. 1996. “Estimating Effect Size under Publication Bias: Small Sample Properties and Robustness of a Random Effects Selection Model.” Journal of Educational and Behavioral Statistics 21 (4): 299–332.
146. Heffetz, Ori, and Katrina Ligett. 2014. “Privacy and Data-Based Research.” Journal of Economic Perspectives 28 (2): 75–98.
147. Hendry, David F. 1987. “Econometric Methodology: A Personal Perspective.” In Advances in Econometrics: Fifth World Congress: Volume II, edited by Truman F. Bewley, 29–48. Cambridge and New York: Cambridge University Press.
148. Hendry, David F. 1995. Dynamic Econometrics: Advanced Texts in Econometrics. Oxford and New York: Oxford University Press.
149. Henry, Emeric. 2009. “Strategic Disclosure of Research Results: The Cost of Proving Your Honesty.” Economic Journal 119 (539): 1036–64.
150. Henry, Emeric, and Marco Ottaviani. 2014. “Research and the Approval Process.” https://cepr.org/sites/default/files/Henry-submission%20CEPR.pdf.
151. Herndon, Thomas, Michael Ash, and Robert Pollin. 2014. “Does High Public Debt Consistently Stifle Economic Growth? A Critique of Reinhart and Rogoff.” Cambridge Journal of Economics 38 (2): 257–79.
152. Hicks, Joan Hamory, Michael Kremer, and Edward Miguel. 2015. “Commentary: Deworming Externalities and Schooling Impacts in Kenya: A Comment on Aiken et al. (2015) and Davey et al. (2015).” International Journal of Epidemiology 44, (5):1593–96
153. Hirshleifer, Sarojini, David McKenzie, Rita Almeida, and Cristobal Ridao-Cano. 2016. “The Impact of Vocational Training for the Unemployed: Experimental Evidence from Turkey.” Economic Journal 126 (597): 2115–46.
154. Holm, Sture. 1979. “A Simple Sequentially Rejective Multiple Test Procedure.” Scandinavian Journal of Statistics 6 (2): 65–70.
155. Hoxby, Caroline, M. 2000. “Does Competition among Public Schools Benefit Students and Taxpayers?” American Economic Review 90 (5): 1209–38.
156. Hoxby, Caroline, M. 2007. “Does Competition Among Public Schools Benefit Students and Taxpayers? Reply.” American Economic Review 97 (5): 2038–55.
157. Humphreys, Macartan, Raul Sanchez de la Sierra, and Peter van der Windt. 2013. “Fishing, Commitment, and Communication: A Proposal for Comprehensive Nonbinding Research Registration.” Political Analysis 21 (1): 1–20.
158. Hung, H. M. James, Robert T. O’Neill, Peter Bauer, and Karl Kohne. 1997. “The Behavior of the P-Value When the Alternative Hypothesis Is True.” Biometrics 53 (1): 11–22.
159. Hunter, John E. 2001. “The Desperate Need for Replications.” Journal of Consumer Research 28 (1): 149–58.
160. Husereau, Don, et al. 2013. “Consolidated Health Economic Evaluation Reporting Standards (CHEERS)—Explanation and Elaboration: A Report of the ISPOR Health Economic Evaluation Publication Guidelines Good Reporting Practices Task Force.” Value in Health 16 (2): 231–50.
161. Ifcher, John, and Homa Zarghamee. 2011. “Happiness and Time Preference: The Effect of Positive Affect in a Random-Assignment Experiment.” American Economic Review 101 (7): 3109–29.
162. Ioannidis, John P. A. 2005. “Why Most Published Research Findings Are False.” PloS Medicine 2 (8): E124.
163. Ioannidis, John P. A. 2008. “Effectiveness of Antidepressants: An Evidence Myth Constructed from a Thousand Randomized Trials?” Philosophy, Ethics, and Humanities in Medicine 3 (14).
164. Ioannidis, John P. A., T. D. Stanley, and Hristos Doucouliagos. 2017. “The Power of Bias in Economics Research.” Economic Journal 127 (605): F236–65.
165. ISIS-2 (Second International Study of Infarct Survival) Collaborative Group. 1988. “Randomised Trial of Intravenous Streptokinase, Oral Aspirin, Both, or Neither among 17 187 Cases of Suspected Acute Myocardial Infarction: ISIS-2.” The Lancet 332 (8607): 349–60.
166. John, Leslie K. George Loewenstein, and Drazen Prelec. 2012. “Measuring the Prevalence of Questionable Research Practices with Incentives for Truth Telling.” Psychological Science 23 (5): 524–32.
167. Journal of Economic Perspectives. 2011. “Correspondence: David H. Autor and Bruno S. Frey.” Journal of Economic Perspectives 25 (3): 239–40.
168. Journal of Economic Policy Reform. 2010. “Statement of Retraction.” Journal of Economic Policy Reform 13 (4): 387.
169. Jung, Kiju, Sharon Shavitt, Madhu Viswanathan, and Joseph M. Hilbe. 2014. “Female Hurricanes Are Deadlier than Male Hurricanes.” Proceedings of the National Academy of Sciences 111 (24): 8782–87.
170. Kane, Edward J. 1984. “Why Journal Editors Should Encourage the Replication of Applied Econometric Research.” Quarterly Journal of Business and Economics 23 (1): 3–8.
171. Karabag, Solmaz Filiz, and Christian Berggren. 2012. “Retraction, Dishonesty and Plagiarism: Analysis of a Crucial Issue for Academic Publishing, and the Inadequate Responses from Leading Journals in Economics and Management Disciplines.” Journal of Applied Economics and Business Research 2 (3): 172–83.
172. Katz, Larry, Esther Duflo, Pinelopi Goldberg, and Duncan Thomas. 2013. “AEA E-Mail Announcement.” https://aeaweb.org/announcements/20131118_rct_email.php.
173. King, Gary. 1995. “Replication, Replication.” PS: Political Science and Politics 28 (3): 444–52.
174. Kirsch, Irving, Brett J. Deacon, Tania B. Huedo-Me-dina, Alan Scoboria, Thomas J. Moore, and Blair T. Johnson. 2008. “Initial Severity and Antidepressant Benefits: A Meta-analysis of Data Submitted to the Food and Drug Administration.” PloS Medicine 5 (2): E45.
175. Klein, Joshua R., and Aaron Roodman. 2005. “Blind Analysis in Nuclear and Particle Physics.” Annual Review of Nuclear and Particle Science 55: 141–63.
176. Klein, Richard A., et al. 2014. “Investigating Variation in Replicability: A ‘Many Labs’ Replication Project.” Social Psychology 45 (3): 142–52.
177. Kleinberg, Jon, Himabindu Lakkaraju, Jure Lesk-ovec, Jens Ludwig, and Sendhil Mullainathan. 2015. “Human Decisions and Machine Problems.” Unpublished.
178. Kleinberg, Jon, Jens Ludwig, Sendhil Mullainathan, and Ziad Obermeyer. 2015. “Prediction Policy Problems.” American Economic Review 105 (5): 491–95.
179. Kling, Jeffrey R., Jeffrey B. Liebman, and Lawrence F. Katz. 2007. “Experimental Analysis of Neighborhood Effects.” Econometrica 75 (1): 83–119.
180. Knell, Markus, and Helmut Stix. 2005. “The Income Elasticity of Money Demand: A Meta-analysis of Empirical Results.” Journal of Economic Surveys 19 (3): 513–33.
181. Knittel, Christopher R., and Konstantinos Metaxoglou. 2011. “Challenges in Merger Simulation Analysis.” American Economic Review 101 (3): 56–59.
182. Knittel, Christopher R., and Konstantinos Metaxoglou. 2014. “Estimation of Random-Coefficient Demand Models: Two Empiricists’ Perspective.” Review of Economics and Statistics 96 (1): 34–59.
183. Knuth, Donald E. 1992. “Literate Programming.” Stanford: CSLI Publications.
184. Koenker, Roger, and Achim Zeileis. 2009. “On Reproducible Econometric Research.” Journal of Applied Econometrics 24 (5): 833–47.
185. Kunce, Mitch, Shelby Gerking, and William E. Morgan. 2002. “Effects of Environmental and Land Use Regulation in the Oil and Gas Industry Using the Wyoming Checkerboard as an Experimental Design.” American Economic Review 92 (5): 1588–93.
186. LaCour, Michael J., and Donald P. Green. 2014. “When Contact Changes Minds: An Experiment on Transmission of Support for Gay Equality.” Science 346 (6215): 1366–69.
187. Laine, Christine, et al. 2007. “Clinical Trial Registration—Looking Back and Moving Ahead.” New England Journal of Medicine 356 (26): 2734–36.
188. LaLonde, Robert J. 1986. “Evaluating the Econometric Evaluations of Training Programs with Experimental Data.” American Economic Review 76 (4): 604–20.
189. Leamer, Edward E. 1978. Specification Searches: Ad Hoc Inference with Nonexperimental Data. New York: Wiley.
190. Leamer, Edward E. 1983. “Let’s Take the Con Out of Econometrics.” American Economic Review 73 (1): 31–43.
191. Leamer, Edward E. 2010. “Tantalus on the Road to Asymptopia.” Journal of Economic Perspectives 24 (2): 31–46.
192. Leamer, Edward E. 2016. “S-Values: Conventional Context-Minimal Measures of the Sturdiness of Regression Coefficients.” Journal of Econometrics 193 (1): 147–61.
193. Leamer, Edward E., and Herman Leonard. 1983. “Reporting the Fragility of Regression Estimates.” Review of Economics and Statistics 65 (2): 306–17.
194. Lee, Soohyung, and Azeem M. Shaikh. 2014. “Multiple Testing and Heterogeneous Treatment Effects: Re-evaluating the Effect of Progresa on School Enrollment.” Journal of Applied Econometrics 29 (4): 612–26.
195. Leimer, Dean R., and Selig D. Lesnoy. 1982. “Social Security and Private Saving: New Time-Series Evidence.” Journal of Political Economy 90 (3): 606–29.
196. Levine, David I. 2001. “Editor’s Introduction to ‘The Unemployment Effects of Minimum Wages: Evidence from a Prespecified Research Design’.” Industrial Relations 40 (2): 161–62.
197. Lewis, N. S., et al. 1989. “Searches for Low-Temperature Nuclear Fusion of Deuterium in Palladium.” Nature 340 (6234): 525–30.
198. Libgober, Jonathan. 2015. “False Positives in Scientific Research.” https://papers.ssrn.com/sol3/papers.cfm?abstract_id=2617130.
199. Light, Richard J., and David B. Pillemer. 1984. Summing Up: The Science of Reviewing Research. Cambridge and London: Harvard University Press.
200. Lindsay, D. Stephen. 2015. “Replication in Psychological Science.” Psychological Science 26 (12): 1827–32.
201. List, John A., C. D. Bailey, P. J. Euzent, and T. L. Martin. 2001. “Academic Economists Behaving Badly? A Survey on Three Areas of Unethical Behavior.” Economic Inquiry 39 (1): 162–70.
202. List, John A., Azeem M. Shaikh, and Yang Xu. 2016. “Multiple Hypothesis Testing in Experimental Economics.” National Bureau of Economic Research Working Paper 21875.
203. Loder, Elizabeth, Trish Groves, and Domhnall MacAuley. 2010. “Registration of Observational Studies.” British Medical Journal 340: C950.
204. Longhi, Simonetta, Peter Nijkamp, and Jacques Poot. 2005. “A Meta-analytic Assessment of the Effect of Immigration on Wages.” Journal of Economic Surveys 19 (3): 451–77.
205. Longo, Dan L., and Jeffrey M. Drazen. 2016. “Data Sharing.” New England Journal of Medicine 374 (3): 276–77.
206. Lovell, Michael C. 1983. “Data Mining.” Review of Economics and Statistics 65 (1): 1–12.
207. Lundqvist, Heléne, Matz Dahlberg, and Eva Mörk. 2014. “Stimulating Local Public Employment: Do General Grants Work?” American Economic Journal: Economic Policy 6 (1): 167–92.
208. MacCoun, Robert, and Saul Perlmutter. 2015. “Blind Analysis: Hide Results to Seek the Truth.” Nature 526 (7572): 187–89.
209. Maggioni, Aldo P., Bernadette Darne, Dan Atar, Eric Abadie, Bertram Pitt, and Faiez Zannad. 2007. “FDA and CPMP Rulings on Subgroup Analyses.” Cardiology 107 (2): 97–102.
210. Maley, Steve. 2014. “Statistics Show No Evidence of Gender Bias in the Public’s Hurricane Preparedness.” Proceedings of the National Academy of Sciences 111 (37): E3834.
211. Malin, Bradley, Kathleen Benitez, and Daniel Masys. 2011. “Never Too Old for Anonymity: A Statistical Standard for Demographic Data Sharing via the HIPAA Privacy Rule.” Journal of the American Medical Informatics Association 18 (1): 3–10.
212. Malter, Daniel. 2014. “Female Hurricanes Are Not Deadlier than Male Hurricanes.” Proceedings of the National Academy of Sciences 111 (34): E3496.
213. Mansfield, Edwin, Mark Schwartz, and Samuel Wagner. 1981. “Imitation Costs and Patents: An Empirical Study.” Economic Journal 91 (364): 907–18.
214. Maringer, Marcus, and Diederik A. Stapel. 2009. “Retracted: Correction or Comparison? The Effects of Prime Awareness on Social Judgments.” European Journal of Social Psychology 39 (5): 719–33.
215. Mathieu, Sylvain, Isabelle Boutron, David Moher, Douglas G. Altman, and Philippe Ravaud. 2009. “Comparison of Registered and Published Primary Outcomes in Randomized Controlled Trials.” Journal of the American Medical Association 302 (9): 977–84.
216. McAleer, Michael, Adrian R. Pagan, and Paul A. Volker. 1985. “What Will Take the Con Out of Econometrics?” American Economic Review 75 (3): 293–307.
217. McCloskey, Deirdre N., and Stephen T. Ziliak. 1996. “The Standard Error of Regressions.” Journal of Economic Literature 34 (1): 97–114.
218. McCrary, Justin, Garret Christensen, and Daniele Fanelli. 2016. “Conservative Tests under Satisficing Models of Publication Bias.” PloS One 11 (2).
219. McCullough, B. D. 2007. “Got Replicability? The Journal of Money, Credit and Banking Archive.” Econ Journal Watch 4 (3): 326–37.
220. McCullough, B. D. 2009. “Open Access Economics Journals and the Market for Reproducible Economic Research.” Economic Analysis and Policy 39 (1): 117–26.
221. McCullough, B. D., Kerry Anne McGeary, and Teresa D. Harrison. 2006. “Lessons from the JMCB Archive.” Journal of Money, Credit, and Banking 38 (4): 1093–107.
222. McCullough, B. D., Kerry Anne McGeary, and Teresa D. Harrison. 2008. “Do Economics Journal Archives Promote Replicable Research?” Canadian Journal of Economics 41 (4): 1406–20.
223. McCullough, B. D., and H. D. Vinod. 2003. “Verifying the Solution from a Nonlinear Solver: A Case Study.” American Economic Review 93 (3): 873–92.
224. McCullough, B. D., and H. D. Vinod. 2004a. “Verifying the Solution from a Nonlinear Solver: A Case Study: Reply.” American Economic Review 94 (1): 391–96.
225. McCullough, B. D., and H. D. Vinod. 2004b. “Verifying the Solution from a Nonlinear Solver: A Case Study: Reply.” American Economic Review 94 (1): 400–406.
226. McManus, Walter S. 1985. “Estimates of the Deterrent Effect of Capital Punishment: The Importance of the Researcher’s Prior Beliefs.” Journal of Political Economy 93 (2): 417–25.
227. McNutt, Marcia. 2015. “Editorial Retraction.” Science 348 (6239): 1100.
228. McNutt, Marcia. 2016. “Taking Up TOP.” Science 352 (6290): 1147.
229. Merton, Robert K. 1973. The Sociology of Science: Theoretical and Empirical Investigations. Chicago and London: University of Chicago Press.
230. Mervis, Jeffrey. 2014a. “How Two Economists Got Direct Access to IRS Tax Records.” http://www.sciencemag.org/news/2014/05/how-two-economists-got-direct-access-irs-tax-records.
231. Mervis, Jeffrey. 2014b. “Why Null Results Rarely See the Light of Day.” Science 345 (6200): 992.
232. Miguel, Edward, et al. 2014. “Promoting Transparency in Social Science Research.” Science 343 (6166): 30–31.
233. Miguel, Edward, and Michael Kremer. 2004. “Worms: Identifying Impacts on Education and Health in the Presence of Treatment Externalities.” Econometrica 72 (1): 159–217.
234. Miguel, Edward, and Shanker Satyanath. 2011. “Re-examining Economic Shocks and Civil Conflict.” American Economic Journal: Applied Economics 3 (4): 228–32.
235. Miguel, Edward, Shanker Satyanath, and Ernest Sergenti. 2004. “Economic Shocks and Civil Conflict: An Instrumental Variables Approach.” Journal of Political Economy 112 (4): 725–53.
236. Mittelstaedt, Robert A., and Thomas S. Zorn. 1984. “Econometric Replication: Lessons from the Experimental Sciences.” Quarterly Journal of Business and Economics 23 (1): 9–15.
237. Mizon, Grayham E., and Jean-Francois Richard. 1986. “The Encompassing Principle and Its Application to Testing Non-nested Hypotheses.” Econometrica 54 (3): 657–78.
238. Moffitt, Robert A. 2016. “In Defence of the NSF Economics Program.” Journal of Economic Perspectives 30 (3): 213–34.
239. Moher, David, Alison Jones, Leah Lepage, and the CONSORT Group. 2001. “Use of the CONSORT Statement and Quality of Reports of Randomized Trials: A Comparative Before-and-After Evaluation.” Journal of the American Medical Association 285 (15): 1992–95.
240. Moher, David, Kenneth F. Schulz, and Douglas G. Altman. 2001. “The CONSORT Statement: Revised Recommendations for Improving the Quality of Reports of Parallel Group Randomized Trials.” BMC Medical Research Methodology 1:2.
241. Monogan, James E. 2013. “A Case for Registering Studies of Political Outcomes: An Application in the 2010 House Elections.” Political Analysis 21 (1): 21–37.
242. Montgomery, Paul, et al. 2013. “Protocol for CONSORT-SPI: An Extension for Social and Psychological Interventions.” Implementation Science 8 (1): 99.
243. Mookerjee, Rajen. 2006. “A Meta-analysis of the Export Growth Hypothesis.” Economics Letters 91 (3): 395–401.
244. Morey, Richard D., et al. 2016. “The Peer Reviewers’ Openness Initiative: Incentivizing Open Research Practices through Peer Review.” http://rsos.royalsocietypublishing.org/content/royopensci/3/1/150547. full.pdf.
245. Necker, Sarah. 2014. “Scientific Misbehavior in Economics.” Research Policy 43 (10): 1747–59.
246. Neumark, David. 2001. “The Employment Effects of Minimum Wages: Evidence from a Prespecified Research Design.” Industrial Relations: A Journal of Economy and Society 40 (1): 121–44.
247. Neumark, David, J. M. Ian Salas, and William Wascher. 2014. “Revisiting the Minimum Wage–Employment Debate: Throwing Out the Baby with the Bathwater?” ILR Review 67 (Supplement 3): 608–48.
248. Neumark, David, and William Wascher. 1998. “Is the Time-Series Evidence on Minimum Wage Effects Contaminated By Publication Bias?” Economic Inquiry 36 (3): 458–70
249. Neumark, David, and William Wascher. 2000. “Minimum Wages and Employment: A Case Study of the Fast-Food Industry in New Jersey and Pennsylvania: Comment.” American Economic Review 90 (5): 1362–96.
250. Nijkamp, Peter, and Jacques Poot. 2005. “The Last Word on the Wage Curve?” Journal of Economic Surveys 19 (3): 421–50.
251. Nofsinger, John R. 2009. “Retraction Notice to ‘Social Mood: The Stock Market and Political Cycles’ [J. Socio-Econ. 36 (2007) 734–44].” Journal of Socio-Economics 38 (3): 547.
252. Nosek, Brian A., and Daniël Lakens. 2014. “Registered Reports: A Method to Increase the Credibility of Published Results.” Social Psychology 45 (3): 137–41.
253. Nosek, Brian A., Jeffrey R. Spies, and Matt Motyl. 2012. “Scientific Utopia: II. Restructuring Incentives and Practices to Promote Truth over Publishability.” Perspectives on Psychological Science 7 (6): 615–31.
254. Nosek, Brian A., et al. 2015. “Promoting an Open Research Culture.” Science 348 (6242): 1422–25.
255. O’Brien, Peter C. 1984. “Procedures for Comparing Samples with Multiple Endpoints.” Biometrics 40 (4): 1079–87.
256. Olken, Benjamin A. 2015. “Promises and Perils of Pre-analysis Plans.” Journal of Economic Perspectives 29 (3): 61–80.
257. Olken, Benjamin A., Junko Onishi, and Susan Wong. 2012. “Should Aid Reward Performance? Evidence from a Field Experiment on Health and Education in Indonesia.” National Bureau of Economic Research Working Paper 17892.
258. Open Science Collaboration. 2012. “An Open, Large-Scale, Collaboration Effort to Estimate the Reproducibility of Psychological Science.” Psychological Science 7 (6): 657–60.
259. Open Science Collaboration. 2015. “Estimating the Reproducibility of Psychological Science.” Science 349 (6251): Aac4716.
260. Pagan, Adrian. 1987. “Three Econometric Methodologies: A Critical Appraisal.” Journal of Economic Surveys 1 (1–2): 3–23.
261. Patil, Prasad, Roger D. Peng, and Jeffrey T. Leek. 2016. “What Should Researchers Expect When They Replicate Studies? A Statistical View of Replicability in Psychological Science.” Perspectives on Psychological Science 11 (4): 539–44.
262. Pesaran, Hashem. 2003. “Introducing a Replication Section.” Journal of Applied Econometrics 18 (1): 111.
263. Phillips, P. C. B. 1988. “Reflections on Econometric Methodology.” Economic Record 64 (4): 344–59.
264. Quarterly Journal of Economics. 1984. “Notice to Our Readers.” Quarterly Journal of Economics 99 (2): 383–84.
265. Regional Studies. 2009. “Retraction Statement and Authors’ Apology.” Regional Studies 43 (1): 156.
266. Regional Studies. 2011. “Redundant Publishing.” Regional Studies 45 (2): 282.
267. Reinhart, Carmen M., and Kenneth S. Rogoff. 2010. “Growth in a Time of Debt.” American Economic Review 100 (2): 573–78.
268. Roberts, Colin J. 2005. “Issues in Meta-regression Analysis: An Overview.” Journal of Economic Surveys 19 (3): 295–98.
269. Romano, Joseph P., Azeem M. Shaikh, and Michael Wolf. 2008. “Control of the False Discovery Rate under Dependence Using the Bootstrap and Subsa-mpling.” TEST 17: 417.
270. Rose, Andrew K., and T. D. Stanley. 2005. “A Meta-analysis of the Effect of Common Currencies on International Trade.” Journal of Economic Surveys 19 (3): 347–65.
271. Rosenthal, Robert. 1979. “The ‘File Drawer Problem’ and Tolerance for Null Results.” Psychological Bulletin 86 (3): 638–41.
272. Rothstein, Jesse. 2007. “Does Competition Among Public Schools Benefit Students and Taxpayers? Comment.” American Economic Review 97 (5): 2026–37.
273. Sachs, Jeffrey D., and Andrew M. Warner. 1997. “Natural Resource Abundance and Economic Growth.” https://pdfs.semanticscholar.org/7b14/045909f42117 197b82a910782ab68330a3e7.pdf.
274. Sala-i-Martin, Xavier. 1997. “I Just Ran Two Million Regressions.” American Economic Review 87 (2): 178–83.
275. Sala-i-Martin, Xavier, Gernot Doppelhofer, and Ronald I. Miller. 2004. “Determinants of Long-Term Growth: A Bayesian Averaging of Classical Estimates (BACE) Approach.” American Economic Review 94 (4): 813–35.
276. Schulz, Kenneth F., Douglas G. Altman, and David Moher. 2010. “CONSORT 2010 Statement: Updated Guidelines for Reporting Parallel Group Randomised Trials.” British Medical Journal 340: 332.
277. Schulz, Kenneth F., and David A. Grimes. 2005. “Multiplicity in Randomised Trials II: Subgroup and Interim Analyses.” The Lancet 365 (9471): 1657–61.
278. Schwabish, Jonathan A. 2014. “An Economist’s Guide to Visualizing Data.” Journal of Economic Perspectives 28 (1): 209–34.
279. Schwarz, Gideon. 1978. “Estimating the Dimension of a Model.” Annals of Statistics 6 (2): 461–64.
280. Shachar, Ron, and Barry Nalebuff. 2004. “Verifying the Solution from a Nonlinear Solver: A Case Study: Comment.” American Economic Review 94 (1): 382–90.
281. Shen, Helen. 2014. “Interactive Notebooks: Sharing the Code.” Nature 515 (7525): 151–52.
282. Silberzahn, Raphael, and Eric L. Uhlmann. 2015. “Crowdsourced Research: Many Hands Make Tight Work.” Nature 526 (7572): 189–91.
283. Simmons, Joseph P., Leif D. Nelson, and Uri Simonsohn. 2011. “False-Positive Psychology: Undisclosed Flexibility in Data Collection and Analysis Allows Presenting Anything as Significant.” Psychological Science 22 (11): 1359–66.
284. Simmons, Joseph P., Leif D. Nelson, and Uri Simonsohn. 2012. “A 21 Word Solution.” Dialogue 26 (2): 4–7.
285. Simonsohn, Uri. 2013. “Just Post It: The Lesson from Two Cases of Fabricated Data Detected By Statistics Alone.” Psychological Science 24 (10): 1875–88.
286. Simonsohn, Uri. 2015. “Small Telescopes: Detectability and the Evaluation of Replication Results.” Psychological Science 26 (5): 559–69.
287. Simonsohn, Uri. 2017. “[58] The Funnel Plot Is Invalid: Because of This Crazy Assumption: R(n,d)=0.” http://datacolada.org/58.
288. Simonsohn, Uri, Leif D. Nelson, and Joseph P. Simmons. 2014a. “P-Curve: A Key to the File-Drawer.” Journal of Experimental Psychology: General 143 (2): 534–47.
289. Simonsohn, Uri, Leif D. Nelson, and Joseph P. Simmons. 2014b. “p-Curve and Effect Size: Correcting for Publication Bias Using Only Significant Results.” Perspectives on Psychological Science 9 (6): 666–81.
290. Simonsohn, Uri, Joseph P. Simmons, and Leif D. Nelson. 2015a. “Better P-Curves: Making P-Curve Analysis More Robust to Errors, Fraud, and Ambitious P-Hacking, a Reply to Ulrich and Miller (2015).” Journal of Experimental Psychology: General 144 (6): 1146–52.
291. Simonsohn, Uri, Joseph P. Simmons, and Leif D. Nelson. 2015b. “Specification Curve: Descriptive and Inferential Statistics on All Reasonable Specifications.” SSRN Scholarly Paper ID 2694998.
292. Sinisi, Sandra E., Eric C. Polley, Maya L. Petersen, Soo-Yon Rhee, and Mark J. van der Laan. 2007. “Super Learning: An Application to the Prediction of the HIV-1 Drug Resistance.” Statistical Applications in Genetics and Molecular Biology 6 (1).
293. Siskind, Frederic B. 1977. “Minimum Wage Legislation in the United States: Comment.” Economic Inquiry 15 (1): 135–38.
294. Stanley, T. D. 2005. “Beyond Publication Bias.” Journal of Economic Surveys 19 (3): 309–45.
295. Stanley, T. D. 2008. “Meta-regression Methods for Detecting and Estimating Empirical Effects in the Presence of Publication Selection.” Oxford Bulletin of Economics and Statistics 70 (1): 103–27.
296. Stanley, T. D., and Hristos Doucouliagos. 2010. “Picture This: A Simple Graph That Reveals Much Ado about Research.” Journal of Economic Surveys 24 (1): 170–91.
297. Stanley, T. D., and Hristos Doucouliagos. 2012. Meta-Regression Analysis in Economics and Business. London and New York: Taylor and Francis: Routledge.
298. Stanley, T. D., et al. 2013. “Meta-analysis of Economics Research Reporting Guidelines.” Journal of Economic Surveys 27 (2): 390–94.
299. Steen, R. Grant. 2011. “Retractions in the Scientific Literature: Do Authors Deliberately Commit Research Fraud?” Journal of Medical Ethics 37 (2): 65.
300. Steen, R. Grant, Arturo Casadevall, and Ferric C. Fang. 2013. “Why Has the Number of Scientific Retractions Increased?” PLoS ONE 8 (7): E68397.
301. Sterling, Theodore D. 1959. “Publication Decisions and Their Possible Effects on Inferences Drawn from Tests of Significance—or Vice Versa.” Journal of the American Statistical Association 54 (285): 30–34.
302. Stodden, Victoria, Friedrich Leisch, and Roger D. Peng. 2014. Implementing Reproducible Research. Boca Raton: Taylor and Francis, CRC Press.
303. Sullivan, Ryan, Allan Timmermann, and Halbert White. 1999. “Data-Snooping, Technical Trading Rule Performance, and the Bootstrap.” Journal of Finance 54 (5): 1647–91.
304. Sweeney, Latanya. 2002. “k-Anonymity: A Model for Protecting Privacy.” International Journal on Uncertainty, Fuzziness and Knowledge-Based Systems 10 (5): 557–70.
305. Taichman, Darren B., et al. 2016. “Sharing Clinical Trial Data: A Proposal from the International Committee of Medical Journal Editors.” Journal of the American Medical Association 315 (5): 467–68.
306. Taubman, Sarah L., Heidi L. Allen, Bill J. Wright, Katherine Baicker, and Amy N. Finkelstein. 2014. “Medicaid Increases Emergency-Department Use: Evidence from Oregon’s Health Insurance Experiment.” Science 343 (6168): 263–68.
307. The International Consortium of Investigators for Fairness in Trial Data Sharing. 2016. “Toward Fairness in Data Sharing.” New England Journal of Medicine 375 (5): 405–07.
308. The Lancet. 2010. “Should Protocols for Observational Research Be Registered?” The Lancet 375 (9712): 348.
309. Thomas, Duncan, et al. 2003. “Iron Deficiency and the Well-Being of Older Adults: Early Results from a Randomized Nutrition Intervention.” In Population Association of America Annual Meetings, Minneapolis.
310. Thomas, Duncan, et al. 2006. “Causal Effect of Health on Labor Market Outcomes: Experimental Evidence.” California Center for Population Research.
311. Tufte, Edward R. 2001. The Visual Display of Quantitative Information. Cheshire: Graphics Press.
312. Turner, Erick H., Annette M. Matthews, Eftihia Linar-datos, Robert A. Tell, and Robert Rosenthal. 2008. “Selective Publication of Antidepressant Trials and Its Influence on Apparent Efficacy.” New England Journal of Medicine 358 (3): 252–60.
313. Ulrich, Rolf, and Jeff Miller. 2015. “p-Hacking By Post Hoc Selection with Multiple Opportunities: Detectability By Skewness Test?: Comment on Simonsohn, Nelson, and Simmons (2014).” Journal of Experimental Psychology: General 144 (6): 1137–45.
314. van der Laan, Mark J., Eric C. Polley, and Alan E. Hubbard. 2007. “Super Learner.” Statistical Applications in Genetics and Molecular Biology 6 (1).
315. Viscusi, W. Kip. 2015. “The Role of Publication Selection Bias in Estimates of the Value of a Statistical Life.” American Journal of Health Economics 1 (1): 27–52.
316. Vivalt, Eva. 2015. “The Trajectory of Specification Searching and Publication Bias across Methods and Disciplines.” http://evavivalt.com/wp-content/uploads/2015/09/Trajectory-of-Specification-Search-ing.pdf.
317. von Elm, Erik, Douglas G. Altman, Matthias Egger, Stuart J. Pocock, Peter C. Gøtzsche, and Jan P. Vanden-broucke. 2007. “The Strengthening the Reporting of Observational Studies in Epidemiology (STROBE) Statement: Guidelines for Reporting Observational Studies.” Preventive Medicine 45 (4): 247–51.
318. Wacholder, Sholom, Stephen Chanock, Montserrat Garcia-Closas, Laure El ghormli, and Nathaniel Rothman. 2004. “Assessing the Probability That a Positive Report Is False: An Approach for Molecular Epidemiology Studies.” Journal of the National Cancer Institute 96 (6): 434–42.
319. Walsh, Elias, Sarah Dolfin, and John DiNardo. 2009. “Lies, Damn Lies, and Pre-election Polling.” American Economic Review 99 (2): 316–22.
320. Warren, Elizabeth. 2016. “Strengthening Research through Data Sharing.” New England Journal of Medicine 375 (5): 401–03.
321. Welch, Finis. 1974. “Minimum Wage Legislation in the United States.” Economic Inquiry 12 (3): 285–318.
322. Welch, Finis. 1977. “Minimum Wage Legislation in the United States: Reply.” Economic Inquiry 15 (1): 139–42.
323. Westfall, Peter H., and S. Stanley Young. 1993. Resam-pling-Based Multiple Testing: Examples and Methods for p-Value Adjustment. New York and Chichester: John Wiley and Sons.
324. White, Halbert. 2000. “A Reality Check for Data Snooping.” Econometrica 68 (5): 1097–126.
325. Wydick, Bruce, Elizabeth Katz, and Brendan Janet. 2014. “Do In-Kind Transfers Damage Local Markets? The Case of TOMS Shoe Donations in El Salvador.” Journal of Development Effectiveness 6 (3): 249–67.
326. Xie, Yihui. 2013. Dynamic Documents with R and knitr. New York and London: Taylor and Francis: CRC Press.
327. Xie, Yihui. 2014. “knitr: A Comprehensive Tool for Reproducible Research in R.” In Implementing Reproducible Research, edited by Victoria Stodden, Friedrich Leisch, and Roger D. Peng, 3–32. Boca Raton: Taylor and Francis: CRC Press.
328. Ziliak, Stephen T., and Deirdre N. McCloskey. 2004. “Size Matters: The Standard Error of Regressions in the American Economic Review.” Journal of Socio-Economics 33 (5): 527–46.
329. Ziliak, Stephen T., and Deirdre N. McCloskey. 2008. The Cult of Statistical Significance: How the Standard Error Costs Us Jobs, Justice, and Lives. Ann Arbor: University of Michigan Press.
330. Zimmermann, Christian. 2015. “On the Need for a Replication Journal.” Federal Reserve Bank of St. Louis Research Division Working Paper 2015-016A.