Picture this: A simple graph that reveals much ado about research

Journal of Economic Surveys

Volumn 24, Issue 1, 2010, Pages 170-191

  Stanley, T D ^a   Doucouliagos, Hristos ^b  

^a HENDRIX COLLEGE   (United States)

^b DEAKIN UNIVERSITY   (Australia)

Author keywords

Empirical economics; Funnel graphs; Meta regression analysis; Precision; Publication bias

Indexed keywords

ECONOMICS; EMPIRICAL ANALYSIS; GRAPHICAL METHOD; REGRESSION ANALYSIS;

EID: 73549117223     PISSN: 09500804     EISSN: 14676419     Source Type: Journal    
DOI: 10.1111/j.1467-6419.2009.00593.x     Document Type: Article

References (37)

1. Akerlof GA. Labor contracts as partial gift exchange. Quarterly Journal of Economics 1982, 97:543-569.
2. Akerlof GA. Behavioral macroeconomics and macroeconomic behavior. American Economic Review 2002, 92:411-433.
3. Bateman IJ, Jones AP. Contrasting conventional with multi-level modeling approaches to meta-analysis: expectation consistency in U.K. woodland recreation values. Land Economics 2003, 79:235-258.
4. Card D, Krueger AB. Myth and Measurement: The New Economics of the Minimum Wage 1995a, Princeton, NJ, Princeton University Press
5. Card D, Krueger AB. Time-series minimum-wage studies: a meta-analysis. American Economic Review 1995b, 85:238-243.
6. Dalhuisen J, Florax RJGM, deGroot HLF, Nijkamp P. Price and income elasticities of residential water demand: a meta-analysis. Land Economics 2003, 79:292-308.
7. Davidson R, MacKinnon JG. Econometric Theory and Methods 2004, Oxford, Oxford University Press
8. Djankov S, Murrell P. Enterprise restructuring in transition: a quantitative survey. Journal of Economic Literature 2002, XL:739-792.
9. Doucouliagos CH. Worker participation and productivity in labor-managed and participatory capitalist firms: a meta-analysis. Industrial and Labor Relations Review 2005, 49:58-77.
10. Doucouliagos CH, Laroche P. What do unions do to productivity: a meta-analysis. Industrial Relations 2003, 42:650-691.
11. Doucouliagos CH, Stanley TD. Theory competition and selectivity. 2008a, Deakin Working Paper 2008-06
12. Doucouliagos CH, Stanley TD. A tale of two biases. 2008b, Paper presented at the International Workshop on Meta-Analysis in Economics and Business, Nancy, France, October
13. Doucouliagos CH, Stanley TD. Publication selection bias in minimum-wage research? A meta-regression analysis. British Journal of Industrial Relations 2009, 47:406-428.
14. Friedman M. Do old fallacies ever die?. Journal of Economic Literature 1992, 30:2129-2132.
15. Galton F. Natural Inheritance 1889, London, Macmillan
16. Gemmill MC, Costa-Font J, McGuire A. In search of a corrected prescription drug elasticity estimate: a meta-regression approach. Health Economics 2007, 16:627-643.
17. Krakovsky M. Register or perish. Scientific American 2004, 291(Dec.):18-20.
18. Lau J, Ioannidis JPL, Terin N, Schmid CH, Olkin I. The case of the misleading funnel plot. British Medical Journal 2006, 333:597-600.
19. Light RJ, Pillemer DB. Summing Up: The Science of Reviewing Research 1984, Cambridge, MA, Harvard University Press
20. Marey EJ. La Méthode Graphique Dans les Sciences Expérimentales et Principalement en Physiologie et en Médecine 1885, 2nd edn, Paris, Masson
21. Moreno SG, Sutton AJ, Ades A, Stanley TD, Abrams KR, Peters JL, Cooper NJ. Assessment of regression-based methods to adjust for publication bias through a comprehensive simulation study. BMC Medical Research Methodology 2009, 9:2. http://www.biomedcentral.com/1471-2288/9/2, Available at, last accessed 22 July 2009
22. Roberts CJ, Stanley TD. Meta-Regression Analysis: Issues of Publication Bias in Economics 2005, eds, Oxford, Blackwell
23. Rose AK. One money, one market: estimating the effect of common currencies on trade. Economic Policy 2000, 30:9-45.
24. Rose AK, Stanley TD. A meta-analysis of the effect of common currencies on international trade. Journal of Economic Surveys 2005, 19:347-365.
25. Stanley TD. Galton, Stein, and empirical Bayes: artifacts and connections. Statistica 1989, XLIX:499-511.
26. Stanley TD. Wheat from chaff: meta-analysis as quantitative literature review. Journal of Economic Perspectives 2001, 15:131-150.
27. Stanley TD. Beyond publication selection. Journal of Economic Surveys 2005, 19:309-345.
28. Stanley TD. Meta-regression methods for detecting and estimating empirical effect in the presence of publication bias. Oxford Bulletin of Economics and Statistics 2008, 70:103-127.
29. Stanley TD, Doucouliagos C. Identifying and correcting publication selection bias in the efficiency-wage literature: Heckman meta-regression. 2007, School Working Paper, Economics Series 2007-11, Deakin University
30. Stanley TD, Jarrell SB. Meta-regression analysis: a quantitative method of literature surveys. Journal of Economic Surveys 1989, 3:54-67.
31. Stanley TD, Jarrell SB. Gender wage discrimination bias? A meta-regression analysis. Journal of Human Resources 1998, 33:947-973.
32. Stanley TD, Krassoi Peach E. Efficiency wages, productivity and simultaneity: a meta-regression analysis. Journal of Labor Research 2009, http://www.springerlink.com/content/110581/?Content+Status=Accepted, Available at, last accessed 22 July 2009
33. Stanley TD, Rosenberger RS. Are recreation values systematically underestimated? Reducing publication selection bias for benefit transfer. 2009, http://www.hendrix.edu/maer-network/default.aspx?id=15206, Available at, last accessed 22 July 2009
34. Sterne JAC, Egger M, Smith GD. Investigating and dealing with publication and other biases in meta-analysis. British Medical Journal 2001, 14:101-105.
35. Sutton AJ, Abrams KR, Jones DR, Sheldon TA, Song F. Methods for Meta-analysis in Medical Research 2000, Chichester, Wiley
36. Terrin N, Schmid CH, Lau J. In an empirical evaluation of the funnel plot, researchers could not visually identify publication bias. Journal of Clinical Epidemiology 2005, 58:894-901.
37. Tufte ER. The Visual Display of Quantitative Information 2001, 2nd edn, Cheshire, CT, Graphics Press