What If We Were Texas Sharpshooters? Predictor Reporting Bias in Regression Analysis

Organizational Research Methods

Volumn 16, Issue 3, 2013, Pages 335-363

  Biemann, Torsten ^a  

^a UNIVERSITY OF MANNHEIM   (Germany)

Author keywords

funnel plots; meta analysis; publication bias; regression; reporting bias

Indexed keywords

EID: 84878765822     PISSN: 10944281     EISSN: 15527425     Source Type: Journal    
DOI: 10.1177/1094428113485135     Document Type: Article

References (77)

1. Aguinis H.,Dalton D. R.,Bosco F. A.,Pierce C. A.,Dalton C. M.Meta-analytic choices and judgment calls: Implications for theory building and testing, obtained effect sizes, and scholarly impact.Journal of Management. 2011;37 (1): 5-38.
2. Aguinis H.,Pierce C. A.,Bosco F. A.,Dalton D. R.,Dalton C. M.Debunking myths and urban legends about meta-analysis.Organizational Research Methods. 2011;14 (2): 306-331.
3. Atinc G.,Simmering M. J.,Kroll M. J.Control variable use and reporting in macro and micro management research.Organizational Research Methods. 2012;15:57-74.
4. Banks G. C.,Kepes S.,Banks K. P.Publication bias the antagonist of meta-analytic reviews and effective policymaking.Educational Evaluation and Policy Analysis. 2012;34 (3): 259-277.
5. Barrick M. R.,Mount M. K.The Big Five personality dimensions and job performance: A meta-analysis.Personnel Psychology. 1991;44 (1): 1-26.
6. Becker T. E.Potential problems in the statistical control of variables in organizational research: A qualitative analysis with recommendations.Organizational Research Methods. 2005;8 (3): 274-289.
7. Bedeian A. G.,Taylor S. G.,Miller A. N.Management science on the credibility bubble: Cardinal sins and various misdemeanors.Academy of Management Learning and Education. 2010;9 (4): 715-725.
8. Begg C. B.,Mazumdar M.Operating characteristics of a rank correlation test for publication bias.Biometrics. 1994;50 (4): 1088-1101.
9. Bentler P. M.On tests and indices for evaluating structural models.Personality and Individual Differences. 2007;42 (5): 825-829.
10. Biemann T.,Heidemeier H.Does excluding some groups from research designs improve statistical power?.Small Group Research. 2012;43 (4): 387-409.
11. Blank H.,Musch J.,Pohl R. F.Hindsight bias: On being wise after the event.Social Cognition. 2007;25 (1): 1-9.
12. Chalmers T. C.,Levin H.,Sacks H. S.,Reitman D.,Berrier J.,Nagalingam R.Meta-analysis of clinical trials as a scientific discipline. I: Control of bias and comparison with large co-operative trials.Statistics in Medicine. 1987;6 (3): 315-325.
13. Chatterjee S.,Hadi A. S.Regression analysis by example. New York, NY: John Wiley; 2006:.
14. Cohen J.Statistical power analysis for the behavioral sciences. Hillsdale, NJ: Lawrence Erlbaum; 1988:.
15. Cohen J.,Cohen P.,West S. G.,Aiken L. S.Applied multiple regression/correlation analysis for the behavioral sciences. Hillsdale, NJ: Lawrence Erlbaum; 2003:.
16. Conger A. J.A revised definition for suppressor variables: A guide to their identification and interpretation.Educational and Psychological Measurement. 1974;34 (1): 35-46.
17. Copas J.,Shi J. Q.A sensitivity analysis for publication bias in systematic reviews.Statistical Methods in Medical Research. 2001;10 (4): 251-265.
18. Dalton D. R.,Aguinis H.,Dalton C. M.,Bosco F. A.,Pierce C. A.Revisiting the file drawer problem in meta-analysis: An assessment of published and non-published correlation matrices.Personnel Psychology. 2012;65 (2): 221-249.
19. Dalton D. R.,Dalton C. M.Meta-analyses. Some very good steps toward a bit longer journey.Organizational Research Methods. 2008;11 (1): 127-147.
20. Dana J.,Weber R. A.,Kuang J. X.Exploiting moral wiggle room: Experiments demonstrating an illusory preference for fairness.Economic Theory. 2007;33 (1): 67-80.
21. De Angelis C.,Drazen J. M.,Frizelle F. A.,Haug C.,Hoey J.,Horton R.,.Clinical trial registration: A statement from the International Committee of Medical Journal Editors.Annals of Internal Medicine. 2004;141 (6): 477.
22. Deeks J. J.,Macaskill P.,Irwig L.The performance of tests of publication bias and other sample size effects in systematic reviews of diagnostic test accuracy was assessed.Journal of Clinical Epidemiology. 2005;58:882-893.
23. Dickersin K.,Min Y. I.,Meinert C. L.Factors influencing publication of research results.Journal of the American Medical Association. 1992;267 (3): 374-378.
24. Duval S.,Tweedie R.Trim and fill: A simple funnel-plot-based method of testing and adjusting for publication bias in meta-analysis.Biometrics. 2000;56 (2): 455-463.
25. Easterbrook P. J.,Berlin J. A.,Gopalan R.,Matthews D. R.Publication bias in clinical research.The Lancet. 1991;337 (8746): 867-872.
26. Edwards J. R.To prosper, organizational psychology should ... overcome methodological barriers to progress.Journal of Organizational Behavior. 2008;29 (4): 469-491.
27. Egger M.,Smith G. D.Meta-analysis: Potentials and promise.British Medical Journal. 1997;315 (7119): 1371-1374.
28. Egger M.,Smith G. D.,Schneider M.,Minder C.Bias in meta-analysis detected by a simple, graphical test.British Medical Journal. 1997;315 (7109): 629-634.
29. Friedman L.,Wall M.Graphical views of suppression and multicollinearity in multiple linear regression.American Statistician. 2005;59 (2): 127-136.
30. Gawande A.The cancer-cluster myth.New Yorker. 1999;9:34-37.
31. Gilovich T.,Vallone R.,Tversky A.The hot hand in basketball: On the misperception of random sequences.Cognitive Psychology. 1985;17 (3): 295-314.
32. Glick W. H.,Miller C. C.,Cardinal L. B.Making a life in the field of organization science.Journal of Organizational Behavior. 2007;28 (7): 817-835.
33. Hahn S.,Williamson P. R.,Hutton J. L.Investigation of within-study selective reporting in clinical research: Follow-up of applications submitted to a local research ethics committee.Journal of Evaluation in Clinical Practice. 2002;8 (3): 353-359.
34. Hedges L. V.,Olkin I.Statistical methods for meta-analysis. New York, NY: Academic Press; 1985:.
35. Hedges L. V.,Vevea J. L.Estimating effect size under publication bias: Small sample properties and robustness of a random effects selection model.Journal of Educational and Behavioral Statistics. 1996;21 (4): 299-332.
36. Henson R. K.,Roberts J. K.Use of exploratory factor analysis in published research.Educational and Psychological Measurement. 2006;66 (3): 393-416.
37. Higgins J. P. T.,Thompson S. G.Quantifying heterogeneity in a meta-analysis.Statistics in Medicine. 2002;21:1539-1558.
38. Higgins J. P. T.,Thompson S. G.,Deeks J. J.,Altman D. G.Measuring inconsistency in meta-analyses.British Medical Journal. 2003;327 (7414): 557-560.
39. Horwitz S. K.,Horwitz I. B.The effects of team diversity on team outcomes: A meta-analytic review of team demography.Journal of Management. 2007;33:987-1015.
40. Huedo-Medina T. B.,Sánchez-Meca J.,Marín-Martínez F.,Botella J.Assessing heterogeneity in meta-analysis: Q statistic or I2 index?.Psychological Methods. 2006;11 (2): 193-206.
41. Hunter J. E.,Schmidt F. L.Methods of meta-analysis: Correcting error and bias in research findings. Newbury Park, CA: Sage; 2004:.
42. Hunter J. E.,Schmidt F. L.,Jackson G. B.Meta-analysis: Cumulating research findings across studies. Beverly Hills, CA: Sage; 1982:.
43. Hutton J. L.,Williamson P. R.Bias in meta-analysis due to outcome variable selection within studies.Journal of the Royal Statistical Society: Series C (Applied Statistics). 2000;49 (3): 359-370.
44. Ioannidis J. P.Why most published research findings are false.PLoS Medicine. 2005;2 (8): 696.
45. John L. K.,Loewenstein G.,Prelec D.Measuring the prevalence of questionable research practices with incentives for truth-telling.Psychological Science. 2012;23:524-532.
46. Kepes S.,Banks G. C.,McDaniel M.,Whetzel D. L.Publication bias in the organizational sciences.Organizational Research Methods. 2012;15:.
47. Kerr N. L.HARKing: Hypothesizing after the results are known.Personality and Social Psychology Review. 1998;2 (3): 196-217.
48. Kraemer H. C.,Gardner C.,Brooks J. O., III,Yesavage J. A.Advantages of excluding underpowered studies in meta-analysis: Inclusionist versus exclusionist viewpoints.Psychological Methods. 1998;3 (1): 23-31.
49. Leamer E. E.Let's take the con out of econometrics.American Economic Review. 1983;73 (1): 31-43.
50. Light R. J.,Pillemer D. B.Summing up: the science of reviewing research. Cambridge, MA: Harvard University Press; 1984:.
51. Lipsey M. W.,Wilson D. B.The efficacy of psychological, educational, and behavioral treatment: Confirmation from meta-analysis.American Psychologist. 1993;48 (12): 1181-1209.
52. Macaskill P.,Walter S. D.,Irwig L.A comparison of methods to detect publication bias in meta-analysis.Statistics in Medicine. 2001;20 (4): 641-654.
53. Maxwell S. E.The persistence of underpowered studies in psychological research: Causes, consequences, and remedies.Psychological Methods. 2004;9 (2): 147-163.
54. McGauran N.,Wieseler B.,Kreis J.,Schüler Y. B.,Kölsch H.,Kaiser T.Reporting bias in medical research-A narrative review.Trials. 2010;11:37.
55. Milloy S. J.Science without sense: The risky business of public health research. Washington, DC: Cato Institute; 1995:.
56. Moreno S., Sutton, A., Ades, A., Stanley, T., Abrams, K., Peters, J., & Cooper, N.Assessment of regression-based methods to adjust for publication bias through a comprehensive simulation study.BMC Medical Research Methodology. 2009;9 (1): 2.
57. Nickerson R. S.Confirmation bias: A ubiquitous phenomenon in many guises.Review of General Psychology. 1998;2 (2): 175-220.
58. R: A language and environment for statistical computing. 2010:.
59. Rendina-Gobioff G.Detecting publication bias in random effects meta-analysis: An empirical comparison of statistical methods. Tampa, FL: Unpublished doctoral dissertation). University of South Florida; 2006:.
60. Rothstein H. R.,Sutton A. J.,Borenstein M.Publication bias in meta-analysis. New York, NY: John Wiley; 2005:.
61. Scargle J. D.Publication bias: The "file-drawer problem" in scientific inference.Journal of Scientific Exploration. 2000;14 (2): 94-106.
62. Sedlmeier P.,Gigerenzer G.Do studies of statistical power have an effect on the power of studies?.Psychological Bulletin. 1989;105 (2): 309-316.
63. Selvin H. C.,Stuart A.Data-dredging procedures in survey analysis.American Statistician. 1966;20 (3): 20-23.
64. Shore L. M.,Randel A. E.,Chung B. G.,Dean M. A.,Ehrhart K. H.,Singh G.Inclusion and diversity in work groups: A review and model for future research.Journal of Management. 2011;37 (4): 1262-1289.
65. Simmons J. P.,Nelson L. D.,Simonsohn U.False-positive psychology.Psychological Science. 2011;22 (11): 1359-1366.
66. Song F.,Parekh S.,Hooper L.,Loke Y. K.,Ryder J.,Sutton A. J.,.Dissemination and publication of research findings: An updated review of related biases.Health Technology Assessment. 2010;14 (8): 1-220.
67. Spector P. E.,Brannick M. T.Methodological urban legends: The misuse of statistical control variables.Organizational Research Methods. 2011;14:279-286.
68. Stern J. M.,Simes R. J.Publication bias: Evidence of delayed publication in a cohort study of clinical research projects.British Medical Journal. 1997;315 (7109): 640-645.
69. Sterne J. A.,Egger M.Funnel plots for detecting bias in meta-analysis: Guidelines on choice of axis.Journal of Clinical Epidemiology. 2001;54 (10): 1046-1055.
70. Sterne J. A.,Egger M.,Moher D.Cochrane handbook for systematic reviews of interventions. Higgins J. P.Green S., ed. Chichester, UK: John Wiley; 2008:.
71. Sterne J. A. C.,Sutton A. J.,Ioannidis J. P. A.,Terrin N.,Jones D. R.,Lau J.Recommendations for examining and interpreting funnel plot asymmetry in meta-analyses of randomised controlled trials.British Medical Journal. 2011;343 (7818): 302-307.
72. Sutton A. J.,Duval S. J.,Tweedie R. L.,Abrams K. R.,Jones D. R.Empirical assessment of effect of publication bias on meta-analyses.British Medical Journal. 2000;320 (7249): 1574-1577.
73. Turner E. H.,Matthews A. M.,Linardatos E.,Tell R. A.,Rosenthal R.Selective publication of antidepressant trials and its influence on apparent efficacy.New England Journal of Medicine. 2008;358 (3): 252-260.
74. Viechtbauer W.Conducting meta-analyses in R with the metafor package.Journal of Statistical Software. 2010;36 (3): 1-48.
75. Villar J.,Carroli G.,Belizán J. M.Predictive ability of meta-analyses of randomised controlled trials.The Lancet. 1995;345 (8952): 772-776.
76. Wicherts J. M.,Borsboom D.,Kats J.,Molenaar D.The poor availability of psychological research data for reanalysis.American Psychologist. 2006;61 (7): 726-728.
77. Williamson P. R.,Gamble C.,Altman D. G.,Hutton J. L.Outcome selection bias in meta-analysis.Statistical Methods in Medical Research. 2005;14 (5): 515-524.