Meta-assessment of bias in science

Proceedings of the National Academy of Sciences of the United States of America

Volumn 114, Issue 14, 2017, Pages 3714-3719

  Fanelli, Daniele ^a   Costas, Rodrigo ^b   Ioannidis, John P A ^a,c  

^a STANFORD UNIVERSITY SCHOOL OF MEDICINE   (United States)

^b LEIDEN UNIVERSITY   (Netherlands)

^c STANFORD UNIVERSITY   (United States)

Author keywords

Bias; Integrity; Meta analysis; Meta research; Misconduct

Indexed keywords

ARTICLE; EFFECT SIZE; FEASIBILITY STUDY; HUMAN; META ANALYSIS (TOPIC); ODDS RATIO; PRIORITY JOURNAL; PUBLISHING; RISK FACTOR; SCIENCE; SCIENTIFIC MISCONDUCT; SOCIOLOGY; UNITED STATES; FEMALE; MALE; META ANALYSIS; METHODOLOGY; PERSONNEL; SELECTION BIAS;

FEMALE; HUMANS; MALE; PUBLISHING; RESEARCH DESIGN; RESEARCH PERSONNEL; SCIENTIFIC MISCONDUCT; SELECTION BIAS;

EID: 85016938866     PISSN: 00278424     EISSN: 10916490     Source Type: Journal    
DOI: 10.1073/pnas.1618569114     Document Type: Article

References (51)

1. Song F, et al. (2010) Dissemination and publication of research findings: An updated review of related biases. Health Technol Assess 14(8):iii, ix-xi, 1-193.
2. Chavalarias D, Ioannidis JPA (2010) Science mapping analysis characterizes 235 biases in biomedical research. J Clin Epidemiol 63(11):1205-1215.
3. Young NS, Ioannidis JPA, Al-Ubaydli O (2008) Why current publication practices may distort science. PLoS Med 5(10):e201.
4. Dwan K, et al. (2008) Systematic review of the empirical evidence of study publication bias and outcome reporting bias. PLoS One 3(8):e3081.
5. Fanelli D (2010) "Positive" results increase down the Hierarchy of the Sciences. PLoS One 5(4):e10068.
6. Dubben HH, Beck-Bornholdt HP (2005) Systematic review of publication bias in studies on publication bias. BMJ 331(7514):433-434.
7. Van Bavel JJ, Mende-Siedlecki P, Brady WJ, Reinero DA (2016) Contextual sensitivity in scientific reproducibility. Proc Natl Acad Sci USA 113(23):6454-6459.
8. Sterling TD, Rosenbaum WL, Weinkam JJ (1995) Publication decisions revisited - The effect of the outcome of statistical tests on the decision to publish and vice-versa. Am Stat 49(1):108-112.
9. de Winter J, Dodou D (2014) A surge of p-values between 0.040 and 0.049 in recent decades (but negative results are increasing rapidly too). PeerJ PrePrints (2):e447v443.
10. Pautasso M (2010) Worsening file-drawer problem in the abstracts of natural, medical and social science databases. Scientometrics 85(1):193-202.
11. Fanelli D (2012) Negative results are disappearing from most disciplines and countries. Scientometrics 90(3):891-904.
12. Fanelli D (2010) Do pressures to publish increase scientists' bias? An empirical support from US States Data. PLoS One 5(4):e10271.
13. van Dalen HP, Henkens K (2012) Intended and unintended consequences of a publishor-perish culture: A worldwide survey. J Am Soc Inf Sci Technol 63(7):1282-1293.
14. Fanelli D, Ioannidis JPA (2013) US studies may overestimate effect sizes in softer research. Proc Natl Acad Sci USA 110(37):15031-15036.
15. Ioannidis JPA, Fanelli D, Dunne DD, Goodman SN (2015) Meta-research: Evaluation and improvement of research methods and practices. PLoS Biol 13(10):e1002264.
16. Fanelli D, Costas R, Larivière V (2015) Misconduct policies, academic culture and career stage, not gender or pressures to publish, affect scientific integrity. PLoS One 10(6): e0127556.
17. Sterne JAC, Gavaghan D, Egger M (2000) Publication and related bias in metaanalysis: Power of statistical tests and prevalence in the literature. J Clin Epidemiol 53(11):1119-1129.
18. Schooler J (2011) Unpublished results hide the decline effect. Nature 470(7335):437-437.
19. Ioannidis JPA, Trikalinos TA (2005) Early extreme contradictory estimates may appear in published research: The Proteus phenomenon in molecular genetics research and randomized trials. J Clin Epidemiol 58(6):543-549.
20. Jannot AS, Agoritsas T, Gayet-Ageron A, Perneger TV (2013) Citation bias favoring statistically significant studies was present in medical research. J Clin Epidemiol 66(3):296-301.
21. Lexchin J, Bero LA, Djulbegovic B, Clark O (2003) Pharmaceutical industry sponsorship and research outcome and quality: Systematic review. Br Med J 326(7400):1167-1170B.
22. Martinson BC, Crain AL, Anderson MS, De Vries R (2009) Institutions' expectations for researchers' self-funding, federal grant holding and private industry involvement: Manifold drivers of self-interest and researcher behavior. Acad Med 84(11):1491-1499.
23. Qiu J (2010) Publish or perish in China. Nature 463(7278):142-143.
24. Lee C, Schrank A (2010) Incubating innovation or cultivating corruption? The developmental state and the life sciences in Asia. Soc Forces 88(3):1231-1255.
25. Fanelli D (2012) When East meets West...does bias increase? A preliminary study on South Korea, United States and other countries. 8th International Conference on Webometrics, Informetrics and Scientometrics and 13th COLLNET Meeting, eds Ho-Nam C, Hye-Sun K, Kyung-Ran N, Seon-Hee L, Hye-Jin K, Kretschmer H (KISTI, Seoul, Korea), pp 47-48.
26. Lacetera N, Zirulia L (2011) The economics of scientific misconduct. J Law Econ Organ 27(3):568-603.
27. Fang FC, Bennett JW, Casadevall A (2013) Males are overrepresented among life science researchers committing scientific misconduct. mBio 4(1):e00640-e12.
28. Kaatz A, Vogelman PN, Carnes M (2013) Are men more likely than women To commit scientific misconduct? Maybe, maybe not. mBio 4(2):2.
29. Antes AL, et al. (2007) Personality and ethical decision-making in research: The role of perceptions of self and others. J Empir Res Hum Res Ethics 2(4):15-34.
30. Davis MS, Wester KL, King B (2008) Narcissism, entitlement, and questionable research practices in counseling: A pilot study. J Couns Dev 86(2):200-210.
31. Bailey CD (2015) Psychopathy, Academic accountants' attitudes toward unethical research practices, and publication success. Account Rev 90(4):1307-1332.
32. Fanelli D, Ioannidis JPA (2014) Reply to Nuijten et al.: Reanalyses actually confirm that US studies overestimate effects in softer research. Proc Natl Acad Sci USA 111(7):E714-E715.
33. Fanelli D, Glanzel W (2013) Bibliometric evidence for a hierarchy of the sciences. PLoS One 8(6):e66938.
34. Pfeffer C, Olsen BR (2002) Editorial: Journal of negative results in biomedicine. J Negat Results Biomed 1(1):2.
35. Macaskill P, Walter SD, Irwig L (2001) A comparison of methods to detect publication bias in meta-analysis. Stat Med 20(4):641-654.
36. Ioannidis JPA (2008) Interpretation of tests of heterogeneity and bias in meta-analysis. JEvalClinPract14(5):951-957.
37. Djulbegovic B, Hozo I (2007) When should potentially false research findings be considered acceptable? PLoS Med 4(2):211-217.
38. Pimple KD (2002) Six domains of research ethics. A heuristic framework for the responsible conduct of research. Sci Eng Ethics 8(2):191-205.
39. Fanelli D, Larivière V (2016) Researchers' individual publication rate has not increased in a century. PLoS One 11(3):e0149504.
40. Hayer C-A, et al. (2013) Pressures to publish: Catalysts for the loss of scientific writing integrity? Fisheries 38(8):352-355.
41. de Vries R, Anderson MS, Martinson BC (2006) Normal misbehavior: Scientists talk about the ethics of research. J Empir Res Hum Res Ethics 1(1):43-50.
42. Tijdink JK, Verbeke R, Smulders YM (2014) Publication pressure and scientific misconduct in medical scientists. J Empir Res Hum Res Ethics 9(5):64-71.
43. Vale RD (2015) Accelerating scientific publication in biology. Proc Natl Acad Sci USA 112(44):13439-13446.
44. Caron E, Van Eck, NJ (2014) Large scale author name disambiguation using rule-based scoring and clustering. Proceedings of the 19th International Conference on Science and Technology Indicators, ed Noyons E (Centre for Science and Technology Studies, Leiden University, Leiden, The Netherlands), pp 79-86.
45. Tijssen RJW, van Leeuwen TN, van Wijk E (2009) Benchmarking university-industry research cooperation worldwide: Performance measurements and indicators based on co-authorship data for the world's largest universities. Res Eval 18(1):13-24.
46. Waltman L, van Eck NJ, van Leeuwen TN, Visser MS, van Raan AFJ (2011) Towards a new crown indicator: Some theoretical considerations. J Informetrics 5(1):37-47.
47. Horton J, et al. (2010) Systematic review data extraction: Cross-sectional study showed that experience did not increase accuracy. J Clin Epidemiol 63(3):289-298.
48. Iqbal SA, Wallach JD, Khoury MJ, Schully SD, Ioannidis JPA (2016) Reproducible research practices and transparency across the biomedical literature. PloS Biol 14(1)e1002333.
49. Bates D, Mächler M, Bolker BM, Walker SC (2015) Fitting linear mixed-effects models using lme4. J Stat Softw 67(1):1-48.
50. Thompson SG, Sharp SJ (1999) Explaining heterogeneity in meta-analysis: A comparison of methods. Stat Med 18(20):2693-2708.
51. Johnson PCD (2014) Extension of Nakagawa & Schielzeth's R(2)GLMM to random slopes models. Methods Ecol Evol 5(9):944-946.