Scientific citations favor positive results: a systematic review and meta-analysis

Journal of Clinical Epidemiology

Volumn 88, Issue , 2017, Pages 92-101

  Duyx, Bram ^a,b   Urlings, Miriam J E ^a,b   Swaen, Gerard M H ^a,b   Bouter, Lex M ^c,d   Zeegers, Maurice P ^a,b  

^a MAASTRICHT UNIVERSITY   (Netherlands)

^b MAASTRICHT UNIVERSITY   (Netherlands)

^c VU UNIVERSITY MEDICAL CENTER   (Netherlands)

^d VU UNIVERSITY AMSTERDAM   (Netherlands)

Author keywords

Citation bias; Meta analysis; Outcome bias; Questionable research practices; Research integrity; Systematic review

Indexed keywords

BIOMEDICINE; CITATION ANALYSIS; JOURNAL IMPACT FACTOR; NATURAL SCIENCE; PRIORITY JOURNAL; PUBLISHING; REVIEW; STATISTICAL BIAS; STATISTICAL SIGNIFICANCE; SYSTEMATIC REVIEW; HUMAN; META ANALYSIS; PUBLICATION; STATISTICS AND NUMERICAL DATA;

HUMANS; JOURNAL IMPACT FACTOR; PERIODICALS AS TOPIC; PUBLICATION BIAS;

EID: 85023756946     PISSN: 08954356     EISSN: 18785921     Source Type: Journal    
DOI: 10.1016/j.jclinepi.2017.06.002     Document Type: Review

References (66)

1. Song, F., Parekh, S., Hooper, L., Loke, Y.K., Ryder, J., Sutton, A.J., et al. Dissemination and publication of research findings: an updated review of related biases. Health Technol Assess 14 (2010), 1–193.
2. Steneck, N.H., Fostering integrity in research: definitions, current knowledge, and future directions. Sci Eng Ethics 12 (2006), 53–74.
3. Fanelli, D., How many scientists fabricate and falsify research? A systematic review and meta-analysis of survey data. PLoS One, 4(5), 2009, e5738.
4. Bouter, L.M., Tijdink, J., Axelsen, N., Martinson, B.C., ter Riet, G., Ranking major and minor research misbehaviors: results from a survey among participants of four World Conferences on Research Integrity. Res Integrity Peer Rev, 1(1), 2016, 17.
5. Duyx, B., Urlings, M.J.E., Swaen, G., Bouter, L.M., Zeegers, M.P., Protocol for systematic review of citation bias. 2015 Available at http://hdl.handle.net/10411/20710 Accessed September 30, 2016.
6. Leimu, R., Koricheva, J., What determines the citation frequency of ecological papers?. Trends Ecol Evol 20:1 (2005), 28–32.
7. Hutchison, B.G., Oxman, A.D., Lloyd, S., Comprehensiveness and bias in reporting clinical-trials - study of reviews of pneumococcal vaccine effectiveness. Can Fam Physician 41 (1995), 1356–1360.
8. Andrade, N.S., Flynn, J.P., Bartanusz, V., Twenty-year perspective of randomized controlled trials for surgery of chronic nonspecific low back pain: citation bias and tangential knowledge. Spine J 13:11 (2013), 1698–1704.
9. Callaham, M., Wears, R.L., Weber, E., Journal prestige, publication bias, and other characteristics associated with citation of published studies in peer-reviewed journals. JAMA 287 (2002), 2847–2850.
10. Conen, D., Torres, J., Ridker, P.M., Differential citation rates of major cardiovascular clinical trials according to source of funding—a survey from 2000 to 2005. Circulation 118 (2008), 1321–1327.
11. Etter, J.F., Stapleton, J., Citations to trials of nicotine replacement therapy were biased toward positive results and high-impact-factor journals. J Clin Epidemiol 62 (2009), 831–837.
12. Greenberg, S.A., How citation distortions create unfounded authority: analysis of a citation network. BMJ, 339, 2009.
13. Ioannidis, J.P.A., Contradicted and initially stronger effects in highly cited clinical research. JAMA 294:2 (2005), 218–228.
14. Ioannidis, J.P.A., Panagiotou, O.A., Comparison of effect sizes associated with biomarkers reported in highly cited individual articles and in subsequent meta-analyses. JAMA 305 (2011), 2200–2210.
15. Jannot, A.S., Agoritsas, T., Gayet-Ageron, A., Perneger, T.V., Citation bias favoring statistically significant studies was present in medical research. J Clin Epidemiol 66 (2013), 296–301.
16. Kivimäki, M., Batty, G.D., Kawachi, I., Virtanen, M., Singh-Manoux, A., Brunner, E.J., Don't let the truth get in the way of a good story: an illustration of citation bias in epidemiologic research. Am J Epidemiol 180 (2014), 446–448.
17. Kjaergard, L.L., Gluud, C., Citation bias of hepato-biliary randomized clinical trials. J Clin Epidemiol 55 (2002), 407–410.
18. Nieminen, P., Rucker, G., Miettunen, J., Carpenter, J., Schumacher, M., Statistically significant papers in psychiatry were cited more often than others. J Clin Epidemiol 60 (2007), 939–946.
19. Perino, A.C., Hoang, D.D., Holmes, T.H., Santangeli, P., Heidenreich, P.A., Perez, M.V., et al. Association between success rate and citation count of studies of radiofrequency catheter ablation for atrial fibrillation possible evidence of citation bias. Circ Cardiovasc Qual Outcomes 7 (2014), 687–692.
20. Ravnskov, U., Cholesterol lowering trials in coronary heart-disease - frequency of citation and outcome. BMJ 305 (1992), 15–19.
21. Schrag, M., Mueller, C., Oyoyo, U., Smith, M.A., Kirsch, W.M., Iron, zinc and copper in the Alzheimer's disease brain: a quantitative meta-analysis. Some insight on the influence of citation bias on scientific opinion. Prog Neurobiol 94 (2011), 296–306.
22. Sheth, U., Simunovic, N., Tornetta, P., Einhorn, T.A., Bhandari, M., Poor citation of prior evidence in hip fracture trials. J Bone Joint Surg Am 93A (2011), 2079–2086.
23. Valachis, A., Mauri, D., Neophytou, C., Polyzos, N.P., Tsali, L., Garras, A., et al. Translational medicine and reliability of single-nucleotide polymorphism studies: can we believe in SNP reports or not?. Int J Med Sci 8 (2011), 492–500.
24. Voracek, M., No effects of androgen receptor gene CAG and GGC repeat polymorphisms on digit ratio (2D:4D): a comprehensive meta-analysis and critical evaluation of research. Evol Hum Behav 35 (2014), 430–437.
25. Zhao, J., Schooling, C.M., Promotion of “Low T” and citation bias in testosterone studies. Int J Cardiol 184 (2015), 510–511.
26. Kostoff, R.N., The difference between highly and poorly cited medical articles in the journal Lancet. Scientometrics 72 (2007), 513–520.
27. Bastiaansen, J.A., de Vries, Y.A., Munafo, M.R., Citation distortions in the literature on the serotonin-transporter-linked polymorphic region and amygdala activation. Biol Psychiatry 78 (2015), E35–E36.
28. Useem, J., Brennan, A., LaValley, M., Vickery, M., Ameli, O., Reinen, N., et al. Systematic differences between cochrane and non-cochrane meta-analyses on the same topic: a matched pair analysis. PLoS One, 10, 2015, 17.
29. Kulkarni, A.V., Busse, J.W., Shams, I., Characteristics associated with citation rate of the medical literature. PLoS One, 2, 2007, e403.
30. Peritz, B.C., On the heuristic value of scientific publications and their design—a citation analysis of some clinical-trials. Scientometrics 30 (1994), 175–186.
31. Allareddy, V., Shah, A., Lin, C.-Y., Elangovan, S., Lee, M.K., Chung, K., et al. Randomized trials published in the journal of dental research are cited more often compared with those in other top-tier non-specialty-specific dental journals. J Evid Based Dent Pract 10 (2010), 71–77.
32. Annalingam, A., Damayanthi, H., Jayawardena, R., Ranasinghe, P., Determinants of the citation rate of medical research publications from a developing country. Springerplus, 3, 2014, 140.
33. Koren, G., Nickel, C., Perpetuating fears: bias against the null hypothesis in fetal safety of drugs as expressed in scientific citations. J Popul Ther Clin Pharmacol 18 (2011), e28–e32.
34. Zambon, M., Biondi-Zoccai, G., Bignami, E., Ruggeri, L., Zangrillo, A., Landoni, G., A comprehensive appraisal of meta-analyses focusing on nonsurgical treatments aimed at decreasing perioperative mortality or major cardiac complications. J Anesth 26 (2012), 509–515.
35. Sawin, V.I., Robinson, K.A., Biased and inadequate citation of prior research in reports of cardiovascular trials is a continuing source of waste in research. J Clin Epidemiol 69 (2016), 174–178.
36. Etwel, F., Koren, G., When positive studies of novel therapies are subsequently nullified: cumulative meta-analyses in preeclampsia. Clin Invest Med 38:5 (2015), E274–E284.
37. Nieri, M., Clauser, C., Franceschi, D., Pagliaro, U., Saletta, D., Pini-Prato, G., Randomized clinical trials in implant therapy: relationships among methodological, statistical, clinical, paratextual features and number of citations. Clin Oral Implants Res 18 (2007), 419–431.
38. Peritz, B.C., The use of meta-analyses for citation studies: an illustrative example. Informetrics 91 (1992), 358–376.
39. Misemer, B.S., Platts-Mills, T.F., Jones, C.W., Citation bias favoring positive clinical trials of thrombolytics for acute ischemic stroke: a cross-sectional analysis. Trials, 17, 2016, 473.
40. Trinquart, L., Johns, D.M., Galea, S., Why do we think we know what we know? A metaknowledge analysis of the salt controversy. Int J Epidemiol 45 (2016), 251–260.
41. de Vries, Y.A., Roest, A.M., Franzen, M., Munafo, M.R., Bastiaansen, J.A., Citation bias and selective focus on positive findings in the literature on the serotonin transporter gene (5-HTTLPR), life stress and depression. Psychol Med 46 (2016), 2971–2979.
42. Unger, J.M., Barlow, W.E., Ramsey, S.D., LeBlanc, M., Blanke, C.D., Hershman, D.L., The scientific impact of positive and negative phase 3 cancer clinical trials. JAMA Oncol 2 (2016), 875–881.
43. Aguinis, H., Dalton, D.R., Bosco, F.A., Pierce, C.A., Dalton, C.M., Meta-analytic choices and judgment calls: implications for theory building and testing, obtained effect sizes, and scholarly impact. J Manage 37 (2011), 5–38.
44. Chapman, S., Ragg, M., McGeechan, K., Citation bias in reported smoking prevalence in people with schizophrenia. Aust NZ J Psychiatr 43 (2009), 277–282.
45. Eisend, M., Tarrahi, F., Meta-analysis selection bias in marketing research. Int J Res Mark 31:3 (2014), 317–326.
46. Robins, R.W., Craik, K.H., Is there a citation bias in the judgment and decision literature. Organ Behav Hum Decis Process 54 (1993), 225–244.
47. Christensen-Szalanski, J.J., Beach, L.R., The citation bias: fad and fashion in the judgment and decision literature. Am Psychol 39 (1984), 75–78.
48. Tanner-Smith, E.E., Polanin, J.R., A retrospective analysis of dissemination biases in the brief alcohol intervention literature. Psychol Addict Behav 29:1 (2015), 49–62.
49. Schumm, W.R., Evidence of pro-homosexual bias in social science: citation rates and research on lesbian parenting. Psychol Rep 106:2 (2010), 374–380.
50. Barto, E.K., Rillig, M.C., Dissemination biases in ecology: effect sizes matter more than quality. Oikos 121 (2012), 228–235.
51. Lortie, C.J., Aarssen, L.W., Budden, A.E., Leimu, R., Do citations and impact factors relate to the real numbers in publications? A case study of citation rates, impact, and effect sizes in ecology and evolutionary biology. Scientometrics 94 (2013), 675–682.
52. Fanelli, D., Positive results receive more citations, but only in some disciplines. Scientometrics 94 (2013), 701–709.
53. Kim, M.A., Lee, H.S., Duo-trio difference-preference test with two replications: use of psychological biases for measuring meaningful preference. J Sens Stud 30 (2015), 211–224.
54. Kumar, G.K., Kumar, K.K., Baumgarten, G., Ramkumar, G., Validation of MERRA reanalysis upper-level winds over low latitudes with independent rocket sounding data. J Atmos Solar-Terrestrial Phys 123 (2015), 48–54.
55. Kunz, M., Yu, Z., Frangakis, A.S., M-free: mask-independent scoring of the reference bias. J Struct Biol 192 (2015), 307–311.
56. Greenberg, S.A., Understanding belief using citation networks. J Eval Clin Pract 17 (2011), 389–393.
57. Ioannidis, J.P.A., The mass production of redundant, misleading, and conflicted systematic reviews and meta-analyses. Milbank Q 94 (2016), 485–514.
58. Chalmers, T.C., Frank, C.S., Reitman, D., Minimizing the three stages of publication bias. JAMA 263 (1990), 1392–1395.
59. Zhou, X., Wang, Y., Tsafnat, G., Coiera, E., Bourgeois, F.T., Dunn, A.G., Citations alone were enough to predict favorable conclusions in reviews of neuraminidase inhibitors. J Clin Epidemiol 68 (2015), 87–93.
60. Case, D.O., Higgins, G.M., How can we investigate citation behavior? A study of reasons for citing literature in communication. J Am Soc Inf Sci 51 (2000), 635–645.
61. Yu, G., Li, Y.J., Parameter identification of the observed citation distribution. Scientometrics 71 (2007), 339–348.
62. Akcan, D., Axelsson, S., Bergh, C., Davidson, T., Rosen, M., Methodological quality in clinical trials and bibliometric indicators: no evidence of correlations. Scientometrics 96 (2013), 297–303.
63. Nieminen, P., Carpenter, J., Rucker, G., Schumacher, M., The relationship between quality of research and citation frequency. BMC Med Res Methodol, 6, 2006, 42.
64. West, R., McIlwaine, A., What do citation counts count for in the field of addiction? An empirical evaluation of citation counts and their link with peer ratings of quality. Addiction 97 (2002), 501–504.
65. Giuffrida, M.A., Brown, D.C., Association between article citation rate and level of evidence in the companion animal literature. J Vet Intern Med 26 (2012), 252–258.
66. Bhandari, M., Busse, J., Devereaux, P.J., Montori, V.M., Swiontkowski, M., Tornetta, P., et al. Factors associated with citation rates in the orthopedic literature. Can J Surg 50 (2007), 119–123.