Empirical evaluation of which trial characteristics are associated with treatment effect estimates

Journal of Clinical Epidemiology

Volumn 77, Issue , 2016, Pages 24-37

  Dechartres, Agnes ^a,b,c   Trinquart, Ludovic ^a,b,d   Faber, Timor ^a,b   Ravaud, Philippe ^a,b,c,d  

^a PARIS DESCARTES UNIVERSITY   (France)

^b HÔTEL DIEU   (France)

^c UNIVERSITÉ PARIS DESCARTES   (France)

^d Columbia University ^*  (United States)

Author keywords

Bias; Meta analysis; Meta epidemiology; Methods; Randomized controlled trial; Systematic review

Indexed keywords

ARTICLE; CLINICAL STUDY; CLUSTER ANALYSIS; COCHRANE LIBRARY; CORRELATION ANALYSIS; EMBASE; EPIDEMIOLOGICAL DATA; EVALUATION STUDY; HUMAN; MEDLINE; META ANALYSIS (TOPIC); METHODOLOGY; PRIORITY JOURNAL; RANDOMIZED CONTROLLED TRIAL (TOPIC); STATISTICAL ANALYSIS; SYSTEMATIC REVIEW; TREATMENT OUTCOME; WEB OF SCIENCE; EPIDEMIOLOGY;

EPIDEMIOLOGIC STUDIES; HUMANS; RESEARCH DESIGN;

EID: 84977540744     PISSN: 08954356     EISSN: 18785921     Source Type: Journal    
DOI: 10.1016/j.jclinepi.2016.04.005     Document Type: Article

References (110)

1. [1] Chalmers, I., Altman, D.G., Systematic reviews. 1995, BMJ Publishing Group, London.
2. [2] Dickersin, K., Min, Y.I., Publication bias: the problem that won't go away. Ann N Y Acad Sci 703 (1993), 135–146 discussion 146–138.
3. [3] Dickersin, K., Min, Y.I., Meinert, C.L., Factors influencing publication of research results. Follow-up of applications submitted to two institutional review boards. JAMA 267 (1992), 374–378.
4. [4] Egger, M., Smith, G.D., Bias in location and selection of studies. BMJ 316 (1998), 61–66.
5. [5] Hopewell, S., Loudon, K., Clarke, M.J., Oxman, A.D., Dickersin, K., Publication bias in clinical trials due to statistical significance or direction of trial results. Cochrane Database Syst Rev, 2009, MR000006.
6. [6] Stern, J.M., Simes, R.J., Publication bias: evidence of delayed publication in a cohort study of clinical research projects. BMJ 315 (1997), 640–645.
7. [7] Turner, E.H., Matthews, A.M., Linardatos, E., Tell, R.A., Rosenthal, R., Selective publication of antidepressant trials and its influence on apparent efficacy. N Engl J Med 358 (2008), 252–260.
8. [8] Chan, A.W., Altman, D.G., Identifying outcome reporting bias in randomised trials on PubMed: review of publications and survey of authors. BMJ, 330, 2005, 753.
9. [9] Chan, A.W., Hrobjartsson, A., Haahr, M.T., Gotzsche, P.C., Altman, D.G., Empirical evidence for selective reporting of outcomes in randomized trials: comparison of protocols to published articles. JAMA 291 (2004), 2457–2465.
10. [10] Chan, A.W., Krleza-Jeric, K., Schmid, I., Altman, D.G., Outcome reporting bias in randomized trials funded by the Canadian Institutes of Health Research. CMAJ 171:7 (2004), 735–740.
11. [11] Kirkham, J.J., Dwan, K.M., Altman, D.G., Gamble, C., Dodd, S., Smyth, R., et al. The impact of outcome reporting bias in randomised controlled trials on a cohort of systematic reviews. BMJ, 340, 2010, c365.
12. [12] Chalmers, I., Glasziou, P., Avoidable waste in the production and reporting of research evidence. Lancet 374 (2009), 86–89.
13. [13] Gluud, L.L., Bias in clinical intervention research. Am J Epidemiol 163 (2006), 493–501.
14. [14] Ioannidis, J.P., Greenland, S., Hlatky, M.A., Khoury, M.J., Macleod, M.R., Moher, D., et al. Increasing value and reducing waste in research design, conduct, and analysis. Lancet 383 (2014), 166–175.
15. [15] Ioannidis, J.P., Modeling and research on research. Clin Chem 60 (2014), 1238–1239.
16. [16] Ioannidis, J.P., How to make more published research true. PloS Med, 11(10), 2014, e1001747.
17. [17] Sterne, J.A., Juni, P., Schulz, K.F., Altman, D.G., Bartlett, C., Egger, M., Statistical methods for assessing the influence of study characteristics on treatment effects in 'meta-epidemiological' research. Stat Med 21 (2002), 1513–1524.
18. [18] Higgins, J.P.T., Green, S., (eds.) Cochrane Handbook for Systematic Reviews of Interventions Version 5.1.0 [updated March 2011], 2011, The Cochrane Collaboration Available at http://handbook.cochrane.org/. Accessed May 17, 2016.
19. [19] Schulz, K.F., Chalmers, I., Hayes, R.J., Altman, D.G., Empirical evidence of bias. Dimensions of methodological quality associated with estimates of treatment effects in controlled trials. JAMA 273 (1995), 408–412.
20. [20] Juni, P., Altman, D.G., Egger, M., Systematic reviews in health care: assessing the quality of controlled clinical trials. BMJ 323 (2001), 42–46.
21. [21] Juni, P., Tallon, D., Egger, M., ‘Garbage in- garbage out’? Assessment of the quality of controlled trials in meta-analyses published in leading journals. Proceedings of the 3rd symposium on systematic reviews: beyond the basics, St Catherine's college, Oxford, 2000, Centre for Statistics in Medicine, Oxford, 19.
22. [22] Moher, D., Pham, B., Jones, A., Cook, D.J., Jadad, A.R., Moher, M., et al. Does quality of reports of randomised trials affect estimates of intervention efficacy reported in meta-analyses?. Lancet 352 (1998), 609–613.
23. [23] Pildal, J., Hrobjartsson, A., Jorgensen, K.J., Hilden, J., Altman, D.G., Gotzsche, P.C., Impact of allocation concealment on conclusions drawn from meta-analyses of randomized trials. Int J Epidemiol 36 (2007), 847–857.
24. [24] Tierney, J.F., Stewart, L.A., Investigating patient exclusion bias in meta-analysis. Int J Epidemiol 34 (2005), 79–87.
25. [25] Higgins, J.P., Altman, D.G., Gotzsche, P.C., Juni, P., Moher, D., Oxman, A.D., et al. The Cochrane Collaboration's tool for assessing risk of bias in randomised trials. BMJ, 343, 2011, d5928.
26. [26] Siersma, V., Als-Nielsen, B., Chen, W., Hilden, J., Gluud, L.L., Gluud, C., Multivariable modelling for meta-epidemiological assessment of the association between trial quality and treatment effects estimated in randomized clinical trials. Stat Med 26 (2007), 2745–2758.
27. [27] Welton, N.J., Ades, A.E., Carlin, J.B., Altman, D.G., Sterne, J.A.C., Models for potentially biased evidence in meta-analysis using empirically based priors. J R Stat Soc Ser A Stat Soc 172 (2009), 119–136.
28. [28] Bassler, D., Briel, M., Montori, V.M., Lane, M., Glasziou, P., Zhou, Q., et al. Stopping randomized trials early for benefit and estimation of treatment effects: systematic review and meta-regression analysis. JAMA 303 (2010), 1180–1187.
29. [29] Contopoulos-Ioannidis, D.G., Baltogianni, M.S., Ioannidis, J.P., Comparative effectiveness of medical interventions in adults versus children. J Pediatr 157 (2010), 322–330.e317.
30. [30] Dechartres, A., Boutron, I., Trinquart, L., Charles, P., Ravaud, P., Single-center trials show larger treatment effects than multicenter trials: evidence from a meta-epidemiologic study. Ann Intern Med 155 (2011), 39–51.
31. [31] Dechartres, A., Trinquart, L., Boutron, I., Ravaud, P., Influence of trial sample size on treatment effect estimates: meta-epidemiological study. BMJ, 346, 2013, f2304.
32. [32] Nuesch, E., Trelle, S., Reichenbach, S., Rutjes, A.W., Tschannen, B., Altman, D.G., et al. Small study effects in meta-analyses of osteoarthritis trials: meta-epidemiological study. BMJ, 341, 2010, c3515.
33. [33] Panagiotou, O.A., Contopoulos-Ioannidis, D.G., Ioannidis, J.P., Comparative effect sizes in randomised trials from less developed and more developed countries: meta-epidemiological assessment. BMJ, 346, 2013, f707.
34. [34] Chaimani, A., Vasiliadis, H.S., Pandis, N., Schmid, C.H., Welton, N.J., Salanti, G., Effects of study precision and risk of bias in networks of interventions: a network meta-epidemiological study. Int J Epidemiol 42 (2013), 1120–1131.
35. [35] Savovic, J., Harris, R.J., Wood, L., Beynon, R., Altman, D.G., Als-Nielsen, B., et al. Development of a combined database for meta-epidemiological research. Res Synth Methods 1 (2010), 212–225.
36. [36] Savovic, J., Jones, H.E., Altman, D.G., Harris, R.J., Juni, P., Pildal, J., et al. Influence of reported study design characteristics on intervention effect estimates from randomized, controlled trials. Ann Intern Med 157 (2012), 429–438.
37. [37] Wood, L., Egger, M., Gluud, L.L., Schulz, K.F., Juni, P., Altman, D.G., et al. Empirical evidence of bias in treatment effect estimates in controlled trials with different interventions and outcomes: meta-epidemiological study. BMJ 336 (2008), 601–605.
38. [38] Liberati, A., Altman, D.G., Tetzlaff, J., Mulrow, C., Gotzsche, P.C., Ioannidis, J.P., et al. The PRISMA statement for reporting systematic reviews and meta-analyses of studies that evaluate healthcare interventions: explanation and elaboration. BMJ, 339, 2009, b2700.
39. [39] Booth, A., Clarke, M., Ghersi, D., Moher, D., Petticrew, M., Stewart, L., An international registry of systematic-review protocols. Lancet 377 (2011), 108–109.
40. [40] Herbison, P., Hay-Smith, J., Gillespie, W.J., Adjustment of meta-analyses on the basis of quality scores should be abandoned. J Clin Epidemiol 59 (2006), 1249–1256.
41. [41] Juni, P., Witschi, A., Bloch, R., Egger, M., The hazards of scoring the quality of clinical trials for meta-analysis. JAMA 282 (1999), 1054–1060.
42. [42] Anglemyer, A., Horvath, H.T., Bero, L., Healthcare outcomes assessed with observational study designs compared with those assessed in randomized trials. Cochrane Database Syst Rev, 2014, MR000034.
43. [43] Delgado-Rodriguez, M., Ruiz-Canela, M., De Irala-Estevez, J., Llorca, J., Martinez-Gonzalez, A., Participation of epidemiologists and/or biostatisticians and methodological quality of published controlled clinical trials. J Epidemiol Community Health 55 (2001), 569–572.
44. [44] Als-Nielsen, B., Chen, W., Gluud, L.L., Siersma, V., Hilden, J., Gluud, C., Are trial size and reported methodological quality associated with treatment effects? Observational study of 523 randomised trials. [abstract] 12th Cochrane Colloquium: Bridging the Gaps, 2004, 102–103 Cochrane Database Systematic Review; ​Ottawa, Ontario, Canada.
45. [45] Bafeta, A., Dechartres, A., Trinquart, L., Yavchitz, A., Boutron, I., Ravaud, P., Impact of single centre status on estimates of intervention effects in trials with continuous outcomes: meta-epidemiological study. BMJ, 344, 2012, e813.
46. [46] Balk, E.M., Bonis, P.A., Moskowitz, H., Schmid, C.H., Ioannidis, J.P., Wang, C., et al. Correlation of quality measures with estimates of treatment effect in meta-analyses of randomized controlled trials. JAMA 287 (2002), 2973–2982.
47. [47] Bialy, L., Vandermeer, B., Lacaze-Masmonteil, T., Dryden, D.M., Hartling, L., A meta-epidemiological study to examine the association between bias and treatment effects in neonatal trials. Evid Based Child Health 9 (2014), 1052–1059.
48. [48] Contopoulos-Ioannidis, D.G., Gilbody, S.M., Trikalinos, T.A., Churchill, R., Wahlbeck, K., Ioannidis, J.P., Comparison of large versus smaller randomized trials for mental health-related interventions. Am J Psychiatry 162 (2005), 578–584.
49. [49] Dechartres, A., Altman, D.G., Trinquart, L., Boutron, I., Ravaud, P., Association between analytic strategy and estimates of treatment outcomes in meta-analyses. JAMA 312 (2014), 623–630.
50. [50] Egger, M., Juni, P., Bartlett, C., Holenstein, F., Sterne, J., How important are comprehensive literature searches and the assessment of trial quality in systematic reviews? Empirical study. Health Technol Assess 7 (2003), 1–76.
51. [51] Fenwick, J., Needleman, I.G., Moles, D.R., The effect of bias on the magnitude of clinical outcomes in periodontology: a pilot study. J Clin Periodontol 35 (2008), 775–782.
52. [52] Gartlehner, G., Dobrescu, A., Evans, T.S., Thaler, K., Nussbaumer, B., Sommer, I., et al. Average effect estimates remain similar as evidence evolves from single trials to high-quality bodies of evidence: a meta-epidemiologic study. J Clin Epidemiol 69 (2016), 16–22.
53. [53] Halladay, C.W., Trikalinos, T.A., Schmid, I.T., Schimd, C.H., Dahabreh, I.J., Using data sources beyond PubMed has a modest impact on the results of systematic reviews of therapeutic interventions. J Clin Epidemiol 68 (2015), 1076–1084.
54. [54] Hartling, L., Hamm, M.P., Fernandes, R.M., Dryden, D.M., Vandermeer, B., Quantifying bias in randomized controlled trials in child health: a meta-epidemiological study. PLoS One, 9, 2014, e88008.
55. [55] Hempel, S., Miles, J., Suttorp, M.J., Wang, Z., Johnsen, B., Morton, S., et al. Detection of associations between trial quality and effect sizes. 2012, Agency for Healthcare Research and Quality, Rockville.
56. [56] Hempel, S., Suttorp, M.J., Miles, J.N.V., Wang, Z., Maglione, M., Morton, S., et al. Detection of associations between trial quality and effect sizes. 2011, Agency for Healthcare Research and Quality, Rockville.
57. [57] Herbison, P., Hay-Smith, J., Gillespie, W.J., Different methods of allocation to groups in randomized trials are associated with different levels of bias. A meta-epidemiological study. J Clin Epidemiol 64 (2011), 1070–1075.
58. [58] Hrobjartsson, A., Emanuelsson, F., Skou Thomsen, A.S., Hilden, J., Brorson, S., Bias due to lack of patient blinding in clinical trials. A systematic review of trials randomizing patients to blind and nonblind sub-studies. Int J Epidemiol 43 (2014), 1272–1283.
59. [59] Hrobjartsson, A., Thomsen, A.S., Emanuelsson, F., Tendal, B., Hilden, J., Boutron, I., et al. Observer bias in randomised clinical trials with binary outcomes: systematic review of trials with both blinded and non-blinded outcome assessors. BMJ, 344, 2012, e1119.
60. [60] Hrobjartsson, A., Thomsen, A.S., Emanuelsson, F., Tendal, B., Hilden, J., Boutron, I., et al. Observer bias in randomized clinical trials with measurement scale outcomes: a systematic review of trials with both blinded and nonblinded assessors. CMAJ 185 (2013), E201–E211.
61. [61] Hrobjartsson, A., Thomsen, A.S., Emanuelsson, F., Tendal, B., Rasmussen, J.V., Hilden, J., et al. Observer bias in randomized clinical trials with time-to-event outcomes: systematic review of trials with both blinded and non-blinded outcome assessors. Int J Epidemiol 43 (2014), 937–948.
62. [62] Jacobs, W.C., Kruyt, M.C., Verbout, A.J., Oner, F.C., Effect of methodological quality measures in spinal surgery research: a metaepidemiological study. Spine J 12 (2012), 339–348.
63. [63] Juni, P., Holenstein, F., Sterne, J., Bartlett, C., Egger, M., Direction and impact of language bias in meta-analyses of controlled trials: empirical study. Int J Epidemiol 31 (2002), 115–123.
64. [64] Khan, K.S., Daya, S., Collins, J.A., Walter, S.D., Empirical evidence of bias in infertility research: overestimation of treatment effect in crossover trials using pregnancy as the outcome measure. Fertil Steril 65 (1996), 939–945.
65. [65] Kjaergard, L.L., Villumsen, J., Gluud, C., Reported methodologic quality and discrepancies between large and small randomized trials in meta-analyses. Ann Intern Med 135 (2001), 982–989.
66. [66] Lathyris, D., Panagiotou, O.A., Baltogianni, M., Ioannidis, J.P., Contopoulos-Ioannidis, D.G., Safety of medical interventions in children versus adults. Pediatrics 133 (2014), e666–e673.
67. [67] Lathyris, D.N., Trikalinos, T.A., Ioannidis, J.P., Evidence from crossover trials: empirical evaluation and comparison against parallel arm trials. Int J Epidemiol 36 (2007), 422–430.
68. [68] McAuley, L., Pham, B., Tugwell, P., Moher, D., Does the inclusion of grey literature influence estimates of intervention effectiveness reported in meta-analyses?. Lancet 356 (2000), 1228–1231.
69. [69] Mhaskar, R., Djulbegovic, B., Magazin, A., Soares, H.P., Kumar, A., Published methodological quality of randomized controlled trials does not reflect the actual quality assessed in protocols. J Clin Epidemiol 65 (2012), 602–609.
70. [70] Moher, D., Cook, D.J., Jadad, A.R., Tugwell, P., Moher, M., Jones, A., et al. Assessing the quality of reports of randomised trials: implications for the conduct of meta-analyses. Health Technol Assess 3 (1999), i–iv 1-98.
71. [71] Moher, D., Pham, B., Klassen, T.P., Schulz, K.F., Berlin, J.A., Jadad, A.R., et al. What contributions do languages other than English make on the results of meta-analyses?. J Clin Epidemiol 53 (2000), 964–972.
72. [72] Moher, D., Pham, B., Lawson, M.L., Klassen, T.P., The inclusion of reports of randomised trials published in languages other than English in systematic reviews. Health Technol Assess 7 (2003), 1–90.
73. [73] Ndounga Diakou, L.A., Trinquart, L., Hrobjartsson, A., Barnes, C., Ravaud, P., Boutron, I., Comparison of central adjudication of outcomes and onsite outcome assessment on treatment effect estimates. Cochrane Database Syst Rev, 2016, MR000043.
74. [74] Nuesch, E., Reichenbach, S., Trelle, S., Rutjes, A.W., Liewald, K., Sterchi, R., et al. The importance of allocation concealment and patient blinding in osteoarthritis trials: a meta-epidemiologic study. Arthritis Rheum 61 (2009), 1633–1641.
75. [75] Nuesch E, Trelle S, Reichenbach S, Juni P: Overestimation of treatment benefits in single-centre osteoarthritis trials may be related to sample size: meta-epidemiological study. 19th Cochrane Colloquium Madrid 2011, P2B226, Cochrane Database Systematic Review.
76. [76] Nuesch, E., Trelle, S., Reichenbach, S., Rutjes, A.W., Burgi, E., Scherer, M., et al. The effects of excluding patients from the analysis in randomised controlled trials: meta-epidemiological study. BMJ, 339, 2009, b3244.
77. [77] Papageorgiou, S.N., Antonoglou, G.N., Tsiranidou, E., Jepsen, S., Jager, A., Bias and small-study effects influence treatment effect estimates: a meta-epidemiological study in oral medicine. J Clin Epidemiol 67 (2014), 984–992.
78. [78] Papageorgiou, S.N., Xavier, G.M., Cobourne, M.T., Basic study design influences the results of orthodontic clinical investigations. J Clin Epidemiol 68 (2015), 1512–1522.
79. [79] Patsopoulos, N.A., Ioannidis, J.P., The use of older studies in meta-analyses of medical interventions: a survey. Open Med 3:2 (2009), e62–e68.
80. [80] Pham, B., Klassen, T.P., Lawson, M.L., Moher, D., Language of publication restrictions in systematic reviews gave different results depending on whether the intervention was conventional or complementary. J Clin Epidemiol 58 (2005), 769–776.
81. [81] Sampson, M., Barrowman, N.J., Moher, D., Klassen, T.P., Pham, B., Platt, R., et al. Should meta-analysts search Embase in addition to Medline?. J Clin Epidemiol 56 (2003), 943–955.
82. [82] Savovic, J., Jones, H., Altman, D., Harris, R., Juni, P., Pildal, J., et al. Influence of reported study design characteristics on intervention effect estimates from randomised controlled trials: combined analysis of meta-epidemiological studies. Health Technol Assess 16 (2012), 1–82.
83. [83] Seegers, V., Trinquart, L., Boutron, I., Ravaud, P., Comparison of treatment effect estimates for pharmacological randomized controlled trials enrolling older adults and those including adults: a meta-epidemiological study. PLoS One, 8(5), 2013, e63677.
84. [84] Siontis, K.C., Evangelou, E., Ioannidis, J.P., Magnitude of effects in clinical trials published in high-impact general medical journals. Int J Epidemiol 40 (2011), 1280–1291.
85. [85] Smail-Faugeron, V., Fron-Chabouis, H., Courson, F., Durieux, P., Comparison of intervention effects in split-mouth and parallel-arm randomized controlled trials: a meta-epidemiological study. BMC Med Res Methodol, 14, 2014, 64.
86. [86] Turner, R.M., Bird, S.M., Higgins, J.P., The impact of study size on meta-analyses: examination of underpowered studies in Cochrane reviews. PLoS One, 8, 2013, e59202.
87. [87] Unverzagt, S., Prondzinsky, R., Peinemann, F., Single-center trials tend to provide larger treatment effects than multicenter trials: a systematic review. J Clin Epidemiol 66 (2013), 1271–1280.
88. [88] van Tulder, M.W., Suttorp, M., Morton, S., Bouter, L.M., Shekelle, P., Empirical evidence of an association between internal validity and effect size in randomized controlled trials of low-back pain. Spine (Phila Pa 1976) 34:16 (2009), 1685–1692.
89. [89] Wood, L., Egger, M., Sterne, J.A.C., Association between trial size and treatment effect: meta-epidemiological study. 4th Symposium on Systematic Reviews: Pushing the Boundaries, 2002, St Catherine's College, Oxford, UK.
90. [90] Wood, L., Sterne, J., Juni, P., Egger, M., The smaller the study the larger the effect: meta-epidemiological study of randomised controlled trials. [abstract] XI Cochrane Colloquium: Evidence, Health Care and Culture, 2003, Cochrane Database Systematic Review, Barcelona Spain, 49.
91. [91] Zhang, Z., Xu, X., Ni, H., Small studies may overestimate the effect sizes in critical care meta-analyses: a meta-epidemiological study. Crit Care, 17(1), 2013, R2.
92. [92] Hopewell, S., Impact of grey literature on systematic reviews of randomized trials. Doctor of Philosophy Thesis, 2004, University of Oxford, Oxford, UK.
93. [93] Giraudeau, B., Higgins, J.P., Tavernier, E., Trinquart, L., Sample size calculation for meta-epidemiological studies. Stat Med 35 (2016), 239–250.
94. [94] Bala, M.M., Akl, E.A., Sun, X., Bassler, D., Mertz, D., Mejza, F., et al. Randomized trials published in higher vs. lower impact journals differ in design, conduct, and analysis. J Clin Epidemiol 66 (2013), 286–295.
95. [95] Lee, K.P., Schotland, M., Bacchetti, P., Bero, L.A., Association of journal quality indicators with methodological quality of clinical research articles. JAMA 287 (2002), 2805–2808.
96. [96] Samaan, Z., Mbuagbaw, L., Kosa, D., Borg Debono, V., Dillenburg, R., Zhang, S., et al. A systematic scoping review of adherence to reporting guidelines in health care literature. J Multidiscip Healthc 6 (2013), 169–188.
97. [97] Alba, A.C., Alexander, P.E., Chang, J., MacIsaac, J., DeFry, S., Guyatt, G.H., High statistical heterogeneity is more frequent in meta-analysis of continuous than binary outcomes. J Clin Epidemiol 70 (2016), 129–135.
98. [98] Abraha, I., Cherubini, A., Cozzolino, F., De Florio, R., Luchetta, M.L., Rimland, J.M., et al. Deviation from intention to treat analysis in randomised trials and treatment effect estimates: meta-epidemiological study. BMJ, 350, 2015, h2445.
99. [99] Goodman, S., Dickersin, K., Metabias: a challenge for comparative effectiveness research. Ann Intern Med 155 (2011), 61–62.
100. [100] Hrobjartsson, A., Boutron, I., Turner, L., Altman, D.G., Moher, D., Assessing risk of bias in randomised clinical trials included in Cochrane reviews: the why is easy, the how is a challenge. [editorial] Cochrane Database Syst Rev, 2013, ED000058.
101. [101] Lijmer, J.G., Mol, B.W., Heisterkamp, S., Bonsel, G.J., Prins, M.H., van der Meulen, J.H., et al. Empirical evidence of design-related bias in studies of diagnostic tests. JAMA 282 (1999), 1061–1066.
102. [102] Rutjes, A.W., Reitsma, J.B., Di Nisio, M., Smidt, N., van Rijn, J.C., Bossuyt, P.M., Evidence of bias and variation in diagnostic accuracy studies. CMAJ 174:4 (2006), 469–476.
103. [103] Takwoingi, Y., Leeflang, M.M., Deeks, J.J., Empirical evidence of the importance of comparative studies of diagnostic test accuracy. Ann Intern Med 158 (2013), 544–554.
104. [104] van Enst, W.A., Scholten, R.J., Whiting, P., Zwinderman, A.H., Hooft, L., Meta-epidemiologic analysis indicates that MEDLINE searches are sufficient for diagnostic test accuracy systematic reviews. J Clin Epidemiol 67 (2014), 1192–1199.
105. [105] Bello, S., Krogsboll, L.T., Gruber, J., Zhao, Z.J., Fischer, D., Hrobjartsson, A., Lack of blinding of outcome assessors in animal model experiments implies risk of observer bias. J Clin Epidemiol 67 (2014), 973–983.
106. [106] Crossley, N.A., Sena, E., Goehler, J., Horn, J., van der Worp, B., Bath, P.M., et al. Empirical evidence of bias in the design of experimental stroke studies: a metaepidemiologic approach. Stroke 39 (2008), 929–934.
107. [107] Perel, P., Roberts, I., Sena, E., Wheble, P., Briscoe, C., Sandercock, P., et al. Comparison of treatment effects between animal experiments and clinical trials: systematic review. BMJ, 334, 2007, 197.
108. [108] Tsilidis, K.K., Panagiotou, O.A., Sena, E.S., Aretouli, E., Evangelou, E., Howells, D.W., et al. Evaluation of excess significance bias in animal studies of neurological diseases. PloS Biol, 11(7), 2013, e1001609.
109. [109] Aljasir, B., Ioannidis, J.P., Yurkiewich, A., Moher, D., Higgins, J.P., Arora, P., et al. Assessment of systematic effects of methodological characteristics on candidate genetic associations. Hum Genet 132:2 (2013), 167–178.
110. [110] Evangelou, E., Trikalinos, T.A., Salanti, G., Ioannidis, J.P., Family-based versus unrelated case-control designs for genetic associations. PloS Genet, 2(8), 2006, e123.