Using multiple types of studies in systematic reviews of health care interventions - A systematic review

PLoS ONE

Volumn 8, Issue 12, 2013, Pages

  Peinemann, Frank ^a,b   Tushabe, Doreen Allen ^c   Kleijnen, Jos ^a,d  

^a MAASTRICHT UNIVERSITY   (Netherlands)

^b UNIVERSITY OF COLOGNE   (Germany)

^c UNIVERSITY OF BIRMINGHAM   (United Kingdom)

^d KLEIJNEN SYSTEMATIC REVIEWS LTD   (United Kingdom)

Author keywords

[No Author keywords available]

Indexed keywords

CLINICAL TRIALS AS TOPIC; PUBMED; REPRODUCIBILITY OF RESULTS; RESEARCH DESIGN; REVIEW LITERATURE AS TOPIC;

EID: 84893494232     PISSN: None     EISSN: 19326203     Source Type: Journal    
DOI: 10.1371/journal.pone.0085035     Document Type: Review

References (103)

1. Haynes B (1999) Can it work? Does it work? Is it worth it? The testing of health care interventions is evolving. BMJ 319: 652-653. doi:10.1136/bmj.319. 7211.652. PubMed: 10480802.
2. Djulbegovic B, Paul A (2011) From efficacy to effectiveness in the face of uncertainty: indication creep and prevention creep. JAMA 305: 2005-2006. PubMed: 21586716.
3. Victora CG, Habicht JP, Bryce J (2004) Evidence-based public health: moving beyond randomized trials. Am J Public Health 94: 400-405. doi:10.2105/AJPH.94.3.400. PubMed: 14998803. (Pubitemid 38269492)
4. Djulbegovic M, Djulbegovic B (2011) Implications of the principle of question propagation for comparative-effectiveness and "data mining" research. JAMA 305: 298-299. doi:10.1001/jama.2010.2013. PubMed: 21245185.
5. Vandenbroucke JP (2008) Observational research, randomised trials, and two views of medical science. PLoS Med 5: e67. doi:10.1371/journal.pmed.0050067. PubMed: 18336067.
6. CEBM (2009) Levels of evidence. Oxford: Centre of Evidence-Based Medicine (CEBM).
7. Concato J, Peduzzi P, Huang GD, O'Leary TJ, Kupersmith J (2010) Comparative effectiveness research: what kind of studies do we need? J Investig Med 58: 764-769. PubMed: 20479661.
8. Hoppe DJ, Schemitsch EH, Morshed S, Tornetta P 3rd, Bhandari M (2009) Hierarchy of evidence: where observational studies fit in and why we need them. J Bone Joint Surg Am 91 Suppl 3: 2-9. doi:10.2106/JBJS.I.00549. PubMed: 19411493.
9. NCBI (2011) MeSH: Randomized Controlled Trials as Topic. Bethesda: National Center for Biotechnology Information (NCBI), U.S. National Library of Medicine (NLM)
10. NCBI (2011) MeSH: Epidemiologic Study Characteristics as Topic. Bethesda: National Center for Biotechnology Information, U.S. National Library of Medicine.
11. MacLehose RR, Reeves BC, Harvey IM, Sheldon TA, Russell IT et al. (2000) A systematic review of comparisons of effect sizes derived from randomised and non-randomised studies. Health Technol Assess 4: 1-154.
12. Pibouleau L, Boutron I, Reeves BC, Nizard R, Ravaud P (2009) Applicability and generalisability of published results of randomised controlled trials and non-randomised studies evaluating four orthopaedic procedures: methodological systematic review. BMJ 339: b4538. doi:10.1136/bmj.b4538. PubMed: 19920015.
13. Fraser C, Murray A, Burr J (2006) Identifying observational studies of surgical interventions in MEDLINE and EMBASE. BMC Med Res Methodol 6: 41. doi:10.1186/1471-2288-6-41. PubMed: 16919159.
14. Archampong D, Borowski D, Wille-Jørgensen P, Iversen LH (2012) Workload and surgeon's specialty for outcome after colorectal cancer surgery. Cochrane Database Syst Rev 3: CD005391: CD005391 PubMed: 22419309.
15. Atkins D (2007) Creating and synthesizing evidence with decision makers in mind: integrating evidence from clinical trials and other study designs. Med Care 45: S16-S22. doi:10.1097/MLR.0b013e3180616c3f. PubMed: 17909376.
16. Black N (1996) Why we need observational studies to evaluate the effectiveness of health care. BMJ 312: 1215-1218. doi:10.1136/bmj. 312.7040.1215. PubMed: 8634569.
17. Britton A, McKee M, Black N, McPherson K, Sanderson C et al. (1998) Choosing between randomised and non-randomised studies: a systematic review. Health Technol Assess 2: 1-iv. 9793791.
18. Chambers D, Rodgers M, Woolacott N (2009) Not only randomized controlled trials, but also case series should be considered in systematic reviews of rapidly developing technologies. J Clin Epidemiol 62: 1253-1260 e1254
19. Chambers D, Fayter D, Paton F, Woolacott N (2010) Use of non-randomised evidence alongside randomised trials in a systematic review of endovascular aneurysm repair: strengths and limitations. Eur J Vasc Endovasc Surg 39: 26-34. doi:10.1016/j.ejvs.2009.09.010. PubMed: 19836274.
20. Chou R, Aronson N, Atkins D, Ismaila AS, Santaguida P et al. (2010) AHRQ series paper 4: assessing harms when comparing medical interventions: AHRQ and the effective health-care program. J Clin Epidemiol 63: 502-512. doi:10.1016/j.jclinepi.2008.06.007. PubMed: 18823754.
21. Chumbler NR, Kobb R, Brennan DM, Rabinowitz T (2008) Recommendations for research design of telehealth studies. Telemed J E Health 14: 986-989. doi:10.1089/tmj.2008.0108. PubMed: 19035813.
22. Concato J, Lawler EV, Lew RA, Gaziano JM, Aslan M et al. (2010) Observational methods in comparative effectiveness research. Am J Med 123: e16-e23. doi:10.1016/j.amjmed.2010.04.014. PubMed: 21184862.
23. Essock SM, Drake RE, Frank RG, McGuire TG (2003) Randomized controlled trials in evidence-based mental health care: getting the right answer to the right question. Schizophr Bull 29: 115-123. doi:10.1093/oxfordjournals.schbul. a006981. PubMed: 12908666. (Pubitemid 36871062)
24. Fletcher RH (2002) Evaluation of interventions. J Clin Epidemiol 55: 1183-1190. doi:10.1016/S0895-4356(02)00525-5. PubMed: 12547447.
25. Fletcher AE (2009) Controversy over "contradiction": Should randomized trials always trump observational studies? Am J Ophthalmol 147: 384-386. doi:10.1016/j.ajo.2008.04.024. PubMed: 19217953.
26. Gale RP, Eapen M, Logan B, Zhang MJ, Lazarus HM (2009) Are there roles for observational database studies and structured quantification of expert opinion to answer therapy controversies in transplants? Bone Marrow Transplant 43: 435-446. doi:10.1038/bmt.2008.447. PubMed: 19182830.
27. Grzeskowiak LE, Gilbert AL, Morrison JL (2012) Investigating outcomes associated with medication use during pregnancy: a review of methodological challenges and observational study designs. Reprod Toxicol 33: 280-289. doi:10.1016/j.reprotox.2012.01.006. PubMed: 22329969.
28. Hadley J, Yabroff KR, Barrett MJ, Penson DF, Saigal CS et al. (2010) Comparative effectiveness of prostate cancer treatments: evaluating statistical adjustments for confounding in observational data. J Natl Cancer Inst 102: 1780-1793. doi:10.1093/jnci/djq393. PubMed: 20944078.
29. Hartling L, McAlister FA, Rowe BH, Ezekowitz J, Friesen C et al. (2005) Challenges in systematic reviews of therapeutic devices and procedures. Ann Intern Med 142: 1100-1111. doi:10.7326/0003-4819-142-12-Part-2-200506211-00010. PubMed: 15968035. (Pubitemid 40847713)
30. Hodgson R, Bushe C, Hunter R (2007) Measurement of long-term outcomes in observational and randomised controlled trials. Br J Psychiatry Suppl 50: s78-s84. PubMed: 18019049. (Pubitemid 47204560)
31. Horn SD, Gassaway J, Pentz L, James R (2010) Practice-based evidence for clinical practice improvement: an alternative study design for evidence-based medicine. Stud Health Technol Inform 151: 446-460. PubMed: 20407178.
32. Kovesdy CP, Kalantar-Zadeh K (2012) Observational studies versus randomized controlled trials: avenues to causal inference in nephrology. Adv Chronic Kidney Dis 19: 11-18. doi:10.1053/j.ackd. 2011.09.004. PubMed: 22364796.
33. Lewsey JD, Leyland AH, Murray GD, Boddy FA (2000) Using routine data to complement and enhance the results of randomised controlled trials. Health Technol Assess 4: 1-55. PubMed: 11074392.
34. Linde K, Scholz M, Melchart D, Willich SN (2002) Should systematic reviews include non-randomized and uncontrolled studies? The case of acupuncture for chronic headache. J Clin Epidemiol 55: 77-85. doi:10.1016/S0895-4356(01) 00422-X. PubMed: 11781125. (Pubitemid 34040997)
35. McCarthy CM, Collins ED, Pusic AL (2008) Where do we find the best evidence? Plast Reconstr Surg 122: 1942-1951; discussion: 19050548.
36. Mercer SL, DeVinney BJ, Fine LJ, Green LW, Dougherty D (2007) Study designs for effectiveness and translation research: identifying trade-offs. Am J Prev Med 33: 139-154. doi:10.1016/j.amepre. 2007.04.005. PubMed: 17673103.
37. Mitchell AA, Lesko SM (1995) When a randomised controlled trial is needed to assess drug safety. The case of paediatric ibuprofen. Drug Saf 13: 15-24. doi:10.2165/00002018-199513010-00003. PubMed: 8527016.
38. Norris SL, Atkins D (2005) Challenges in using nonrandomized studies in systematic reviews of treatment interventions. Ann Intern Med 142: 1112-1119. doi:10.7326/0003-4819-142-12-Part-2-200506211-00011. PubMed: 15968036. (Pubitemid 40847714)
39. Norris SL, Atkins D, Bruening W, Fox S, Johnson E et al. (2011) Observational studies in systemic reviews of comparative effectiveness: AHRQ and the Effective Health Care Program. J Clin Epidemiol 64: 1178-1186. doi:10.1016/j.jclinepi.2010.04.027. PubMed: 21636246.
40. Ogilvie D, Egan M, Hamilton V, Petticrew M (2005) Systematic reviews of health effects of social interventions: 2. Best available evidence: how low should you go? J Epidemiol Community Health 59: 886-892. doi:10.1136/jech.2005. 034199. PubMed: 16166365. (Pubitemid 41360916)
41. Olivier P, Montastruc JL (2006) The nature of the scientific evidence leading to drug withdrawals for pharmacovigilance reasons in France. Pharmacoepidemiol Drug Saf 15: 808-812. doi:10.1002/pds.1248. PubMed: 16700082. (Pubitemid 44767097)
42. Reeves BC, van Binsbergen J, van Weel C (2005) Systematic reviews incorporating evidence from nonrandomized study designs: reasons for caution when estimating health effects. Eur J Clin Nutr 59 Suppl 1: S155-S161. doi:10.1038/sj.ejcn.1602049. PubMed: 16052184. (Pubitemid 41195197)
43. Rosendaal FR (2001) Bridging case-control studies and randomized trials. Curr Control Trials Cardiovasc Med 2: 109-110. doi:10.1186/ CVM-2-3-109. PubMed: 11806781.
44. Sharma V, Minhas R (2012) Explanatory models are needed to integrate RCT and observational data with the patient's unique biology. J R Soc Med 105: 11-24. doi:10.1258/jrsm.2011.110236. PubMed: 22275494.
45. Shrier I, Boivin JF, Steele RJ, Platt RW, Furlan A et al. (2007) Should meta-analyses of interventions include observational studies in addition to randomized controlled trials? A critical examination of underlying principles. Am J Epidemiol 166: 1203-1209. doi:10.1093/aje/kwm189. PubMed: 17712019. (Pubitemid 350022749)
46. Silverman SL (2009) From randomized controlled trials to observational studies. Am J Med 122: 114-120. doi:10.1016/j.amjmed.2008.09.030. PubMed: 19185083.
47. Vandenbroucke JP (1998) Observational research and evidence-based medicine: What should we teach young physicians? J Clin Epidemiol 51: 467-472. doi:10.1016/S0895-4356(98)00025-0. PubMed: 9635995.
48. Vandenbroucke JP (2004) When are observational studies as credible as randomised trials? Lancet 363: 1728-1731. doi:10.1016/S0140-6736(04)16261-2. PubMed: 15158638.
49. Vandenbroucke JP (2009) The HRT controversy: observational studies and RCTs fall in line. Lancet 373: 1233-1235. doi:10.1016/S0140-6736(09)60708-X. PubMed: 19362661.
50. Vandenbroucke JP (2011) Why do the results of randomised and observational studies differ? BMJ 343: d7020. doi:10.1136/bmj.d7020. PubMed: 22065658.
51. Wilcken B (2001) Rare diseases and the assessment of intervention: what sorts of clinical trials can we use? J Inherit Metab Dis 24: 291-298. doi:10.1023/A:1010387522195. PubMed: 11405347.
52. Zlowodzki M, Jonsson A, Bhandari M (2006) Common pitfalls in the conduct of clinical research. Med Princ Pract 15: 1-8. doi:10.1159/000089379. PubMed: 16340221. (Pubitemid 41790159)
53. Kunz R, Oxman AD (1998) The unpredictability paradox: review of empirical comparisons of randomised and non-randomised clinical trials. BMJ 317: 1185-1190. doi:10.1136/bmj.317.7167.1185. PubMed: 9794851. (Pubitemid 28488507)
54. Kunz R, Vist G, Oxman AD (2007) Randomisation to protect against selection bias in healthcare trials. Cochrane Database Syst Rev: MR: 000012. PubMed: 2149141517443633.
55. Odgaard-Jensen J, Vist GE, Timmer A, Kunz R, Akl EA et al. (2011) Randomisation to protect against selection bias in healthcare trials. Cochrane Database Syst Rev: MR: 000012. PubMed: 21491415.
56. Savović J, Jones HE, Altman DG, Harris RJ, Jüni P et al. (2012) Influence of reported study design characteristics on intervention effect estimates from randomized, controlled trials. Ann Intern Med 157: 429-438. doi:10.7326/0003-4819-157-6-201209180-00537. PubMed: 22945832.
57. Ioannidis JP (2005) Contradicted and initially stronger effects in highly cited clinical research. JAMA 294: 218-228. doi:10.1001/jama. 294.2.218. PubMed: 16014596.
58. Ioannidis JP, Haidich AB, Pappa M, Pantazis N, Kokori SI et al. (2001) Comparison of evidence of treatment effects in randomized and nonrandomized studies. JAMA 286: 821-830. doi:10.1001/jama. 286.7.821. PubMed: 11497536.
59. Reeves BC, Deeks JJ, Higgins JPT, Wells GA (2011) Chapter 13: Including non-randomized studies. In: JPT HigginsS Green. Cochrane Handbook for Systematic Reviews of Interventions Version 510 [updated March 2011]. Oxford, UK: The Cochrane Collaboration
60. Higgins JPT, Altman DG, Sterne JAC (2011) Chapter 8: Assessing risk of bias in included studies. In: JPT HigginsS Green. Cochrane Handbook for Systematic Reviews of Interventions Version 510 [updated March 2011]: The Cochrane Collaboration
61. Hadley G, Derry S, Moore RA, Wee B (2009) Can observational studies provide a realistic alternative to randomized controlled trials in palliative care? J Pain Palliat Care Pharmacother 23: 106-113. doi:10.1080/ 15360280902899921. PubMed: 19492211.
62. Abraham NS, Byrne CJ, Young JM, Solomon MJ (2010) Meta-analysis of well-designed nonrandomized comparative studies of surgical procedures is as good as randomized controlled trials. J Clin Epidemiol 63: 238-245. doi:10.1016/j.jclinepi.2009.04.005. PubMed: 19716267.
63. Algra AM, Rothwell PM (2012) Effects of regular aspirin on long-term cancer incidence and metastasis: a systematic comparison of evidence from observational studies versus randomised trials. Lancet Oncol 13: 518-527. doi:10.1016/S1470-2045(12)70112-2. PubMed: 22440112.
64. Antman K, Amato D, Wood W, Carson J, Suit H et al. (1985) Selection bias in clinical trials. J Clin Oncol 3: 1142-1147. PubMed: 4020412. (Pubitemid 15004838)
65. Aslani N, Brown CJ (2010) Does mesh offer an advantage over tissue in the open repair of umbilical hernias? A systematic review and meta-analysis. Hernia 14: 455-462. doi:10.1007/s10029-010-0705-9. PubMed: 20635190.
66. Benis MM (2002) Are pacifiers associated with early weaning from breastfeeding? Adv Neonatal Care 2: 259-266. doi:10.1016/S1536-0903(02)70003-9. PubMed: 12881939.
67. Benson K, Hartz AJ (2000) A comparison of observational studies and randomized, controlled trials. N Engl J Med 342: 1878-1886. doi:10.1056/ NEJM200006223422506. PubMed: 10861324. (Pubitemid 30419053)
68. Bhandari M, Tornetta P 3rd, Ellis T, Audige L, Sprague S et al. (2004) Hierarchy of evidence: differences in results between non-randomized studies and randomized trials in patients with femoral neck fractures. Arch Orthop Trauma Surg 124: 10-16. doi:10.1007/s00402-003-0559-z. PubMed: 14576955. (Pubitemid 38263421)
69. Carroll D, Tramèr M, McQuay H, Nye B, Moore A (1996) Randomization is important in studies with pain outcomes: systematic review of transcutaneous electrical nerve stimulation in acute postoperative pain. Br J Anaesth 77: 798-803. doi:10.1093/bja/77.6.798. PubMed: 9014639. (Pubitemid 27023321)
70. (1984) Coronary artery surgery study (CASS): a randomized trial of coronary artery bypass surgery. Comparability of entry characteristics and survival in randomized patients and nonrandomized patients meeting randomization criteria. J Am Coll Cardiol 3: 114-128. doi:10.1016/S0735-1097(84)80437-4. PubMed: 6361099.
71. Cheng Y, Xiong XZ, Wu SJ, Lin YX, Cheng NS (2012) Laparoscopic vs. open cholecystectomy for cirrhotic patients: a systematic review and meta-analysis. Hepatogastroenterology 59: 1727-1734. PubMed: 22193435.
72. Choi HJ, Hahn S, Lee J, Park BJ, Lee SM et al. (2012) Surfactant lavage therapy for meconium aspiration syndrome: a systematic review and meta-analysis. Neonatology 101: 183-191. doi:10.1159/000329822. PubMed: 22067375.
73. Clagett GP, Youkey JR, Brigham RA, Orecchia PM, Salander JM et al. (1984) Asymptomatic cervical bruit and abnormal ocular pneumoplethysmography: a prospective study comparing two approaches to management. Surgery 96: 823-830. PubMed: 6387988. (Pubitemid 15209009)
74. Colditz GA, Miller JN, Mosteller F (1989) How study design affects outcomes in comparisons of therapy. I: Medical. Stat Med 8: 441-454. doi:10.1002/sim.4780080408. PubMed: 2727468. (Pubitemid 19105892)
75. Conaty S, Watson L, Dinnes J, Waugh N (2004) The effectiveness of pneumococcal polysaccharide vaccines in adults: a systematic review of observational studies and comparison with results from randomised controlled trials. Vaccine 22: 3214-3224. doi:10.1016/j.vaccine. 2003.08.050. PubMed: 15297076.
76. Concato J, Shah N, Horwitz RI (2000) Randomized, controlled trials, observational studies, and the hierarchy of research designs. N Engl J Med 342: 1887-1892. doi:10.1056/NEJM200006223422507. PubMed: 10861325. (Pubitemid 30419054)
77. Deeks JJ, Dinnes J, D'Amico R, Sowden AJ, Sakarovitch C et al. (2003) Evaluating non-randomised intervention studies. Health Technol Assess 7: iii-iix, 1-173 PubMed: 14499048.
78. Edwards JP, Kelly EJ, Lin Y, Lenders T, Ghali WA et al. (2012) Meta-analytic comparison of randomized and nonrandomized studies of breast cancer surgery. Can J Surg 55: 155-162. doi:10.1503/cjs. 023410. PubMed: 22449722.
79. Flossmann E, Rothwell PM, British Doctors Aspirin T, the UKTIAAT (2007) Effect of aspirin on long-term risk of colorectal cancer: consistent evidence from randomised and observational studies. Lancet 369: 1603-1613. (Pubitemid 46710123)
80. Franklin ME, Abramowitz JS, Kozak MJ, Levitt JT, Foa EB (2000) Effectiveness of exposure and ritual prevention for obsessive-compulsive disorder: randomized compared with nonrandomized samples. J Consult Clin Psychol 68: 594-602. doi:10.1037/0022-006X. 68.4.594. PubMed: 10965635.
81. Furlan AD, Tomlinson G, Jadad AA, Bombardier C (2008) Examining heterogeneity in meta-analysis: comparing results of randomized trials and nonrandomized studies of interventions for low back pain. Spine (Phila Pa 1976) 33: 339-348. PubMed: 18303468.
82. Furlan AD, Tomlinson G, Jadad AA, Bombardier C (2008) Methodological quality and homogeneity influenced agreement between randomized trials and nonrandomized studies of the same intervention for back pain. J Clin Epidemiol 61: 209-231. doi:10.1016/j.jclinepi.2007.04.019. PubMed: 18226744.
83. Golder S, Loke YK, Bland M (2011) Meta-analyses of adverse effects data derived from randomised controlled trials as compared to observational studies: methodological overview. PLoS Med 8: e1001026. PubMed: 21559325.
84. Gross CP, Garg PP, Krumholz HM (2005) The generalizability of observational data to elderly patients was dependent on the research question in a systematic review. J Clin Epidemiol 58: 130-137. doi:10.1016/j.jclinepi.2004. 10.001. PubMed: 15680745. (Pubitemid 40174920)
85. Guyatt GH, DiCenso A, Farewell V, Willan A, Griffith L (2000) Randomized trials versus observational studies in adolescent pregnancy prevention. J Clin Epidemiol 53: 167-174. doi:10.1016/S0895-4356(99)00160-2. PubMed: 10729689. (Pubitemid 30151222)
86. Hannan EL (2008) Randomized clinical trials and observational studies: guidelines for assessing respective strengths and limitations. JACC Cardiovasc Interv 1: 211-217. doi:10.1016/j.jcin.2008.01.008. PubMed: 19463302.
87. Hlatky MA, Califf RM, Harrell FE Jr., Lee KL, Mark DB et al. (1988) Comparison of predictions based on observational data with the results of randomized controlled clinical trials of coronary artery bypass surgery. J Am Coll Cardiol 11: 237-245. doi:10.1016/0735-1097(88)90086-1. PubMed: 3276752. (Pubitemid 18051344)
88. Kuss O, Legler T, Börgermann J (2011) Treatments effects from randomized trials and propensity score analyses were similar in similar populations in an example from cardiac surgery. J Clin Epidemiol 64:1076-1084. doi:10.1016/j.jclinepi.2011.01.005. PubMed: 21482068.
89. Lawlor DA, Davey Smith G, Kundu D, Bruckdorfer KR, Ebrahim S (2004) Those confounded vitamins: what can we learn from the differences between observational versus randomised trial evidence? Lancet 363: 1724-1727. doi:10.1016/S0140-6736(04)16260-0. PubMed: 15158637. (Pubitemid 38670332)
90. Müeller D, Sauerland S, Neugebauer EA, Immenroth M (2010) Reported effects in randomized controlled trials were compared with those of nonrandomized trials in cholecystectomy. J Clin Epidemiol 63: 1082-1090. doi:10.1016/j.jclinepi.2009.12.009. PubMed: 20346627.
91. Naudet F, Maria AS, Falissard B (2011) Antidepressant response in major depressive disorder: a meta-regression comparison of randomized controlled trials and observational studies. PLOS ONE 6: e20811. doi:10.1371/journal.pone. 0020811. PubMed: 21687681.
92. Papanikolaou PN, Christidi GD, Ioannidis JP (2006) Comparison of evidence on harms of medical interventions in randomized and nonrandomized studies. CMAJ 174: 635-641. doi:10.1503/cmaj. 050873. PubMed: 16505459.
93. Phillips AN, Grabar S, Tassie JM, Costagliola D, Lundgren JD et al. (1999) Use of observational databases to evaluate the effectiveness of antiretroviral therapy for HIV infection: comparison of cohort studies with randomized trials. EuroSIDA, the French Hospital Database on HIV and the Swiss HIV Cohort Study Groups. AIDS 13: 2075-2082. doi:10.1097/00002030-199910220- 00010. PubMed: 10546860. (Pubitemid 29509584)
94. (1994) Worldwide collaborative observational study and meta-analysis on allogenic leukocyte immunotherapy for recurrent spontaneous abortion. Recurrent Miscarriage Immunotherapy Trialists Group. Am J Reprod Immunol 32: 55-72.
95. Rovers MM, Straatman H, Ingels K, van der Wilt GJ, van den Broek P et al. (2001) Generalizability of trial results based on randomized versus nonrandomized allocation of OME infants to ventilation tubes or watchful waiting. J Clin Epidemiol 54: 789-794. doi:10.1016/S0895-4356(01)00340-7. PubMed: 11470387. (Pubitemid 32722182)
96. Schmoor C, Caputo A, Schumacher M (2008) Evidence from nonrandomized studies: a case study on the estimation of causal effects. Am J Epidemiol 167: 1120-1129. doi:10.1093/aje/kwn010. PubMed: 18334500. (Pubitemid 351601242)
97. Shea MK, Houston DK, Nicklas BJ, Messier SP, Davis CC et al. (2010) The effect of randomization to weight loss on total mortality in older overweight and obese adults: the ADAPT Study. J Gerontol A Biol Sci Med Sci 65: 519-525. PubMed: 20080875.
98. Shikata S, Nakayama T, Noguchi Y, Taji Y, Yamagishi H (2006) Comparison of effects in randomized controlled trials with observational studies in digestive surgery. Ann Surg 244: 668-676. doi:10.1097/01.sla.0000225356.04304. bc. PubMed: 17060757. (Pubitemid 44629938)
99. Tzoulaki I, Siontis KC, Ioannidis JP (2011) Prognostic effect size of cardiovascular biomarkers in datasets from observational studies versus randomised trials: meta-epidemiology study. BMJ 343: d6829. doi:10.1136/bmj. d6829. PubMed: 22065657.
100. Vis AN, Roemeling S, Reedijk AM, Otto SJ, Schröder FH (2008) Overall survival in the intervention arm of a randomized controlled screening trial for prostate cancer compared with a clinically diagnosed cohort. Eur Urol 53: 91-98. doi:10.1016/j.eururo.2007.06.001. PubMed: 17583416. (Pubitemid 350166711)
101. Vist GE, Bryant D, Somerville L, Birminghem T, Oxman AD (2008) Outcomes of patients who participate in randomized controlled trials compared to similar patients receiving similar interventions who do not participate. Cochrane Database Syst Rev: MR: 000009.
102. Wilkes MM, Navickis RJ, Chan WW, Lewiecki EM (2010) Bisphosphonates and osteoporotic fractures: a cross-design synthesis of results among compliant/persistent postmenopausal women in clinical practice versus randomized controlled trials. Osteoporos Int 21: 679-688. doi:10.1007/s00198-009-0991-1. PubMed: 19572092.
103. Wolfe F, Michaud K, Dewitt EM (2004) Why results of clinical trials and observational studies of antitumour necrosis factor (anti-TNF) therapy differ: methodological and interpretive issues. Ann Rheum Dis 63 Suppl 2: ii13-ii17. PubMed: 15479864. (Pubitemid 39424652)