Healthcare outcomes assessed with observational study designs compared with those assessed in randomized trials

Cochrane Database of Systematic Reviews

Volumn 2014, Issue 4, 2014, Pages

  Anglemyer, Andrew ^a   Horvath, Hacsi T ^a   Bero, Lisa ^a  

^a UNIVERSITY OF CALIFORNIA   (United States)

Author keywords

[No Author keywords available]

Indexed keywords

BIBLIOGRAPHIC DATABASE; CLINICAL EFFECTIVENESS; CLINICAL OUTCOME; CLINICAL RESEARCH; DRUG THERAPY; EVIDENCE BASED MEDICINE; HEALTHCARE OUTCOME; HUMAN; INTERMETHOD COMPARISON; META ANALYSIS (TOPIC); METHODOLOGY; OBSERVATIONAL METHOD; OUTCOME ASSESSMENT; PRIORITY JOURNAL; PROPENSITY SCORE; PROSPECTIVE STUDY; QUANTITATIVE ANALYSIS; RANDOMIZED CONTROLLED TRIAL (TOPIC); REVIEW; STATISTICAL ANALYSIS; STUDY DESIGN; SYSTEMATIC REVIEW; SYSTEMATIC REVIEW (TOPIC); COMPARATIVE STUDY; OBSERVATIONAL STUDY; PROCEDURES;

HUMANS; META-ANALYSIS AS TOPIC; OBSERVATIONAL STUDY AS TOPIC; OUTCOME ASSESSMENT (HEALTH CARE); RANDOMIZED CONTROLLED TRIALS AS TOPIC;

EID: 84901502488     PISSN: None     EISSN: 1469493X     Source Type: Journal    
DOI: 10.1002/14651858.MR000034.pub2     Document Type: Review

References (73)

1. Benson K, Hartz A. A comparison of observational studies and randomized, controlled trials. New England Journal of Medicine 2000;342(25):1878-86.
2. Beynon R, Harris R, Sterne JAC, et al.The quantification of bias in randomised and non-randomised studies: the BRANDO NRS Study [Poster]. 16th Cochrane Colloquium. Freiburg im Breisgau, Germany, 3-7 October, 2008.
3. Bhandari M, Tornetta PIII, Ellis T, Audige L, Sprague S, Kuo JC, et al.Hierarchy of evidence: differences in results between non-randomized studies and randomized trials in patients with femoral neck fractures. Archives of Orthopaedic and Trauma Surgery 2004;124(1):10-6.
4. Concato J, Shah N, Horwitz R. Randomized, controlled trials, observational studies, and the hierarchy of research designs. New England Journal of Medicine 2000;342(25):1887-92.
5. Edwards J, Kelly E, Lin Y, Lenders T, Ghali W, Graham A. Meta-analytic comparison of randomized and nonrandomized studies of breast cancer surgery. Canadian Journal of Surgery 2012;55(3):155-62.
6. Furlan A, Tomlinson G, Jadad A, Bombardier C. Examining heterogeneity in meta-analysis: comparing results of randomized trials and nonrandomized studies of interventions for low back pain. Spine 2008;33(3):339-48.
7. Golder S, Loke YK, Bland M. Meta-analyses of adverse effects data derived from randomised controlled trials as compared to observational studies: methodological review. PLoS Medicine 2011;8(5):e1001026.
8. Ioannidis J, Haidich A, Pappa M, Pantazis N, Kokori SI, Tektonidou MG, et al.Comparison of evidence of treatment effects in randomized and nonrandomized studies. JAMA 2001;286(7):821-30.
9. Kuss O, Legler T, Boergermann J. Treatments effects from randomized trials and propensity score analyses were similar in similar populations in an example from cardiac surgery. Journal of Clinical Epidemiology 2011;64:1076-84.
10. Lonjon G, Boutron I, Trinquart L, Ahmad N, Aim F, Nizard R, et al.Comparison of treatment effect estimates from prospective nonrandomized studies with propensity score analysis and randomized controlled trials of surgical procedures. Annals of Surgery 2013;259(1):18-25.
11. Müeller D, Sauerland S, Neugebauer EA, Immenroth M. Reported effects in randomized controlled trials were compared with those of nonrandomized trials in cholecystectomy. Journal of Clinical Epidemiology 2010;63(10):1082-90.
12. Naudet F, Maria AS, Falissard B. Antidepressant response in major depressive disorder: a meta-regression comparison of randomized controlled trials and observational studies. PLoS One 2011;6(6):e20811.
13. Oliver S, Bagnall AM, Thomas J, Shepherd J, Sowden A, White I, et al.Randomised controlled trials for policy interventions: a review of reviews and meta-regression. Health Technology Assessment 2010;14(16):1.
14. Papanikolaou P, Christidi G, Ioannidis J. Comparison of evidence on harms of medical interventions in randomized and nonrandomized studies. CMAJ: Canadian Medical Association Journal 2006;174(5):635-41.
15. Shikata S, Nakayama T, Noguchi Y, Taji Y, Yamagishi H. Comparison of effects in randomized controlled trials with observational studies in digestive surgery. Annals of Surgery 2006;244(5):668-76.
16. Ather S, Bangalore S, Vemuri S, Cao LB, Bozkurt B, Messerli FH. Trials on the effect of cardiac resynchronization on arterial blood pressure in patients with heart failure. American Journal of Cardiology 2011;107(4):561-78.
17. Begg C, Pilote L. A model for incorporating historical controls into a meta-analysis. Biometrics 1991;47(3):899-906.
18. Beyersmann J, Gastmeier P, Wolkewitz M, Schumacher M. An easy mathematical proof showed that time-dependent bias inevitably leads to biased effect estimation. Journal of Clinical Epidemiology 2008;61(12):1216-21.
19. Bosco J, Silliman R, Thwin S, Geiger AM, Buist DS, Prout MN, et al.A most stubborn bias: no adjustment method fully resolves confounding by indication in observational studies. Journal of Clinical Epidemiology 2010;3(1):64-74.
20. Britton A, McKee M, Black N, McPherson K, Sanderson C, Bain C. Choosing between randomised and non-randomised studies: a systematic review. Health Technology Assessment 1998;2(13):1-124.
21. Chambers D, Fayter D, Paton F, Woolacott N. Use of non-randomised evidence alongside randomised trials in a systematic review of endovascular aneurysm repair: strengths and limitations. European Journal of Vascular and Endovascular Surgery 2010;39(1):26-34.
22. Coulam CB, Clark DA, Collins J, Scott JR, Schlesselman JS, Aoki K, et al. Recurrent Miscarriage Immunotherapy Trialists Group. Worldwide collaborative observational study and meta-analysis on allogenic leukocyte immunotherapy for recurrent spontaneous abortion. American Journal of Reproductive Immunology 1994;32(2):55-72.
23. Dahabreh I, Sheldrick R, Paulus J, Chung M, Varvarigou V, Jafri H, et al.Do observational studies using propensity score methods agree with randomized trials? A systematic comparison of studies on acute coronary syndromes. European Heart Journal 2012;33:1893-901.
24. Deeks JJ, D'Amico R, Sakarovitch C, et al.Are comparability of case-mix and the use of statistical adjustment markers of quality in non-randomised studies? An empirical investigation. 4th Symposium on Systematic Reviews: Pushing the Boundaries. Oxford, UK, 2002.
25. Deeks JJ, Dinnes J, D'Amico R, Sowden AJ, Sakarovitch C, Song F, et al. International Stroke Trial Collaborative Group, European Carotid Surgery Trial Collaborative Group. Evaluating non-randomised intervention studies. Health Technology Assessment 2003;7(27):1-173.
26. Diehl L, Perry D. A comparison of randomized concurrent control groups with matched historical control groups: are historical controls valid?. Journal of Clinical Oncology 1986;4(7):1114-20.
27. Diez P, Vogelius IS, Bentzen SM. A new method for synthesizing radiation dose-response data from multiple trials applied to prostate cancer:. International Journal of Radiation Oncology, Biology, Physics 2010;77(4):1066-71.
28. Flossmann E, Rothwell P. Effect of aspirin on long-term risk of colorectal cancer: consistent evidence from randomised and observational studies. Lancet 2007;369:1603-13.
29. Hallstrom A, Anderson Jl, Cobb La, Friedman PL, Herre JM, Klein RC, et al.Advantages and disadvantages of trial designs: a review of analysis methods for ICD studies. Pacing and Clinical Electrophysiology: PACE 2000;23(6):1029-38.
30. Henry D, Moxey A, O'Connell D. Agreement between randomized and non-randomized studies: the effects of bias and confounding. 9th Cochrane Colloquium. Lyon, France, 9-13 October, 2001.
31. Hlatky MA, Califf RM, Harrell FE Jr, Lee KL, Mark DB, Pryor DB. Comparison of predictions based on observational data with the results of randomized controlled clinical trials of coronary artery bypass surgery. Journal of the American College of Cardiology 1988;11(2):237-45.
32. Ioannidis JP. Contradicted and initially stronger effects in highly cited clinical research. JAMA 2005;294(2):218-28.
33. Labarere J, Bosson JL, Sevestre MA, Delmas AS, Dupas S, Thenault MH, et al.Graduated compression stocking thromboprophylaxis for elderly inpatients. Journal of General Internal Medicine 2006;21(12):1282-7.
34. LaTorre G, de Waure C, Specchia ML, Nicolotti N, Capizzi S, Bilotta A, et al.Does quality of observational studies affect the results of a meta-analysis?: the case of cigarette smoking and pancreatic cancer. Pancreas 2009;38(3):241-7.
35. Linde K, Streng A, Hoppe A, Weidenhammer W, Wagenpfeil S, Melchart D. Randomized trial vs. observational study of acupuncture for migraine found that patient characteristics differed but outcomes were similar. Journal of Clinical Epidemiology 2007;60(3):280-7.
36. Lipsey M, Wilson D. The efficacy of psychological, educational, and behavioral treatment. American Psychologist 1993;48(12):1181-209.
37. Loke Y, Cavallazzi R, Singh S. Risk of fractures with inhaled corticosteroids in COPD: systematic review and meta-analysis of randomised controlled trials and observational studies. Thorax 2011;66:699-708.
38. MacLehose RR, Reeves BC, Harvey IM, Sheldon TA, Russell IT, Black AM. A systematic review of comparisons of effect sizes derived from randomised and non-randomised studies. Health Technology Assessment 2000;4(34):1-154.
39. Mak A, Cheung MW, Chun-Man Ho R, Ai-Cia Cheak A, Chak Sing Lau C. Bisphosphonates and atrial fibrillation: Bayesian meta-analyses of randomized controlled trials and observational studies. BMC Musculoskeletal Disorders 2009;10:113.
40. McCarron CE, Pullenayegum EM, Thabane L, Goeree R, Tarride JE. The importance of adjusting for potential confounders in Bayesian hierarchical models synthesising evidence from randomised and non-randomised studies: an application comparing treatments for abdominalaortic aneurysms. BMC Medical Research Methodology 2010;10:64.
41. McKee M, Britton A, Black N, McPherson K, Sanderson C, Bain C. Interpreting the evidence: choosing between randomised and non-randomised studies. BMJ 1999;319:312-5.
42. Moreira RF, Foltran FA, Albuquerque-Sendin F, Mancini MC, Coury HJCG. Comparison of randomized and non- randomized controlled trials evidence regarding the effectiveness of workplace exercise on musculoskeletal pain control. Work 2012;41(Suppl 1):4782-9.
43. Ni Chroinin D, Asplund K, Asberg S, Callaly E, Cuadrado-Godia E, Diez-Tejedor E, et al.Statin therapy and outcome after ischemic stroke: systematic review and meta-analysis of observational studies and randomized trials. Stroke 2013;44(2):448-56.
44. Nixdorf D, Moana-Filho E, Law A, McGuire LA, Hodges JS, John MT. Frequency of persistent tooth pain after root canal therapy: a systematic review and meta-analysis. Journal of Endodotics 2010;36(2):224-30.
45. Ottenbacher K. Impact of random assignment on study outcome: an empirical examination. Controlled Clinical Trials 1992;13:50-61.
46. Papanastassiou I, Phillips F, van Meirhaeghe J, Berenson J, Andersson G, Chung G, et al.Comparing effects of kyphoplasty, vertebroplasty, and non- surgical management in a systematic review of randomized and non-randomized controlled studies. European Spine Journal 2012;21:1826-43.
47. Phillips AN, Grabar S, Tassie JM, Costagliola D, Lundgren JD, Egger M. Use of observational databases to evaluate the effectiveness of antiretroviral therapy for HIV infection: comparison of cohort studies with randomized trials. AIDS 1999;13(15):2075-82.
48. Pratt N, Roughead E, Salter A, Ryan P. Choice of observational study design impacts on measurement of antipsychotic risks in the elderly: a systematic review. BMC Medical Research Methodology 2012;12(72):1-19.
49. Pyorala S, Huttunen N, Uhari M. A review and meta-analysis of hormonal treatment of cryptorchidism. Journal of Clinical Endocrinology and Metabolism 1995;80(9):2795-9.
50. Schmoor C, Caputo A, Schumacher M. Evidence from nonrandomized studies: a case study on the estimation of causal effects. American Journal of Epidemiology 2008;167(9):1120-9.
51. Scott P, Kingsley G, Smith C, Choy EH, Scott DL. Non-steroidal anti-inflammatory drugs and myocardial infarctions: comparative systematic review of evidence from observational studies and randomised controlled trials. Annals of the Rheumatic Diseases 2007;66(10):1296-304.
52. Shah B, Laupacis A, Hux J, Austin PC. Propensity score methods gave similar results to traditional regression modeling in observational studies: a systematic review. Journal of Clinical Epidemiology 2005;58(6):550-9.
53. Shepherd J, Bagnall A, Colquitt J. 'Sometimes similar, sometimes different': a systematic review of meta-analyses of randomised and non-randomised policy intervention studies. 14th Cochrane Colloquium,. Dublin, Ireland, 23-26 October, 2006.
54. Steinberg K, Smith J, Thacker S, Stroup DF. Breast cancer risk and duration of estrogen use: the role of study design in meta-analysis. Epidemiology 1994;5(4):415-21.
55. Stukel T, Fisher E, Wennberg D, Alter DA, Gottlieb DJ, Vermeulen MJ. Analysis of observational studies in the presence of treatment selection bias: effects of invasive cardiac management on AMI survival using propensity score and instrumental variable methods. JAMA 2007;297(3):278-85.
56. Ward LC, Fielding J, Dunn J, Kelly KA. The selection of cases for randomised trials:a registry survey of concurrent trial and non-trial patients. British Journal of Cancer 1992;66(5):943-50.
57. Watson A, Vandekerckhove P, Lilford R, Vail A, Brosens I, Hughes E. A meta-analysis of the therapeutic role of oil soluble contrast media at hysterosalpingography: a surprising result?. Fertility and Sterility 1994;61(3):470-7.
58. Williams PT, Fortmann SP, Farquhar JW, Varady A, Mellen S. A comparison of statistical methods for evaluating risk factor changes in community-based studies: an example from the Stanford Three-Community Study. Journal of Chronic Diseases 1981;34(11):565-71.
59. Wilson D, Lipsey M. The role of method in treatment effectiveness research: evidence from meta-analysis. Psychological Methods 2001;6(4):413-29.
60. Altman D, Bland J. Interaction revisited: the difference between two estimates. BMJ 2003;326:219.
61. Dorsey ER, de Roulet J, Thompson JP, Reminick JL, Thai A, White-Stellato Z, et al.Funding of US Biomedical Research, 2003-2008. JAMA 2010;303(2):137-43.
62. Higgins JPT, Green S (editors). Cochrane Handbook for Systematic Reviews of Interventions. Chichester: John Wiley & Sons Ltd, 2011.
63. Institute of Medicine. Initial National Priorities for Comparitive Effectiveness Research. Institute of Medicine, Washington DC 2009.
64. Kamerow D. PCORI: odd name, important job, potential trouble. BMJ 2011;342:d2635.
65. Kunz R, Oxman AD. The unpredictability paradox: review of empirical comparisons of randomised and non-randomised clinical trials. BMJ 1998;317(7167):1185-90.
66. Kunz R, Vist G, Oxman AD. Randomisation to protect against selection bias in healthcare trials. Cochrane Database of Systematic Reviews 2002, Issue 4. [DOI:].
67. Lundh A, Sismondo S, Lexchin J, Busuioc OA, Bero L. Industry sponsorship and research outcome. Cochrane Database of Systematic Reviews 2012, Issue 12. [DOI:].
68. Montori VM, Wilczynski NL, Morgan D, Haynes RB, Hedges Team. Optimal search strategies for retrieving systematic reviews from Medline: analytical survey. BMJ 2005 Jan 8;330(7482):68.
69. Odgaard-Jensen J, Timmer A, Kunz R, Akl EA, Schünemann H, Briel M, et al.Randomisation to protect against selection bias in healthcare trials. Cochrane Database of Systematic Reviews 2011, Issue 4. [DOI:].
70. Patient Centered Outcomes Research Institute (PCORI). PCORI Methodology Standards. http://www.pcori.org/assets/PCORI-Methodology-Standards.pdf December 14, 2012.
71. Reeves B, Higgins J, Ramsay C, Shea B, Tugwall P, Wells G. An introduction to methodological issues when including non-randomised studies in systematic reviews on the effects of interventions. Research Synthesis Methods 2013;4:1-11.
72. Sacks H, Chalmers T, Smith HJ. Randomized versus historical controls for clinical trials. American Jourrnal of Medicine 1982;72(2):233-40.
73. Sampson M, McGowan J, Cogo E, Grimshaw J, Moher D, Lefebvre C. An evidence-based practice guideline for the peer review of electronic search strategies. Journal of Clinical Epidemiology 2009;62(9):944-52.