Comparison of Pooled Risk Estimates for Adverse Effects from Different Observational Study Designs: Methodological Overview

PLoS ONE

Volumn 8, Issue 8, 2013, Pages

  Golder, Su ^a   Loke, Yoon K ^b   Bland, Martin ^a  

^a UNIVERSITY OF YORK   (United Kingdom)

^b UNIVERSITY OF EAST ANGLIA   (United Kingdom)

Author keywords

[No Author keywords available]

Indexed keywords

NONSTEROID ANTIINFLAMMATORY AGENT; ORAL CONTRACEPTIVE AGENT;

ADVERSE DRUG REACTION; ARTICLE; CASE CONTROL STUDY; CEREBROVASCULAR ACCIDENT; COHORT ANALYSIS; CONFERENCE PAPER; CROSS-SECTIONAL STUDY; EVIDENCE BASED MEDICINE; GASTROINTESTINAL DISEASE; HUMAN; MEDICAL LITERATURE; META ANALYSIS (TOPIC); OBSERVATIONAL STUDY; RANDOMIZED CONTROLLED TRIAL (TOPIC); RISK ASSESSMENT; UNSPECIFIED SIDE EFFECT; VENOUS THROMBOEMBOLISM;

ANTI-INFLAMMATORY AGENTS, NON-STEROIDAL; CASE-CONTROL STUDIES; CONTRACEPTIVES, ORAL, HORMONAL; HORMONE REPLACEMENT THERAPY; HUMANS; META-ANALYSIS AS TOPIC; OBSERVATIONAL STUDY AS TOPIC; ODDS RATIO; RESEARCH DESIGN; RISK ASSESSMENT;

EID: 84882601540     PISSN: None     EISSN: 19326203     Source Type: Journal    
DOI: 10.1371/journal.pone.0071813     Document Type: Article

References (91)

1. Loke Y, Golder S, Vandenbrouche J, (2011) Comprehensive evaluations of the adverse effects of drugs: importance of appropriate study selection and data sources. Ther Adv Drug Saf 2: 59-68.
2. Chou R, Helfand M, (2005) Challenges in systematic reviews that assess treatment harms. Ann Intern Med 142: 1090-1099.
3. Mittmann N, Liu BA, Knowles SR, Shear NH, (1999) Meta-analysis and adverse drug reactions. CMAJ 160: 987.
4. Golder S, Loke YK, Bland M (2011) Meta-analyses of Adverse Effects Data Derived from Randomised Controlled Trials as Compared to Observational Studies: Methodological Overview. PloS Med 8.
5. Ioannidis JP, Mulrow CD, Goodman SN, (2006) Adverse events: the more you search, the more you find. Ann Intern Med 144: 298-300.
6. McDonagh M, Peterson K, Carson S (2006) The impact of including non-randomized studies in a systematic review: a case study. 14th Cochrane Colloquium; October 23-26; Dublin, Ireland.
7. Jacob RF, Lloyd PM, (2000) How to evaluate a dental article about harm. J Prosthet Dent 84: 8-16.
8. Psaty BM, Koepsell TD, Lin D, Weiss NS, Siscovick DS, et al. (1999) Assessment and control for confounding by indication in observational studies. J Am Geriatr Soc 47: 749-754.
9. Skegg DC, (2001) Evaluating the safety of medicines, with particular reference to contraception. Stat Med 20: 3557-3569.
10. Vandenbroucke JP, (2006) What is the best evidence for determining harms of medical treatment? CMAJ 174: 645-646.
11. Chou R, Aronsonb N, Atkinsc D, Ismailad AS, Santaguidad P, et al. (2010) AHRQ Series Paper 4: Assessing harms when comparing medical interventions: AHRQ and the Effective Health-Care Program J Clin Epidemiol. 63: 502-512.
12. Andrews N, Miller E, Taylor B, Lingam R, Simmons A, et al. (2002) Recall bias, MMR, and autism. Arch Dis Child 87: 493-494.
13. Kallen BAJ, (2005) Methodological issues in the epidemiological study of the teratogenicity of drugs. Congenit Anom 45: 44-51.
14. Levine M, Walter S, Lee H, Haines T, Holbrook A, et al. (1994) User's guides to the medical literature, IV: how to use an article about harm. JAMA 271: 1615-1619.
15. Blomgren KJ, Sundstrom A, Steineck G, Wiholm BE, (2006) Interviewer variability - quality aspects in a case-control study. Eur J Epidemiol 21: 267-267.
16. Kim CJ, Berlin JA (2006) The use of meta-analysis in pharmacoepidemiology. In: Strom BL, Kimmel SE, editors. Textbook of pharmacoepidemiology. Chichester: John Wiley & Sons Ltd.
17. Kaufman DW, (2008) Interpretation of associations in pharmacoepidemiology. Semin Hematol 45: 181-188.
18. Britton A, McKee M, Black N, McPherson K, Sanderson C, et al. (1998) Choosing between randomised and non-randomised studies: a systematic review. Health Technol Assess 2: 1-124.
19. Concato J, Shah N, Horwitz RI, (2000) Randomised, controlled trials, observational studies, and the hierarchy of research designs. NEJM 342: 1887-1892.
20. Ioannidis JP, Haidich AB, Pappa M, Pantazis N, Kokori SI, et al. (2001) Comparison of evidence of treatment effects in randomised and non-randomised studies. JAMA 286: 821-830.
21. MacLehose RR, Reeves BC, Harvey IM, Sheldon TA, Russell IT, et al. (2000) A systematic review of comparisons of effect sizes derived from randomised and non-randomised studies. Health Technol Assess 4.
22. Shepherd J, Bagnall A-M, Colquitt J, Dinnes J, Duffy S, et al. (2006) 'Sometimes similar, sometimes different': a systematic review of meta-analyses of random and non-randomized policy intervention studies. 14th Cochrane Colloquium; 23-26 October; Dublin, Ireland.
23. Shikata S, Nakayama T, Noguchi Y, Taji Y, Yamagishi H, (2006) Comparison of effects in randomized controlled trials with observational studies in digestive surgery. Ann Surg 244: 668-676.
24. Oliver S, Bagnall AM, Thomas J, Shepherd J, Sowden A, et al. (2010) Randomised controlled trials for policy interventions: a review of reviews and meta-regression. Health Technol Assess 14: 1-192.
25. Golder S, Loke YK, Zorzela L, (2013) Some improvements are apparent in identifying adverse effects in systematic reviews from 1994 to 2011. J Clin Epidemiol 66: 253-260.
26. Higgins JPT, Thompson SG, Deeks JJ, Altman DG (2003) Measuring inconsistency in meta-analyses. BMJ 327.
27. Perera R, Heneghan C, (2008) Interpretating meta-analysis in systematic reviews. Evid Based Med 13: 67-69.
28. Davies HTO, (1998) Interpreting measures of treatment effect. Hosp Med 59: 499-501.
29. Golder S, Loke YK, Bland M, (2010) Unpublished data can be of value in systematic reviews of adverse effects: methodological overview. J Clin Epidemiol 63: 1071-1081.
30. Borenstein M, Hedges LV, Higgins JPT, Rothstein H (2009) Introduction to Meta-Analysis. Chichester: Wiley.
31. Bland M (2000) An Introduction to Medical Statistics, Oxford: Oxford University Press.
32. Deeks J, Higgins J, Altman DG (2011) Chapter 9: Analysing data and undertaking meta-analyses. In: Higgins JP GS, editor. Cochrane Handbook for Systematic Reviews of Interventions Version 5.1.0 (updated March 2011). The Cochrane Collaboration.
33. Peters JL, Sutton AJ, Jones DR, Abrams KR, Rushton L, (2008) Contour-enhanced meta-analysis funnel plots help distinguish publication bias from other causes of asymmetry. J Clin Epidemiol 61: 991-996.
34. Alghamdi AA, Moussa F, Fremes SE, (2007) Does the use of preoperative aspirin increase the risk of bleeding in patients undergoing coronary artery bypass grafting surgery? Systematic review and meta-analysis. J Card Surg 22: 247-256.
35. Browning DR, Martin RM, (2007) Statins and risk of cancer: a systematic review and metaanalysis. Int J Cancer 120: 833-843.
36. Canonico M, Plu-Bureau G, Lowe GDO, Scarabin PY, (2008) Hormone replacement therapy and risk of venous thromboembolism in postmenopausal women: systematic review and meta-analysis. BMJ 336: 1227-1231.
37. Chou R, Fu R, Carson S, Saha S, Helfand M (2006) Empirical evaluation of the association between methodological shortcomings and estimates of adverse events. Rockville, MD: Agency for Healthcare Research and Quality (AHRQ).
38. Chou R, Fu R, Carson S, Saha S, Helfand M, (2007) Methodological shortcomings predicted lower harm estimates in one of two sets of studies of clinical interventions. J Clin Epidemiol 60: 18-28.
39. Col NF, Kim JA, Chlebowski RT, (2005) Menopausal hormone therapy after breast cancer: a meta-analysis and critical appraisal of the evidence. Breast cancer res 7: R535-540.
40. Cosmi B, Castelvetri C, Milandri M, Rubboli A, Conforti A (2000) The evaluation of rare adverse drug events in Cochrane reviews: the incidence of thrombotic thrombocytopenic purpura after ticlopidine plus aspirin for coronary stenting. 8th Annual Cochrane Colloquium; October; Cape Town; South Africa.
41. Cutler C, Giri S, Jeyapalan S, Paniagua D, Viswanathan A, et al. (2001) Acute and chronic graft-versus-host disease after allogeneic peripheral-blood stem-cell and bone marrow transplantation: a meta-analysis. J Clin Oncol 19: 3685-3691.
42. Jensen P, Mikkelsen T, Kehlet H, (2002) Postherniorrhaphy urinary retention - effect of local, regional, and general anesthesia: a review. Reg Anesth Pain Med 27: 612-617.
43. Loe SM, Sanchez-Ramos L, Kaunitz AM, (2005) Assessing the neonatal safety of indomethacin tocolysis: a systematic review with meta-analysis. Obstet Gynecol 106: 173-179.
44. Loke YK, Derry S, Aronson JK, (2004) A comparison of three different sources of data in assessing the frequencies of adverse reactions to amiodarone. Br J Clin Pharmacol 57: 616-621.
45. McAlister FA, Clark HD, Wells PS, Laupacis A, (1998) Perioperative allogenic blood transfusion does not cause adverse sequelae in patients with cancer: a meta-analysis of unconfounded studies. Br J Surg 85: 171-178.
46. McGettigan P, Henry D. Cardiovascular ischaemia with anti-inflammatory drugs. University of York: York, UK.
47. Nalysnyk L, Fahrbach K, Reynolds MW, Zhao SZ, Ross S, (2003) Adverse events in coronary artery bypass graft (CABG) trials: a systematic review and analysis. Heart 89: 767-772.
48. Papanikolaou PN, Christidi GD, Ioannidis JPA, (2006) Comparison of evidence on harms of medical interventions in randomized and nonrandomized studies. CMAJ 174: 635-641.
49. Ross SD, DiGeorge A, Connelly JE, Whitting GW, McDonnell N, (1998) Safety of GM-CSF in patients with AIDS: A review of the literature. Pharmacotherapy 18: 1290-1297.
50. Siegel CA, Marden SM, Persing SM, Larson RJ, Sands BE, (2009) Risk of lymphoma associated with combination anti-tumor necrosis factor and immunomodulator therapy for the treatment of Crohn's disease: a meta-analysis. Clin Gastroenterol Hepatol 7: 874-881.
51. Singh S, Loke YK, Furberg CD, (2007) Thiazolidinediones and heart failure: A teleo-analysis. Diabetes Care 30: 2148-2153.
52. Torloni MR, Vedmedovska N, Merialdi M, Betran AP, Allen T, et al. (2009) Safety of ultrasonography in pregnancy: WHO systematic review of the literature and meta-analysis. Ultrasound Obstet Gynecol 33: 599-608.
53. Tramer MR, Moore RA, McQuay HJ, (1997) Propofol and bradycardia: causation, frequency and severity. Br J Anaesth 78: 642-651.
54. Tramer MR, Moore RA, Reynolds DJ, McQuay HJ, (2000) Quantitative estimation of rare adverse events which follow a biological progression: a new model applied to chronic NSAID use. Pain 85: 169-182.
55. Vohra S, Johnston BC, Cramer K, Humphreys K, (2007) Adverse events associated with pediatric spinal manipulation: a systematic review. Pediatrics 119: e275-283.
56. Wang T, Collet JP, Shapiro S, Ware MA, (2008) Adverse effects of medical cannabinoids: a systematic review. CMAJ 178: 1669-1678.
57. Jones G, Riley M, Couper D, Dwyer T, (1999) Water fluoridation, bone mass and fracture: a quantitative overview of the literature. Aust N Z J Public Health 23: 34-40.
58. Bollini P, Garcia RLA, Pérez GS, Walker AM, (1992) The impact of research quality and study design on epidemiologic estimates of the effect of nonsteroidal anti-inflammatory drugs on upper gastrointestinal tract disease. Arch Intern Med 152: 1289-1295.
59. Agency for Healthcare Research and Quality (2002) Hormone Replacement Therapy and Risk of Venous Thromboembolism. Rockville, MD: Agency for Healthcare Research and Quality.
60. Capurso G, Schünemann HJ, Terrenato I, Moretti A, Koch M, et al. (2007) Meta-analysis: the use of non-steroidal anti-inflammatory drugs and pancreatic cancer risk for different exposure categories. Aliment Pharmacol Ther 26: 1089-1099.
61. Chan WS, Ray J, Wai EK, Ginsburg S, Hannah ME, et al. (2004) Risk of stroke in women exposed to low-dose oral contraceptives: a critical evaluation of the evidence. Arch Intern Med 164: 741-747.
62. Dolovich LR, Addis A, Vaillancourt JM, Power JD, Koren G, et al. (1998) Benzodiazepine use in pregnancy and major malformations or oral cleft: meta-analysis of cohort and case-control studies. BMJ 317: 839-843.
63. Douketis JD, Ginsberg JS, Holbrook A, Crowther M, Duku EK, et al. (1997) A reevaluation of the risk for venous thromboembolism with the use of oral contraceptives and hormone replacement therapy. Arch Intern Med 157: 1522-1530.
64. Garg PP, Kerlikowske K, Subak L, Grady D, (1998) Hormone replacement therapy and the risk of epithelial ovarian carcinoma: a meta-analysis. Obstet Gynecol 92: 472-479.
65. Gillum LA, Mamidipudi SK, Johnston SC, (2000) Ischemic stroke risk with oral contraceptives: a meta-analysis. JAMA 284: 72-78.
66. Grady D, Gebretsadik T, Kerlikowske K, Ernster V, Petitti D, (1995) Hormone replacement therapy and endometial cancer risk: a meta-analysis. Obstet Gynecol 85: 304-313.
67. Henry D, McGettigan P (2003) Epidemiology overview of gastrointestinal and renal toxicity of NSAIDs. Int J Clin Pract Suppl 135: 43-49.
68. Johnston SC, Colford JM Jr, Gress DR, (1998) Oral contraceptives and the risk of subarachnoid hemorrhage. Neurology 51: 411-418.
69. Koster T, Small RA, Rosendaal FR, Helmerhorst FM, (1995) Oral contraceptives and venous thromboembolism: a quantitative discussion of the uncertainties. J Intern Med 238: 31-37.
70. Leipzig RM, Cumming RG, Tinetti ME, (1999) Drugs and falls in older people: a systematic review and meta-analysis: I. Psychotropic drugs. J Am Geriatr Soc 47: 30-39.
71. Leipzig RM, Cumming RG, Tinetti ME, (1999) Drugs and falls in older people: a systematic review and meta-analysis: II. Cardiac and analgesic drugs. J Am Geriatr Soc 47: 40-50.
72. Loke YK, Singh S, Furberg CD, (2008) Long-term use of thiazolidinediones and fractures in type 2 diabetes: A meta-analysis. CMAJ 180: 32-39.
73. MacLennan SC, MacLennan AH, Ryan P, (1995) Colorectal cancer and oestrogen replacement therapy. A meta-analysis of epidemiological studies. Med J Aust 162: 491-493.
74. McGettigan P, Henry D, (2006) Cardiovascular risk and inhibition of cyclooxygenase: a systematic review of the observational studies of selective and nonselective inhibitors of cyclooxygenase 2. JAMA 296: 1633-1644.
75. Oger E, Scarabin PY, (1999) Assessment of the risk for venous thromboembolism among users of hormone replacement therapy. Drugs Aging 14: 55-61.
76. Ofman JJ, MacLean CH, Straus WL, Morton SC, Berger ML, et al. (2002) A meta-analysis of severe upper gastrointestinal complications of non-steroidal anti-inflammatory drugs. J Rheumatol 29: 804-812.
77. Salhab M, Al Sarakbi W, Mokbel K, (2005) In vitro fertilization and breast cancer risk: a review. Int J Fertil Womens Med 50: 259-266.
78. Scott PA, Kingsley GH, Smith CM, Choy EH, Scott DL, (2007) Non-steroidal anti-inflammatory drugs and myocardial infarctions: comparative systematic review of evidence from observational studies and randomised controlled trials. Ann Rheum Dis 66: 1296-1304.
79. Scott PA, Kingsley GH, Scott DL, (2008) Non-steroidal anti-inflammatory drugs and cardiac failure: meta-analysis of observational studies and randomised controlled trials. Eur J Heart Fail 10: 1102-1107.
80. Smith JS, Green J, Berrington de Gonzalez A, Appleby P, Peto J, et al. (2003) Cervical cancer and use of hormonal contraceptives: a systematic review. Lancet 36: 1159-1167.
81. Takkouche B, Montes-Martínez A, Gill SS, Etminan M, (2007) Psychotropic medications and the risk of fracture: a meta-analysis. Drug Saf 30: 171-184.
82. Woolcott JC, Richardson KJ, Wiens MO, Patel B, Marin J, et al. (2009) Meta-analysis of the impact of 9 medication classes on falls in elderly persons. Arch Intern Med 169: 1952-1960.
83. Schwarz EB, Moretti ME, Nayak S, Koren G, (2008) Risk of hypospadias in offspring of women using loratadine during pregnancy: a systematic review and meta-analysis. Drug Saf 31: 775-788.
84. Bergendal A, Odlind V, Persson I, Kieler H, (2009) Limited knowledge on progestogen-only contraception and risk of venous thromboembolism. Acta Obstet Gynecol Scand 88: 261-266.
85. Bager P, Whohlfahrt J, Westergaard T, (2008) Caesarean delivery and risk of atopy and allergic disease: meta-analysis. Clin Exp Allergy 38: 634-642.
86. Lewallen S, Courtright P, (1998) Epidemiology in Practice: Case-Control Studies. Community Eye Health 11: 57-58.
87. Levine MA, Hamet P, Novosel S, Jolain B, (1997) A prospective comparison of four study designs used in assessing safety and effectiveness of drug therapy in hypertension management. Am J Hypertens 10: 1191-1200.
88. Rothwell PM, (2005) External validity of randomised controlled trials: "to whom do the results of this trial apply?". Lancet 365: 82-93.
89. Brewer T, Colditz GA, (1999) Postmarketing surveillance and adverse drug reactions: current perspectives and future needs. JAMA 281: 824-829.
90. Ross SD, (2001) Drug-related adverse events: a readers' guide to assessing literature reviews and meta-analyses. Arch Intern Med 161: 1041-1046.
91. Sutton AJ, Cooper NJ, Lambert PC, Jones DR, Abrams KR, et al. (2002) Meta-analysis of rare and adverse event data. Expert Rev Pharmacoecon Outcomes Res 2: 367-379.