Risk of Bias in Systematic Reviews of Non-Randomized Studies of Adverse Cardiovascular Effects of Thiazolidinediones and Cyclooxygenase-2 Inhibitors: Application of a New Cochrane Risk of Bias Tool

PLoS Medicine

Volumn 13, Issue 4, 2016, Pages

  Bilandzic, Anja ^a   Fitzpatrick, Tiffany ^b   Rosella, Laura ^a,c,d   Henry, David ^a,d,e  

^a UNIVERSITY OF TORONTO   (Canada)

^b Support Unit   (Canada)

^c PUBLIC HEALTH ONTARIO   (Canada)

^d INSTITUTE FOR CLINICAL EVALUATIVE SCIENCES   (Canada)

^e UNIVERSITY OF TORONTO   (Canada)

Author keywords

[No Author keywords available]

Indexed keywords

2,4 THIAZOLIDINEDIONE DERIVATIVE; CELECOXIB; CYCLOOXYGENASE 2 INHIBITOR; DICLOFENAC; NAPROXEN; PIOGLITAZONE; PIROXICAM; ROFECOXIB; ROSIGLITAZONE; ANTIDIABETIC AGENT;

ARTICLE; CARDIOVASCULAR DISEASE; CARDIOVASCULAR DISEASE ASSESSMENT; CARDIOVASCULAR EFFECT; CARDIOVASCULAR RISK; GENETIC HETEROGENEITY; HEART FAILURE; HEART INFARCTION; HUMAN; META ANALYSIS; MORTALITY; OUTCOME ASSESSMENT; RANDOMIZED CONTROLLED TRIAL (TOPIC); SENSITIVITY ANALYSIS; SYSTEMATIC REVIEW; CARDIOVASCULAR DISEASES; CHEMICALLY INDUCED; EVIDENCE BASED MEDICINE; LITERATURE; META ANALYSIS (TOPIC); OBSERVER VARIATION; ODDS RATIO; REPRODUCIBILITY; RISK ASSESSMENT; RISK FACTOR; SELECTION BIAS; STATISTICAL BIAS; STATISTICS AND NUMERICAL DATA;

BIAS (EPIDEMIOLOGY); CARDIOVASCULAR DISEASES; CONFOUNDING FACTORS (EPIDEMIOLOGY); CYCLOOXYGENASE 2 INHIBITORS; EVIDENCE-BASED MEDICINE; HUMANS; HYPOGLYCEMIC AGENTS; META-ANALYSIS AS TOPIC; OBSERVER VARIATION; ODDS RATIO; REPRODUCIBILITY OF RESULTS; REVIEW LITERATURE AS TOPIC; RISK ASSESSMENT; RISK FACTORS; SELECTION BIAS; THIAZOLIDINEDIONES;

EID: 84964788275     PISSN: 15491277     EISSN: 15491676     Source Type: Journal    
DOI: 10.1371/journal.pmed.1001987     Document Type: Article

References (71)

1. Van Spall HC, Toren A, Kiss A, Fowler RA, Eligibility criteria of randomized controlled trials published in high-impact general medical journals: a systematic sampling review. JAMA. 2007;297:1233–1240. doi: 10.1001/jama.297.11.123317374817
2. Rothwell PM, External validity of randomised controlled trials: “To whom do the results of this trial apply?”. Lancet. 2005;365:82–93. doi: 10.1016/S0140-6736(04)17670-815639683
3. Black N, Why we need observational studies to evaluate the effectiveness of health care. BMJ. 1996;312:1215–1218. 8634569
4. Schneeweiss S, Avorn J, A review of uses of health care utilization databases for epidemiologic research on therapeutics. J Clin Epidemiol. 58:323–337. doi: 10.1016/j.jclinepi.2004.10.01215862718
5. Austin PC, An introduction to propensity score methods for reducing the effects of confounding in observational studies. Multivariate Behav Res. 2011;46:399–424. doi: 10.1080/00273171.2011.56878621818162
6. Golder S, Loke YK, Bland M, Meta-analyses of adverse effects data derived from randomised controlled trials as compared to observational studies: methodological overview. PLoS Med. 2011;8:e1001026. doi: 10.1371/journal.pmed.100102621559325
7. Sox HC, Greenfield S, Comparative effectiveness research: a report from the Institute of Medicine. Ann Intern Med. 2009;151:203–205. 19567618
8. Egger M, Schneider M, Davey SG, Spurious precision? Meta-analysis of observational studies. BMJ. 1998;316:140–144. 9462324
9. Higgins JPT, Altman DG, Gøtzsche PC, Jüni P, Moher D, Oxman AD, et al. The Cochrane Collaboration’s tool for assessing risk of bias in randomised trials. BMJ. 2011;343:d5928. doi: 10.1136/bmj.d592822008217
10. Deeks J, Dinnes J, D’Amico R, Sowden A, Sakarovitch C, Evaluating non-randomised intervention studies. Health Technol Assess. 2003;7:186. doi: 10.3310/hta7270
11. Sanderson S, Tatt ID, Higgins JP, Tools for assessing quality and susceptibility to bias in observational studies in epidemiology: a systematic review and annotated bibliography. Int J Epidemiol. 2007;36:666–676. doi: 10.1093/ije/dym01817470488
12. Wells GA, Shea B, O’Connell D, Peterson J, Welch V, Losos M, et al. The Newcastle–Ottawa Scale (NOS) for assessing the quality of nonrandomised studies in meta-analyses. 2014 [cited 26 Aug 2015]. Ottawa: Ottawa Hospital Research Institute. Available: http://www.ohri.ca/programs/clinical_epidemiology/oxford.asp.
13. Downs SH, Black N, The feasibility of creating a checklist for the assessment of the methodological quality both of randomised and non-randomised studies of health care interventions. J Epidemiol Community Health. 1998;52:377–384. doi: 10.1136/jech.52.6.3779764259
14. Scottish Intercollegiate Guidelines Network. Critical appraisal: notes and checklists. 2015 [cited 10 Oct 2015]. Available: http://www.sign.ac.uk/methodology/checklists.html.
15. Jüni P, Witschi A, Bloch R, Egger M, The hazards of scoring the quality of clinical trials for meta-analysis. JAMA. 1999;282:1054–1060. doi: 10.1001/jama.282.11.105410493204
16. Sterne J, Higgins J, Reeves B, editors. A Cochrane risk of bias assessment tool: for non-randomized studies of interventions (ACROBAT-NRSI). Version 1.0.0, 24 September 2014. Available: http://www.riskofbias.info.
17. Loke YK, Kwok CS, Singh S, Comparative cardiovascular effects of thiazolidinediones: systematic review and meta-analysis of observational studies. BMJ. 2011;342:d1309. doi: 10.1136/bmj.d130921415101
18. McGettigan P, Henry D, Cardiovascular risk and inhibition of cyclooxygenase: a systematic review of the observational studies of selective and nonselective inhibitors of cyclooxygenase 2. JAMA. 2006;296:1633–1644. doi: 10.1001/jama.296.13.jrv6001116968831
19. McGettigan P, Henry D, Cardiovascular risk with non-steroidal anti-inflammatory drugs: systematic review of population-based controlled observational studies. PLoS Med. 2011;8:e1001098. doi: 10.1371/journal.pmed.100109821980265
20. Graham DJ, Ouellet-Hellstrom R, MaCurdy TE, Ali F, Sholley C, Worrall C, et al. Risk of acute myocardial infarction, stroke, heart failure, and death in elderly medicare patients treated with rosiglitazone or pioglitazone. JAMA. 2010;304:411–418. doi: 10.1001/jama.2010.92020584880
21. Singh G, Mithal A, Triadafilopoulos G, Both selective COX-2 inhibitors and non-selective NSAIDs increase the risk of acute myocardial infarction in patients with arthritis: selectivity is with the patient, not the drug class. Ann Rheum Dis. 2005;64(Suppl III):85.
22. Sturkenboom MC, Dieleman J, Verhamme K, Straus S, Vander Hoeven-Borgman M, van der Lei J, Cardiovascular events during use of COX-2 selective and non-selective NSAIDs. Pharmacoepidemiol Drug Saf. 2005;14:S57.
23. Cicchetti DV, Allison T, A new procedure for assessing reliability of scoring EEG sleep recordings. Am J EEG Technol. 1971;11:101–109.
24. Bilik D, McEwen LN, Brown MB, Selby JV, Karter AJ, Marrero DG, et al. Thiazolidinediones, cardiovascular disease and cardiovascular mortality: Translating Research Into Action For Diabetes (TRIAD). Pharmacoepidemiol Drug Saf. 2010;19:715–721. doi: 10.1002/pds.195420583206
25. Brownstein JS, Murphy SN, Goldfine AB, Grant RW, Sordo M, Gainer V, et al. Rapid identification of myocardial infarction risk associated with diabetes medications using electronic medical records. Diabetes Care. 2010;33:526–531. doi: 10.2337/dc09-150620009093
26. Dormuth CR, Maclure M, Carney G, Schneeweiss S, Bassett K, Wright JM, Rosiglitazone and myocardial infarction in patients previously prescribed metformin. PLoS ONE. 2009;4:e6080. doi: 10.1371/journal.pone.000608019562036
27. Hsiao F-Y, Huang W-F, Wen Y-W, Chen P-F, Kuo K, Tsai Y-W, Thiazolidinediones and cardiovascular events in patients with type 2 diabetes mellitus: a retrospective cohort study of over 473,000 patients using the National Health Insurance database in Taiwan. Drug Saf. 2009;32:675–690. doi: 10.2165/00002018-200932080-0000619591532
28. Juurlink DN, Gomes T, Lipscombe LL, Austin PC, Hux JE, Mamdani MM, Adverse cardiovascular events during treatment with pioglitazone and rosiglitazone: population based cohort study. BMJ. 2009;339:b2942. doi: 10.1136/bmj.b294219690342
29. Koro CE, Fu Q, Stender M, An assessment of the effect of thiazolidinedione exposure on the risk of myocardial infarction in type 2 diabetic patients. Pharmacoepidemiol Drug Saf. 2008;17:989–996. doi: 10.1002/pds.164018759378
30. Lipscombe LL, Gomes T, Lévesque LE, Hux JE, Juurlink DN, Alter DA, Thiazolidinediones and cardiovascular outcomes in older patients with diabetes. JAMA. 2007;298:2634–2643. doi: 10.1001/jama.298.22.263418073359
31. Margolis DJ, Hofstad O, Strom BL, Association between serious ischemic cardiac outcomes and medications used to treat diabetes. Pharmacoepidemiol Drug Saf. 2008;17:753–759. doi: 10.1002/pds.163018613215
32. Pantalone K, Kattan M, Yu C, Wells B, Arrigain S, Jain A, et al. The risk of developing coronary artery disease or congestive heart failure, and overall mortality, in type 2 diabetic patients receiving rosiglitazone, pioglitazone, metformin, or sulfonylureas: a retrospective analysis. Acta Diabetol. 2009;46:145–154. doi: 10.1007/s00592-008-0090-319194648
33. Stockl KM, Le L, Zhang S, Harada ASM, Risk of acute myocardial infarction in patients treated with thiazolidinediones or other antidiabetic medications. Pharmacoepidemiol Drug Saf. 2009;18:166–174. doi: 10.1002/pds.170019109802
34. Tzoulaki I, Molokhia M, Curcin V, Little MP, Millett CJ, Ng A, et al. Risk of cardiovascular disease and all cause mortality among patients with type 2 diabetes prescribed oral antidiabetes drugs: retrospective cohort study using UK general practice research database. BMJ. 2009;339:b4731. doi: 10.1136/bmj.b473119959591
35. Walker AM, Koro CE, Landon J, Coronary heart disease outcomes in patients receiving antidiabetic agents in the PharMetrics database 2000–2007. Pharmacoepidemiol Drug Saf. 2008;17:760–768. doi: 10.1002/pds.159818383443
36. Wertz DA, Chang C-L, Sarawate CA, Willey VJ, Cziraky MJ, Bohn RL, Risk of cardiovascular events and all-cause mortality in patients treated with thiazolidinediones in a managed-care population. Circ Cardiovasc Qual Outcomes. 2010;3:538–545. doi: 10.1161/circoutcomes.109.91146120736441
37. Winkelmayer WC, Setoguchi S, Levin R, Solomon DH, Comparison of cardiovascular outcomes in elderly patients with diabetes who initiated rosiglitazone vs pioglitazone therapy. Arch Intern Med. 2008;168:2368–2375. doi: 10.1001/archinte.168.21.236819029503
38. Ziyadeh N, McAfee AT, Koro C, Landon J, Arnold Chan K, The thiazolidinediones rosiglitazone and pioglitazone and the risk of coronary heart disease: a retrospective cohort study using a US health insurance database. Clin Ther. 2009;31:2665–2677. doi: 10.1016/j.clinthera.2009.11.00320110009
39. Bak S, Andersen M, Tsiropoulos I, García Rodríguez LA, Hallas J, Christensen K, et al. Risk of stroke associated with nonsteroidal anti-inflammatory drugs: a nested case-control study. Stroke. 2003;34:379–386. doi: 10.1161/01.str.0000053029.45352.a0
40. Curtis JP, Wang Y, Portnay EL, Masoudi FA, Havranek EP, Krumholz HM, Aspirin, ibuprofen, and mortality after myocardial infarction: retrospective cohort study. BMJ. 2003;327:1322–1323. 14656840
41. Fischer LM, Schlienger RG, Matter CM, Jick H, Meier CR, Current use of nonsteroidal antiinflammatory drugs and the risk of acute myocardial infarction. Pharmacotherapy. 2005;25:503–510. doi: 10.1592/phco.25.4.503.6102115977911
42. García Rodríguez LA, Varas C, Patrono C, Differential effects of aspirin and non-aspirin nonsteroidal antiinflammatory drugs in the primary prevention of myocardial infarction in postmenopausal women. Epidemiology. 2000;11:382–387. 10874543
43. García Rodríguez LA, Varas-Lorenzo C, Maguire A, González-Pérez A, Nonsteroidal antiinflammatory drugs and the risk of myocardial infarction in the general population. Circulation. 2004;109:3000–3006. doi: 10.1161/01.cir.0000132491.96623.0415197149
44. Gislason GH, Jacobsen S, Rasmussen JN, Rasmussen S, Buch P, Friberg J, et al. Risk of death or reinfarction associated with the use of selective cyclooxygenase-2 inhibitors and nonselective nonsteroidal antiinflammatory drugs after acute myocardial infarction. Circulation. 2006;113:2906–2913. doi: 10.1161/circulationaha.106.61621916785336
45. Graham DJ, Campen D, Hui R, Spence M, Cheetham C, Levy G, et al. Risk of acute myocardial infarction and sudden cardiac death in patients treated with cyclo-oxygenase 2 selective and non-selective non-steroidal anti-inflammatory drugs: nested case-control study. Lancet. 2005;365:475–481. doi: 10.1016/S0140-6736(05)70270-115705456
46. Hippisley-Cox J, Coupland C, Risk of myocardial infarction in patients taking cyclo-oxygenase-2 inhibitors or conventional non-steroidal anti-inflammatory drugs: population based nested case-control analysis. BMJ. 2005;330:1366. doi: 10.1136/bmj.330.7504.136615947398
47. Johnsen SP, Larsson H, Tarone RE, McLaughlin JK, Nørgård B, Friis S, et al. Risk of hospitalization for myocardial infarction among users of rofecoxib, celecoxib, and other nsaids: a population-based case-control study. Arch Intern Med. 2005;165:978–984. doi: 10.1001/archinte.165.9.97815883235
48. Kimmel SE, Berlin JA, Reilly M, Jaskowiak J, Kishel L, Chittams J, et al. The effects of nonselective non-aspirin non-steroidal anti-inflammatory medications on the risk of nonfatal myocardial infarction and their interaction with aspirin. J Am Coll Cardiol. 2004;43:985–990. doi: 10.1016/j.jacc.2003.08.06415028354
49. Kimmel SE, Berlin JA, Reilly M, Jaskowiak J, Kishel L, Chittams J, et al. Patients exposed to rofecoxib and celecoxib have different odds of nonfatal myocardial infarction. Ann Intern Med. 2005;142:157–164. doi: 10.7326/0003-4819-142-3-200502010-0000515684203
50. Lévesque LE, Brophy JM, Zhang B, The risk for myocardial infarction with cyclooxygenase-2 inhibitors: a population study of elderly adults. Ann Intern Med. 2005;142:481–489. doi: 10.7326/0003-4819-142-7-200504050-0011315809459
51. MacDonald TM, Wei L, Effect of ibuprofen on cardioprotective effect of aspirin. Lancet. 2003;361:573–574. doi: 10.1016/S0140-6736(03)12509-312598144
52. Mamdani M, Rochon P, Juurlink DN, Anderson GM, Kopp A, Naglie G, et al. Effect of selective cyclooxygenase 2 inhibitors and naproxen on short-term risk of acute myocardial infarction in the elderly. Arch Intern Med. 2003;163:481–486. doi: 10.1001/archinte.163.4.48112588209
53. McGettigan P, Han P, Henry D, Cyclooxygenase-2 inhibitors and coronary occlusion—exploring dose–response relationships. Br J Clin Pharmacol. 2006;62:358–365. doi: 10.1111/j.1365-2125.2006.02660.x16934052
54. Ray WA, Stein CM, Hall K, Daugherty JR, Griffin MR, Non-steroidal anti-inflammatory drugs and risk of serious coronary heart disease: an observational cohort study. Lancet. 2002;359:118–123. doi: 10.1016/S0140-6736(02)07370-111809254
55. Ray WA, Stein CM, Daugherty JR, Hall K, Arbogast PG, Griffin MR, COX-2 selective non-steroidal anti-inflammatory drugs and risk of serious coronary heart disease. Lancet. 2002;360:1071–1073. doi: 10.1016/S0140-6736(02)11131-712383990
56. Schlienger RG, Jick H, Meier CR, Use of nonsteroidal anti-inflammatory drugs and the risk of first-time acute myocardial infarction. Br J Clin Pharmacol. 2002;54:327–332. doi: 10.1046/j.1365-2125.2002.01637.x12236854
57. Solomon DH, Glynn RJ, Levin R, Avorn J, Nonsteroidal anti-inflammatory drug use and acute myocardial infarction. Arch Intern Med. 2002;162:1099–1104. doi: 10.1001/archinte.162.10.109912020178
58. Solomon DH, Schneeweiss S, Glynn RJ, Kiyota Y, Levin R, Mogun H, et al. Relationship between selective cyclooxygenase-2 inhibitors and acute myocardial infarction in older adults. Circulation. 2004;109:2068–2073. doi: 10.1161/01.cir.0000127578.21885.3e15096449
59. Watson DJ, Rhodes T, Cai B, Guess HA, Lower risk of thromboembolic cardiovascular events with naproxen among patients with rheumatoid arthritis. Arch Intern Med. 2002;162:1105–1110. doi: 10.1001/archinte.162.10.110512020179
60. Landis JR, Koch GG, The measurement of observer agreement for categorical data. Biometrics. 1977;33:159–174. doi: 10.2307/2529310843571
61. Nissen SE, Wolski K, Rosiglitazone revisited: an updated meta-analysis of risk for myocardial infarction and cardiovascular mortality. Arch Intern Med. 2010;170:1191–1201. doi: 10.1001/archinternmed.2010.20720656674
62. Katikireddi SV, Egan M, Petticrew M, How do systematic reviews incorporate risk of bias assessments into the synthesis of evidence? A methodological study. J Epidemiol Community Health. 2015;69:189–195. doi: 10.1136/jech-2014-20471125481532
63. Bernatsky S, Slim Z, Yuwan M, ACPA and future onset of rheumatoid arthritis among individuals with undifferentiated arthritis and arthritis free individuals: a systematic review of cohort studies. J Autoimmune Dis Rheumatol. 2015;3:30–40.
64. Kalkhoran S, Glantz S, E-cigarettes and smoking cessation in real-world and clinical settings: a systematic review and meta-analysis. Lancet Respir Med. 2016;4:116–128. doi: 10.1016/s2213-2600(15)00521-426776875
65. Emmett C, Close H, Yiannakou Y, Mason J, Trans-anal irrigation therapy to treat adult chronic functional constipation: systematic review and meta-analysis. BMC Gastroenterol. 2015;15:139. doi: 10.1186/s12876-015-0354-726474758
66. Staton C, Vissoci J, Gong E, Toomey N, Wafula R, Abdelgadir J, et al. Road traffic injury prevention initiatives: a systematic review and metasummary of effectiveness in low and middle income countries. PLoS ONE. 2016;11:e0144971. doi: 10.1371/journal.pone.014497126735918
67. Thomson H, Campbell M, Craig P, Hilton Boon M, Katikireddi V, editors. ACROBAT-NRSI for public health: reporting on feasibility and utility of applying ACROBAT to studies of housing improvement. Cochrane Colloquium 2015; 3–7 Oct 2015; Vienna, Austria.
68. Morgan R, Thayer K, Guyatt G, Blain R, Eftim S, Ross P, et al. Assessing the usability of ACROBAT-NRSI for studies of exposure and intervention in environmental health research. Cochrane Colloquium 2015; 3–7 Oct 2015; Vienna, Austria.
69. Couto E, Pike E, Torkilseng E, Klemp M. Inter-rater reliability of the Risk of Bias Assessment Tool: for Non-Randomized Studies of Interventions (ACROBAT-NRSI). Cochrane Colloquium 2015; 3–7 Oct 2015; Vienna, Austria.
70. Whiting P, Savović J, Higgins J, Caldwell D, Reeves B, Shea B, et al. ROBIS: a new tool to assess risk of bias in systematic reviews was developed. J Clin Epidemiol. 2016;69:225–234. doi: 10.1016/j.jclinepi.2015.06.00526092286
71. Shea B, AMSTAR Development Group. AMSTAR: helping decision makers distinguish high and low quality systematic reviews that include non-randomized studies. Cochrane Colloquium 2015; 3–7 Oct 2015; Vienna, Austria.