Poor agreement in significant findings between meta-analyses and subsequent large randomized trials in perioperative medicine

British Journal of Anaesthesia

Volumn 117, Issue 4, 2016, Pages 431-441

  Sivakumar, H ^a   Peyton, P J ^b,c,d  

^a PETER MACCALLUM CANCER CENTRE   (Australia)

^b AUSTIN HEALTH   (Australia)

^c UNIVERSITY OF MELBOURNE   (Australia)

^d Institute for Breathing and Sleep (IBAS)   (Australia)

Author keywords

anaesthesia; clinical trials; meta analysis

Indexed keywords

ANESTHESIA; HUMAN; MEDICINE; MEDLINE; META ANALYSIS (TOPIC); PERIOPERATIVE PERIOD; PREDICTIVE VALUE; PRIORITY JOURNAL; RANDOMIZED CONTROLLED TRIAL (TOPIC); RELIABILITY; REVIEW; SENSITIVITY ANALYSIS; SYSTEMATIC REVIEW;

HUMANS; META-ANALYSIS AS TOPIC; PERIOPERATIVE CARE; RANDOMIZED CONTROLLED TRIALS AS TOPIC;

EID: 84995555549     PISSN: 00070912     EISSN: 14716771     Source Type: Journal    
DOI: 10.1093/bja/aew170     Document Type: Review

References (97)

1. Rerkasem K, Rothwell PM. Meta-analysis of small randomized controlled trials in surgery may be unreliable. Br J Surg 2010; 97: 466-9
2. Fisher DM. Surrogate end points. Are they meaningful? Anesthesiology 1994; 81: 795-6
3. Myles PS. Why we need large randomized studies in anaesthesia. Br J Anaesth 1999; 83: 833-4
4. Merlin T, Weston A, Tooher R. Extending an evidence hierarchy to include topics other than treatment: revising the Australian 'levels of evidence'. BMC Med Res Methodol 2009; 9: 34
5. LeLorier J, Gregoire G, Benhaddad A, Lapierre J, Derderian F. Discrepancies between meta-analyses and subsequent large randomized, controlled trials. N Engl J Med 1997; 337: 536-42
6. Villar J, Carroli G, Belizan JM. Predictive ability of meta-analyses of randomized controlled trials. Lancet 1995; 345: 772-6
7. Cappelleri JC, Ioannidis JP, Schmid CH, et al. Large trials vs meta-analysis of smaller trials: how do their results compare? JAMA 1996; 276: 1332-8
8. Feinstein AR. Meta-analysis: statistical alchemy for the 21st century. J Clin Epidemiol 1995; 48: 71-9
9. Meinert CL. Meta-analysis: science or religion? Control Clin Trials 1989; 10: 257S-63S
10. Abramson JH. Meta-analysis: a reviewof pros and cons. Public Health Rev 1990; 18: 1-47
11. Bailar JC 3rd. The practice of meta-analysis. J Clin Epidemiol 1995; 48: 149-57
12. Afshari A, Wetterslev J. When may systematic reviews and meta-analyses be considered reliable? Eur J Anaesthesiol 2015; 32: 85-7
13. Whiting PF, Rutjes AW, Westwood ME, et al. QUADAS-2: a revised tool for the quality assessment of diagnostic accuracy studies. Ann Intern Med 2011; 155: 529-36
14. NewmanMF, Ferguson TB, White JA, et al. Effect of adenosineregulating agent acadesine on morbidity and mortality associated with coronary artery bypass grafting: the RED-CABG randomized controlled trial. JAMA 2012; 308: 157-64
15. Mangano DT. Effects of acadesine on myocardial infarction, stroke, and death following surgery. A meta-analysis of the 5 international randomized trials. The Multicenter Study of Perioperative Ischemia (McSPI) Research Group. JAMA 1997; 277: 325-32
16. Mangano DT, Miao Y, Tudor IC, Dietzel C. Post-reperfusion myocardial infarction: long-term survival improvement using adenosine regulation with acadesine. J Am Coll Cardiol 2006; 48: 206-14
17. Zhang C, Xu L, Ma YQ, et al. Bispectral index monitoring prevent awareness during total intravenous anesthesia: a prospective, randomized, double-blinded, multi-center controlled trial. Chin Med J (Engl) 2011; 124: 3664-9
18. Punjasawadwong Y, Phongchiewboon A, BunchungmongkolN. Bispectral index for improving anaesthetic delivery and postoperative recovery. Cochrane Database Syst Rev 2014; 6: CD003843
19. Park WY, Thompson JS, Lee KK. Effect of epidural anesthesia and analgesia on perioperative outcome: a randomized, controlled Veterans Affairs cooperative study. Ann Surg 2001; 234: 560-9
20. Beattie WS, Badner NH, Choi P. Epidural analgesia reduces postoperative myocardial infarction: a meta-analysis. Anesth Analg 2001; 93: 853-8
21. Rodgers A, Walker N, Schug S, et al. Reduction of postoperative mortality and morbidity with epidural or spinal anaesthesia: results from overview of randomized trials. Br Med J 2000; 321: 1493
22. Ballantyne JC, Carr DB, deFerranti S, et al. The comparative effects of postoperative analgesic therapies on pulmonary outcome: cumulative meta-analyses of randomized, controlled trials. Anesth Analg 1998; 86: 598-612
23. Fergusson DA, Hebert PC, Mazer CD, et al. A comparison of aprotinin and lysine analogues in high-risk cardiac surgery. N Engl J Med 2008; 358: 2319-31
24. Brown JR, Birkmeyer NJ, O'Connor GT. Meta-analysis comparing the effectiveness and adverse outcomes of antifibrinolytic agents in cardiac surgery. Circulation 2007; 115: 2801-13
25. Umscheid CA, Kohl BA, Williams K. Antifibrinolytic use in adult cardiac surgery. Curr Opin Hematol 2007; 14: 455-67
26. Carless PA, Moxey AJ, Stokes BJ, Henry DA. Are antifibrinolytic drugs equivalent in reducing blood loss and transfusion in cardiac surgery? A meta-analysis of randomized head-tohead trials. BMC Cardiovasc Disord 2005; 5: 19
27. Lewis SC,WarlowCP, BodenhamAR, et al. General anaesthesia versus local anaesthesia for carotid surgery (GALA): a multicentre, randomized controlled trial. Lancet 2008; 372: 2132-42
28. Guay J. Regional or general anesthesia for carotid endarterectomy? Evidence from published prospective and retrospective studies. J Cardiothorac Vasc Anesth 2007; 21: 127-32
29. Rerkasem K, Bond R, Rothwell PM. Local versus general anaesthesia for carotid endarterectomy. Cochrane Database Syst Rev 2004; 2: CD000126
30. Mozaffarian D, Marchioli R, Macchia A, et al. Fish oil and postoperative atrial fibrillation: the Omega-3 Fatty Acids for Prevention of Post-operative Atrial Fibrillation (OPERA) randomized trial. JAMA 2012; 308: 2001-11
31. Armaganijan L, Lopes RD, Healey JS, Piccini JP, Nair GM, Morillo CA. Do omega-3 fatty acids prevent atrial fibrillation after open heart surgery? A meta-analysis of randomized controlled trials. Clinics (Sao Paulo) 2011; 66: 1923-8
32. Takagi H, Umemoto T. Preoperative n-3 Polyunsatured fatty acids do not reduce postoperative atrial fibrillation in cardiac surgery. Angiology 2011; 62: 276-7
33. Devereaux PJ, Yang H, Yusuf S, et al. Effects of extended-release metoprolol succinate in patients undergoing non-cardiac surgery (POISE trial): a randomized controlled trial. Lancet 2008; 371: 1839-47
34. Wiesbauer F, Schlager O, Domanovits H, et al. Perioperative beta-blockers for preventing surgery-related mortality and morbidity: a systematic review and meta-analysis. Anesth Analg 2007; 104: 27-41
35. Schouten O, Shaw LJ, Boersma E, et al. A meta-analysis of safety and effectiveness of perioperative beta-blocker use for the prevention of cardiac events in different types of noncardiac surgery. Coron Artery Dis 2006; 17: 173-9
36. McGory ML, Maggard MA, Ko CY. A meta-analysis of perioperative beta blockade: what is the actual risk reduction? Surgery 2005; 138: 171-9
37. Stevens RD, Burri H, Tramer MR. Pharmacologic myocardial protection in patients undergoing noncardiac surgery: a quantitative systematic review. Anesth Analg 2003; 97: 623-33
38. Myburgh JA, Finfer S, Bellomo R, et al. Hydroxyethyl starch or saline for fluid resuscitation in intensive care. N Engl J Med 2012; 367: 1901-11
39. Perel P, Roberts I. Colloids versus crystalloids for fluid resuscitation in critically ill patients. Cochrane Database Syst Rev 2012; 6: CD000567
40. Wiedermann CJ. Systematic review of randomized clinical trials on the use of hydroxyethyl starch for fluid management in sepsis. BMC Emerg Med 2008; 8: 1
41. Finfer S, Bellomo R, Boyce N, French J, Myburgh J, Norton R. A comparison of albumin and saline for fluid resuscitation in the intensive care unit. N Engl J Med 2004; 350: 2247-56
42. Albumin Reviewers (Alderson P, Bunn F, Lefebvre C, et al.) Human albumin solution for resuscitation and volume expansion in critically ill patients. Cochrane Database Syst Rev 2011; 10: CD001208
43. Human albumin administration in critically ill patients: systematic review of randomized controlled trials. Br Med J 1998; 317: 235-40
44. Dieleman JM, Nierich AP, Rosseel PM, et al. Intraoperative high-dose dexamethasone for cardiac surgery: a randomized controlled trial. JAMA 2012; 308: 1761-7
45. Cappabianca G, Rotunno C, de Luca Tupputi Schinosa L, Ranieri VM, Paparella D. Protective effects of steroids in cardiac surgery: a meta-analysis of randomized double-blind trials. J Cardiothorac Vasc Anesth 2011; 25: 156-65
46. Yi K, Tian JH, Zhang B, Song B, Yang KH. Effect of dexamethasone on atrial fibrillation after cardiac surgery: a meta-analysis. Zhonghua Wai Ke Za Zhi 2010; 48: 1174-7
47. Ho KM, Tan JA. Benefits and risks of corticosteroid prophylaxis in adult cardiac surgery: a dose-response meta-analysis. Circulation 2009; 119: 1853-66
48. Whitlock RP, Chan S, Devereaux PJ, et al. Clinical benefit of steroid use in patients undergoing cardiopulmonary bypass: a meta-analysis of randomized trials. Eur Heart J 2008; 29: 2592-600
49. Marik PE, FrommR. The efficacy and dosage effect of corticosteroids for the prevention of atrial fibrillation after cardiac surgery: a systematic review. J Crit Care 2009; 24: 458-63
50. Lamy A, Devereaux PJ, Prabhakaran D, et al. Off-pump or onpump coronary-artery bypass grafting at 30 days. N Engl J Med 2012; 366: 1489-97
51. Shroyer AL, Grover FL, Hattler B, et al. On-pump versus offpump coronary-artery bypass surgery. N Engl J Med 2009; 361: 1827-37
52. Afilalo J, Rasti M, Ohayon SM, Shimony A, Eisenberg MJ. Off-pump vs. on-pump coronary artery bypass surgery: an updated meta-analysis and meta-regression of randomized trials. Eur Heart J 2012; 33: 1257-67
53. Godinho AS, Alves AS, Pereira AJ, Pereira TS. On-pump versus off-pump coronary-artery bypass surgery: a meta-analysis. Arq Bras Cardiol 2012; 98: 87-94
54. Moller CH, Penninga L, Wetterslev J, Steinbruchel DA, Gluud C. Off-pump versus on-pump coronary artery bypass grafting for ischaemic heart disease. Cochrane Database Syst Rev 2012; 3: CD007224
55. Seabra VF, Alobaidi S, Balk EM, Poon AH, Jaber BL. Off-pump coronary artery bypass surgery and acute kidney injury: a meta-analysis of randomized controlled trials. Clin J Am Soc Nephrol 2010; 5: 1734-44
56. Feng ZZ, Shi J, Zhao XW, Xu ZF. Meta-analysis of on-pump and off-pump coronary arterial revascularization. Ann Thorac Surg 2009; 87: 757-65
57. Moller CH, Penninga L, Wetterslev J, Steinbruchel DA, Gluud C. Clinical outcomes in randomized trials of off-vs. on-pump coronary artery bypass surgery: systematic review with meta-analyses and trial sequential analyses. Eur Heart J 2008; 29: 2601-16
58. Sedrakyan A, Wu AW, Parashar A, Bass EB, Treasure T. Offpump surgery is associated with reduced occurrence of stroke and other morbidity as compared with traditional coronary artery bypass grafting: a meta-analysis of systematically reviewed trials. Stroke 2006; 37: 2759-69
59. Cheng DC, Bainbridge D, Martin JE, Novick RJ. Does off-pump coronary artery bypass reduce mortality, morbidity, and resource utilization when compared with conventional coronary artery bypass? A meta-analysis of randomized trials. Anesthesiology 2005; 102: 188-203
60. Wijeysundera DN, Beattie WS, Djaiani G, et al. Off-pump coronary artery surgery for reducing mortality and morbidity: meta-analysis of randomized and observational studies. J Am Coll Cardiol 2005; 46: 872-82
61. Chen YB, Shu J, Yang WT, et al. Meta-analysis of randomized trials comparing the effectiveness of on-pump and offpump coronary artery bypass. Chin Med J (Engl) 2012; 125: 338-44
62. Nigwekar SU, Kandula P, Hix JK, Thakar CV. Off-pump coronary artery bypass surgery and acute kidney injury: a meta-analysis of randomized and observational studies. Am J Kidney Dis 2009; 54: 413-23
63. Meyhoff CS, Wetterslev J, Jorgensen LN, et al. Effect of high perioperative oxygen fraction on surgical site infection and pulmonary complications after abdominal surgery: the PROXI randomized clinical trial. JAMA 2009; 302: 1543-50
64. Qadan M, Akca O, Mahid SS, Hornung CA, Polk HC Jr. Perioperative supplemental oxygen therapy and surgical site infection: a meta-analysis of randomized controlled trials. Arch Surg 2009; 144: 359-66
65. Sandham JD, Hull RD, Brant RF, et al. A randomized, controlled trial of the use of pulmonary-artery catheters in high-risk surgical patients. N Engl J Med 2003; 348: 5-14
66. Barone JE, Tucker JB, Rassias D, Corvo PR. Routine perioperative pulmonary artery catheterization has no effect on rate of complications in vascular surgery: a meta-analysis. Am Surg 2001; 67: 674-9
67. Ivanov RI, Allen J, Sandham JD, Calvin JE. Pulmonary artery catheterization: a narrative and systematic critique of randomized controlled trials and recommendations for the future. New Horiz 1997; 5: 268-76
68. Devereaux PJ, Mrkobrada M, Sessler DI, et al. Aspirin in patients undergoing noncardiac surgery. N Engl J Med 2014; 370: 1494-503
69. Collaborative overview of randomized trials of antiplatelet therapy-III: Reduction in venous thrombosis and pulmonary embolism by antiplatelet prophylaxis among surgical and medical patients. Antiplatelet Trialists' Collaboration. Br Med J 1994; 308: 235-46
70. Prevention of pulmonary embolism and deep vein thrombosis with lowdose aspirin: Pulmonary Embolism Prevention (PEP) trial. Lancet 2000; 355: 1295-302
71. Devereaux PJ, Sessler DI, Leslie K, et al. Clonidine in patients undergoing noncardiac surgery. N Engl J Med 2014; 370: 1504-13
72. Wijeysundera DN, Bender JS, Beattie WS. Alpha-2 adrenergic agonists for the prevention of cardiac complications among patients undergoing surgery. Cochrane Database Syst Rev 2009; 4: CD004126
73. Wijeysundera DN, Naik JS, Beattie WS. Alpha-2 adrenergic agonists to prevent perioperative cardiovascular complications: a meta-analysis. Am J Med 2003; 114: 742-52
74. Devereaux PJ, Beattie WS, Choi PT, et al. How strong is the evidence for the use of perioperative beta blockers in noncardiac surgery? Systematic review and meta-analysis of randomized controlled trials. Br Med J 2005; 331: 313-21
75. Dieleman JM, van Paassen J, van Dijk D, et al. Prophylactic corticosteroids for cardiopulmonary bypass in adults. Cochrane Database Syst Rev 2011; 5: CD005566
76. Tangkanakul C, Counsell CE, Warlow CP. Local versus general anaesthesia in carotid endarterectomy: a systematic review of the evidence. Eur J Vasc Endovasc Surg 1997; 13: 491-9
77. Kuss O, von Salviati B, Borgermann J. Off-pump versus on-pump coronary artery bypass grafting: a systematic review and meta-analysis of propensity score analyses. J Thorac Cardiovasc Surg 2010; 140: 829-35
78. Reston JT, Tregear SJ, Turkelson CM. Meta-analysis of shortterm and mid-term outcomes following off-pump coronary artery bypass grafting. Ann Thorac Surg 2003; 76: 1510-5
79. Athanasiou T, Aziz O, Mangoush O, et al. Does off-pump coronary artery bypass reduce the incidence of post-operative atrial fibrillation? A question revisited. Eur J Cardiothorac Surg 2004; 26: 701-10
80. Landis JR, Koch GG. The measurement of observer agreement for categorical data. Biometrics 1977; 33: 159-74
81. Perneger TV. What's wrong with Bonferroni adjustments. Br Med J 1998; 316: 1236-8
82. Bland JM, Altman DG. Multiple significance tests: the Bonferroni method. Br Med J 1995; 310: 170
83. De Oliveira GS Jr, Chang R, Kendall MC, Fitzgerald PC, McCarthy RJ. Publication bias in the anesthesiology literature. Anesth Analg 2012; 114: 1042-8
84. Dickersin K, Chan S, Chalmers TC, Sacks HS, Smith H Jr. Publication bias and clinical trials. Control Clin Trials 1987; 8: 343-53
85. Scherer RW, Dickersin K, Langenberg P. Full publication of results initially presented in abstracts. A meta-analysis. JAMA 1994; 272: 158-62
86. Egger M, Davey Smith G, Schneider M, Minder C. Bias in metaanalysis detected by a simple, graphical test. Br Med J 1997; 315: 629-34
87. Peters JL, Sutton AJ, Jones DR, Abrams KR, Rushton L. Comparison of two methods to detect publication bias in meta-analysis. JAMA 2006; 295: 676-80
88. Ioannidis JP, Cappelleri JC, Lau J. Issues in comparisons between meta-analyses and large trials. JAMA 1998; 279: 1089-93
89. Sterne J, Egger M, Moher D. Chapter 10: Addressing reporting biases In: JPT H, S G, eds. Cochrane Handbook for Systematic Reviews of Interventions. Version 5.0.1 ed. The Cochrane Collaboration, 2008
90. Sutton AJ, Duval SJ, Tweedie RL, Abrams KR, Jones DR. Empirical assessment of effect of publication bias on metaanalyses. Br Med J 2000; 320: 1574-7
91. Kicinski M, Springate DA, Kontopantelis E. Publication bias in meta-analyses from the Cochrane Database of Systematic Reviews. Statistics in Medicine 2015; 34: 2781-95
92. Egger M, Smith GD. Misleading meta-analysis. Br Med J 1995; 310: 752-4
93. Chong SW, Collins NF, Wu C, Liskaser G, Peyton PJ. The Relationship of Study Findings to Publication Outcome in Anesthesia Research: a retrospective observational study examining publication bias. Can J Anesth 2016; 63: 682-90
94. Pereira TV, Ioannidis JPA. Statistically significant metaanalyses of clinical trials have modest credibility and inflated effects. J Clin Epidemiol 2011; 64: 1060-9
95. Hennekens CH, Demets D. The need for large-scale randomized evidence without undue emphasis on small trials, meta-analyses, or subgroup analyses. JAMA 2009; 302: 2361-2
96. Devereaux PJ, Chan MT, Eisenach J, Schricker T, Sessler DI. The need for large clinical studies in perioperative medicine. Anesthesiology 2012; 116: 1169-75
97. Ioannidis JP. Why most published research findings are false. PLoS Med 2005; 2: e124