Methodologies for systematic reviews with meta-analysis of randomised clinical trials in pain, anaesthesia, and perioperative medicine

British Journal of Anaesthesia

Volumn 126, Issue 4, 2021, Pages 903-911

  Doleman, Brett ^a   Mathiesen, Ole ^b,c   Jakobsen, Janus C ^d,e   Sutton, Alex J ^f   Freeman, Suzanne ^f   Lund, Jonathan N ^a   Williams, John P ^a  

^a UNIVERSITY OF NOTTINGHAM   (United Kingdom)

^b UNIVERSITY OF COPENHAGEN   (Denmark)

^c ZEALAND UNIVERSITY HOSPITAL   (Denmark)

^d Copenhagen Trial Unit ^*  (Denmark)

^e UNIVERSITY OF SOUTHERN DENMARK   (Denmark)

^f UNIVERSITY OF LEICESTER   (United Kingdom)

Author keywords

meta analysis; methodology; perioperative medicine; postoperative pain; publication bias; systematic review

Indexed keywords

ANESTHESIA; FALSE POSITIVE RESULT; HUMAN; META ANALYSIS (TOPIC); METHODOLOGY; PERIOPERATIVE MEDICINE; POSTOPERATIVE PAIN; PRIORITY JOURNAL; PUBLICATION BIAS; RANDOM ERROR; RANDOMIZED CONTROLLED TRIAL (TOPIC); REVIEW; SEQUENTIAL ANALYSIS; STATISTICAL BIAS; SYSTEMATIC REVIEW (TOPIC); PROCEDURES;

ANESTHESIA; HUMANS; META-ANALYSIS AS TOPIC; PERIOPERATIVE MEDICINE; RANDOMIZED CONTROLLED TRIALS AS TOPIC; SYSTEMATIC REVIEWS AS TOPIC;

EID: 85100530277     PISSN: 00070912     EISSN: 14716771     Source Type: Journal    
DOI: 10.1016/j.bja.2021.01.004     Document Type: Review

References (83)

1. Ioannidis, J.P., The mass production of redundant, misleading, and conflicted systematic reviews and meta-analyses. Milbank Q 94 (2016), 485–514.
2. IntHout, J., Ioannidis, J.P., Borm, G.F., Obtaining evidence by a single well-powered trial or several modestly powered trials. Stat Methods Med Res 25 (2016), 538–552.
3. Doleman, B., Williams, J.P., Lund, J., Why most published meta-analysis findings are false. Tech Coloproctol 23 (2019), 925–928.
4. LeLorier, J., Gregoire, G., Benhaddad, A., Lapierre, J., Derderian, F., Discrepancies between meta-analyses and subsequent large randomized, controlled trials. N Engl J Med 337 (1997), 536–542.
5. Villar, J., Carroli, G., Belizan, J.M., Predictive ability of meta-analyses of randomised controlled trials. Lancet 345 (1995), 772–776.
6. Sivakumar, H., Peyton, P.J., Poor agreement in significant findings between meta-analyses and subsequent large randomized trials in perioperative medicine. Br J Anaesth 117 (2016), 431–441.
7. Conway, A., Conway, Z., Soalheira, K., Sutherland, J., High quality of evidence is uncommon in Cochrane systematic reviews in anaesthesia, critical care and emergency medicine. Eur J Anaesthesiol 34 (2017), 808–813.
8. Brinck, E.C., Tiippana, E., Heesen, M., et al. Perioperative intravenous ketamine for acute postoperative pain in adults. Cochrane Database Syst Rev, 12, 2018, CD012033.
9. Booth, A., Clarke, M., Dooley, G., et al. The nuts and bolts of PROSPERO: an international prospective register of systematic reviews. Syst Rev 1 (2012), 1–9.
10. Allers, K., Hoffmann, F., Mathes, T., Pieper, D., Systematic reviews with published protocols compared to those without: more effort, older search. J Clin Epidemiol 95 (2018), 102–110.
11. Page, M.J., Shamseer, L., Tricco, A.C., Registration of systematic reviews in PROSPERO: 30,000 records and counting. Syst Rev, 7, 2018, 32.
12. Sideri, S., Papageorgiou, S.N., Eliades, T., Registration in the international prospective register of systematic reviews (PROSPERO) of systematic review protocols was associated with increased review quality. J Clin Epidemiol 100 (2018), 103–110.
13. Tricco, A.C., Cogo, E., Page, M.J., et al. A third of systematic reviews changed or did not specify the primary outcome: a PROSPERO register study. J Clin Epidemiol 79 (2016), 46–54.
14. Koensgen, N., Rombey, T., Allers, K., Mathes, T., Hoffmann, F., Pieper, D., Comparison of non-Cochrane systematic reviews and their published protocols: differences occurred frequently but were seldom explained. J Clin Epidemiol 110 (2019), 34–41.
15. Moher, D., Liberati, A., Tetzlaff, J., Altman, D.G., Prisma Group. Preferred reporting items for systematic reviews and meta-analyses: the PRISMA statement. PLoS Med, 6, 2009, e1000097.
16. Panic, N., Leoncini, E., De Belvis, G., Ricciardi, W., Boccia, S., Evaluation of the endorsement of the preferred reporting items for systematic reviews and meta-analysis (PRISMA) statement on the quality of published systematic review and meta-analyses. PLoS One, 8, 2013, e83138.
17. Oh, J.H., Shin, W.J., Park, S., Chung, J.S., Reporting and methodologic evaluation of meta-analyses published in the anesthesia literature according to AMSTAR and PRISMA checklists: a preliminary study. Korean J Anesthesiol 70 (2017), 446–455.
18. Møller, M.H., Patient-important outcomes and core outcome sets: increased attention needed!. Br J Anaesth 122 (2019), 408–410.
19. Myles, P.S., Myles, D.B., Galagher, W., et al. Measuring acute postoperative pain using the visual analog scale: the minimal clinically important difference and patient acceptable symptom state. Br J Anaesth 118 (2017), 424–429.
20. Myles, P.S., Boney, O., Botti, M., et al. Systematic review and consensus definitions for the Standardised Endpoints in Perioperative Medicine (StEP) initiative: patient comfort. Br J Anaesth 120 (2018), 705–711.
21. Buggy, D.J., Freeman, J., Johnson, M.Z., et al. Systematic review and consensus definitions for standardised endpoints in perioperative medicine: postoperative cancer outcomes. Br J Anaesth 121 (2018), 38–44.
22. McIlroy, D.R., Bellomo, R., Billings, F.T. 4th, et al. Systematic review and consensus definitions for the Standardised Endpoints in Perioperative Medicine (StEP) initiative: renal endpoints. Br J Anaesth 121 (2018), 1013–1024.
23. Barnes, J., Hunter, J., Harris, S., et al. Systematic review and consensus definitions for the Standardised Endpoints in Perioperative Medicine (StEP) initiative: infection and sepsis. Br J Anaesth 122 (2019), 500–508.
24. Moonesinghe, S.R., Jackson, A.I., Boney, O., et al. Systematic review and consensus definitions for the Standardised Endpoints in Perioperative Medicine initiative: patient-centred outcomes. Br J Anaesth 123 (2019), 664–670.
25. Schuster Bruce, C., Brhlikova, P., Heath, J., McGettigan, P., The use of validated and nonvalidated surrogate endpoints in two European Medicines Agency expedited approval pathways: a cross-sectional study of products authorised 2011–2018. PLoS Med, 16, 2019, e1002873.
26. Fabritius, M.L., Mathiesen, O., Wetterslev, J., Dahl, J.B., Post-operative analgesia: focus has been on benefit—are we forgetting the harm?. Acta Anaesthesiol Scand 60 (2016), 839–841.
27. Edwards, J.E., McQuay, H.J., Moore, R.A., Collins, S.L., Reporting of adverse effects in clinical trials should be improved: lessons from acute postoperative pain. J Pain Symptom Manage 18 (1999), 427–437.
28. Fabritius, M.L., Strøm, C., Koyuncu, S., et al. Benefit and harm of pregabalin in acute pain treatment: a systematic review with meta-analyses and trial sequential analyses. Br J Anaesth 119 (2017), 775–791.
29. Mills, E.J., Thorlund, K., Ioannidis, J.P., Demystifying trial networks and network meta-analysis. BMJ, 346, 2013, f2914.
30. Martinez, V., Beloeil, H., Marret, E., Fletcher, D., Ravaud, P., Trinquart, L., Non-opioid analgesics in adults after major surgery: systematic review with network meta-analysis of randomized trials. Br J Anaesth 118 (2017), 22–31.
31. Faltinsen, E.G., Storebø, O.J., Jakobsen, J.C., Boesen, K., Lange, T., Gluud, C., Network meta-analysis: the highest level of medical evidence?. BMJ Evid Based Med 23 (2018), 56–59.
32. Greenfield, M.L., Mhyre, J.M., Mashour, G.A., Blum, J.M., Yen, E.C., Rosenberg, A.L., Improvement in the quality of randomized controlled trials among general anesthesiology journals 2000 to 2006: a 6-year follow-up. Anesth Analg 108 (2009), 1916–1921.
33. Kunz, R., Vist, G.E., Oxman, A.D., Randomisation to protect against selection bias in healthcare trials. Cochrane Database Syst Rev, 2, 2007, MR000012.
34. Kjaergard, L.L., Villumsen, J., Gluud, C., Reported methodologic quality and discrepancies between large and small randomized trials in meta-analyses. Ann Intern Med 135 (2001), 982–989.
35. Nüesch, E., Reichenbach, S., Trelle, S., et al. The importance of allocation concealment and patient blinding in osteoarthritis trials: a meta-epidemiologic study. Arthritis Rheum 61 (2009), 1633–1641.
36. Carroll, D., Tramer, M., McQuay, H., Nye, B., Moore, A., Randomization is important in studies with pain outcomes: systematic review of transcutaneous electrical nerve stimulation in acute postoperative pain. Br J Anaesth 77 (1996), 798–803.
37. Montori, V.M., Bhandari, M., Devereaux, P.J., Manns, B.J., Ghali, W.A., Guyatt, G.H., In the dark: the reporting of blinding status in randomized controlled trials. J Clin Epidemiol 55 (2002), 787–790.
38. Barcot, O., Boric, M., Dosenovic, S., Pericic, T.P., Cavar, M., Puljak, L., Risk of bias assessments for blinding of participants and personnel in Cochrane reviews were frequently inadequate. J Clin Epidemiol 113 (2019), 104–113.
39. Vase, L., Petersen, G.L., Riley, J.L. 3rd, Price, D.D., Factors contributing to large analgesic effects in placebo mechanism studies conducted between 2002 and 2007. Pain 145 (2009), 36–44.
40. Hróbjartsson, A., Thomsen, A.S., Emanuelsson, F., et al. Observer bias in randomized clinical trials with measurement scale outcomes: a systematic review of trials with both blinded and nonblinded assessors. CMAJ 185 (2013), E201–E211.
41. Jüni, P., Egger, M., Commentary: empirical evidence of attrition bias in clinical trials. Int J Epidemiol 34 (2005), 87–88.
42. Doleman, B., Heinink, T.P., Read, D.J., Faleiro, R.J., Lund, J.N., Williams, J.P., A systematic review and meta-regression analysis of prophylactic gabapentin for postoperative pain. Anaesthesia 70 (2015), 1186–1204.
43. De Oliveira, G.S. Jr., Jung, M.J., McCarthy, R.J., Discrepancies between randomized controlled trial registry entries and content of corresponding manuscripts reported in anesthesiology journals. Anesth Analg 121 (2015), 1030–1033.
44. Jones, P.M., Chow, J.T., Arango, M.F., et al. Comparison of registered and reported outcomes in randomized clinical trials published in anesthesiology journals. Anesth Analg 125 (2017), 1292–1300.
45. Sterne, J.A., Savović, J., Page, M.J., et al. RoB 2: a revised tool for assessing risk of bias in randomised trials. BMJ, 366, 2019, l4898.
46. Detweiler, B.N., Kollmorgen, L.E., Umberham, B.A., Hedin, R.J., Vassar, B.M., Risk of bias and methodological appraisal practices in systematic reviews published in anaesthetic journals: a meta-epidemiological study. Anaesthesia 71 (2016), 955–968.
47. Heesen, M., Klimek, M., Imberger, G., Hoeks, S.E., Rossaint, R., Straube, S., Co-administration of dexamethasone with peripheral nerve block: intravenous vs perineural application: systematic review, meta-analysis, meta-regression and trial-sequential analysis. Br J Anaesth 120 (2018), 212–227.
48. 2 is not an absolute measure of heterogeneity. Res Synth Methods 8 (2017), 5–18.
49. 2 in assessing heterogeneity may mislead. BMC Med Res Methodol, 8, 2008, 79.
50. Choi, P.T., Halpern, S.H., Malik, N., Jadad, A.R., Tramèr, M.R., Walder, B., Examining the evidence in anesthesia literature: a critical appraisal of systematic reviews. Anesth Analg 92 (2001), 700–709.
51. Borenstein, M., Hedges, L.V., Higgins, J.P., Rothstein, H.R., A basic introduction to fixed-effect and random-effects models for meta-analysis. Res Synth Methods 1 (2010), 97–111.
52. Jakobsen, J.C., Wetterslev, J., Winkel, P., Lange, T., Gluud, C., Thresholds for statistical and clinical significance in systematic reviews with meta-analytic methods. BMC Med Res Methodol, 14, 2014, 120.
53. Riley, R.D., Higgins, J.P., Deeks, J.J., Interpretation of random effects meta-analyses. BMJ, 342, 2011, d549.
54. Thompson, S.G., Higgins, J.P., How should meta-regression analyses be undertaken and interpreted?. Stat Med 21 (2002), 1559–1573.
55. Berlin, J.A., Santanna, J., Schmid, C.H., Szczech, L.A., Feldman, H.I., Individual patient-versus group-level data meta-regressions for the investigation of treatment effect modifiers: ecological bias rears its ugly head. Stat Med 21 (2002), 371–387.
56. Doleman, B., Sutton, A.J., Sherwin, M., Lund, J.N., Williams, J.P., Baseline morphine consumption may explain between-study heterogeneity in meta-analyses of adjuvant analgesics and improve precision and accuracy of effect estimates. Anesth Analg 126 (2018), 648–660.
57. Doleman, B., Lund, J.N., Williams, J.P., Clinically significant reductions in morphine consumption need to take account of baseline risk: presentation of a novel meta-analysis methodology. Br J Anaesth 120 (2018), 414–415.
58. Verret, M., Lauzier, F., Zarychanski, R., et al. Perioperative use of gabapentinoids for the management of postoperative acute pain: a systematic review and meta-analysis. Anesthesiology 133 (2020), 265–279.
59. Friedrich, J.O., Adhikari, N.K., Beyene, J., The ratio of means method as an alternative to mean differences for analyzing continuous outcome variables in meta-analysis: a simulation study. BMC Med Res Methodol, 8, 2008, 32.
60. Wetterslev, J., Jakobsen, J.C., Gluud, C., Trial sequential analysis in systematic reviews with meta-analysis. BMC Med Res Methodol 17 (2017), 1–8.
61. Imberger, G., Thorlund, K., Gluud, C., Wetterslev, J., False-positive findings in Cochrane meta-analyses with and without application of trial sequential analysis: an empirical review. BMJ Open, 6, 2016, e011890.
62. Castellini, G., Bruschettini, M., Gianola, S., Gluud, C., Moja, L., Assessing imprecision in Cochrane systematic reviews: a comparison of GRADE and trial sequential analysis. Syst Rev, 7, 2018, 110.
63. Doleman, B., Leonardi-Bee, J., Heinink, T.P., Bhattacharjee, D., Lund, J.N., Williams, J.P., Pre-emptive and preventive opioids for postoperative pain in adults undergoing all types of surgery. Cochrane Database Syst Rev, 12, 2018, CD012624.
64. Pedersen, C., Troensegaard, H., Laigaard, J., et al. Differences in patient characteristics and external validity of randomized clinical trials on pain management following total hip and knee arthroplasty: a systematic review. Reg Anesth Pain Med 45 (2020), 709–715.
65. Barazanchi, A.W., MacFater, W.S., Rahiri, J.L., et al. Evidence-based management of pain after laparoscopic cholecystectomy: a PROSPECT review update. Br J Anaesth 121 (2018), 787–803.
66. Hopewell, S., Loudon, K., Clarke, M.J., Oxman, A.D., Dickersin, K., Publication bias in clinical trials due to statistical significance or direction of trial results. Cochrane Database Syst Rev, 1, 2009, MR000006.
67. Duyx, B., Urlings, M.J., Swaen, G.M., Bouter, L.M., Zeegers, M.P., Scientific citations favor positive results: a systematic review and meta-analysis. J Clin Epidemiol 88 (2017), 92–101.
68. Tramèr, M.R., Reynolds, D.J., Moore, R.A., McQuay, H.J., Impact of covert duplicate publication on meta-analysis: a case study. BMJ 315 (1997), 635–640.
69. Hedin, R.J., Umberham, B.A., Detweiler, B.N., Kollmorgen, L., Vassar, M., Publication bias and nonreporting found in majority of systematic reviews and meta-analyses in anesthesiology journals. Anesth Analg 123 (2016), 1018–1025.
70. Chong, S.W., Collins, N.F., Wu, C.Y., Liskaser, G.M., Peyton, P.J., The relationship between study findings and publication outcome in anesthesia research: a retrospective observational study examining publication bias. Can J Anaesth 63 (2016), 682–690.
71. De Oliveira, G.S. Jr., Chang, R., Kendall, M.C., Fitzgerald, P.C., McCarthy, R.J., Publication bias in the anesthesiology literature. Anesth Analg 114 (2012), 1042–1048.
72. Egger, M., Smith, G.D., Schneider, M., Minder, C., Bias in meta-analysis detected by a simple, graphical test. BMJ 315 (1997), 629–634.
73. Mayo-Wilson, E., Li, T., Fusco, N., et al. Cherry-picking by trialists and meta-analysts can drive conclusions about intervention efficacy. J Clin Epidemiol 91 (2017), 95–110.
74. Egger, M., Juni, P., Bartlett, C., Holenstein, F., Sterne, J., How important are comprehensive literature searches and the assessment of trial quality in systematic reviews? Empirical study. Health Technol Assess 7 (2003), 1–76.
75. Doleman, B., Lund, J.N., Williams, J.P., Detection and prevention of publication bias in meta-analyses of postoperative analgesics: a meta-epidemiological study. Anaesthesia, 72(S3), 2017, 20, 10.1111/anae.13974.
76. Biocic, M., Fidahic, M., Cikes, K., Puljak, L., Comparison of information sources used in Cochrane and non-Cochrane systematic reviews: a case study in the field of anesthesiology and pain. Res Synth Methods 10 (2019), 597–605.
77. Peters, J.L., Sutton, A.J., Jones, D.R., Abrams, K.R., Rushton, L., Contour-enhanced meta-analysis funnel plots help distinguish publication bias from other causes of asymmetry. J Clin Epidemiol 61 (2008), 991–996.
78. Terrin, N., Schmid, C.H., Lau, J., In an empirical evaluation of the funnel plot, researchers could not visually identify publication bias. J Clin Epidemiol 58 (2005), 894–901.
79. Sterne, J.A., Sutton, A.J., Ioannidis, J.P., et al. Recommendations for examining and interpreting funnel plot asymmetry in meta-analyses of randomised controlled trials. BMJ, 343, 2011, d4002.
80. Doleman, B., Freeman, S.C., Lund, J.N., Williams, J.P., Sutton, A.J., Funnel plots may show asymmetry in the absence of publication bias with continuous outcomes dependent on baseline risk: presentation of a new publication bias test. Res Synth Methods 11 (2020), 522–534.
81. Zwetsloot, P.P., Van Der Naald, M., Sena, E.S., et al. Standardized mean differences cause funnel plot distortion in publication bias assessments. eLife 6 (2017), 1–20.
82. Peters, J.L., Sutton, A.J., Jones, D.R., Abrams, K.R., Rushton, L., Comparison of two methods to detect publication bias in meta-analysis. JAMA 295 (2006), 676–680.
83. Hunter, J.P., Saratzis, A., Sutton, A.J., Boucher, R.H., Sayers, R.D., Bown, M.J., In meta-analyses of proportion studies, funnel plots were found to be an inaccurate method of assessing publication bias. J Clin Epidemiol 67 (2014), 897–903.