The fragility of statistically significant findings from randomized trials in spine surgery: A systematic survey

Spine Journal

Volumn 15, Issue 10, 2015, Pages 2188-2197

  Evaniew, Nathan ^a   Files, Carly ^a   Smith, Christopher ^a   Bhandari, Mohit ^a   Ghert, Michelle ^a   Walsh, Michael ^a   Devereaux, Philip J ^a   Guyatt, Gordon ^a  

^a MCMASTER UNIVERSITY   (Canada)

Author keywords

Clinical epidemiology; Fragility Index; Outcomes; Randomized controlled trials; Spine surgery; Statistical significance

Indexed keywords

BONE FRAGILITY; CLINICAL ASSESSMENT TOOL; DATA BASE; DEGENERATION; EVIDENCE BASED MEDICINE; FISHER EXACT TEST; FRAGILITY INDEX; HEALTH SURVEY; HUMAN; INTERVENTION STUDY; PATIENT HISTORY OF ORTHOPEDIC SURGERY; PRIORITY JOURNAL; RANDOMIZED CONTROLLED TRIAL (TOPIC); REVIEW; SPINE SURGERY; STATISTICAL SIGNIFICANCE; SYSTEMATIC REVIEW; TREATMENT OUTCOME; ADVERSE EFFECTS; META ANALYSIS; NEUROSURGERY; ORTHOPEDIC SURGERY; SPINAL INJURIES; STANDARDS; STATISTICS AND NUMERICAL DATA;

HUMANS; NEUROSURGICAL PROCEDURES; ORTHOPEDIC PROCEDURES; RANDOMIZED CONTROLLED TRIALS AS TOPIC; SPINAL INJURIES;

EID: 84942506120     PISSN: 15299430     EISSN: 18781632     Source Type: Journal    
DOI: 10.1016/j.spinee.2015.06.004     Document Type: Article

References (78)

1. M. Bhandari, G.H. Guyatt, and M.F. Swiontkowski User's guide to the orthopaedic literature: how to use an article about a surgical therapy J Bone Joint Surg Am 83-A 2001 916 926
2. M. Bhandari, J.W. Busse, D. Jackowski, V.M. Montori, H. Schunemann, S. Sprague, and et al. Association between industry funding and statistically significant pro-industry findings in medical and surgical randomized trials CMAJ 170 2004 477 480
3. SPRINT Investigators M. Bhandari, P. Tornetta 3rd, S.A. Rampersad, S. Sprague, D. Heels-Ansdell, D.W. Sanders, and et al. (Sample) size matters! an examination of sample size from the SPRINT trial study to prospectively evaluate reamed intramedullary nails in patients with tibial fractures J Orthop Trauma 27 2013 183 188
4. M. Bhandari, V.M. Montori, and E.H. Schemitsch The undue influence of significant p-values on the perceived importance of study results Acta Orthop 76 2005 291 295
5. J. van Oldenrijk, Y. van Berkel, G.M. Kerkhoffs, M. Bhandari, and R.W. Poolman Do authors report surgical expertise in open spine surgery related randomized controlled trials? A systematic review on quality of reporting Spine 38 2013 857 864
6. C.S. Bailey, C.G. Fisher, and M.F. Dvorak Type II error in the spine surgical literature Spine 29 2004 1146 1149
7. D. Bassler, M. Briel, V.M. Montori, M. Lane, P. Glasziou, Q. Zhou, and et al. Stopping randomized trials early for benefit and estimation of treatment effects: systematic review and meta-regression analysis JAMA 303 2010 1180 1187
8. V.M. Montori, P.J. Devereaux, N.K. Adhikari, K.E. Burns, C.H. Eggert, M. Briel, and et al. Randomized trials stopped early for benefit: a systematic review JAMA 294 2005 2203 2209
9. M. Walsh, S.K. Srinathan, D.F. McAuley, M. Mrkobrada, O. Levine, C. Ribic, and et al. The statistical significance of randomized controlled trial results is frequently fragile: a case for a fragility index J Clin Epidemiol 67 2014 622 628
10. B.H. Stromqvist, S. Berg, P. Gerdhem, R. Johnsson, A. Moller, T. Sahlstrand, and et al. X-stop versus decompressive surgery for lumbar neurogenic intermittent claudication: randomized controlled trial with 2-year follow-up Spine 38 2013 1436 1442
11. S.S. Bederman, and J.G. Wright Randomized trials in surgery: how far have we come? J Bone Joint Surg Am 94 2012 2 6
12. OE - OrthoEvidence [homepage on the Internet] 2014 Available at: www.myorthoevidence.com Accessed September 1, 2014
13. Higgins JP, Greene S, eds. Cochrane handbook for systematic reviews of interventions. Version 5.1.0. The Cochrane Collaboration: 2011. Available at: www.cochrane-handbook.org. Accessed September 1, 2014.
14. P.J. Devereaux, M. Bhandari, M. Clarke, V.M. Montori, D.J. Cook, S. Yusuf, and et al. Need for expertise based randomised controlled trials BMJ 330 2005 88
15. G.H. Guyatt, A.D. Oxman, R. Kunz, D. Atkins, J. Brozek, G. Vist, and et al. GRADE guidelines: 2. framing the question and deciding on important outcomes J Clin Epidemiol 64 2011 395 400
16. Science citation index [homepage on the Internet] 2014 Available at: http://ip-science.thomsonreuters.com/cgi-bin/jrnlst/jloptions.cgi?PC=K Accessed November 1, 2014
17. J. Sim, and C.C. Wright The kappa statistic in reliability studies: use, interpretation, and sample size requirements Phys Ther 85 2005 257 268
18. J.E. Kuhn, M.L. Greenfield, and E.M. Wojtys A statistics primer. Statistical tests for discrete data Am J Sports Med 25 1997 585 586
19. S. Lydersen, V. Pradhan, P. Senchaudhuri, and P. Laake Choice of test for association in small sample unordered r x c tables Stat Med 26 2007 4328 4343
20. S. Hopewell, M. Clarke, C. Lefebvre, and R. Scherer Handsearching versus electronic searching to identify reports of randomized trials Cochrane Database Syst Rev 2007 MR000001
21. M.M. Bala, E.A. Akl, X. Sun, D. Bassler, D. Mertz, F. Mejza, and et al. Randomized trials published in higher vs. lower impact journals differ in design, conduct, and analysis J Clin Epidemiol 66 2013 286 295
22. F. Farrokhyar, P.J. Karanicolas, A. Thoma, M. Simunovic, M. Bhandari, P.J. Devereaux, and et al. Randomized controlled trials of surgical interventions Ann Surg 251 2010 409 416
23. S.J. Pocock Current issues in the design and interpretation of clinical trials Br Med J (Clin Res Ed) 290 1985 39 42
24. A.R. Feinstein The unit fragility index: an additional appraisal of "statistical significance" for a contrast of two proportions J Clin Epidemiol 43 1990 201 209
25. S.D. Walter Statistical significance and fragility criteria for assessing a difference of two proportions J Clin Epidemiol 44 1991 1373 1378
26. Tornetta P III, Bhandari M, Parisien RL, Dashe J, Cronin P. In: Statistical significance in trauma research: too unstable to trust? 2014 Annual Meeting of the Orthopaedic Trauma Association: Tampa, Florida, 2014. p. 354.
27. J.P. Ioannidis Why most published research findings are false PLoS Med 2 2005 e124
28. S. Sprague, P. Leece, M. Bhandari, P. Tornetta 3rd, E. Schemitsch, M.F. Swiontkowski, and et al. Limiting loss to follow-up in a multicenter randomized trial in orthopedic surgery Control Clin Trials 24 2003 719 725
29. E.A. Akl, M. Briel, J.J. You, X. Sun, B.C. Johnston, J.W. Busse, and et al. Potential impact on estimated treatment effects of information lost to follow-up in randomised controlled trials (LOST-IT): systematic review BMJ 344 2012 e2809
30. B.A. Zelle, M. Bhandari, A.I. Sanchez, C. Probst, and H.C. Pape Loss of follow-up in orthopaedic trauma: is 80% follow-up still acceptable? J Orthop Trauma 27 2013 177 181
31. D.O. Scharfstein, J. Hogan, and A. Herman On the prevention and analysis of missing data in randomized clinical trials: the state of the art J Bone Joint Surg Am 94 2012 80 84
32. M. Kau, S. Sprague, T. Ignacy, A. Thoma, M. Bhandari, and F. Farrokhyar How to optimize participant retention and complete follow-up in surgical research Can J Surg 57 2014 420 427
33. H. Khan, N. Hussain, and M. Bhandari The influence of large clinical trials in orthopedic trauma: do they change practice? J Orthop Trauma 27 2013 e268 e274
34. S.E. Straus, J. Tetroe, and I. Graham Defining knowledge translation CMAJ 181 2009 165 168
35. V.K. Noonan, D.L. Wolfe, N.P. Thorogood, S.E. Park, J.T. Hsieh, J.J. Eng, and et al. Knowledge translation and implementation in spinal cord injury: a systematic review Spinal Cord 52 2014 578 587
36. P.J. Devereaux, M.T. Chan, J. Eisenach, T. Schricker, and D.I. Sessler The need for large clinical studies in perioperative medicine Anesthesiology 116 2012 1169 1175
37. S. Yusuf, R. Collins, and R. Peto Why do we need some large, simple randomized trials? Stat Med 3 1984 409 422
38. L. Mbuagbaw, F. Morfaw, J.E. Kunda, J.K. Mukonzo, J. Kastner, S. Zhang, and et al. Tips for charting the course of a successful health research career J Multidiscip Healthc 6 2013 163 168
39. L. Thabane, L. Mbuagbaw, S. Zhang, Z. Samaan, M. Marcucci, C. Ye, and et al. A tutorial on sensitivity analyses in clinical trials: the what, why, when and how BMC Med Res Methodol 13 2013 92
40. Y.S. Bai, Y.F. Niu, Z.Q. Chen, X.D. Zhu, L.K. Gabriel, H.K. Wong, and et al. Comparison of the pedicle screws placement between electronic conductivity device and normal pedicle finder in posterior surgery of scoliosis J Spinal Disord Tech 26 2013 316 320
41. S. Berg, T. Tullberg, B. Branth, C. Olerud, and H. Tropp Total disc replacement compared to lumbar fusion: a randomised controlled trial with 2-year follow-up Eur Spine J 18 2009 1512 1519
42. J.E. Bible, K.R. O'Neill, C.G. Crosby, J.G. Schoenecker, M.J. McGirt, and C.J. Devin Implant contamination during spine surgery Spine J 13 2013 637 640
43. J. Blasco, A. Martinez-Ferrer, J. Macho, L. San Roman, J. Pomes, J. Carrasco, and et al. Effect of vertebroplasty on pain relief, quality of life, and the incidence of new vertebral fractures: a 12-month randomized follow-up, controlled trial J Bone Miner Res 27 2012 1159 1166
44. J.K. Burkus, R.W. Haid, V.C. Traynelis, and P.V. Mummaneni Long-term clinical and radiographic outcomes of cervical disc replacement with the prestige disc: results from a prospective randomized controlled clinical trial J Neurosurg Spine 13 2010 308 318
45. Y. Chen, X. Wang, X. Lu, L. Yang, H. Yang, W. Yuan, and et al. Comparison of titanium and polyetheretherketone (PEEK) cages in the surgical treatment of multilevel cervical spondylotic myelopathy: a prospective, randomized, control study with over 7-year follow-up Eur Spine J 22 2013 1539 1546
46. L. Cheng, L. Nie, and L. Zhang Posterior lumbar interbody fusion versus posterolateral fusion in spondylolisthesis: a prospective controlled study in the Han nationality Int Orthop 33 2009 1043 1047
47. L. Cheng, L. Nie, M. Li, Y. Huo, and X. Pan Superiority of the bryan ((R)) disc prosthesis for cervical myelopathy: a randomized study with 3-year follow up Clin Orthop Relat Res 469 2011 3408 3414
48. D. Coric, P.D. Nunley, R.D. Guyer, D. Musante, C.N. Carmody, C.R. Gordon, and et al. Prospective, randomized, multicenter study of cervical arthroplasty: 269 patients from the Kineflex|C artificial disc investigational device exemption study with a minimum 2-year follow-up: clinical article J Neurosurg Spine 15 2011 348 358
49. J.R. Dimar 2nd, S.D. Glassman, J.K. Burkus, P.W. Pryor, J.W. Hardacker, and L.Y. Carreon Clinical and radiographic analysis of an optimized rhBMP-2 formulation as an autograft replacement in posterolateral lumbar spine arthrodesis J Bone Joint Surg Am 91 2009 1377 1386
50. M. Engquist, H. Lofgren, B. Oberg, A. Holtz, A. Peolsson, A. Soderlund, and et al. Surgery versus nonsurgical treatment of cervical radiculopathy: a prospective, randomized study comparing surgery plus physiotherapy with physiotherapy alone with a 2-year follow-up Spine 38 2013 1715 1722
51. M.R. Farrokhi, E. Alibai, and Z. Maghami Randomized controlled trial of percutaneous vertebroplasty versus optimal medical management for the relief of pain and disability in acute osteoporotic vertebral compression fractures J Neurosurg Spine 14 2011 561 569
52. V.T. Gauger, T.D. Voepel-Lewis, C.N. Burke, A.J. Kostrzewa, M.S. Caird, D.S. Wagner, and et al. Epidural analgesia compared with intravenous analgesia after pediatric posterior spinal fusion J Pediatr Orthop 29 2009 588 593
53. R. Hart, M. Komzak, F. Okal, D. Nahlik, P. Jajtner, and M. Puskeiler Allograft alone versus allograft with bone marrow concentrate for the healing of the instrumented posterolateral lumbar fusion Spine J 14 2014 1318 1324
54. B. He, L. Yan, H. Guo, T. Liu, X. Wang, and D. Hao The difference in superior adjacent segment pathology after lumbar posterolateral fusion by using 2 different pedicle screw insertion techniques in 9-year minimum follow-up Spine 39 2014 1093 1098
55. A. Hiller, I. Helenius, E. Nurmi, P.J. Neuvonen, M. Kaukonen, T. Hartikainen, and et al. Acetaminophen improves analgesia but does not reduce opioid requirement after major spine surgery in children and adolescents Spine 37 2012 E1225 E1231
56. R.J. Hurlbert, D. Alexander, S. Bailey, J. Mahood, E. Abraham, R. McBroom, and et al. rhBMP-2 for posterolateral instrumented lumbar fusion: a multicenter prospective randomized controlled trial Spine 38 2013 2139 2148
57. T. Jiya, T. Smit, J. Deddens, and M. Mullender Posterior lumbar interbody fusion using nonresorbable poly-ether-ether-ketone versus resorbable poly-L-lactide-co-D,L-lactide fusion devices: a prospective, randomized study to assess fusion and clinical outcome Spine 34 2009 233 237
58. D.F. Kallmes, B.A. Comstock, P.J. Heagerty, J.A. Turner, D.J. Wilson, T.H. Diamond, and et al. A randomized trial of vertebroplasty for osteoporotic spinal fractures N Engl J Med 361 2009 569 579
59. C.A. Klazen, A. Venmans, J. de Vries, W.J. van Rooij, F.H. Jansen, M.C. Blonk, and et al. Percutaneous vertebroplasty is not a risk factor for new osteoporotic compression fractures: results from VERTOS II AJNR Am J Neuroradiol 31 2010 1447 1450
60. P. Korovessis, K. Vardakastanis, T. Repantis, and V. Vitsas Balloon kyphoplasty versus KIVA vertebral augmentation-comparison of 2 techniques for osteoporotic vertebral body fractures: a prospective randomized study Spine 38 2013 292 299
61. H. Lofgren, M. Engquist, P. Hoffmann, B. Sigstedt, and L. Vavruch Clinical and radiological evaluation of trabecular metal and the Smith-Robinson technique in anterior cervical fusion for degenerative disease: a prospective, randomized, controlled study with 2-year follow-up Eur Spine J 19 2010 464 473
62. D. Murrey, M. Janssen, R. Delamarter, J. Goldstein, J. Zigler, B. Tay, and et al. Results of the prospective, randomized, controlled multicenter food and drug administration investigational device exemption study of the ProDisc-C total disc replacement versus anterior discectomy and fusion for the treatment of 1-level symptomatic cervical disc disease Spine J 9 2009 275 286
63. K. Nagahama, M. Kanayama, D. Togawa, T. Hashimoto, and A. Minami Does alendronate disturb the healing process of posterior lumbar interbody fusion? A prospective randomized trial J Neurosurg Spine 14 2011 500 507
64. S.V. Nandyala, A. Marquez-Lara, S.J. Fineberg, M. Pelton, and K. Singh Prospective, randomized, controlled trial of silicate-substituted calcium phosphate versus rhBMP-2 in a minimally invasive transforaminal lumbar interbody fusion Spine 39 2014 185 191
65. K.R. O'Neill, D.T. Lockney, J.E. Bible, C.G. Crosby, and C.J. Devin Bupivacaine for pain reduction after iliac crest bone graft harvest Orthopedics 37 2014 e428 e434
66. S. Ohtori, M. Suzuki, T. Koshi, M. Takaso, M. Yamashita, K. Yamauchi, and et al. Single-level instrumented posterolateral fusion of the lumbar spine with a local bone graft versus an iliac crest bone graft: a prospective, randomized study with a 2-year follow-up Eur Spine J 20 2011 635 639
67. F.M. Phillips, J.Y. Lee, F.H. Geisler, A. Cappuccino, C.D. Chaput, J.G. DeVine, and et al. A prospective, randomized, controlled clinical investigation comparing PCM cervical disc arthroplasty with anterior cervical discectomy and fusion. 2-year results from the US FDA IDE clinical trial Spine 38 2013 E907 E918
68. T.R. Pitzen, J. Chrobok, J. Stulik, S. Ruffing, J. Drumm, L. Sova, and et al. Implant complications, fusion, loss of lordosis, and outcome after anterior cervical plating with dynamic or rigid plates: two-year results of a multi-centric, randomized, controlled study Spine 34 2009 641 646
69. M. Putzier, P. Strube, J.F. Funk, C. Gross, H.J. Monig, C. Perka, and et al. Allogeneic versus autologous cancellous bone in lumbar segmental spondylodesis: a randomized prospective study Eur Spine J 18 2009 687 695
70. F. Ringel, C. Stuer, A. Reinke, A. Preuss, M. Behr, F. Auer, and et al. Accuracy of robot-assisted placement of lumbar and sacral pedicle screws: a prospective randomized comparison to conventional freehand screw implantation Spine 37 2012 E496 E501
71. G.U. Roh, S.Y. Yang, J.K. Shim, and Y.L. Kwak Efficacy of palonosetron versus ramosetron on preventing opioid-based analgesia-related nausea and vomiting after lumbar spinal surgery: a prospective, randomized, and double-blind trial Spine 39 2014 E543 E549
72. S. Ruetten, M. Komp, H. Merk, and G. Godolias Surgical treatment for lumbar lateral recess stenosis with the full-endoscopic interlaminar approach versus conventional microsurgical technique: a prospective, randomized, controlled study J Neurosurg Spine 10 2009 476 485
73. R.C. Sasso, P.A. Anderson, K.D. Riew, and J.G. Heller Results of cervical arthroplasty compared with anterior discectomy and fusion: four-year clinical outcomes in a prospective, randomized controlled trial J Bone Joint Surg Am 93 2011 1684 1692
74. C. Skold, H. Tropp, and S. Berg Five-year follow-up of total disc replacement compared to fusion: a randomized controlled trial Eur Spine J 22 2013 2288 2295
75. J.S. Thalgott, M.E. Fogarty, J.M. Giuffre, S.D. Christenson, A.K. Epstein, and C. Aprill A prospective, randomized, blinded, single-site study to evaluate the clinical and radiographic differences between frozen and freeze-dried allograft when used as part of a circumferential anterior lumbar interbody fusion procedure Spine 34 2009 1251 1256
76. H. Wu, Z.L. Gao, J.C. Wang, Y.P. Li, P. Xia, and R. Jiang Pedicle screw placement in the thoracic spine: a randomized comparison study of computer-assisted navigation and conventional techniques Chin J Traumatol 13 2010 201 205
77. Z. Yang, J. Tan, Y. Xu, H. Sun, L. Xie, R. Zhao, and et al. Treatment of MM-associated spinal fracture with percutaneous vertebroplasty (PVP) and chemotherapy Eur Spine J 21 2012 912 919
78. J.E. Zigler, R. Delamarter, D. Murrey, J. Spivak, and M. Janssen ProDisc-C and anterior cervical discectomy and fusion as surgical treatment for single-level cervical symptomatic degenerative disc disease: five-year results of a Food and Drug Administration study Spine 38 2013 203 209