Paradoxical interpretations of noninferiority studies: Violating the excluded middle

Future Virology

Volumn 7, Issue 11, 2012, Pages 1055-1063

  DiNubile, Mark J ^a   Sklar, Peter ^a   Lupinacci, Robert J ^a   Eron, Joseph J ^b  

^a MERCK RESEARCH LABORATORIES   (United States)

^b UNIVERSITY OF NORTH CAROLINA   (United States)

Author keywords

clinical trials; confidence intervals; noninferiority margins; statistical inference

Indexed keywords

AMPHOTERICIN B DEOXYCHOLATE; ANTIRETROVIRUS AGENT; CREATININE; EMTRICITABINE PLUS TENOFOVIR DISOPROXIL; GENTAMICIN; PAROMOMYCIN; PLACEBO; RALTEGRAVIR; TOBRAMYCIN;

ANTIBIOTIC THERAPY; ANTIVIRAL THERAPY; CREATININE BLOOD LEVEL; DRUG EFFICACY; DRUG RESEARCH; DRUG RESPONSE; DRUG SAFETY; DRUG TREATMENT FAILURE; HUMAN; HUMAN IMMUNODEFICIENCY VIRUS INFECTION; PRIORITY JOURNAL; REVIEW; VISCERAL LEISHMANIASIS;

EID: 84870216704     PISSN: 17460794     EISSN: 17460808     Source Type: Journal    
DOI: 10.2217/fvl.12.101     Document Type: Review

References (49)

1. Kaji AH, Lewis RJ. Are we looking for superiority, equivalence, or noninferiority? Asking the right question and answering it correctly. Ann. Emerg. Med. 55, 408-411 (2010).
2. Powers JH. Noninferiority and equivalence trials: deciphering 'similarity' of medical interventions. Stat. Med. 27, 343-352 (2008). •• Comprehensive critique from a worldrenowned expert emphasizing that fundamental misunderstandings about the design and interpretation of noninferiority studies extend 'beyond mere semantics'. (Pubitemid 351227871)
3. Tunes DS, Logan BR, Klein JP. Methods for equivalence and noninferiority testing. Biol. Blood Marrow Transplant 15(1 Suppl.), 120-127 (2009).
4. D'Agostino RB Sr, Massaro JM, Sullivan LM. Non-inferiority trials: design concepts and issues - the encounters of academic consultants in statistics. Stat. Med. 22, 169-186 (2003). (Pubitemid 36163579)
5. Djulbegovic B, Clarke M. Scientific and ethical issues in equivalence trials. JAMA 285(9), 1206-1208 (2001). (Pubitemid 32198488)
6. Le Henanff A, Giraudeau B, Baron G, Ravaud P. Quality of reporting of noninferiority and equivalence randomized trials. JAMA 295(10), 1147-1151 (2006). (Pubitemid 43352344)
7. Gotzsche PC. Lessons from and cautions about noninferiority and equivalence randomized trials. JAMA 295(10), 1172-1174 (2006). (Pubitemid 43352349)
8. Hilton JF. Designs of superiority and noninferiority trials for binary responses are noninterchangeable. Biom. J. 48(6), 934-947 (2006). •An inherent lack of symmetry exists between superiority and noninferiority hypotheses as usually formulated.
9. Tsong Y, Zhang J. Simultaneous test for superiority and noninferiority hypotheses in active-controlled clinical trials. J. Biopharm. Stat. 17, 247-257 (2007). (Pubitemid 46423596)
10. Fleming TR, Powers JH. Issues in noninferiority trials: the evidence in community-acquired pneumonia. Clin. Infect. Dis. 47(Suppl. 3), S108-S1020 (2008).
11. Powers JH, Ross DB, Brittain E, Albrecht R, Goldberger MJ. The United States Food and Drug Administration and noninferiority margins in clinical trials of antimicrobial agents. Clin. Infect. Dis. 34(6), 879-881 (2002). (Pubitemid 34194867)
12. Shlaes DM, Moellering RC, Jr. The United States Food and Drug Administration and the end of antibiotics. Clin. Infect. Dis. 34(3), 420-422 (2002). •• Regulatory requirements mandating smaller noninferiority margins for antibiotic trials could have an unintended chilling effect on the development of novel agents by pharmaceutical companies, despite increasing antimicrobial resistance. (Pubitemid 34062252)
13. Spellberg B, Talbot GH, Boucher HW et al. Antimicrobial agents for complicated skin and skin-structure infections: justification of noninferiority margins in the absence of placebo-controlled trials. Clin. Infect. Dis. 49, 383-291 (2009).
14. Tuma RS. Trend toward noninferiority trials may mean more difficult interpretation of trial results. J. Natl Cancer Inst. 99, 1746-1748 (2007).
15. Gandjour A. Economic evaluations alongside equivalence and noninferiority trials. Value Health 12(4), 629 (2009).
16. Kaul S, Diamond GA. Good enough: a primer on the analysis and interpretation of noninferiority trials. Ann. Intern. Med. 145, 62-69 (2006). •• Noninferiority designs are frequently used when placebo-controlled studies are deemed unethical. Although superiority trials are a more straightforward method to demonstrate efficacy for an investigational drug over placebo, they do not account for the established efficacy of the current standard of care. On the other hand, noninferiority margins should be chosen so as to ensure the new drug is better than placebo. Clinical judgment usually dictates the degree of inferiority that is medically acceptable, but potential safety and tolerability advantages of the new drug versus the standard bearer that may counterbalance modestly inferior efficacy are still not well established during drug-development programs as protocols are being written. (Pubitemid 46780603)
17. Tsou HH, Hsiao CF, Chow SC, Yue L, Xu Y, Lee S. Mixed noninferiority margin and statistical tests in active controlled trials. J. Biopharm. Stat. 17(2), 339-357 (2007). (Pubitemid 46423601)
18. Freidlin B, Korn EL, George SL, Gray R. Randomized clinical trial design for assessing noninferiority when superiority is expected. J. Clin. Oncol. 25, 5019-5023 (2007). (Pubitemid 350220436)
19. Logan BR, Tamhane AC. Superiority inferences on individual endpoints following noninferiority testing in clinical trials. Biom. J. 50(5), 693-703 (2008).
20. Ng TH. Simultaneous testing of noninferiority and superiority increases the false discovery rate. J. Biopharm. Stat. 17(2), 259-264 (2007).
21. Ng TH. Noninferiority hypotheses and choice of noninferiority margin. Stat. Med. 27(26), 5392-5406 (2008).
22. Nie L, Soon G. A covariate-adjustment regression model approach to noninferiority margin definition. Stat. Med. 29(10), 1107-1113 (2010).
23. Kaul S, Diamond GA, Weintraub WS. Trials and tribulations of non-inferiority: the ximelagatran experience. J. Am. Coll. Cardiol. 46(11), 1986-1995 (2005). (Pubitemid 41713579)
24. Lange S, Freitag G. Choice of delta: requirements and reality - xresults of a systematic review. Biom. J. 47, 12-27 (2005). (Pubitemid 40390435)
25. Smith CR, Lipsky JJ, Laskin OL et al. Double-blind comparison of the nephrotoxicity and auditory toxicity of gentamicin and tobramycin. N. Engl. J. Med. 302(20), 1106-1109 (1980). (Pubitemid 10107526)
26. Mora-Duarte J, Betts R, Rotstein C et al. Comparison of caspofungin and amphotericin B for invasive candidiasis. N. Engl. J. Med. 347(25), 2020-2029 (2002). (Pubitemid 35461658)
27. Walsh TJ, Teppler H, Donowitz GR et al. Caspofungin versus liposomal amphotericin B for empirical antifungal therapy in patients with persistent fever and neutropenia. N. Engl. J. Med. 351(14), 1391-1402 (2004).
28. Beasley BN, Unger EF, Temple R. Anticoagulant options - why the FDA approved a higher but not a lower dose of dabigatran. N. Engl. J. Med. 364(19), 1788-1790 (2011).
29. Connolly SJ, Ezekowitz MD, Yusuf S et al. Dabigatran versus warfarin in patients with atrial fibrillation. N. Engl. J. Med. 361(12), 1139-1151 (2009).
30. N. Engl. J. Med. 363, 1877 (2010); eratum.
31. Piaggio G, Elbourne DR, Altman DG, Pocock SJ, Evans SJ. Reporting of noninferiority and equivalence randomized trials: an extension of the CONSORT statement. JAMA 295(10), 1152-1160 (2006). (Pubitemid 43352345)
32. Soonawala D, Middelburg RA, Egger M, Vandenbroucke JP, Dekkers OM. Efficacy of experimental treatments compared with standard treatments in non-inferiority trials: a meta-analysis of randomized controlled trials. Int. J. Epidemiol. 39(6), 1567-1581 (2010).
33. Helgason CM. Artificial adaptive systems and fuzzy measures for translation of clinical trial results to the bedside. Curr. Treat. Options Cardiovasc. Med. 11(3), 187-190 (2009).
34. Sundar S, Jha TK, Thakur CP, Sinha PK, Bhattacharya SK. Injectable paromomycin for visceral leishmaniasis in India. N. Engl. J. Med. 356(25), 2571-2581 (2007). (Pubitemid 46961002)
35. Eron JJ, Young B, Cooper DA et al. Switch to a raltegravir-based regimen versus continuation of a lopinavir-ritonavir-based regimen in stable HIV-infected patients with suppressed viraemia (SWITCHMRK 1 and 2): two multicentre, double-blind, randomised controlled trials. Lancet 375(9712), 396-407 (2010).
36. Eron JJ, Rockstroh JK, Reynes J et al. Raltegravir once or twice daily in treatmentnaive HIV-1 infected patients. Lancet Infect. Dis. 11, 70196-70197 (2011).
37. Garattini S, Bertele V. Non-inferiority trials are unethical because they disregard patients' interests. Lancet 370(9602), 1875-1877 (2007). (Pubitemid 350180009)
38. Hilton JF. Noninferiority trial designs for odds ratios and risk differences. Stat. Med. 29(9), 982-993 (2010).
39. Hrobjartsson A, Gotzsche PC. Is the placebo powerless? An analysis of clinical trials comparing placebo with no treatment. N. Engl. J. Med. 344(21), 1594-1602 (2001). (Pubitemid 32479913)
40. Wechsler ME, Kelley JM, Boyd IO et al. Active albuterol or placebo, sham acupuncture, or no intervention in asthma. N. Engl. J. Med. 365(2), 119-126 (2011).
41. Fleming TR, Odem-Davis K, Rothman M, Shen YL. Some essential considerations in the design and conduct of non-inferiority trials. Clin Trials 8(4), 432-439 (2011). •• A call for caution focusing on the introduction of bias into noninferiority trial designs is followed by a plea to pursue alternative paradigms where feasible.
42. Braitman LE. Confidence intervals assess both clinical significance and statistical significance. Ann. Intern. Med. 114(6), 515-517 (1991). •Explains that not all statistically significant differences are clinically significant. Conversely, lack of evidence of a statistically significant difference is not necessarily evidence of lack of a clinically significant difference.
43. Freedman B. Equipoise and the ethics of clinical research. N. Engl. J. Med. 317(3), 141-145 (1987). (Pubitemid 17092151)
44. Djulbegovic B. Uncertainty and equipoise: at interplay between epistemology, decision making and ethics. Am. J. Med. Sci. 342(4), 282-289 (2011).
45. Flandre P. Statistical methods in recent HIV noninferiority trials: reanalysis of 11 trials. PLoS One 6(9), e22871 (2011). •Noninferiority trials have emerged as the design du jour for antiretroviral drug studies, using a difference in the proportions of responders between the two treatment arms as the primary efficacy outcome. From an analysis of 11 recent studies, the author concludes that the specific statistical methodology used to compute confidence intervals may alter the conclusions of such trials.
46. Tanaka S, Kinjo Y, Kataoka Y, Yoshimura K, Teramukai S. Statistical issues and recommendations for noninferiority trials in oncology: a systematic review. Clin. Cancer Res. 18(7), 1837-1847 (2012).
47. Head SJ, Kaul S, Bogers AJ, Kappetein AP. Non-inferiority study design: lessons to be learned from cardiovascular trials. Eur. Heart J. 33(11), 1318-1324 (2012).
48. Wangge G, Roes KC, de Boer A, Hoes AW, Knol MJ. The challenges of determining noninferiority margins: a case study of noninferiority randomized controlled trials of novel oral anticoagulants. CMAJ, doi:10.1503/cmaj.120142 (2012) (Epub ahead of print).
49. Demets DL. Current development in clinical trials: issues old and new. Stat. Med. doi:10.1002/sim.5405 (2012) (Epub ahead of print). • Selection of a noninferiority margin can be impacted by assumptions that can neither be validated nor quantified (as well as by practical considerations). As a starting point the noninferiority margin can be based on the efficacy difference that would affect clinical practice. To avoid misleadingly optimistic claims about the test drug, the comparator must represent the most efficacious standard in use at the initiation of the trial.