Balancing benefit and risk of medicines: A systematic review and classification of available methodologies

Pharmacoepidemiology and Drug Safety

Volumn 23, Issue 7, 2014, Pages 667-678

  Mt Isa, Shahrul ^a   Hallgreen, Christine E ^a   Wang, Nan ^a   Callréus, Torbjörn ^b   Genov, Georgy ^c   Hirsch, Ian ^d   Hobbiger, Stephen F ^e   Hockley, Kimberley S ^a   Luciani, Davide ^f   Phillips, Lawrence D ^c   Quartey, George ^g   Sarac, Sinan B ^b   Stoeckert, Isabelle ^h   Tzoulaki, Ioanna ^a   Micaleff, Alain ⁱ   Ashby, Deborah ^a   Amzal, Billy ^j   Ashworth, Simon ^j   Asiimwe, Alex ^j   Bring, Johan ^j   Callreus, Torbjorn ^j   Chan, Edmond Kakit ^j   Dierig, Christoph ^j   Downey, Gerald ^j   Gelb, David ^j   Goginsky, Alesia ^j   Hermann, Richard ^j   Hobbiger, Steve ^j   Hockley, Kimberley ^j   Hughes, Diana ^j   Juhaeri, Juhaeri ^j   Kuhls, Silvia ^j   Lieftucht, Alfons ^j   Metcalf, Marilyn ^j   Mwangi, Jeremiah ^j   Nixon, Richard ^j   Noel, Rebecca ^j   Pears, John ^j   Peters, Ruth ^j   Phillips, Lawrence ^j   Shepherd, Susan ^j   Swain J , Elizabeth ^j   Thomson, Andrew ^j   Titeux, Laurence ^j   Ham, Rianne van den ^j   Staa, Tjeerd van ^j   Waddingham, Edward ^j   Wise, Lesley ^j   more..

^a IMPERIAL COLLEGE LONDON   (United Kingdom)

^b Danish Health and Medicines Authority   (Denmark)

^c EUROPEAN MEDICINES AGENCY   (United Kingdom)

^d ASTRAZENECA   (United Kingdom)

^e GLAXOSMITHKLINE   (United Kingdom)

^f MARIO NEGRI INSTITUTE FOR PHARMACOLOGICAL RESEARCH   (Italy)

^g GENENTECH INC   (United States)

^h BAYER PHARMA AG   (Germany)

ⁱ MERCK KGAA   (Germany)

^j NONE

more..

Author keywords

Benefit risk; Decision making; Framework; Medicines; Pharmacoepidemiology; Qualitative; Quantitative; Review

Indexed keywords

HUMAN; MEDLINE; PRIORITY JOURNAL; PUBLICATION; QUALITATIVE ANALYSIS; QUANTITATIVE ANALYSIS; REVIEW; RISK BENEFIT ANALYSIS; SYSTEMATIC REVIEW; TAXONOMY; CLASSIFICATION; DECISION MAKING; DRUG-RELATED SIDE EFFECTS AND ADVERSE REACTIONS; PROCEDURES; QUALITY OF LIFE; RISK ASSESSMENT; STATISTICAL MODEL;

DRUG;

DECISION MAKING; DRUG-RELATED SIDE EFFECTS AND ADVERSE REACTIONS; HUMANS; MODELS, STATISTICAL; PHARMACEUTICAL PREPARATIONS; QUALITY OF LIFE; RISK ASSESSMENT;

EID: 84903794459     PISSN: 10538569     EISSN: 10991557     Source Type: Journal    
DOI: 10.1002/pds.3636     Document Type: Review

References (117)

1. Eddy DM. The confidence profile method: a Bayesian method for assessing health technologies. Oper Res 1989; 37(2): 210-228.
2. Droitcour J, Silberman G, Chelimsky E. A new form of meta-analysis for combining results from randomized clinical trials and medical-practice databases. Int J Technol Assess Health Care 1993; 9(3): 440-449.
3. Lumley T. Network meta-analysis for indirect treatment comparisons. Stat Med 2002; 21(16): 2313-2324.
4. Lu G, Ades AE. Combination of direct and indirect evidence in mixed treatment comparisons. Stat Med 2004; 23(20): 3105-3124.
5. CIOMS Working Group IV. Benefit-risk Balance for Marketed Drugs. Evaluating Safety Signals. Council for International Organizations of Medical Sciences: Geneva, 1998.
6. CHMP. Report of the CHMP working group on benefit-risk assessment models and methods. January 2007. Report No.: EMEA/CHMP/15404/2007. http://www.ema.europa.eu/docs/en_GB/document_library/Regulatory_and_procedural_guideline/2010/01/WC500069668.pdf.
7. Mt-Isa S, Wang N, Hallgreen CE, et al. Review of Methodologies for Benefit and Risk Assessment of Medication. IMI-PROTECT: London, 2013. Report No.: 1. http://www.imi-protect.eu/documents/ShahruletalReviewofmethodologiesforbenefitandriskassessmentofmedicationMay2013.pdf (accessed 8 July 2013).
8. Phillips LD, Fasolo B, Zafiropoulos N, Beyer A. Benefit-risk Methodology Project Work Package 2 Report: Applicability of Current Tools and Processes for Regulatory Benefit-risk Assessment. European Medicines Agency: London, 2010. Report No.: EMA/549682/2010. http://www.ema.europa.eu (accessed 8 July 2013).
9. Guo JJ, Pandey S, Doyle J, Bian B, Lis Y, Raisch DW. A review of quantitative risk-benefit methodologies for assessing drug safety and efficacy-report of the ISPOR risk-benefit management working group. Value Health 2010; 13(5): 657-666.
10. Garrison LP, Towse A, Bresnahan BW. Assessing a structured, quantitative health outcomes approach to drug risk-benefit analysis. Health Aff (Millwood) 2007; 26(3): 684-695.
11. Holden WL. Benefit-risk analysis: a brief review and proposed quantitative approaches. Drug Saf 2003; 26(12): 853-862.
12. Boada J, Boada C, Garcia MM, Rodriguez C, Garcia M, Fernandez E. Net efficacy adjusted for risk: further developments. Expert Opin Drug Saf 2009; 8(6): 649-654.
13. Briggs AH, O'Brien BJ, Blackhouse G. Thinking outside the box: recent advances in the analysis and presentation of uncertainty in cost-effectiveness studies. Annu Rev Public Health 2002; 23: 377-401.
14. Carrin G. Economic evaluation of health care interventions: a review of alternative methods. Soc Sci Med 1984; 19(10): 1015-1030.
15. Cates CJ. Simpson's paradox and calculation of number needed to treat from meta-analysis. BMC Med Res Methodol 2002; 2: 1.
16. Chuang-Stein C, Entsuah R, Pritchett Y. Measures for conducting comparative benefit: risk assessment. Drug Inf J 2008; 42(3): 223-233.
17. Citrome L. Relative vs. absolute measures of benefit and risk: what's the difference? Acta Psychiatr Scand 2010; 121(2): 94-102.
18. Fletcher A. Pressure to treat and pressure to cost: a review of cost-effectiveness analysis. J Hypertens 1991; 9(3): 193-198.
19. Hahn S, Whitehead A. An illustration of the modelling of cost and efficacy data from a clinical trial. Stat Med 2003; 22(6): 1009-1024.
20. Harris RP, Helfand M, Woolf SH, et al. Current methods of the US Preventive Services Task Force: a review of the process. Am J Prev Med 2001; 20(3 Suppl): 21-35.
21. Hoch JS, Dewa CS. A clinician's guide to correct cost-effectiveness analysis: think incremental not average. Can J Psychiatry 2008; 53(4): 267-274.
22. Khan AA, Perlstein I, Krishna R. The use of clinical utility assessments in early clinical development. AAPS J 2009; 11(1): 33-38.
23. Linkov I, Satterstrom FK, Kiker G, Batchelor C, Bridges T, Ferguson E. From comparative risk assessment to multi-criteria decision analysis and adaptive management: recent developments and applications. Environ Int 2006; 32(8): 1072-1093.
24. Linnerooth J. The value of human life: a review of the models. Econ Inq 1979; 17(1): 52-74.
25. McAlearney AS, Schweikhart SB, Pathak DS. Quality-adjusted life-years and other health indices: a comparative analysis. Clin Ther 1999; 21(9): 1605-1629.
26. Mussen F, Salek S, Walker S. A quantitative approach to benefit-risk assessment of medicines-part 1: the development of a new model using multi-criteria decision analysis. Pharmacoepidemiol Drug Saf 2007; 16(Suppl 1): S2-S15.
27. Ouellet D. Benefit-risk assessment: the use of clinical utility index. Expert Opin Drug Saf 2010; 9(2): 289-300.
28. Pennington C, DeRienzo NR. An effective process for making decisions about major operating room purchases. AORN J 2010; 91(3): 341-349.
29. Rich KM, Miller GY, Winter-Nelson A. A review of economic tools for the assessment of animal disease outbreaks. Rev Sci Tech 2005; 24(3): 833-845.
30. Ried W. QALYs versus HYEs-what's right and what's wrong. A review of the controversy. J Health Econ 1998; 17(5): 607-625.
31. Sassi F. Calculating QALYs, comparing QALY and DALY calculations. Health Policy Plan 2006; 21(5): 402-408.
32. Shih YC. Bayesian approach in pharmacoeconomics: relevance to decision-makers. Expert Rev Pharmacoecon Outcomes Res 2003; 3(3): 237-250.
33. Wong EY, Ponce RA, Farrow S, Bartell SM, Lee RC, Faustman EM. Comparative risk and policy analysis in environmental health. Risk Anal 2003; 23(6): 1337-1349.
34. Boyd CM, Singh S, Varadhan R, et al. Methods for Benefit and Harm Assessment in Systematic Reviews.: Rockville MD, 2012.
35. Brass EP, Lofstedt R, Renn O. Improving the decision-making process for nonprescription drugs: a framework for benefit-risk assessment. Clin Pharmacol Ther 2011; 90(6): 791-803.
36. Brass EP, Lofstedt R, Renn O. A decision-analysis tool for benefit-risk assessment of nonprescription drugs. J Clin Pharmacol 2013; 53(5): 475-482.
37. Curtin F. Assessing the benefit: risk ratio of a drug-randomized and naturalistic evidence. Dialogues Clin Neurosci 2011; 13(2): 183-190.
38. Hellberg RS, DeWitt CAM, Morrissey MT. Risk-benefit analysis of seafood consumption: a review. Compre Rev Food Sci Food Saf 2012; 11(5): 490-517.
39. Hoekstra J, Hart A, Boobis A, et al. BRAFO tiered approach for benefit-risk assessment of foods. Food Chem Toxicol 2012; 50(Suppl 4): S684-S698.
40. Kodell RL, Chen JJ. Quantitative benefit-risk analysis for evaluating drug therapies. J Biopharm Stat 2013; 23(1): 231-238.
41. Levitan B, Mussen F. Evaluating benefit-risk during and beyond drug development: an industry view. Regul Rapporteur 2012; 9(6): 10-15.
42. Liberti L, McAuslane JN, Walker S. Standardizing the benefit-risk assessment of new medicines: practical applications of frameworks for the pharmaceutical healthcare professional. Pharm Med 2011; 25(3): 139-146.
43. Magnússon SH, Gunnlaugsdóttir H, van Loveren H, et al. State of the art in benefit-risk analysis: food microbiology. Food Chem Toxicol 2012; 50(1): 33-39.
44. Puhan MA, Singh S, Weiss CO, Varadhan R, Boyd CM. A framework for organizing and selecting quantitative approaches for benefit-harm assessment. Bmc Med Res Method 2012; 12: 173.
45. Quartey G, Wang J. Statistical aspects in comparative benefit-risk assessment: challenges and opportunities for pharmaceutical statisticians. Pharm Stat 2012; 11(1): 82-85.
46. Sashegyi A. A benefit-risk model to facilitate DMC-sponsor communication and decision making. Drug Inf J 2011; 45(6): 749-757.
47. van Valkenhoef G, Tervonen T, Zhao J, de Brock B, Hillege HL, Postmus D. Multicriteria benefit-risk assessment using network meta-analysis. J Clin Epidemiol 2012; 65(4): 394-403.
48. van Valkenhoef G, Tervonen T, Zwinkels T, de Brock B, Hillege H. ADDIS: a decision support system for evidence-based medicine. Decis Support Syst 2013; 55(2): 459-475.
49. Mussen F, Salek S, Walker S. Benefit-risk Appraisal of Medicines. John Wiley & Sons, Ltd: West Sussex, 2009.
50. Sonnenberg FA, Beck JR. Markov models in medical decision making: a practical guide. Med Decis Making 1993; 13(4): 322-338.
51. Tom E, Schulman KA. Mathematical models in decision analysis. Infect Control Hosp Epidemiol 1997; 18(1): 65-73.
52. Sarac SB, Rasmussen CH, Rasmussen MA, et al. A comprehensive approach to benefit-risk assessment in drug development. Basic Clin Pharmacol Toxicol 2012; 111(1): 65-72.
53. Coplan PM, Noel RA, Levitan BS, Ferguson J, Mussen F. Development of a framework for enhancing the transparency, reproducibility and communication of the benefit-risk balance of medicines. Clin Pharmacol Ther 2011; 89(2): 312-315.
54. CIRS Workshop Synopsis on Benefit-Risk. Building the Benefit-risk Toolbox: Are There Enough Common Elements Across the Different Methodologies to Enable a Consensus on a Scientifically Acceptable Framework for Making Benefit-risk Decisions? Washington, DC: Centre for Innovation in Regulatory Science; June 2012. http://cirsci.org/system/files/private/CIRS_June_2012_Workshop_Synopsis.pdf (accessed 8 July 2013).
55. Schulzer M, Mancini GB. 'Unqualified success' and 'unmitigated failure': number-needed-to-treat-related concepts for assessing treatment efficacy in the presence of treatment-induced adverse events. Int J Epidemiol 1996; 25(4): 704-712.
56. Johnson FR, Ozdemir S, Mansfield C, Hass S, Siegel CA, Sands BE. Are adult patients more tolerant of treatment risks than parents of juvenile patients? Risk Anal 2009; 29(1): 121-136.
57. Ashby D, Smith AF. Evidence-based medicine as Bayesian decision-making. Stat Med 2000; 19(23): 3291-3305.
58. Mt-Isa S, Tzoulaki I, Callreus T, Micaleff A, Ashby D. Weighing benefit-risk of medicines: concepts and approaches. Drug Discovery Today: Technol 2011; 8(1): e29-e35.
59. Dodgson J, Spackman M, Pearman A et al. Multi-criteria analysis: a manual. 2000. London, Department of the Environment, Transport and the Regions.
60. Keeney RL, Raiffa H. Decisions with Multiple Objectives: Preference and Value Tradeoffs. John Wiley: New York, 1976.
61. Chuang-Stein C. A New proposal for benefit-less-risk analysis in clinical trials. Control Clin Trials 1994; 15: 30-43.
62. Holden WL, Juhaeri J, Dai W. Benefit-risk analysis: a proposal using quantitative methods. Pharmacoepidemiol Drug Saf 2003; 12(7): 611-616.
63. Holden WL, Juhaeri J, Dai W. Benefit-risk analysis: examples using quantitative methods. Pharmacoepidemiol Drug Saf 2003; 12(8): 693-697.
64. Beckmann J. Basic aspects of risk-benefit analysis. Semin Thromb Hemost 1999; 25(1): 89-95.
65. Thompson JP, Noyes K, Dorsey ER, Schwid SR, Holloway RG. Quantitative risk-benefit analysis of natalizumab. Neurology 2008; 71(5): 357-364.
66. Nixon RM, Bansback N, Brennan A. Using mixed treatment comparisons and meta-regression to perform indirect comparisons to estimate the efficacy of biologic treatments in rheumatoid arthritis. Stat Med 2007; 26(6): 1237-1254.
67. Levitan BS, Andrews EB, Gilsenan A, et al. Application of the BRAT framework to case studies: observations and insights. Clin Pharmacol Ther 2011; 89(2): 217-224.
68. Sutton AJ, Cooper NJ, Abrams KR, Lambert PC, Jones DR. A Bayesian approach to evaluating net clinical benefit allowed for parameter uncertainty. J Clin Epidemiol 2005; 58(1): 26-40.
69. Korting H, Schafer-Korting M. The benefit-risk ratio. A handbook for the rational use of potentially hazardous drugs. CRC Press LLC: Boca Raton, 1999.
70. Payne JT, Loken MK. A survey of the benefits and risks in the practice of radiology. CRC Crit Rev Clin Radiol Nucl Med 1975; 6(3): 425-439.
71. Boada JN, Boada C, Garcia-Saiz M, Garcia M, Fernandez E, Gomez E. Net efficacy adjusted for risk (NEAR): a simple procedure for measuring risk:benefit balance. PLoS One 2008; 3(10): e3580.
72. Louviere JJ, Hensher DA, Swait J. Stated choice methods: analysis and application. Cambridge University Press: Cambridge, 2000.
73. Louviere JJ, Flynn TN, Carson RT. Discrete choice experiments are not conjoint analysis. J Choice Model 2010; 3(3): 57-72.
74. Laupacis A, Sackett DL, Roberts RS. An assessment of clinically useful measures of the consequences of treatment. N Engl J Med 1988; 318(26): 1728-1733.
75. GAO/PEMD. Cross design synthesis. A new strategy for medical effectiveness research. Washington, D.C.: United States General Accounting Office/ Program Evaluation and Methodology Division; March 1992. Report No.: B-244808
76. Deal L, Gold BD, Gremse DA, et al. Age-specific questionnaires distinguish GERD symptom frequency and severity in infants and young children: development and initial validation. J Pediatr Gastroenterol Nutr 2005; 41(2): 178-185.
77. Walker S, McAuslane N, Liberti L, Salek S. Measuring benefit and balancing risk: strategies for the benefit-risk assessment of new medicines in a risk-averse environment. Clin Pharmacol Ther 2009; 85(3): 241-246.
78. Boers M, Brooks P, Fries JF, Simon LS, Strand V, Tugwell P. A first step to assess harm and benefit in clinical trials in one scale. J Clin Epidemiol 2010: 63-6.
79. Edwards R, Wiholm BE, Martinez C. Concepts in risk-benefit assessment. A simple merit analysis of a medicine? Drug Saf 1996; 15(1): 1-7.
80. Eddy DM, Hasselblad V, McGivney W, Hendee W. The value of mammography screening in women under age 50years. JAMA: J Am Med Assoc 1988; 259(10): 1512-1519.
81. Ades AE, Sutton AJ. Multiparameter evidence synthesis in epidemiology and medical decision-making: current approaches. J R Stat Soc A Stat Soc 2006; 169(1): 5-35.
82. Hunink M, Glasziou P, Siegel J, et al. Decision making in health and medicine: integrating evidence and values. Cambridge University Press: Cambridge, 2001.
83. Hammond JS, Keeney RL, Raiffa H. Smart Choices: A Practical Guide to Making Better Decisions. Harvard Business School Press: Boston, MA, 2002.
84. Ouellet D, Werth J, Parekh N, Feltner D, McCarthy B, Lalonde RL. The use of a clinical utility index to compare insomnia compounds: a quantitative basis for benefit-risk assessment. Clin Pharmacol Ther 2009; 85(3): 277-282.
85. Renard D, Wu K, Wada R, Flesch G. Using desirability indices for decision making in drug development. Population Approach Group Europe (PAGE); 2009 Jun 23; St. Petersburg, Russia.: Novartis Pharma AG, Basel, Switzerland 2009.
86. Lynd LD, O'Brien BJ. Advances in risk-benefit evaluation using probabilistic simulation methods: an application to the prophylaxis of deep vein thrombosis. J Clin Epidemiol 2004; 57(8): 795-803.
87. van Staa TP, Smeeth L, Persson I, Parkinson J, Leufkens HG. What is the harm-benefit ratio of Cox-2 inhibitors? Int J Epidemiol 2008; 37(2): 405-413.
88. Smith RD. Construction of the contingent valuation market in health care: a critical assessment. Health Econ 2003; 12(8): 609-628.
89. Mitchell RC, Carson RT. Using surveys to value public goods: the contingent valuation method. The Johns Hopkins University Press: Washington, D.C., 2005.
90. Havet N, Morelle M, Remonnay R, Carrere MO. Cancer patients' willingness to pay for blood transfusions at home: results from a contingent valuation study in a French cancer network. Eur J Health Econ 2012; 13(3): 289-300.
91. Pearl J. Probabilistic Reasoning in Intelligent Systems: Networks of Plausible Inference. Morgan Kaufmann Publishers, Inc.: San Francisco, CA, 1988.
92. Jensen FV. An introduction to Bayesian Networks (1st edn). Springer-Verlag New York, Inc: Secaucus, NJ, 1996.
93. Jensen FV, Nielsen TD. Bayesian Networks and Decision Graphs. Springer: New York, 2007.
94. Darwiche A. Bayesian networks. Commun. ACM 2010; 53(12): 80-90.
95. Gelber RD, Cole BF, Gelber S, Aron G. Comparing treatments using quality-adjusted survival: the Q-twist method. Amer. Statistician 1995; 49(2): 161-169.
96. Goldhirsch A, Gelber RD, Simes RJ, Glasziou P, Coates AS. Costs and benefits of adjuvant therapy in breast cancer: a quality-adjusted survival analysis. J Clin Oncol 1989; 7(1): 36-44.
97. Ryan M, Gerard K, Amaya-Amaya M. Using Discrete Choice Experiments to Value Health and Health Care. Springer: Dordrecht, The Netherlands, 2008.
98. Ryan M, Hughes J. Using conjoint analysis to assess women's preferences for miscarriage management. Health Econ 1997; 6(3): 261-273.
99. Ryan M, Bate A, Eastmond CJ, Ludbrook A. Use of discrete choice experiments to elicit preferences. Qual Health Care 2001; 10: I55-I60.
100. Guyatt GH, Sinclair J, Cook DJ, Glasziou P. Users' guides to the medical literature: XVI. How to use a treatment recommendation. Evidence-Based Medicine Working Group and the Cochrane Applicability Methods Working Group. JAMA 1999; 281(19): 1836-1843.
101. Raiffa H. Decision analysis: introductory lectures on choices under uncertainty. McGraw Hill: Reading, MA, 1968.
102. Spiegelhalter D, Abrams K, Myles JP. Bayesian Approaches to Clinical Trials and Health-care Evaluation. John Wiley & Sons Ltd.: Chicester, West Sussex, 2004.
103. Frey P. Benefit-risk Considerations in CDER: Development of a Qualitative Framework. US Food and Drug Administration: Philadelphia, PA, 2012. http://www.fda.gov/downloads/AboutFDA/CentersOffices/OfficeofMedicalProductsandTobacco/CDER/UCM317788.pdf (accessed 08 July 2013).
104. Jenkins J. A United States Regulator's Perspective on Risk-Benefit Considerations. Shady Grove Conference Center: Rockville, MD, 2010.
105. Tervonen T, Figueira JR. A survey on stochastic multicriteria acceptability analysis methods. J Multi-Criteria Decis Anal 2008; 15(1-2): 1-14.
106. Tervonen T, van Valkenhoef G, Buskens E, Hillege HL, Postmus D. A stochastic multicriteria model for evidence-based decision making in drug benefit-risk analysis. Statist Med 2011; 30(12): 1419-1428.
107. Lahdelma R, Hokkanen J, Salminen P. SMAA-stochastic multiobjective acceptability analysis. Eur J Oper Res 1998; 106(1): 137-143.
108. Chuang-Stein C, Mohberg NR, Sinkula MS. Three measures for simultaneously evaluating benefits and risks using categorical data from clinical trials. Stat Med 1991; 10(9): 1349-1359.
109. Attia J, Page J, Heller RF, Dobson AJ. Impact numbers in health policy decisions. J Epidemiol Community Health 2002; 56(8): 600-605.
110. Heller RF, Dobson AJ, Attia J, Page J. Impact numbers: measures of risk factor impact on the whole population from case-control and cohort studies. J Epidemiol Community Health 2002; 56(8): 606-610.
111. Heller RF, Buchan I, Edwards R, Lyratzopoulos G, McElduff P, Leger SS. Communicating risks at the population level: application of population impact numbers. BMJ 2003; 327(7424): 1162-1165.
112. Verma A, Torun P, Harris E, et al. Population impact analysis: a framework for assessing the population impact of a risk or intervention. J Public Health (Oxf) 2012; 34(1): 83-89.
113. Lynd LD, Najafzadeh M, Colley L, et al. Using the incremental net benefit framework for quantitative benefit-risk analysis in regulatory decision-making-a case study of alosetron in irritable bowel syndrome. Value Health 2010; 13(4): 411-417.
114. Minelli C, Abrams KR, Sutton AJ, Cooper NJ. Benefits and harms associated with hormone replacement therapy: clinical decision analysis. BMJ 2004; 328(7436): 371.
115. Lynd LD, Marra CA, Najafzadeh M, Sadatsafavi M. A quantitative evaluation of the regulatory assessment of the benefits and risks of rofecoxib relative to naproxen: an application of the incremental net-benefit framework. Pharmacoepidemiol Drug Saf 2010; 19(11): 1172-1180.
116. Riegelman R, Schroth WS. Adjusting the number needed to treat: incorporating adjustments for the utility and timing of benefits and harms. Med Decis Making 1993; 13(3): 247-252.
117. Altman DG. Confidence intervals for the number needed to treat. BMJ 1998; 317(7168): 1309-1312.