Evidence-based decision making: When should we wait for more information?

Health Affairs

Volumn 27, Issue 6, 2008, Pages 1642-1653

  Chalkidou, Kalipso ^b   Lord, Joanne ^a   Fischer, Alastair ^b   Littlejohns, Peter ^b  

^a BRUNEL UNIVERSITY   (United Kingdom)

^b NICE ^*

Author keywords

[No Author keywords available]

Indexed keywords

CYCLOOXYGENASE 2 INHIBITOR; HYDROXYMETHYLGLUTARYL COENZYME A REDUCTASE INHIBITOR; NATALIZUMAB;

BIOMEDICAL TECHNOLOGY ASSESSMENT; CANCER SURGERY; COLORECTAL CANCER; COLORECTAL SURGERY; COST BENEFIT ANALYSIS; COST EFFECTIVENESS ANALYSIS; DECISION MAKING; DRUG ELUTING STENT; DRUG SAFETY; DRUG WITHDRAWAL; EMPHYSEMA; EVIDENCE BASED PRACTICE; HEALTH CARE COST; HUMAN; INTERMETHOD COMPARISON; LAPAROSCOPIC SURGERY; LUNG RESECTION; MEDICAL ETHICS; MEDICAL RESEARCH; MEDICAL TECHNOLOGY; NATIONAL HEALTH SERVICE; PATIENT SAFETY; REIMBURSEMENT; REVIEW; THERAPY DELAY; TREATMENT WITHDRAWAL; UNITED KINGDOM; UNITED STATES; UTERINE CERVIX CANCER; UTERINE CERVIX CYTOLOGY;

DECISION MAKING; DIFFUSION OF INNOVATION; EVIDENCE-BASED MEDICINE; GREAT BRITAIN; HEALTH SERVICES ACCESSIBILITY; HUMANS; REIMBURSEMENT MECHANISMS; TECHNOLOGY, MEDICAL; UNITED STATES;

EID: 56649092736     PISSN: 02782715     EISSN: 02782715     Source Type: Journal    
DOI: 10.1377/hlthaff.27.6.1642     Document Type: Review

References (45)

1. A.M. Garber, "Evidence-Based Coverage Policy," Health Affairs 20, no. 5 (2001): 62-82.
2. "Subject to and in accordance with such directions as the Secretary of State may give, the Institute shall perform...such functions in connection with the promotion of clinical excellence, and the effective use of available resources in the health service." National Health Service, England and Wales, "Directions and Consolidating Directions to the National Institute for Health and Clinical Excellence 2005," p. 2, https://www.nice.org.uk/ niceMedia/pdf/DirectionFromSecretaryOfState2005.pdf (accessed 8 August 2008).
3. Garber, "Evidence-Based Coverage Policy."
4. S.R. Tunis, "Reflections on Science, Judgment, and Value in Evidence-Based Decision Making: A Conversation with David Eddy," Health Affairs 26, no. 4 (2007): w500-w515 (published online 19 June 2007; 10.1377/hlthaff.26.4.w500).
5. NICE's approval of Herceptin for use in women with advanced breast cancer triggered the first ever appeal against a positive decision by a public local payer. M.L. Harding, "Drug Funding: Hewitt, Herceptin, and the 100m Pound Sterling Bill PCTs Can't Afford to Pay," Health Service Journal 115, no. 5985 (2005): 12-13.
6. J.S. Skinner, D.O. Staiger, and E.S. Fisher, "Is Technological Change in Medicine Always Worth It? The Case of Acute Myocardial Infarction," Health Affairs 25 (2006): w34-w47 (published online 7 February 2006; 10.1377/hlthaff.25.w34).
7. R. Hill et al., "Coronary Artery Stents: A Rapid Systematic Review and Economic Evaluation," Health Technology Assessment 8, no. 35 (2004): 1-242. NICE is currently reviewing its guidance; preliminary recommendations are against the routine use of drug-eluting stents in the NHS on cost-effectiveness grounds.
8. See "Coronary Artery Disease - Drug-Eluting Stents," February 2007, http://guidance.nice.org.uk/page.aspx?o=239915 (accessed 24 July 2008).
9. I. Evans et al., Testing Treatments: Better Research for Better Healthcare (London: British Library, 2006).
10. T. Carino, S. Sheingold, and S. Tunis, "Using Clinical Trials as a Condition of Coverage: Lessons from the National Emphysema Treatment Trial," Clinical Trials 1, no. 1 (2004): 108-114;
11. and Institute of Medicine, Adopting New Medical Technology (Washington: National Academies Press, 1994).
12. S.R. Tunis and S.D. Pearson, "Coverage Options for Promising Technologies: Medicare's 'Coverage with Evidence Development'," Health Affairs 25, no. 5 (2006): 1218-1230.
13. Also see Centers for Medicare and Medicaid Services, "National Coverage Determinations with Data Collection as a Condition of Coverage: Coverage with Evidence Development," 12 July 2006, http://www.cms.hhs.gov/ mcd/ncpc_view_document.asp?id=8 (accessed 29 July 2008).
14. We do not know whether this is the correct rate or whether the most appropriate technologies were selected for OIR. K. Chalkidou, A. Hoy, and P. Littlejohns, "Making a Decision to Wait for More Evidence: When the National Institute for Health and Clinical Excellence Recommends a Technology Only in the Context of Research," Journal of the Royal Society of Medicine 100, no. 10 (2007): 453-460.
15. A decision-making framework is illustrated in Supplemental Exhibit 1, online at http://content.healthaffairs.org/cgi/content/full/27/6/1642/DC1.
16. There always is a degree of discretion when determining the nature and height of the evidential hurdles applied. For example, not including cost-effectiveness considerations in CMS decisions is a political determination (as is including them in the context of NICE and the NHS). We do not wish to argue for the "right" set of criteria; we are merely highlighting the importance of making these criteria transparent and ensuring that they are consistently applied.
17. M. Gold et al., Cost-Effectiveness in Health and Medicine (Oxford and New York: Oxford University Press, 1996).
18. K. Claxton, "The Irrelevance of Inference: A Decision-Making Approach to the Stochastic Evaluation of Health Care Technologies," Journal of Health Economics 18, no. 3 (1999): 341-364.
19. See the online Technical Appendix, as in Note 12.
20. J. Chilcott et al., The Role of Modelling in Prioritising and Planning Clinical Trials, August 2003, http://www.hta.ac.uk/fullmono/mon723.pdf (accessed 15 September 2008).
21. S. Griffin et al., "Dangerous Omissions: The Opportunity Costs of Decision Uncertainty," Presentation at the Twenty-eighth Annual Meeting of the Society for Medical Decision Making, Boston, Massachusetts, October 2006. Fixed irretrievable costs of implementation (sunk costs) should not be used to argue against reversing a decision once implemented, but they can and should be considered before they have been incurred.
22. M. Amram and N. Kulatilaka, Real Options: Managing Strategic Investment in an Uncertain World (Boston: Harvard Business School Press, 1999);
23. and S. Palmer and P.C. Smith, "Incorporating Option Values into the Economic Evaluation of Health Care Technologies," Journal of Health Economics 19, no. 5 (2000): 755-766.
24. Ibid.
25. Detailed descriptions of all three case studies and relevant references can be found online, as in Note 12.
26. I. Chalmers, "Addressing Uncertainties about the Effects of Treatments Offered to NHS Patients: Whose Responsibility?" Journal of the Royal Society of Medicine 100, no. 10 (2007): 440-441.
27. These are illustrated in Supplemental Exhibit 1, online as in Note 12.
28. S.C. Hammill et al., "Review of the Registry's First Year, Data Collected, and Future Plans," Heart Rhythm 4, no. 9 (2007): 1260-1263;
29. and M.J. Lindsay et al., "The National Oncologic PET Registry: Expanded Medicare Coverage for PET under Coverage with Evidence Development," American Journal of Roentgenology 188, no. 4 (2007): 1109-1113.
30. See, for example, J.R. Berger and I.J. Koralnik, "Progressive Multifocal Leukoencephalopathy and Natalizumab - Unforeseen Consequences," New England Journal of Medicine 353, no. 4 (2005): 414-416;
31. and G.D. Curfman, S. Morrissey, and J.M. Drazen, "Expression of Concern Reaffirmed," New England Journal of Medicine 354, no. 11 (2006): 1193.
32. S.D. Pearson, F.G. Miller, and E.J. Emanuel, "Medicare's Requirement for Research Participation as a Condition of Coverage: Is It Ethical?" Journal of the American Medical Association 296, no. 8 (2006): 988-991.
33. NICE Citizens' Council, Only in Research, January 2007, http://www.nice.org.uk/media/129/29/OIRReport300407.pdf (accessed 11 September 2008).
34. I. Chalmers, "Well Informed Uncertainties about the Effects of Treatments," British Medical Journal 328, no. 7438 (2004): 475-476.
35. See Evans et al., Testing Treatments.
36. C.P. Gross et al., "Does Random Treatment Assignment Cause Harm to Research Participants?" PLoS Medicine 3, no. 6 (2006): e188;
37. and G.E. Vist et al., "Systematic Review to Determine Whether Participation in a Trial Influences Outcome," British Medical Journal 330, no. 7501 (2005): 1175.
38. S. Hammill, S. Phurrough, and R. Brindis, "The National ICD Registry: Nowand into the Future," Heart Rhythm 3, no. 4 (2006): 470-473.
39. The first was the prospective cohort study into photodynamic therapy for age-related macular degeneration where NHS R&D, local NHS commissioners (the equivalent of U.S. private insurers), the professional association of ophthalmologists, and methodologists worked to address a "coverage" question. See London School of Hygiene and Tropical Medicine, "Verteporfin Photodynamic Therapy Cohort Study - UK," http://www.lshtm.ac.uk/hsru/vpdt (accessed 29 July 2008).
40. The second was the Department of Health-funded research into liquid-based cytology for cervical screening. Privately funded research into the use of proprietary software for cognitive behavioral therapy is possibly a third (responsive) case. S.M. Moss et al., Evaluation of HPV/LBC Cervical Screening Pilot Studies: First Report to the Department of Health on Evaluation of LBC (London: Institute of Cancer Research, 2003).
41. See K. Chalkidou et al., "Evidence-Informed Evidence-Making," Journal of Health Services Research and Policy 13, no. 3 (2008): 167-173.
42. M.R. Gillick, "Medicare Coverage for Technological Innovations - Time for New Criteria?" New England Journal of Medicine 350, no. 21 (2004): 2199-2203.
43. Sean Tunis, Center for Medical Technology Policy and former overseer of CMS coverage policy, personal communication, May 2008.
44. Cooksey Review, http://www.hm-treasury.gov.uk/independent_reviews/ cooksey_review/cookseyreview_index.cfm (accessed 11 September 2008).
45. G.R. Wilensky, "Developing a Center for Comparative Effectiveness Information," Health Affairs 25 (2006): w572-w585 (published online 7 November 2006; 10.1377/hlthaff.25.w572).