When is evidence sufficient?

Health Affairs

Volumn 24, Issue 1, 2005, Pages 93-101

  Claxton, Karl ^a   Cohen, Joshua T ^b   Neumann, Peter J ^c  

^a UNIVERSITY OF YORK   (United Kingdom)

^b HARVARD CENTER FOR RISK ANALYSIS   (United States)

^c HARVARD SCHOOL OF PUBLIC HEALTH   (United States)

Author keywords

[No Author keywords available]

Indexed keywords

CLINICAL TRIAL; CONTROLLED CLINICAL TRIAL; COST BENEFIT ANALYSIS; DECISION MAKING; EVIDENCE BASED MEDICINE; HEALTH CARE COST; HEALTH CARE POLICY; HUMAN; MEDICAL INFORMATION; MEDICAL RESEARCH; METHODOLOGY; QUANTITATIVE ANALYSIS; RANDOMIZED CONTROLLED TRIAL; REVIEW;

DECISION MAKING, ORGANIZATIONAL; EVIDENCE-BASED MEDICINE; GREAT BRITAIN; HEALTH POLICY; RANDOMIZED CONTROLLED TRIALS; UNITED STATES;

EID: 12844268669     PISSN: 02782715     EISSN: None     Source Type: Journal    
DOI: 10.1377/hlthaff.24.1.93     Document Type: Review

References (23)

1. See, for example, K. Claxton, "The Irrelevance of Inference: A Decision Making Approach to the Stochastic Evaluation of Health Care Technologies," Journal of Health Economics 18, no. 3 (1999): 341-364; K. Claxton, L. Lacey, and S. Walker, "Selecting Treatments: A Decision Theoretic Approach," Journal of the Royal Statistical Society 163, no. 2 (2000): 211-225; and K. Claxton and K. Thompson, "A Dynamic Programming Approach to the Efficient Design of Clinical Trials," Journal of Health Economics 20, no. 5 (2001): 797-822.
2. See, for example, K. Claxton, "The Irrelevance of Inference: A Decision Making Approach to the Stochastic Evaluation of Health Care Technologies," Journal of Health Economics 18, no. 3 (1999): 341-364; K. Claxton, L. Lacey, and S. Walker, "Selecting Treatments: A Decision Theoretic Approach," Journal of the Royal Statistical Society 163, no. 2 (2000): 211-225; and K. Claxton and K. Thompson, "A Dynamic Programming Approach to the Efficient Design of Clinical Trials," Journal of Health Economics 20, no. 5 (2001): 797-822.
3. See, for example, K. Claxton, "The Irrelevance of Inference: A Decision Making Approach to the Stochastic Evaluation of Health Care Technologies," Journal of Health Economics 18, no. 3 (1999): 341-364; K. Claxton, L. Lacey, and S. Walker, "Selecting Treatments: A Decision Theoretic Approach," Journal of the Royal Statistical Society 163, no. 2 (2000): 211-225; and K. Claxton and K. Thompson, "A Dynamic Programming Approach to the Efficient Design of Clinical Trials," Journal of Health Economics 20, no. 5 (2001): 797-822.
4. R.P. Harris et al., "Current Methods of the U.S. Preventive Services Task Force: A Review of the Process," American Journal of Preventive Medicine 30, no. 3, Supp. (2001): 21-35.
5. Agency for Healthcare Research and Quality, "Evidence-based Practice Centers," RFP no. AHRQ-02-0003, 18 January 2002, www.ahrq.gov/downloads/ pub/contract/rfp20003.pdf (15 November 2004); BlueCross BlueShield Association Technology Evaluation Center, "TEC Criteria," 2004, www.bluecares.com/tec/teccriteria.html (12 October 2004); Canadian Task Force on the Periodic Health Examination, "Methodology," June 1998, www.ctfphc.org/Methodology.html (12 October 2004); and D.C. McCrory and S.Z. Lewis, "Methodology and Grading for Pulmonary Hypertension Evidence Review and Guideline Development," Chest 126, no. 1 Supp. (2004): 11S-13S.
6. Agency for Healthcare Research and Quality, "Evidence-based Practice Centers," RFP no. AHRQ-02-0003, 18 January 2002, www.ahrq.gov/downloads/ pub/contract/rfp20003.pdf (15 November 2004); BlueCross BlueShield Association Technology Evaluation Center, "TEC Criteria," 2004, www.bluecares.com/tec/teccriteria.html (12 October 2004); Canadian Task Force on the Periodic Health Examination, "Methodology," June 1998, www.ctfphc.org/Methodology.html (12 October 2004); and D.C. McCrory and S.Z. Lewis, "Methodology and Grading for Pulmonary Hypertension Evidence Review and Guideline Development," Chest 126, no. 1 Supp. (2004): 11S-13S.
7. Agency for Healthcare Research and Quality, "Evidence-based Practice Centers," RFP no. AHRQ-02-0003, 18 January 2002, www.ahrq.gov/downloads/ pub/contract/rfp20003.pdf (15 November 2004); BlueCross BlueShield Association Technology Evaluation Center, "TEC Criteria," 2004, www.bluecares.com/tec/teccriteria.html (12 October 2004); Canadian Task Force on the Periodic Health Examination, "Methodology," June 1998, www.ctfphc.org/Methodology.html (12 October 2004); and D.C. McCrory and S.Z. Lewis, "Methodology and Grading for Pulmonary Hypertension Evidence Review and Guideline Development," Chest 126, no. 1 Supp. (2004): 11S-13S.
8. Agency for Healthcare Research and Quality, "Evidence-based Practice Centers," RFP no. AHRQ-02-0003, 18 January 2002, www.ahrq.gov/downloads/ pub/contract/rfp20003.pdf (15 November 2004); BlueCross BlueShield Association Technology Evaluation Center, "TEC Criteria," 2004, www.bluecares.com/tec/teccriteria.html (12 October 2004); Canadian Task Force on the Periodic Health Examination, "Methodology," June 1998, www.ctfphc.org/Methodology.html (12 October 2004); and D.C. McCrory and S.Z. Lewis, "Methodology and Grading for Pulmonary Hypertension Evidence Review and Guideline Development," Chest 126, no. 1 Supp. (2004): 11S-13S.
9. M.R. Gold et al., Cost-Effectiveness in Health and Medicine (New York: Oxford University Press, 1996); and B. O'Brien and A. Gafni, "When Do the Dollars Make Sense? Toward a Conceptual Framework for Contingent Valuation Studies in Health Care," Medical Decision Making 16, no. 3 (1996): 288-299.
10. M.R. Gold et al., Cost-Effectiveness in Health and Medicine (New York: Oxford University Press, 1996); and B. O'Brien and A. Gafni, "When Do the Dollars Make Sense? Toward a Conceptual Framework for Contingent Valuation Studies in Health Care," Medical Decision Making 16, no. 3 (1996): 288-299.
11. M.C. Weinstein and H.V. Fineberg, Clinical Decision Analysts (Philadelphia: W.B. Saunders Company, 1980).
12. J.K. Hammitt and J.A.K. Cave, Research Planning for Food Safety: A Value of Information Approach (Santa Monica, Calif.: RAND, 1991).
13. K. Claxton et al., "Bayesian Value-of-Information Analysis: An Application to a Policy Model of Alzheimer's Disease," International Journal of Technology Assessment in Health Care 17, no. 1 (2001): 38-55.
14. P.J. Neumann et al., "Cost-Effectiveness of Donepezil in the Treatment of Mild or Moderate Alzheimer's Disease," Neurology 52, no. 11 (1999): 1138-1145.
15. Claxton and colleagues' VOI analysis is only a starting point because it evaluates the value of perfect information. That is, it estimates how much the decisionmaker's performance would be expected to improve if various questions could be resolved definitively. Of course, in the real world, information is rarely perfect, and the analysis would have to take into account the fact that although additional evidence reduces uncertainty, it does not eliminate it. VOI techniques to evaluate the expected value of imperfect information (or "sample" information) have been used in other settings. Once the value of the imperfect information is estimated, it can be compared with the cost of its acquisition, and a research plan can be developed. See A.E. Ades, G. Lu, and K. Claxton, "Expected Value of Sample Information Calculations in Medical Decision Modeling," Medical Decision Making 24, no. 2 (2004): 207-227.
16. J. Hjelmgren, F. Berggren, and F. Andersson, "Health Economic Guidelines - Similarities, Differences, and Some Implications," Value in Health 4, no. 3 (2001): 225-250.
17. National Institute for Clinical Excellence, "Guide to the Methods of Technology Appraisal" (London: NICE, 2004).
18. Ibid., Section 5.2.1.1.
19. Ibid., Section 5.9.6.1.
20. K. Claxton et al., "A Pilot Study of Value of Information Analysis to Support Research Recommendations for the National Institute for Clinical Excellence" (London: NICE, 2004).
21. K. Claxton et al., "A Pilot Study on the Use of Decision Theory and Value of Information Analysis as Part of the NHS Health Technology Assessment Programme," Health Technology Assessment 8, no. 31 (2004): 1-103, iii-x.
22. Although the key challenges are more general and relevant to estimating cost, effect, and decision uncertainty, several issues specific to VOI must be addressed. These include (1) estimating the effective population that may benefit from additional evidence, including estimating time horizons for different technologies and incorporating these factors into VOI estimates; (2) estimating overall VOI based on VOI estimates for patient subgroups; (3) accounting for correlations between inputs in VOI estimates; and (4) developing methods to assess the stability of VOI estimates and the required number of iterations.
23. S.R. Tunis, D.B. Stryer, and C.M. Clancy, "Practical Clinical Trials: Increasing the Value of Clinical Research for Decision Making in Clinical and Health Policy," Journal of the American Medical Association 290, no. 12 (2003): 1624-1632.