Improving the quality of discrete-choice experiments in health: how can we assess validity and reliability?

Expert Review of Pharmacoeconomics and Outcomes Research

Volumn 17, Issue 6, 2017, Pages 531-542

  Janssen, Ellen M ^a   Marshall, Deborah A ^b   Hauber, A Brett ^c   Bridges, John F P ^a  

^a JOHNS HOPKINS BLOOMBERG SCHOOL OF PUBLIC HEALTH   (United States)

^b UNIVERSITY OF CALGARY   (Canada)

^c RTI HEALTH SOLUTIONS   (United States)

Author keywords

Benefit risk assessment; discrete choice experiment; patient preferences; reliability; stated preference methods; validity

Indexed keywords

EXERCISE; HUMAN; PATIENT PREFERENCE; READING; RELIABILITY; RISK ASSESSMENT; STUDY DESIGN; VALIDITY; BIOMEDICAL TECHNOLOGY ASSESSMENT; DECISION MAKING; HEALTH CARE DELIVERY; METHODOLOGY; PROCEDURES; REPRODUCIBILITY; VALIDATION STUDY;

CHOICE BEHAVIOR; DECISION MAKING; DELIVERY OF HEALTH CARE; HUMANS; PATIENT PREFERENCE; REPRODUCIBILITY OF RESULTS; RESEARCH DESIGN; TECHNOLOGY ASSESSMENT, BIOMEDICAL; VALIDATION STUDIES AS TOPIC;

EID: 85032950804     PISSN: 14737167     EISSN: 17448379     Source Type: Journal    
DOI: 10.1080/14737167.2017.1389648     Document Type: Review

References (74)

1. Institute of Medicine (US) Committee on Quality of Health Care in America. Crossing the quality chasm: a new health system for the 21st century. Washington (DC): The National Academy of Sciences; 2001.
2. Epstein RM, Street RL., The values and value of patient-centered care. Ann Fam Med. 2011 Mar;9(2):100–103.
3. Frank L, Basch E, Selby JV., The PCORI perspective on patient-centered outcomes research. JAMA. 2014 Oct 15;312(15):1513–1514. DOI:10.1001/jama.2014.11100.
4. Domecq JP, Prutsky G, Elraiyah T, et al. Patient engagement in research: a systematic review. BMC Health Serv Res. 2014 Feb;26(14):89.
5. Bridges JF, Jones C. Patient-based health technology assessment: a vision of the future. Int J Technol Assess Health Care. 2007 Winter;23(1):30–35. DOI:10.1017/s0266462307051549.
6. Ho M, Saha A, McCleary KK, et al. A framework for incorporating patient preferences regarding benefits and risks into regulatory assessment of medical technologies. Value Health. 2016 Sep–Oct;19(6):746–750. DOI:10.1016/j.jval.2016.02.019.
7. Muhlbacher AC, Bridges JF, Bethge S, et al. Preferences for antiviral therapy of chronic hepatitis C: a discrete choice experiment. Eur J Health Econ. 2016 Feb 04. DOI:10.1007/s10198-016-0763-8.
8. European Medicines Agency (EMA). Benefit-risk methodology project. Work package 4 report: benefit-risk tools and processes; 2012. [cited 2017 Oct 12]. Available from: http://www.ema.europa.eu/docs/en_GB/document_library/Report/2012/03/WC500123819.pdf.
9. Medical Device Innovation Consortium. A Framework for Incorporating Information on Patient Preferences Regarding Benefit and Risk into Regulatory Assessments of New Medical Technology; 2015 [cited 2017]. Available from: http://mdic.org/wp-content/uploads/2015/05/MDIC_PCBR_Framework_Web1.pdf
10. Guo JJ, Pandey S, Doyle J, et al. A review of quantitative risk-benefit methodologies for assessing drug safety and efficacy-report of the ISPOR risk-benefit management working group. Value Health. 2010;13(5):657–666.
11. Hauber BA, Fairchild AO, Johnson FR. Quantifying benefit-risk preferences for medical interventions: an overview of a growing empirical literature. Appl Health Econ Health Policy. 2013;11(4):319–329.
12. Mühlbacher AC, Juhnke C, Beyer AR, et al. Patient-focused benefit-risk analysis to inform regulatory decisions: the European Union perspective. Value Health. 2016;19(6):734–740.
13. Australian Pharmaceutical Benefits Advisory Committee (PBAC). Appendix 6 - Including non health outcomes in a supplementary analysis; 2016 [cited 2017]. Available from: https://pbac.pbs.gov.au/appendixes/appendix-6-including-nonhealth-outcomes-in-a-supplementary-analysis.html
14. United States Food and Drug Administration (FDA). Patient Preference Information–Voluntary Submission, Review in Premarket Approval Applications, Humanitarian Device Exemption Applications, and De Novo Requests, and Inclusion in Decision Summaries and Device Labeling; 2016 [cited 2017 Aug23]. Available from: http://www.fda.gov/downloads/medicaldevices/deviceregulationandguidance/guidancedocuments/ucm446680.pdf
15. Ozdemir S. Improving the validity of stated-preference data in health research: the potential of the time-to-think approach. Patient. 2015 Jun;8(3):247–255. DOI:10.1007/s40271-014-0084-x.
16. Adamowicz W, Swait J, Boxall P, et al. Perceptions versus objective measures of environmental quality in combined revealed and stated preference models of environmental valuation. J Environ Econ Manage. 1997;32(1):65–84.
17. Adamowicz W, Louviere J, Williams M. Combining revealed and stated preference methods for valuing environmental amenities. J Environ Econ Manage. 1994;26(3):271–292.
18. Gurmankin AD, Baron J, Hershey JC, et al. The role of physicians’ recommendations in medical treatment decisions. Med Decis Making. 2002 May-Jun;22(3):262–271.
19. MacLeod TE, Harris AH, Mahal A. Stated and revealed preferences for funding new high-cost cancer drugs: a critical review of the evidence from patients, the public and payers. Patient. 2016 Jun;9(3):201–222. DOI:10.1007/s40271-015-0139-7.
20. Sacristan JA, Lizan L, Comellas M, et al. Perceptions of oncologists, healthcare policy makers, patients and the general population on the value of pharmaceutical treatments in oncology. Adv Ther. 2016 Nov;33(11):2059–2068.
21. Rakotonarivo OS, Schaafsma M, Hockley N. A systematic review of the reliability and validity of discrete choice experiments in valuing non-market environmental goods. J Environ Manage. 2016;183:98–109. DOI:10.1016/j.jenvman.2016.08.032.
22. Clark MD, Determann D, Petrou S, et al. Discrete choice experiments in health economics: a review of the literature. Pharmacoeconomics. 2014 Sep;32(9):883–902. DOI:10.1007/s40273-014-0170-x.
23. De Bekker-Grob EW, Ryan M, Gerard K. Discrete choice experiments in health economics: a review of the literature. Health Econ. 2012 Feb;21(2):145–172. DOI:10.1002/hec.1697.
24. Ryan M, Gerard K. Using discrete choice experiments to value health care programmes: current practice and future research reflections. Appl Health Econ Health Policy. 2003;2(1):55–64.
25. Ryan M, Scott DA, Reeves C, et al. Eliciting public preferences for healthcare: a systematic review of techniques. Health Technol Assess. 2001;5(5):1–186.
26. Lancsar E, Louviere J. Conducting discrete choice experiments to inform healthcare decision making: a user’s guide. Pharmacoeconomics. 2008;26(8):661–677.
27. Muhlbacher A, Johnson FR. Choice experiments to quantify preferences for health and healthcare: state of the practice. Appl Health Econ Health Policy. 2016 Jun;14(3):253–266. DOI:10.1007/s40258-016-0232-7.
28. Ryan M, Bate A, Eastmond CJ, et al. Use of discrete choice experiments to elicit preferences. Qual Health Care. 2001 Sep;10(Suppl 1):i55–i60.
29. Johnson FR, Mohamed AF, Ozdemir S, et al. How does cost matter in health-care discrete-choice experiments? Health Econ. 2011 Mar;20(3):323–330.
30. Lagarde M. Investigating attribute non-attendance and its consequences in choice experiments with latent class models. Health Econ. 2013 May;22(5):554–567. DOI:10.1002/hec.2824.
31. Lancsar E, Swait J. Reconceptualising the external validity of discrete choice experiments. Pharmacoeconomics. 2014 Oct;32(10):951–965. DOI:10.1007/s40273-014-0181-7.
32. Miguel FS, Ryan M, Amaya-Amaya M. ‘Irrational’ stated preferences: a quantitative and qualitative investigation. Health Econ. 2005 Mar;14(3):307–322. DOI:10.1002/hec.912.
33. Ozdemir S, Mohamed AF, Johnson FR, et al. Who pays attention in stated-choice surveys? Health Econ. 2010 Jan;19(1):111–118. DOI:10.1002/hec.1452.
34. Telser H, Zweifel P. Validity of discrete-choice experiments evidence for health risk reduction. Appl Econ. 2007;39(1):69–78.
35. Trochim W, Kane M. Concept mapping: an introduction to structured conceptualization in health care. Int J Qual Health Care. 2005 Jun;17(3):187–191. DOI:10.1093/intqhc/mzi038.
36. De Bekker-Grob EW, Hol L, Donkers B, et al. Labeled versus unlabeled discrete choice experiments in health economics: an application to colorectal cancer screening. Value Health. 2010 Mar-Apr;13(2):315–323. DOI:10.1111/j.1524-4733.2009.00670.x.
37. Muhlbacher AC, Rudolph I, Lincke HJ, et al. Preferences for treatment of attention-deficit/hyperactivity disorder (ADHD): a discrete choice experiment. BMC Health Serv Res. 2009;9:149. DOI:10.1186/1472-6963-9-149.
38. Kenny P, Hall J, Viney R, et al. Do participants understand a stated preference health survey? A qualitative approach to assessing validity. Int J Technol Assess Health Care. 2003 Fall;19(4):664–681.
39. Bijlenga D, Birnie E, Bonsel GJ. Feasibility, reliability, and validity of three health-state valuation methods using multiple-outcome vignettes on moderate-risk pregnancy at term. Value Health. 2009 Jul-Aug;12(5):821–827. DOI:10.1111/j.1524-4733.2009.00503.x.
40. Hollin IL, Caroline Y, Hanson C, et al. Developing a patient-centered benefit-risk survey: a community-engaged process. Value Health. 2016 Sep - Oct;19(6):751–757. DOI:10.1016/j.jval.2016.02.014.
41. Janssen EM, Segal JB, Bridges JF. A framework for instrument development of a choice experiment: an application to type 2 diabetes. Patient. 2016 Oct;9(5):465–479. DOI:10.1007/s40271-016-0170-3.
42. Mark TL, Swait J. Using stated preference and revealed preference modeling to evaluate prescribing decisions. Health Econ. 2004 Jun;13(6):563–573. DOI:10.1002/hec.845.
43. Janssen EM, Hauber AB, Bridges JFP. Conducting a discrete choice experiment study following recommendations for good research practices: an application for eliciting patient preferences for diabetes treatments. Value in Health. 2017. DOI:10.1016/j.jval.2017.07.001
44. Veldwijk J, Essers BA, Lambooij MS, et al. Survival or mortality: does risk attribute framing influence decision-making behavior in a discrete choice experiment? Value Health. 2016 Mar-Apr;19(2):202–209. DOI:10.1016/j.jval.2015.11.004.
45. Rockers PC, Jaskiewicz W, Wurts L, et al. Preferences for working in rural clinics among trainee health professionals in Uganda: a discrete choice experiment. BMC Health Serv Res. 2012 Jul;23(12):212.
46. Kruk ME, Paczkowski MM, Tegegn A, et al. Women’s preferences for obstetric care in rural Ethiopia: a population-based discrete choice experiment in a region with low rates of facility delivery. J Epidemiol Commun Health. 2010 Nov;64(11):984–988. DOI:10.1136/jech.2009.087973.
47. Ho MP, Gonzalez JM, Lerner HP, et al. Incorporating patient-preference evidence into regulatory decision making. Surg Endosc. 2015;29(10):2984–2993. DOI:10.1007/s00464-014-4044-2.
48. Bridges JF, Mohamed AF, Finnern HW, et al. Patients’ preferences for treatment outcomes for advanced non-small cell lung cancer: a conjoint analysis. Lung Cancer. 2012 Jul;77(1):224–231. DOI:10.1016/j.lungcan.2012.01.016.
49. Kimberlin CL, Winterstein AG. Validity and reliability of measurement instruments used in research. Am J Health Syst Pharm. 2008 Dec 01;65(23):2276–2284. DOI:10.2146/ajhp070364.
50. McFadden D. Conditional logit analysis of qualitative choice behavior. In: Zarembka P, editor. Frontiers in econometrics. New York (NY): Academic Press; 1974. p. 105–142.
51. Hougaard JL, Tjur T, Osterdal LP. On the meaningfulness of testing preference axioms in stated preference discrete choice experiments. Eur J Health Econ. 2012 Aug;13(4):409–417. DOI:10.1007/s10198-011-0312-4.
52. Bijlenga D, Bonsel GJ, Birnie E. Eliciting willingness to pay in obstetrics: comparing a direct and an indirect valuation method for complex health outcomes. Health Econ. 2011 Nov;20(11):1392–1406. DOI:10.1002/hec.1678.
53. Hollin IL, Peay HL, Bridges JF. Caregiver preferences for emerging duchenne muscular dystrophy treatments: a comparison of best-worst scaling and conjoint analysis. Patient. 2015;8(1):19–27.
54. Doiron D, Yoo HI. Temporal stability of stated preferences: the case of junior nursing jobs. Health Econ. 2016Apr;14:802–809. DOI:10.1002/hec.3350.
55. Orme B. Including holdout choice tasks in conjoint studies. Washington: Sawtooth Software Inc.; 2015.
56. Hauber AB, Gonzalez JM, Groothuis-Oudshoorn CG, et al. Statistical methods for the analysis of discrete choice experiments: a report of the ISPOR conjoint analysis good research practices task force. Value Health. 2016 Jun;19(4):300–315. DOI:10.1016/j.jval.2016.04.004.
57. Sen A. Internal consistency of choice. Econometrica. 1993;61:495–521.
58. Marshall D, Bridges JF, Hauber B, et al. Conjoint analysis applications in health - how are studies being designed and reported?: an update on current practice in the published literature between 2005 and 2008. Patient. 2010 Dec 1;3(4):249–256. DOI:10.2165/11539650-000000000-00000.
59. Joy SM, Little E, Maruthur NM, et al. Patient preferences for the treatment of type 2 diabetes: a scoping review. Pharmacoeconomics. 2013;31(10):877–892.
60. Bridges JF. Stated preference methods in health care evaluation: an emerging methodological paradigm in health economics. Appl Health Econ Health Policy. 2003;2(4):213–224.
61. Ryan M, Farrar S. Using conjoint analysis to elicit preferences for health care. Bmj. 2000 Jun 3;320(7248):1530–1533.
62. dosReis S, Castillo WC, Ross M, et al. Attribute development using continuous stakeholder engagement to prioritize treatment decisions: a framework for patient-centered research. Value Health. 2016 Sep–Oct;19(6):758–766. DOI:10.1016/j.jval.2016.02.013.
63. Coast J, Al-Janabi H, Sutton EJ, et al. Using qualitative methods for attribute development for discrete choice experiments: issues and recommendations. Health Econ. 2012;21(6):730–741.
64. De Bekker-Grob EW, Donkers B, Jonker MF, et al. Sample size requirements for discrete-choice experiments in healthcare: a practical guide. Patient. 2015 Oct;8(5):373–384.
65. Lancsar E, Louviere J, Flynn T. Several methods to investigate relative attribute impact in stated preference experiments. Soc Sci Med. 2007 Apr;64(8):1738–1753. DOI:10.1016/j.socscimed.2006.12.007.
66. Bridges JF, Hauber AB, Marshall D, et al. Conjoint analysis applications in health–a checklist: a report of the ISPOR good research practices for conjoint analysis task force. Value Health. 2011 Jun;14(4):403–413. DOI:10.1016/j.jval.2010.11.013.
67. Johnson FR, Lancsar E, Marshall D, et al. Constructing experimental designs for discrete-choice experiments: report of the ISPOR Conjoint Analysis Experimental Design Good Research Practices Task Force. Value Health. 2013;16(1):3–13.
68. Viney R, Lancsar E, Louviere J. Discrete choice experiments to measure consumer preferences for health and healthcare. Expert Rev Pharmacoecon Outcomes Res. 2002 Aug;2(4):319–326. DOI:10.1586/14737167.2.4.319.
69. Lancsar E, Louviere J. Deleting ‘irrational’ responses from discrete choice experiments: a case of investigating or imposing preferences? Health Econ. 2006 Aug;15(8):797–811. DOI:10.1002/hec.1104.
70. Thurstone LL. A law of comparative judgment. Psychol Rev. 1927;34(4): 273–286.
71. Kahneman D, Tversky A. Prospect theory: an analysis of decision under risk. Econometrica. 1979;47(2):263–291.
72. Loomes G, Sugden R. A rationale for preference reversal. Am Econ Rev. 1983;73(3):428–432.
73. Loomes G, Sugden R. Testing for regret and disappointment in choice under uncertainty. Econ J. 1987;97:118–129.
74. Durbin J, Watson GS. Testing for serial correlation in least squares regression: I. Biometrika. 1950;37(3–4):409–428.