Ordinal preference elicitation methods in health economics and health services research: Using discrete choice experiments and ranking methods

British Medical Bulletin

Volumn 103, Issue 1, 2012, Pages 21-44

  Ali, Shehzad ^a   Ronaldson, Sarah ^a  

^a UNIVERSITY OF YORK   (United Kingdom)

Author keywords

best worst scaling; discrete choice experiments; economic evaluation; ordinal preference methods; ranking exercises; stated preference

Indexed keywords

ATTITUDE; BEHAVIOR; COGNITION; COST UTILITY ANALYSIS; ECONOMIC EVALUATION; HEALTH ECONOMICS; HEALTH SERVICE; HUMAN; MEDICAL RESEARCH; METHODOLOGY; ORDINAL METHOD; POLICY; PRIORITY JOURNAL; REVIEW; TASK PERFORMANCE; THINKING;

CHOICE BEHAVIOR; COST-BENEFIT ANALYSIS; HEALTH SERVICES; HEALTH SERVICES RESEARCH; HUMANS; QUALITY-ADJUSTED LIFE YEARS; RESEARCH DESIGN;

EID: 84865746487     PISSN: 00071420     EISSN: 14718391     Source Type: Journal    
DOI: 10.1093/bmb/lds020     Document Type: Review

References (97)

1. Drummond MF, Sculpher MJ, Torrance GW. Methods for the Economic Evaluation of Health Care Programmes. USA: Oxford University Press, 2005.
2. Logan AG, Milne B, Achber C et al. Cost-effectiveness of a worksite hypertension treatment program. Hypertension 1981;3:211-8.
3. Hull R, Hirsh J, Sackett DL et al. Cost effectiveness of clinical diagnosis, venography, and noninvasive testing in patients with symptomatic deep-vein thrombosis. N Engl J Med 1981;304:1561-7.
4. Sculpher MJ, Buxton MJ. The episode-free day as a composite measure of effectiveness: an illustrative economic evaluation of formoterol versus salbutamol in asthma therapy. Pharmacoeconomics 1993;4:345.
5. Mark DB, Hlatky MA, Califf RM et al. Cost effectiveness of thrombolytic therapy with tissue plasminogen activator as compared with streptokinase for acute myocardial infarction. N Engl J Med 1995;332:1418-24.
6. Whitehead SJ, Ali S. Health outcomes in economic evaluation: the QALY and utilities. Br Med Bull 2010;96:5-21.
7. Flynn T, Louviere J, Peters T et al. Estimating preferences for a dermatology consultation using best-worst scaling: comparison of various methods of analysis. BMC Med Res Methodol 2008;8:76.
8. Kiiskinen U, Suominen-Taipale AL, Cairns J. Think twice before you book? Modelling the choice of public vs private dentist in a choice experiment. Health Econ 2010;19:670-82.
9. Ryan M. Using conjoint analysis to take account of patient preferences and go beyond health outcomes: an application to in vitro fertilisation. Soc Sci Med 1999;48:535-46.
10. Gerard K, Lattimer V, Turnbull J et al. Reviewing emergency care systems 2: measuring patient preferences using a discrete choice experiment. Emerg Med J 2004;21:692.
11. Scott A, Watson MS, Ross S. Eliciting preferences of the community for out of hours care provided by general practitioners: a stated preference discrete choice experiment. Soc Sci Med 2003;56:803-14.
12. Lancsar E, Louviere JJ. Estimating individual level discrete choice models and welfare measures using best-worst choice experiments and sequential best-worst MNL. University of Technology, Centre for the Study of Choice (Censoc) 2008,1-24.
13. Hjelmgren J, Anell A. Population preferences and choice of primary care models: a discrete choice experiment in Sweden. Health Policy 2007;83:314-22.
14. Rubin G, Bate A, George A et al. Preferences for access to the GP: a discrete choice experiment. Br J Gen Pract 2006;56:743.
15. Hanson K, McPake B, Nakamba P et al. Preferences for hospital quality in Zambia: results from a discrete choice experiment. Health Econ 2005;14:687-701.
16. Doyle S, Lloyd A, Birt J et al. Willingness to pay for obesity pharmacotherapy. Obesity 2012; doi:10.1038/oby.2011.387.
17. King MT, Hall J, Lancsar E et al. Patient preferences for managing asthma: results from a discrete choice experiment. Health Econ 2007;16:703-17.
18. Bech M, Kjaer T, Lauridsen J. Does the number of choice sets matter? Results from a web survey applying a discrete choice experiment. Health Econ 2011;20:273-86.
19. Meenakshi J, Banerji A, Manyong V et al. Using a discrete choice experiment to elicit the demand for a nutritious food: willingness-to-pay for orange maize in rural Zambia. J Health Econ 2012;31:62-71.
20. Benjamin L, Cotté FE, Philippe C et al. Physicians' preferences for prescribing oral and intravenous anticancer drugs: a discrete choice experiment. Eur J Cancer 2012;48:912-20.
21. Kolstad JR. How to make rural jobs more attractive to health workers. Findings from a discrete choice experiment in Tanzania. Health Econ 2011;20:196-211.
22. Potoglou D, Burge P, Flynn T et al. Best-worst scaling vs. discrete choice experiments: an empirical comparison using social care data. Soc Sci Med 2011;72:1717-27.
23. Ryan M, Netten A, Ska°tun D et al. Using discrete choice experiments to estimate a preference-based measure of outcome-an application to social care for older people. J Health Econ 2006;25:927-44.
24. Nieboer AP, Koolman X, Stolk EA. Preferences for long-term care services: willingness to pay estimates derived from a discrete choice experiment. Soc Sci Med 2010;70:1317-25.
25. Ungar WJ. Challenges in health state valuation in paediatric economic evaluation: are QALYs contraindicated? Pharmacoeconomics 2011;29:641-52.
26. Becker K, Zweifel P. Age and choice in health insurance: evidence from a discrete choice experiment. Patient: Patient-Centered Outcomes Res 2008;1:27-40.
27. Brau R, Lippi Bruni M. Eliciting the demand for long-term care coverage: a discrete choice modelling analysis. Health Econ 2008;17:411-33.
28. Lancsar EJ, Hall JP, King M et al. Using discrete choice experiments to investigate subject preferences for preventive asthma medication. Respirology 2007;12:127-36.
29. McIntosh E, Clarke PM, Frew EJ. Applied Methods of Cost-Benefit Analysis in Health Care. Oxford, United Kingdom: Oxford University Press, 2010.
30. Hauber A, Mohamed A, Johnson F et al. Treatment preferences and medication adherence of people with Type 2 diabetes using oral glucose-lowering agents. Diab Med 2009;26:416-24.
31. Ubach C, Scott A, French F et al. What do hospital consultants value about their jobs? A discrete choice experiment. BMJ 2003;326:1432.
32. Mangham LJ, Hanson K, McPake B. How to do (or not to do). . .Designing a discrete choice experiment for application in a low-income country. Health Policy Plan 2009;24:151.
33. Kruk ME, Johnson JC, Gyakobo M et al. Rural practice preferences among medical students in Ghana: a discrete choice experiment. Bull World Health Organ 2010;88:333-41.
34. Green C, Gerard K. Exploring the social value of health-care interventions: a stated preference discrete choice experiment. Health Econ 2009;18:951-76.
35. Ratcliffe J, Bekker HL, Dolan P et al. Examining the attitudes and preferences of health care decision-makers in relation to access, equity and cost-effectiveness: a discrete choice experiment. Health Policy 2009;90:45-57.
36. Mark TL, Swait J. Using stated preference and revealed preference modeling to evaluate prescribing decisions. Health Econ 2004;13:563-73.
37. Viney R, Lancsar E, Louviere J. Discrete choice experiments to measure consumer preferences for health and healthcare. Exp Rev Pharmacoecon Outcomes Res 2002;2:319-26.
38. Donaldson C, Gerard K, Mitton C et al. Economics of Health Care Financing: The Visible Hand. London: Macmillan, 1993.
39. Lancsar E, Louviere J. Conducting discrete choice experiments to inform healthcare decision making: a user's guide. Pharmacoeconomics 2008;26:661-77.
40. Adamowicz W, Louviere J, Williams M. Combining revealed and stated preference methods for valuing environmental amenities. J Environ Econ Manage 1994;26:271-92.
41. Whitehead JC, Pattanayak SK, Van Houtven GL et al. Combining revealed and stated preference data to estimate the nonmarket value of ecological services: an assessment of the state of the science. J Econ Surv 2008;22:872-908.
42. Ratcliffe J, Brazier J, Tsuchiya A et al. Using DCE and ranking data to estimate cardinal values for health states for deriving a preference-based single index from the sexual quality of life questionnaire. Health Econ 2009;18:1261-76.
43. Lloyd AJ. Threats to the estimation of benefit: are preference elicitation methods accurate? Health Econ 2003;12:393-402.
44. Torrance GW, Feeny D, Furlong W. Visual analog scales. Do they have a role in the measurement of preferences for health states? Med Decis Making 2001;21:329-34.
45. Brazier J, Ratcliffe J. Measuring and Valuing Health Benefits for Economic Evaluation. USA: Oxford University Press, 2007.
46. de Bekker-Grob EW, Ryan M, Gerard K. Discrete choice experiments in health economics: a review of the literature. Health Econ 2012;21:145-72.
47. Ratcliffe J. The use of conjoint analysis to elicit willingness-to-pay values. Int J Technol Assess Health Care 2000;16:270-90.
48. Louviere J, Lancsar E. Choice experiments in health: the good, the bad, the ugly and toward a brighter future. Health Econ Pol Law 2009;4:527-46.
49. Craig BM, Busschbach JJV, Salomon JA. Keep it simple: ranking health states yields values similar to cardinal measurement approaches. J Clin Epidemiol 2009;62:296-305.
50. Hershey JC, Kunreuther HC, Schoemaker PJH. Sources of bias in assessment procedures for utility functions. Manage Sci 1982;936-54.
51. Schoemaker PJH. The expected utility model: its variants, purposes, evidence and limitations. J Econ Lit 1982;529-63.
52. Brazier J, Rowen D, Yang Y et al. Using rank and discrete choice data to estimate health state utility values on the QALY scale. 2009.
53. Bleichrodt H. A new explanation for the difference between time trade-off utilities and standard gamble utilities. Health Econ 2002;11:447-56.
54. Flynn TN, Louviere JJ, Peters TJ et al. Best-worst scaling: what it can do for health care research and how to do it. J Health Econ 2007;26:171-89.
55. Salomon JA. Reconsidering the use of rankings in the valuation of health states: a model for estimating cardinal values from ordinal data. Popul Health Metr 2003;1:12.
56. Moshkovich HM, Mechitov AI, Olson DL. Ordinal judgments in multiattribute decision analysis. Eur J Oper Res 2002;137:625-41.
57. Coast J, Flynn T, Sutton E et al. Investigating Choice Experiments for Preferences of Older People (ICEPOP): evaluative spaces in health economics. J Health Serv Res Policy 2008;13(suppl. 3):31-37.
58. Bridges JFP, Hauber AB, Marshall D et al. Conjoint analysis applications in health-a checklist: a report of the ISPOR Good Research Practices for Conjoint Analysis Task Force. Value Health 2011.
59. Thurstone LL. A law of comparative judgment. Psychol Rev 1927;34:273.
60. Fanshel S, Bush JW. A health-status index and its application to health-services outcomes. Oper Res 1970;1021-66.
61. Bradley RA, Terry ME. Rank analysis of incomplete block designs: I. The method of paired comparisons. Biometrika 1952;39:324-45.
62. Kind P. A comparison of two models for scaling health indicators. Int J Epidemiol 1982;11:271-75.
63. Louviere JJ, Woodworth G. Design and analysis of simulated consumer choice or allocation experiments: an approach based on aggregate data. J Market Res 1983;350-67.
64. McFadden D. Conditional Logit Analysis of Qualitative Choice Behavior. Berkeley, California, USA: University of California BIoU, Regional D, 1973.
65. Propper C. Contingent valuation of time spent on NHS waiting lists. Econ J 1990;100:193-9.
66. Ryan M, Hughes J. Using conjoint analysis to assess women's preferences for miscarriage management. Health Econ 1997;6:261-73.
67. Kind P. Applying paired comparisons models to EQ-5D valuations-deriving TTO utilities from ordinal preference data. EQ-5D Concepts and Methods: A Developmental History. The Netherlands: Springer, 2005,201-20.
68. McCabe C, Brazier J, Gilks P et al. Using rank data to estimate health state utility models. J Health Econ 2006;25:418-31.
69. Carson RT, Louviere JJ. A common nomenclature for stated preference elicitation approaches. Environ Res Econ 2011;49:539-59.
70. Ryan M, Farrar S. Using conjoint analysis to elicit preferences for health care. BMJ 2000;320:1530.
71. Ryan M, Gerard K. Using discrete choice experiments to value health care programmes: current practice and future research reflections. Appl Health Econ Health Policy 2003;2:55-64.
72. Scott A. Eliciting GPs' preferences for pecuniary and non-pecuniary job characteristics. J Health Econ 2001;20:329-47.
73. Bansback N, Brazier J, Tsuchiya A et al. Using a discrete choice experiment to estimate societal health state utility values. HEDS Discussion Paper 10/03. 2010,1-34.
74. Ryan M, Ska°tun D, Major K. Using discrete choice experiments to go beyond clinical outcomes when evaluating clinical practice. Using Discrete Choice Experiments to Value Health and Health Care 2008,101-16.
75. Hole AR. A comparison of approaches to estimating confidence intervals for willingness to pay measures. Health Econ 2007;16:827-40.
76. Lancsar E, Wildman J, Donaldson C et al. Deriving distributional weights for QALYs through discrete choice experiments. J Health Econ 2011;30:466-78.
77. Buxton MJ, Chambers JD. What values do the public want their health care systems to use in evaluating technologies? Eur J Health Econ 2011;12:285-8.
78. Amaya-Amaya M, Gerard K, Ryan M. Discrete choice experiments in a nutshell. Using Discrete Choice Experiments to Value Health and Health Care. Dordrecht: Springer, 2008,13-46.
79. Coast J, Horrocks S. Developing attributes and levels for discrete choice experiments using qualitative methods. J Health Serv Res Policy 2007;12:25-30.
80. Coast J, Al-Janabi H, Sutton E et al. Using qualitative methods for attribute development for discrete choice experiments: issues and recommendations. Health Econ 2012;21:730-41.
81. Louviere JJ, Carson R, Pihlens D. Design of discrete choice experiments: a discussion of issues that matter in future applied research. J Choice Model 2011;4:1-8.
82. Lancaster KJ. A new approach to consumer theory. J Polit Econ 1966;74:132-57.
83. Hole AR. Modelling heterogeneity in patients' preferences for the attributes of a general practitioner appointment. J Health Econ 2008;27:1078-94.
84. Ryan M, Watson V, Entwistle V. Rationalising the 'irrational': a think aloud study of discrete choice experiment responses. Health Econ 2009;18:321-36.
85. Lanscar E, Savage E. Deriving welfare measures from discrete choice experiments: inconsistency between current methods and random utility and welfare theory. Health Econ 2004;13:901-37.
86. Santos Silva JMC. Deriving welfare measures in discrete choice experiments: a comment to Lancsar and Savage (2). Health Econ 2004;13:913-8.
87. Lagarde M, Blaauw D. A review of the application and contribution of discrete choice experiments to inform human resources policy interventions. Hum Resour Health 2009;7:62-72.
88. Fok D, Paap R, Van Dijk B. A rank ordered logit model with unobserved heterogeneity in ranking capabilities. J Appl Econom 2012;27:831-46.
89. Slothuus U, Larsen ML, Junker P. The contingent ranking method-a feasible and valid method when eliciting preferences for health care? Soc Sci Med 2002;54:1601-9.
90. Chapman RG, Staelin R. Exploiting rank ordered choice set data within the stochastic utility model. J Market Res 1982;19:288-301.
91. Chiang J, Chib S, Narasimhan C. Markov chain Monte Carlo and models of consideration set and parameter heterogeneity. J Econom 1999;89:223-48.
92. Louviere J, Woodworth G. Best-worst scaling: a model for largest difference judgments. Working paper, Faculty of Business, University of Alberta, 1990.
93. Finn A, Louviere JJ. Determining the appropriate response to evidence of public concern: the case of food safety. J Public Policy Market 1992;11:12-25.
94. Marley AAJ, Flynn TN, Louviere JJ. Probabilistic models of set-dependent and attribute-level best-worst choice. J Math Psychol 2008;52:281-96.
95. Flynn TN. Valuing citizen and patient preferences in health: recent developments in three types of bestworst scaling. Exp Rev Pharmacoecon Outcomes Res 2010;10:259-67.
96. Fischer GW, Hawkins SA. Strategy compatibility, scale compatibility, and the prominence effect. J Exper Psychol Hum Percep Perform 1993;19:580.
97. Lancsar E, Louviere J. Deleting 'irrational' responses from discrete choice experiments: a case of investigating or imposing preferences?. Health economics 2006;15:797-811.