Using conjoint analysis to develop a system of scoring policymakers' use of research in policy and program development

Health Research Policy and Systems

Volumn 13, Issue 1, 2015, Pages

  Makkar, Steve R ^a   Williamson, Anna ^a   Turner, Tari ^b   Redman, Sally ^a   Louviere, Jordan ^c  

^a SAX INSTITUTE   (Australia)

^b WORLD VISION AUSTRALIA   (Australia)

^c UNIVERSITY OF SOUTH AUSTRALIA   (Australia)

Author keywords

Conjoint analysis; Evidence based policy; Evidence informed policy; Health policy; Knowledge translation; Measurement; Policymaker; Research; Use; Utilisation

Indexed keywords

ARTICLE; CONJOINT ANALYSIS; CONTROLLED STUDY; CORRELATION COEFFICIENT; DECISION MAKING; DOCUMENTATION; FEMALE; HEALTH CARE MANAGEMENT; HEALTH CARE ORGANIZATION; HEALTH CARE POLICY; HEALTH CARE UTILIZATION; HEALTH SURVEY; HUMAN; MALE; MEDICAL RESEARCH; PRACTICE GUIDELINE; PROGRAM DEVELOPMENT; SCORING SYSTEM; STATISTICAL ANALYSIS; ADMINISTRATIVE PERSONNEL; CHECKLIST; EVIDENCE BASED MEDICINE; HEALTH SERVICES RESEARCH; MANAGEMENT; TRANSLATIONAL RESEARCH;

ADMINISTRATIVE PERSONNEL; CHECKLIST; EVIDENCE-BASED MEDICINE; HEALTH POLICY; HEALTH SERVICES RESEARCH; HUMANS; POLICY MAKING; PROGRAM DEVELOPMENT; TRANSLATIONAL MEDICAL RESEARCH;

EID: 84938600809     PISSN: None     EISSN: 14784505     Source Type: Journal    
DOI: 10.1186/s12961-015-0022-y     Document Type: Article

References (88)

1. Hausman A. Implications of evidence-based practice for community health. Am J Community Psychol. 2002;30(3):453-67.
2. Black N. Evidence based policy: proceed with care. Brit Med J. 2001;323(7307):275-9.
3. Lemay MA, Sa C. The use of academic research in public health policy and practice. Res Evaluat. 2014;23:79-88.
4. Jacobson N, Butterill D, Goering P. Consulting as a strategy for knowledge transfer. Milbank Q. 2005;83(2):299-321.
5. Lavis JN, Ross SE, Hurley JE, Hohenadel JM, Stoddart GL, Woodward CA, et al. Examining the role of health services research in public policymaking. Milbank Q. 2002;80(1):125-54.
6. LaRocca R, Yost J, Dobbins M, Ciliska D, Butt M. The effectiveness of knowledge translation strategies used in public health: a systematic review. BMC Public Health. 2012;12:751.
7. El-Jardali F, Lavis JN, Moat KA, Pantoja T, Ataya N. Capturing lessons learned from evidence-to-policy initiatives through structured reflection. Health Res Policy Syst. 2014;12:2.
8. Brownson RC, Chriqui JF, Stamatakis KA. Understanding evidence-based public health policy. Am J Public Health. 2009;99(9):1576-83.
9. Hanney SR, Gonzalez-Block MA, Buxton MJ, Kogan M. The utilisation of health research in policy-making: concepts, examples and methods of assessment. Health Res Policy Syst. 2003;1:2.
10. Buchan H. Gaps between best evidence and practice: causes for concern. Med J Australia. 2004;180:S48-9.
11. Bowen S, Erickson T, Martens PJ, Crockett S. More than "using research": the real challenges in promoting evidence-informed decision-making. Health Policy. 2009;4(3):87-102.
12. Fielding JE, Briss PA. Promoting evidence-based public health policy: can we have better evidence and more action? Health Aff (Millwood). 2006;4:969-78.
13. Andre FE, Booy R, Bock HL, Clemens J, Datta SK, John TJ, et al. Vaccination greatly reduces disease, disability, death and inequity worldwide. Bull World Health Organ. 2008;86(2):140-6.
14. Hanna JN, Hills SL, Humphreys JL. Impact of hepatitis A vaccination of Indigenous children on notifications of hepatitis A in north Queensland. Med J Aust. 2004;181(9):482-5.
15. Morrato EH, Elias M, Gericke CA. Using population-based routine data for evidence-based health policy decisions: lessons from three examples of setting and evaluating national health policy in Australia, the UK and the USA. J Public Health. 2007;19(4):463-71.
16. Milat A, Laws R, King L, Newson R, Rychetnik L, Rissel C, et al. Policy and practice impacts of applied research: a case study analysis of the New South Wales Health promotion Demonstration Research Grants Scheme 2000-2006. Health Res Policy Syst. 2013;11:5.
17. Chagnon F, Poullot L, Malo C, Gervais MJ, Pigeon ME. Comparison of determinants of research knowledge utilization by practitioners and administrators in the field of child and family social services. Implement Sci. 2010;5:41.
18. Lavis JN, Lomas J, Hamid M, Sewenkambo NK. Assessing country-level efforts to link research to action. Bull World Health Organ. 2006;84(8):620-8.
19. Amara N, Ouimet M, Landry R. New evidence on instrumental, conceptual, and symbolic utilization of university research in government agencies. Sci Commun. 2004;26(1):75-106.
20. Campbell DM, Redman S, Jorm L, Cooke M, Zwi AB, Rychetnik L. Increasing the use of evidence in health policy: practice and views of policy makers and researchers. Aust New Zealand Health Policy. 2009;6:21.
21. Dobbins M, Cockerill R, Barnsley J. Factors affecting the utilization of systematic reviews: a study of public health decision makers. Int J Technol Assess Health Care. 2001;17:203-14.
22. El-Jardali F, Lavis JN, Ataya N, Jamal D. Use of health systems and policy research evidence in the health policymaking in eastern Mediterranean countries: views and practices of researchers. Implement Sci. 2012;7:2.
23. Ritter A. How do drug policy makers access research evidence? Int J Drug Policy. 2009;20:70-5.
24. Elshaug AG, Hiller JE, Tunis SR, Moss JR. Challenged in Australian policy processes for disinvestment from existing, ineffective health care practices. Aust New Zealand Health Policy. 2007;4:23.
25. Jackson Jr WL, Sales JF. Potentially ineffective care: time for earnest reexamination. Crit Care Res Pract. 2014;2014:1-6.
26. Erixon F, van der Marel E. What is driving the rise in health care expenditures? An inquiry into the nature and causes of the cost disease. In: ECIPE working papers, vol. 5. Brussels, Belgium: European Centre for International Political Economy; 2011.
27. Hansen J. Health services research in Europe: evaluating and improving its contribution to health care policy. J Health Serv Res Policy. 2011;16 Suppl 2:1.
28. Redman S, Turner T, Davies H, Haynes A, Williamson A, Milat A, et al. The SPIRIT Action Framework: a structured approach to selecting and testing strategies to increase the use of research in policy. Soc Sci Med. 2015;136-137:147-55.
29. Brownson RC, Fielding JE, Maylan CM. Evidence-based public health: a fundamental concept for public health practice. Annu Rev Public Health. 2009;30:175-201.
30. Dobbins M, Ciliska D, Cockerill R, Barnsley J, DiCenso A. A framework for the dissemination and utilization of research for health-care policy and practice. Online J Knowl Synth Nurs. 2002;9:149-60.
31. de Goede J, van Bon-Martens MJ, Putters K, van Oers HA. Looking for interaction: quantitative measurement of research utilization by Dutch local health officials. Health Res Policy Syst. 2012;10:9.
32. Landry R, Amara N, Lamari M. Utilization of social science research knowledge in Canada. Res Policy. 2001;30:333-49.
33. Landry R, Amara N, Lamari M. Climbing the ladder of research utilization. Sci Commun. 2001;22(4):396-422.
34. Landry R, Lamari M, Amara N. The extent and determinants of the utilization of university research in government agencies. Public Adm Rev. 2003;63(2):192-205.
35. Squires JE, Estabrooks CA, Newburn-Cook CV, Gierl M. Validation of the conceptual research utilization scale: an application of the standards for educational and psychological testing in healthcare. BMC Health Serv Res. 2011;11:107.
36. Squires JE, Estabrooks CA, O'Rourke HM, Gustavsson P, Newburn-Cook CV, Wallin L. A systematic review of the psychometric properties of self-report research utilization measures used in healthcare. Implement Sci. 2011;6:83.
37. Zardo P, Collie A. Measuring use of research evidence in public health policy: a policy content analysis. Implement Sci. 2014;14:496.
38. Walker I, Hulme C. Concrete words are easier to recall than abstract words: evidence for a semantic contribution to short-term serial recall. J Exper Psychol. 1999;25(5):1256-71.
39. Wattenmaker WD, Shoben EJ. Context and the recallability of concrete and abstract sentences. J Exper Psychol. 1987;13(1):140-50.
40. Weiss C. Knowledge creep and decision accretion. Sci Commun. 1980;1(3):381-404.
41. Beyer JM, Trice HM. The utilization process: a conceptual framework and synthesis of empirical findings. Adm Sci Q. 1982;27(4):591-622.
42. Haynes A, Turner T, Redman S, Milat AJ, Moore G. Developing definitions for a knowledge exchange intervention in health policy and program agencies: reflections on process and value. Int J Soc Res Meth. 2014;18(2):145-59.
43. Weiss C, Bucuvalas MJ. Social science research and decision-making. New York: Columbia University Press; 1980.
44. Beyer JM. Research utilisation: bridging a gap between communities. J Manage Inquiry. 1997;20:385.
45. Sumner A, Crichton J, Theobald S, Zulu E, Parkhurst J. What shapes research impact on policy? Understanding research uptake in sexual and reproductive health policy processes in resource poor contexts. Health Res Policy Syst. 2011;9 Suppl 1:53.
46. Weiss CH. The many meanings of research utilization. Public Adm Rev. 1979;39(5):426-31.
47. Liverani M, Hawkins B, Parkhurst JO. Political and institutional influences on the use of evidence in public health policy. PLoS ONE. 2013;8(10):e77404.
48. Makkar SR, Williamson A, Turner T, Redman S, Louviere J. Using conjoint analysis to develop a system to score research engagement actions by health decision makers. Health Res Policy Syst. 2015;13:22.
49. Ginsburg LR, Lewis S, Zackheim L, Casebeer A. Revisiting interaction in knowledge interaction. Implement Sci. 2007;2:34.
50. Hair JF, Black WC, Babin BJ, Anderson RE, Tatham RL. Multivariate data analysis. 6th ed. Pearson: New Delhi; 2006.
51. Ryan M, McIntosh E, Shackley P. Methodological issues in the application of conjoint analysis in health care. Health Econ. 1998;7:373-8.
52. Ryan M. Using conjoint analysis to take account of patient preferences and go beyond health outcomes: an application to in vitro fertilisation. Soc Sci Med. 1999;48:535-46.
53. Ryan M. A role for conjoint analysis in technology assessment in health care? Int J Technol Assess Health Care. 1999;15(3):443-57.
54. Bridges JF, Hauber AB, Marshall D, Lloyd A, Prosser LA, Regier DA, et al. Conjoint analysis applications in health - a checklist: a report of the ISPOR Good Research Practices for Conjoint Analysis Task Force. Value Health. 2011;14(4):403-13.
55. Carson RT, Louviere J. A common nomenclature for stated preference elicitation approaches. Environ Resource Econ. 2011;49:539-59.
56. Ryan M, Farrar S. Using conjoint analysis to elicit preferences for health care. Brit Med J. 2000;320:1530-3.
57. Ryan M, Hughes J. Using conjoint analysis to assess preferences for miscarriage management. Health Econ. 1997;6:216-73.
58. San Miguel F, Ryan M, McIntosh E. Applying conjoint analysis in economic evaluations: an application to menorrhagia. Appl Econ. 2010;32:823-33.
59. Farrar S, Ryan M. Response-ordering effects: a methodological issue of conjoint analysis. Health Econ. 1999;8:75-9.
60. Lancsar E, Louviere J. Conducting discrete choice experiments to inform healthcare decision making: a user's guide. Pharmacoeconomics. 2008;26(8):661-77.
61. Oppewal H, Louviere JJ, Timmermans HJP. Modeling hierarchical conjoint processes with integrated choice experiments. J Market Res. 1994;31(1):91-105.
62. Louviere J. Hierarchical information integration: a new method for the design and analysis of complex multiattribute judgment problems. Adv Consum Res. 1984;11:148-55.
63. Haynes A, Turner T, Redman S, Milat A, Moore G. Developing definitions for a knowledge exchange intervention in health policy and program agencies: reflections on process and value. Int J Soc Res Meth. 2015;18(2):145-59.
64. Bak A, Bartlomowicz T. Conjoint analysis method and its implementation in conjoint R package. Wroclaw: University of Economics; 2009.
65. Survey Monkey Inc. http://www.surveymonkey.com .
66. Norusis MJ. SPSS 15.0 Advanced statistical procedures companion. Prentice Hall: Upper Saddle River, NJ; 2007.
67. Ghisletta P, Spini D. An introduction to generalized estimating equations and an application to assess selectivity effects in a longitudinal study on very old individuals. J Educ Behav Stat. 2004;29(4):421-37.
68. Lipsitz SR, Fitzmaurice GM, Orav EJ, Laird NM. Performance of generalized estimating equations in practical situations. Biometrics. 1994;50(1):270-8.
69. Orme D. Getting started with conjoint analysis: strategies for product design and pricing research. 2nd ed. Madison, WI: Research Publishers LLC; 2010.
70. Elliott H, Popay J. How are policy makers using evidence? Models of research utilisation and local NHS policy making. J Epidemiol Commun Health. 2000;54:461-8.
71. Lavis JN, Davies H, Oxman A, Denis JL, Golden-Biddle K, Ferlie E. Towards systematic reviews that inform health care management and policy-making. J Health Serv Res Policy. 2005;10 Suppl 1:35-48.
72. Weiss CH, Bucuvalas MJ. Truth tests and utility tests: decision-makers' frames of reference for social science research. Am Sociol Rev. 1980;45(2):302-13.
73. Anderson M, Cosby J, Swan B, Moore H, Broekhoven M. The use of research in local health service agencies. Soc Sci Med. 1999;49(8):1007-19.
74. Orton L, Lloyd-Williams F, Taylor-Robinson D, O'Flaherty M, Capewell S. The use of research evidence in public health decision making processes: systematic review. PLoS ONE. 2011;6(7):e21704.
75. Hennink M, Stephenson R. Using research to inform health policy: barriers and strategies in developing countries. J Health Commun. 2006;10:163-80.
76. Buse K, Mays N, Walt G. Making health policy. Glasgow: Open University Press; 2005.
77. Ellen ME, Lavis JN, Ouimet M, Grimshaw J, Bedard PO. Determining research knowledge infrastructure for healthcare systems: a qualitative study. Implement Sci. 2011;6:60.
78. Ettelt S, Mays N. Health services research in Europe and its use for informing policy. J Health Serv Res Policy. 2011;16 Suppl 2:48-60.
79. Hyder AA, Corluka A, Winch PJ, El-Shinnawy A, Ghassany H, Malekafzali H, et al. National policymakers speak out: are researchers giving them what they need? Health Policy Plann. 2011;26:73-82.
80. Kothari A, Edwards N, Hamel N, Judd M. Is research working for you? Validating a tool to examine the capacity of health organizations to use research. Implement Sci. 2009;4:46. doi: 10.1186/1748-5908-4-46 .
81. Green J, Thorogood N. Qualitative methods for health research. London: SAGE; 2005.
82. Brook RH, Chassin MR, Fink A, Solomon DH, Kosecoff J, Park RE. A method for the detailed assessment of the appropriateness of medical technologies. Int J Technol Assess Health Care. 1986;2(1):53-3.
83. Hsu CC, Sanford BA. The Delphi technique: making sense of consensus. Pract Assess Res Evaluation. 2007;12:10.
84. Shekelle P. The appropriateness method. Med Decis Making. 2004;24:228.
85. Wortman PM, Smyth JM, Langenbrunner JC, Yeaton WH. Consensus among experts and research synthesis: a comparison of methods. Int J Technol Assess Health Care. 1998;14(1):109-22.
86. Investigators CIPHER. Supporting policy in health with research: an intervention trial (SPIRIT)-protocol for a stepped wedge trial. BMJ Open. 2014;4(7):e005293.
87. Oliver K, Lorenc T, Innvaer S. New directions in evidence-based policy research: a critical analysis of the literature. Health Res Policy Syst. 2014;12:34.
88. Marshall D, Bridges JF, Hauber AB, Cameron R, Donnalley L, Fyie K, et al. Conjoint analysis applications in health - how are studies being designed and reported? Patient. 2010;3(4):249-56.