Acceptability of an online modified Delphi panel approach for developing health services performance measures: results from 3 panels on arthritis research

Journal of Evaluation in Clinical Practice

Volumn 23, Issue 2, 2017, Pages 354-360

  Khodyakov, Dmitry ^a   Grant, Sean ^a   Barber, Claire E H ^b   Marshall, Deborah A ^b   Esdaile, John M ^c   Lacaille, Diane ^c  

^a RAND CORPORATION   (United States)

^b UNIVERSITY OF CALGARY   (Canada)

^c ARTHRITIS RESEARCH CENTRE OF CANADA   (Canada)

Author keywords

Delphi; expert panel; health care; health services research; online modified Delphi; performance measures

Indexed keywords

ARTHRITIS; ARTICLE; DELPHI STUDY; HEALTH CARE ORGANIZATION; HEALTH CARE PLANNING; HEALTH CARE QUALITY; HEALTH CARE SURVEY; HEALTH SERVICE; HUMAN; ONLINE ANALYSIS; PERFORMANCE MEASUREMENT SYSTEM; PROGRAM ACCEPTABILITY; PROGRAM DEVELOPMENT; ALBERTA; HEALTH SERVICES RESEARCH; INTERNET; ORGANIZATION AND MANAGEMENT; STANDARDS; TOTAL QUALITY MANAGEMENT;

ALBERTA; ARTHRITIS; DELPHI TECHNIQUE; HEALTH SERVICES RESEARCH; HUMANS; INTERNET; QUALITY IMPROVEMENT; QUALITY INDICATORS, HEALTH CARE;

EID: 84994010353     PISSN: 13561294     EISSN: 13652753     Source Type: Journal    
DOI: 10.1111/jep.12623     Document Type: Article

References (34)

1. McGlynn EA, Asch SM. Developing a clinical performance measure. Am J Prev Med. 1998;14(3 Suppl):14–21.
2. Kotter T, Blozik E, Scherer M. Methods for the guideline-based development of quality indicators: a systematic review. Implement Sci. 2012;7:21.
3. Kötter T, Schaefer FA, Scherer M, Blozik E. Involving patients in quality indicator development: a systematic review. Patient Prefer Adher. 2013;7:259.
4. Boulkedid R, Abdoul H, Loustau M, Sibony O, Alberti C. Using and reporting the Delphi method for selecting healthcare quality indicators: a systematic review. PLoS One. 2011;6(6):e20476.
5. Normand S-LT, McNeil BJ, Peterson LE, Palmer RH. Eliciting expert opinion using the Delphi technique: identifying performance indicators for cardiovascular disease. Int J Qual Health Care. 1998;10(3):247–260.
6. Wollersheim H, Hermens R, Hulscher M, et al. Clinical indicators: development and applications. Neth J Med. 2007;65(1):15–22.
7. Scott DA, Reeves C, Bate A, et al. Eliciting public preferences for healthcare: a systematic review of techniques. Health Technol Assess. 2001;5(5):1–186.
8. Sinha IP, Smyth RL, Williamson PR. Using the Delphi technique to determine which outcomes to measure in clinical trials: recommendations for the future based on a systematic review of existing studies. PLoS Med. 2011;8(1):e1000393.
9. Okoli C, Pawlowski S. The Delphi method as a research tool: an example, design considerations and applications. Inf Manag. 2004;42(1):15–29.
10. Snyder-Halpern R, Thompson CP, Schaffer J. Comparison of mailed vs. internet applications of the Delphi technique in clinical informatics research. Proceedings of the American Medical Informatics Association. 2000; 809–813
11. Donohoe H, Stellefson M, Tennant B. Advantages and limitations of the e-Delphi technique: implications for health education researchers. Am J Health Educ. 2012;43(1):38–46.
12. Elwyn G, O'Connor A, Stacey D, et al. Developing a quality criteria framework for patient decision aids: online international Delphi consensus process. BMJ. 2006;333(7565):417–423.
13. Khodyakov D, Hempel S, Rubenstein L, et al. Conducting online expert panels: a feasibility and experimental replicability study. BMC Med Res Methodol. 2011;11(1):174.
14. Dalal S, Khodyakov D, Srinivasan R, Straus S, Adams J. ExpertLens: a system for eliciting opinions from a large pool of non-collocated experts with diverse knowledge. Tech Forecasting Soc Chang. 2011;78(8):1426–1444.
15. Deshpande AM, Shiffman RN, Nadkarni PM. Metadata-driven Delphi rating on the Internet. Comput Methods Prog Biomed. 2005;77(1):49–56.
16. Ayala GX, Elder JP. Qualitative methods to ensure acceptability of behavioral and social interventions to the target population. J Public Health Dent. 2011;71:S69–S79.
17. Bolger F, Wright G. Improving the Delphi process: lessons from social psychological research. Tech Forcasting Soc Chang. 2011;78:1500–1513.
18. Barber C, Marshall D, Alvarez N, et al. Development of cardiovascular quality indicators for rheumatoid arthritis: results from an international expert panel using a novel online process. J Rheumatol. 2015;42(9):1548–1555.
19. Barber C, Marshall D, Mosher D, et al. Development of system-level performance measures for evaluation of models of care for inflammatory arthritis in Canada. J Rheumatol. 2016. doi: 10.3899/jrheum.150839
20. Barber C, Patel JN, Woodhouse L, et al. Development of key performance indicators to evaluate centralized intake for patients with osteoarthritis and rheumatoid arthritis. Arthritis Res Ther. 2015;17(322):1–12.
21. Ohno-Machado L, Agha Z, Bell DS, et al. pSCANNER: patient-centered Scalable National Network for Effectiveness Research. J Am Med Inform Assoc. 2014;21(4):621–626.
22. Jones MM, Pickett J, Chataway J, et al. Mapping Pathways: Developing Evidence-Based, People-Centred Strategies for the Use of Antiretrovirals as Prevention. Cambridge: RAND Europe; 2013.
23. Khodyakov D, Savitsky TD, Dalal S. Collaborative learning framework for online stakeholder engagement. Health Expect. 2015. doi: 10.1111/hex.12383
24. Rubenstein L, Khodyakov D, Hempel S, et al. How can we recognize continuous quality improvement? Int J Qual Health Care. 2014;26(1):6–15.
25. Claassen CA, Pearson JL, Khodyakov D, et al. Reducing the burden of suicide in the U.S.: the aspirational research goals of the National Action Alliance for Suicide Prevention Research Prioritization Task Force. Am J Prev Med. 2014;47(3):309–314.
26. Khodyakov D, Mikesell L, Schraiber R, Booth M, Bromley E. On using ethical principles of community-engaged research in translational science. Translational Research 2016;171:52–62.
27. Khodyakov D, Stockdale S, Smith N, Booth M, Altman L, Rubenstein L. Patient engagement in the process of planning and designing outpatient care improvements at the Veterans Administration Healthcare System: findings from an online expert panel. Health Expect. 2016. doi: 10.1111/hex.12444
28. Olaniran BA. A model of group satisfaction in computer-mediated communication and face-to-face meetings. Behav Inform Technol. 1996;15(1):24–36.
29. Bailey JE, Pearson SW. Development of a tool for measuring and analyzing computer user satisfaction. Manag Sci. 1983;29(5):530–545.
30. Hiltz SR, Johnson K. User satisfaction with computer-mediated communication systems. Manag Sci. 1990;36(6):739–764.
31. McKenna HP. The selection by ward managers of an appropriate nursing model for long-stay psychiatric patient care. J Adv Nurs. 1989;14(9):762–775.
32. Loughlin KG, Moore LF. Using Delphi to achieve congruent objectives and activities in a pediatrics department. Acad Med. 1979;54(2):101–106.
33. Murphy MK, Black NA, Lamping DL, et al. Consensus development methods, and their use in clinical guideline development. Health Technol Assess. 1998;2(3):1–88.
34. Campbell SM, Braspenning J, Hutchinson A, Marshall M. Research methods used in developing and applying quality indicators in primary care. Qual Saf Health Care. 2002;11(4):358–364.