Conducting Online Expert panels: A feasibility and experimental replicability study

BMC Medical Research Methodology

Volumn 11, Issue , 2011, Pages

  Khodyakov, Dmitry ^a   Hempel, Susanne ^a   Rubenstein, Lisa ^a,b   Shekelle, Paul ^a,c   Foy, Robbie ^d   Salem Schatz, Susanne ^e   O'Neill, Sean ^a,f   Danz, Margie ^a,b   Dalal, Siddhartha ^a  

^a RAND CORPORATION   (United States)

^b VETERANS AFFAIRS MEDICAL CENTER   (United States)

^c VA GREATER LOS ANGELES HEALTHCARE SYSTEM   (United States)

^d UNIVERSITY OF LEEDS   (United Kingdom)

^e Independent Consultant   (United States)

^f NORTHWESTERN UNIVERSITY   (United States)

Author keywords

[No Author keywords available]

Indexed keywords

ADVISORY COMMITTEE; ARTICLE; CAPACITY BUILDING; CONSENSUS; COOPERATION; DECISION MAKING; EXPERT WITNESS; FEASIBILITY STUDY; HEALTH SERVICES RESEARCH; HUMAN; INTERNET; METHODOLOGY; ONLINE SYSTEM; ORGANIZATION AND MANAGEMENT; QUESTIONNAIRE; REPRODUCIBILITY; STANDARD; STATISTICS; TOTAL QUALITY MANAGEMENT; UTILIZATION REVIEW;

ADVISORY COMMITTEES; CAPACITY BUILDING; CONSENSUS; COOPERATIVE BEHAVIOR; DECISION MAKING; EXPERT TESTIMONY; FEASIBILITY STUDIES; HEALTH SERVICES RESEARCH; HUMANS; INTERNET; ONLINE SYSTEMS; QUALITY IMPROVEMENT; QUESTIONNAIRES; REPRODUCIBILITY OF RESULTS; STATISTICS AS TOPIC;

EID: 84155176746     PISSN: None     EISSN: 14712288     Source Type: Journal    
DOI: 10.1186/1471-2288-11-174     Document Type: Article

References (30)

1. Qualitative research: consensus methods for medical and health services research. Jones J, Hunter D, British Medical Journal 1995 311 376 380 10.1136/bmj.311.7001.376 7640549
2. Consensus Methods: Characteristics and Guidelines for Use. Fink A, Kosecoff JB, Chassin MR, Brook RH, Santa Monica, CA: RAND 1991 vol. N-3367-HHS
3. RAND/UCLA Appropriateness Method (RAM). Fitch K, Bernstein SJ, Aguilar MD, Burnand B, LaCalle JR, Lazaro P, Loo Mvh, McDonnell J, Vader JP, Kahan JP, Santa Monica: RAND Corporation 2001 109
4. Consensus methods: characteristics and guidelines for use. Fink A, Kosecoff J, Chassin M, Brook RH, American Journal of Public Health 1984 74 979 983 10.2105/AJPH.74.9.979 6380323 (Pubitemid 14059576)
5. Consensus development methods: a review of best practice in creating clinical guidelines. Black N, Murphy M, Lamping D, McKee M, Sanderson C, Askham J, Marteau T, Journal of health services research & policy 1999 4 236 248 22200445 (Pubitemid 29473735)
6. The Delphi technique: a worthwhile research approach for nursing? McKenna HP, Journal of Advanced Nursing 1994 19 1221 1225 10.1111/j.1365-2648.1994. tb01207.x 7930104
7. Developing clinical guidelines: a challenge to current methods. Raine R, Sanderson C, Black N, British Medical Journal 2005 331 631 10.1136/bmj.331.7517. 631 16166137
8. Developing a quality criteria framework for patient decision aids: online international Delphi consensus process. Elwyn G, O'Connor A, Stacey D, Volk R, Edwards A, Coulter A, Thomson R, Barratt A, Barry M, Bernstein S, British Medical Journal 2006 333 417 423 10.1136/bmj.38926.629329.AE 16908462 (Pubitemid 44320126)
9. Expert consensus for discharge referral decisions using online Delphi. Bowles KH, Holmes JH, Naylor MD, Liberatore M, Nydick R, AMIA Annual Symposium Proceedings 2003 2003 106 109
10. The potential influence of small group processes on guideline development. Pagliari C, Grimshaw J, Eccles M, Journal of Evaluation in Clinical Practice 2001 7 165 173 10.1046/j.1365-2753.2001.00272.x 11489041 (Pubitemid 32489009)
11. The equalization phenomenon: Status effects in computer-mediated and face-to-face decision-making groups. Dubrovsky VJ, Kiesler S, Sethna BN, Human-Computer Interaction 1991 6 119 146 10.1207/s15327051hci0602-2 (Pubitemid 23602922)
12. Consensus development methods, and their use in clinical guideline development. Murphy MK, Black NA, Lamping DL, McKee CM, Sanderson CFB, Askham J, Health Technology Assessment 1998 2
13. Comparison of mailed vs. Internet applications of the Delphi technique in clinical informatics research. Snyder-Halpern R, Thompson C, Schaffer J, Proceedings of the AMIA Symposium 2000 809 813
14. Brown R, Group processes: Dynamics within and between groups Blackwell Pub 2000
15. Wainfan L, Davis PK, Challenges in virtual collaboration: Videoconferencing, audioconferencing, and computer-mediated communications Santa Monica: RAND Corporation 2004
16. Computer-based Delphi processes. Turoff M, Hiltz SR, Gazing into the oracle: the Delphi method and its application to social policy and public health Jessica Kingsley Publishers, Adler M, Ziglio E, 1996 56 89
17. An examination of asynchronous communication experiences and perspectives of students in an online course: A case study. Vonderwell S, The Internet and Higher Education 2003 6 77 90 10.1016/S1096-7516(02)00164-1
18. A critical review of the Delphi technique as a research methodology for nursing. Keeney S, Hasson F, McKenna H, International Journal of Nursing Studies 2001 38 195 200 10.1016/S0020-7489(00)00044-4 11223060 (Pubitemid 33660489)
19. The reproducibility of a method to identify the overuse and underuse of medical procedures. Shekelle P, Kahan J, Bernstein S, Leape L, Kamberg C, Park R, New England Journal of Medicine 1998 338 1888 1895 10.1056/ NEJM199806253382607 9637810 (Pubitemid 28287863)
20. The national drug-abuse policy Delphi. Jillson IA, The Delphi Method: Techniques and Applications, Linstone H, Turoff M, 2002 119 154
21. Finding order in heterogeneity: types of quality-improvement intervention publications. Rubenstein L, Hempel S, Farmer M, Asch S, Yano E, Dougherty D, Shekelle P, Quality and Safety in Health Care 2008 17 403 408 10.1136/qshc.2008.028423 19064654
22. Identifying quality improvement intervention evaluations: is consensus achievable? Danz M, Rubenstein L, Hempel S, Foy R, Suttorp M, Farmer M, Shekelle P, Quality and Safety in Health Care 2010 19 279 283 10.1136/qshc.2009.036475 20630931
23. Identifying continuous quality improvement publications: What makes an improvement intervention "CQI"? O'Neill SM, Hempel S, Lim Y-W, Danz M, Foy R, Suttorp MJ, Shekelle PG, Rubenstein LV, BMJ Quality and Safety 2011 doi:10.1136/bmjqs.2010.050880
24. ExpertLens: A system for eliciting opinions from a large pool of non-collocated experts with diverse knowledge. Dalal SR, Khodyakov D, Srinivasan R, Straus SG, Adams J, Technological Forecasting & Social Change 2011 78 1426 1444 10.1016/j.techfore.2011.03.021 22195377
25. The Montreal definition and classification of gastroesophageal reflux disease: a global evidence-based consensus. Vakil N, van Zanten SV, Kahrilas P, Dent J, Jones R, American Journal of Gastroenterology 2006 101 1900 1920 10.1111/j.1572-0241.2006.00630.x 16928254 (Pubitemid 44166604)
26. An experimental study of determinants of the extent of disagreement within clinical guideline development groups. Hutchings A, Raine R, Sanderson C, Black N, Quality and Safety in Health Care 2005 14 240 245 10.1136/qshc.2004. 013227 16076786 (Pubitemid 41133176)
27. Integration and generalization of kappas for multiple raters. Conger AJ, Psychological Bulletin 1980 88 322 328
28. The measurement of observer agreement for categorical data. Landis JR, Koch GG, Biometrics 1977 33 159 174 10.2307/2529310 843571 (Pubitemid 8054849)
29. How do stakeholder groups vary in a Delphi technique about primary mental health care and what factors influence their ratings? Campbell S, Shield T, Rogers A, Gask L, Quality and Safety in Health Care 2004 13 428 434 10.1136/qshc.2003.007815 15576704 (Pubitemid 39647353)
30. Information overload and the message dynamics of online interaction spaces: A theoretical model and empirical exploration. Jones Q, Ravid G, Rafaeli S, Information Systems Research 2004 15 194 210 10.1287/isre.1040.0023 (Pubitemid 39068903)