Integrating usability testing and think-aloud protocol analysis with "near-live" clinical simulations in evaluating clinical decision support

International Journal of Medical Informatics

Volumn 81, Issue 11, 2012, Pages 761-772

  Li, Alice C ^a   Kannry, Joseph L ^a   Kushniruk, Andre ^b   Chrimes, Dillon ^b   McGinn, Thomas G ^c   Edonyabo, Daniel ^a   Mann, Devin M ^d  

^a MOUNT SINAI SCHOOL OF MEDICINE   (United States)

^b UNIVERSITY OF VICTORIA   (Canada)

^c DONALD AND BARBARA ZUCKER SCHOOL OF MEDICINE AT HOFSTRA NORTHWELL   (United States)

^d BOSTON UNIVERSITY SCHOOL OF MEDICINE   (United States)

Author keywords

Clinical decision support; Clinical prediction rules; Clinical simulations; Electronic health records; Evidence based medicine; Usability

Indexed keywords

CLINICAL DECISION SUPPORT; CLINICAL SIMULATION; ELECTRONIC HEALTH RECORD; EVIDENCE-BASED MEDICINE; PREDICTION RULES; USABILITY;

DECISION SUPPORT SYSTEMS; FORECASTING; RECORDS MANAGEMENT;

USABILITY ENGINEERING;

ARTICLE; CLINICAL DECISION MAKING; CLINICAL PROTOCOL; COMPUTER PROGRAM; ELECTRONIC MEDICAL RECORD; EVIDENCE BASED MEDICINE; GENERAL PRACTITIONER; HEALTH CARE MANAGEMENT; HEALTH CARE UTILIZATION; PERCEPTION; PRIMARY MEDICAL CARE; PRIORITY JOURNAL; QUALITATIVE ANALYSIS; SIMULATION; WORKFLOW;

COMPUTER SIMULATION; DECISION SUPPORT SYSTEMS, CLINICAL; ELECTRONIC HEALTH RECORDS; EVIDENCE-BASED MEDICINE; HUMANS; MEDICAL INFORMATICS; PHYSICIAN'S PRACTICE PATTERNS; PRIMARY HEALTH CARE;

EID: 84867142200     PISSN: 13865056     EISSN: 18728243     Source Type: Journal    
DOI: 10.1016/j.ijmedinf.2012.02.009     Document Type: Article

References (32)

1. Blumenthal D. Stimulating the adoption of health information technology. N. Engl. J. Med. 2009, 360(15):1477-1479.
2. Committee on Quality of Health Care in America IoM Crossing the Quality Chasm: A New Health System for the 21st Century 2001.
3. Garg A.X., Adhikari N.K., McDonald H., et al. Effects of computerized clinical decision support systems on practitioner performance and patient outcomes: a systematic review. JAMA 2005, 293(10):1223-1238.
4. Hunt D.L., Haynes R.B., Hanna S.E., Smith K. Effects of computer-based clinical decision support systems on physician performance and patient outcomes: a systematic review. JAMA 1998, 280(15):1339-1346.
5. Randolph A.G., Haynes R.B., Wyatt J.C., Cook D.J., Guyatt G.H. Users' Guides to the Medical Literature: XVIII. How to use an article evaluating the clinical impact of a computer-based clinical decision support system. JAMA 1999, 282(1):67-74.
6. Eslami S., Abu-Hanna A., de Keizer N.F. Evaluation of outpatient computerized physician medication order entry systems: a systematic review. J. Am. Med. Inform. Assoc. 2007, 14(4):400-406.
7. Lo H.G., Newmark L.P., Yoon C., et al. Electronic health records in specialty care: a time-motion study. J. Am. Med. Inform. Assoc. 2007, 14(5):609-615.
8. Welch W.P., Bazarko D., Ritten K., Burgess Y., Harmon R., Sandy L.G. Electronic health records in four community physician practices: impact on quality and cost of care. J. Am. Med. Inform. Assoc. 2007, 14(3):320-328.
9. Romano M.J., Stafford R.S. Electronic health records and clinical decision support systems: impact on national ambulatory care quality. Arch. Intern. Med. 2011, 171(10):897-903.
10. Saleem J.J., Patterson E.S., Militello L., et al. Impact of clinical reminder redesign on learnability, efficiency, usability, and workload for ambulatory clinic nurses. J. Am. Med. Inform. Assoc. 2007, 14(5):632-640.
11. Patterson E.S., Doebbeling B.N., Fung C.H., Militello L., Anders S., Asch S.M. Identifying barriers to the effective use of clinical reminders: bootstrapping multiple methods. J. Biomed. Inform. 2005, 38(3):189-199.
12. Saleem J.J., Patterson E.S., Militello L., Render M.L., Orshansky G., Asch S.M. Exploring barriers and facilitators to the use of computerized clinical reminders. J. Am. Med. Inform. Assoc. 2005, 12(4):438-447.
13. Bates D.W., Kuperman G.J., Wang S., et al. Ten commandments for effective clinical decision support: making the practice of evidence-based medicine a reality. J. Am. Med. Inform. Assoc. 2003, 10(6):523-530.
14. Mollon B., Chong J., Holbrook A.M., Sung M., Thabane L., Foster G. Features predicting the success of computerized decision support for prescribing: a systematic review of randomized controlled trials. BMC Med. Inform. Decis. Mak. 2009, 9:11.
15. Russ A.L., Zillich A.J., McManus M.S., Doebbeling B.N., Saleem J.J. A human factors investigation of medication alerts: barriers to prescriber decision-making and clinical workflow. AMIA Annu. Symp. Proc. 2009, 2009:548-552.
16. McDonnell C., Werner K., Wendel L. Electronic health record usability. Vendor Practices and Perspectives: AHRQ 2010, James Bell Associates, The Altarum Institute, Rockville.
17. Armijo D., McDonnell C., Werner K. Electronic health record usability. Interface Design Considerations: AHRQ 2009, James Bell Associates, The Altarum Institute, Rockville.
18. Kannry J., Kushniruk A., Koppel R. Meaningful Usability: Health Care Information Technology for the Rest of Us. Medical Informatics: An Executive Primer 2011, HIMSS, Chicago. 2nd ed.
19. Weir C.R., Nebeker J.J., Hicken B.L., Campo R., Drews F., Lebar B. A cognitive task analysis of information management strategies in a computerized provider order entry environment. J. Am. Med. Inform. Assoc. 2007, 14(1):65-75.
20. Kushniruk A.W., Kaufman D.R., Patel V.L., Levesque Y., Lottin P. Assessment of a computerized patient record system: a cognitive approach to evaluating medical technology. MD Comput. 1996, 13(5):406-415.
21. Horsky J., Kaufman D.R., Oppenheim M.I., Patel V.L. A framework for analyzing the cognitive complexity of computer-assisted clinical ordering. J. Biomed. Inform. 2003, 36(1/2):4-22.
22. Rose A.F., Schnipper J.L., Park E.R., Poon E.G., Li Q., Middleton B. Using qualitative studies to improve the usability of an EMR. J. Biomed. Inform. 2005, 38(1):51-60.
23. Borycki E., Kushniruk A. Use of clinical simulations to evaluate the impact of health information systems an ubiquitous computing devices and software upon healthcare work. Ubiquitous Health and Medical Informatics: The Ubiquity 2.0 Trend and Beyond 2010, 552-573. IGI Global, Hershey, PA. S. Mohammed, J. Fiaidhi (Eds.).
24. Kushniruk A.W., Borycki E.M., Kuwata S., Kannry J. Emerging approaches to usability evaluation of health information systems: towards in situ analysis of complex healthcare systems and environments. Studies in Health Technology and Informatics 2011, 169:915-919.
25. McGinn T.G., Deluca J., Ahlawat S.K., Mobo B.H., Wisnivesky J.P. Validation and modification of streptococcal pharyngitis clinical prediction rules. Mayo Clin. Proc. 2003, 78(3):289-293.
26. Walsh B.T., Bookheim W.W., Johnson R.C., Tompkins R.K. Recognition of streptococcal pharyngitis in adults. Arch. Intern. Med. 1975, 135(11):1493-1497.
27. Heckerling P.S., Tape T.G., Wigton R.S., et al. Clinical prediction rule for pulmonary infiltrates. Ann. Intern. Med. 1990, 113(9):664-670.
28. Daniels J., Fels S., Kushniruk A., Lim J., Ansermino J.M. A framework for evaluating usability of clinical monitoring technology. J. Clin. Monit. Comput. 2007, 21(5):323-330.
29. Kushniruk A.W., Patel V.L. Cognitive and usability engineering methods for the evaluation of clinical information systems. J. Biomed. Inform. 2004, 37(1):56-76.
30. Patel V.L., Kushniruk A.W., Yang S., Yale J.F. Impact of a computer-based patient record system on data collection, knowledge organization, and reasoning. J. Am. Med. Inform. Assoc. 2000, 7(6):569-585.
31. Mann D.M., Kannry J.L., Edonyabo D., et al. Rationale, design, and implementation protocol of an electronic health record integrated clinical prediction rule (iCPR) randomized trial in primary care. Implement Sci. 2011, 6(1):109.
32. Horsky J., McColgan K., Pang J.E., et al. Complementary methods of system usability evaluation: surveys and observations during software design and development cycles. J. Biomed. Inform. 2010, 43(5):782-790.