User-centered design improves the usability of drug-drug interaction alerts: Experimental comparison of interfaces

Journal of Biomedical Informatics

Volumn 66, Issue , 2017, Pages 204-213

  Luna, Daniel R ^a,b   Rizzato Lede, Daniel A ^a   Otero, Carlos M ^a   Risk, Marcelo R ^b,c   González Bernaldo de Quirós, Fernán ^a  

^a HOSPITAL ITALIANO DE BUENOS AIRES   (Argentina)

^b INSTITUTO TECNOLÓGICO DE BUENOS AIRES   (Argentina)

^c Facultad de Ciencias Naturales e Instituto M Lillo   (Argentina)

Author keywords

Clinical decision support systems; Drug interactions; Human computer interaction; Participatory design; Usability; User centered design

Indexed keywords

ARTIFICIAL INTELLIGENCE; DECISION SUPPORT SYSTEMS; HUMAN COMPUTER INTERACTION; SOFTWARE ENGINEERING; USER CENTERED DESIGN;

CLINICAL DECISION SUPPORT SYSTEMS; DRUG-DRUG INTERACTIONS; ELECTRONIC HEALTH RECORD; ENGINEERING TECHNIQUES; EXPERIMENTAL COMPARISON; PARTICIPATORY DESIGN; USABILITY; USER CENTERED DESIGN(UCD);

DRUG INTERACTIONS;

ADULT; ARTICLE; CLINICAL DECISION SUPPORT SYSTEM; CLINICAL EFFECTIVENESS; COMPUTER INTERFACE; DRUG DATABASE; DRUG INTERACTION; ELECTRONIC HEALTH RECORD; EVALUATION STUDY; FEMALE; HEALTH CARE QUALITY; HEALTH PRACTITIONER; HUMAN; MALE; METHODOLOGY; PARTICIPATORY DESIGN; PHYSICIAN; PRESCRIPTION; PRODUCTIVITY; SATISFACTION; SOFTWARE DESIGN; USER CENTERED DESIGN; VIGNETTE; PHYSICIAN ORDER ENTRY SYSTEM; SOFTWARE;

DECISION SUPPORT SYSTEMS, CLINICAL; DRUG INTERACTIONS; ELECTRONIC HEALTH RECORDS; HUMANS; MEDICAL ORDER ENTRY SYSTEMS; SOFTWARE; USER-COMPUTER INTERFACE;

EID: 85010297356     PISSN: 15320464     EISSN: None     Source Type: Journal    
DOI: 10.1016/j.jbi.2017.01.009     Document Type: Article

References (68)

1. [1] Makary, M.A., Daniel, M., Medical error — the third leading cause of death in the US. BMJ 2139 (2016), 1–5, 10.1136/bmj.i2139.
2. [2] Kohn, L.T., Corrigan, J.M., Donaldson, M.S., To err is human: building a safer health system. 1999, National Academies Press, Institute of Medicine.
3. [3] Leape, L.L., Bates, D.W., Cullen, D.J., Cooper, J., Demonaco, H.J., Gallivan, T., et al. Systems analysis of adverse drug events. ADE Prevention Study Group. JAMA 274 (1995), 35–43.
4. [4] Bright, T.J., Wong, A., Dhurjati, R., Bristow, E., Bastian, L., Coeytaux, R.R., et al. Effect of clinical decision-support systems: a systematic review. Ann Intern Med 157 (2012), 29–43, 10.7326/0003-4819-157-1-201207030-00450.
5. [5] Kuperman, G.J., Bobb, A., Payne, T.H., Avery, A.J., Gandhi, T.K., Burns, G., et al. Medication-related clinical decision support in computerized provider order entry systems: a review. J Am Med Informatics Assoc 14 (2007), 29–40, 10.1197/jamia.M2170.
6. [6] Glassman, P.a., Simon, B., Belperio, P., Lanto, A., Improving recognition of drug interactions: benefits and barriers to using automated drug alerts. Med Care 40 (2002), 1161–1171, 10.1097/01.MLR.0000036429.31445.E0.
7. [7] Greenes, R., Clinical decision support: the road ahead. 2014, Academic Press, 10.1016/B978-0-12-398476-0.00029-4.
8. [8] Chaudhry, B., Wang, J., Wu, S., Maglione, M., Mojica, W., Roth, E., et al. Systematic review: impact of health information technology on quality, efficiency, and costs of medical care. Ann Intern Med 144 (2006), 742–752.
9. [9] Coleman, J.J., van der Sijs, H., Haefeli, W.E., Slight, S.P., McDowell, S.E., Seidling, H.M., et al. On the alert: future priorities for alerts in clinical decision support for computerized physician order entry identified from a European workshop. BMC Med Inform Decis Mak, 13, 2013, 111, 10.1186/1472-6947-13-111.
10. [10] Xie, M., Weinger, M.B., Gregg, W.M., Johnson, K.B., Presenting multiple drug alerts in an ambulatory electronic prescribing system. Appl Clin Inform 5 (2014), 334–348, 10.4338/ACI-2013-10-RA-0092.
11. [11] van der Sijs, H., Aarts, J., Vulto, A., Berg, M., Overriding of drug safety alerts in computerized physician order entry. J Am Med Informatics Assoc 13 (2006), 138–147, 10.1197/jamia.M1809.
12. [12] Schpilberg M, Bernaldo de Quiros FG, Luna D, Gomez A, Martinez M, Cifarelli G. Creación de un sistema para la detección de interacciones farmacológicas en una Historia Clínica Electrónica. 4to Simp. Informática en Salud - 30 JAIIO, vol. 30; 2001. p. 105–13.
13. [13] Luna, D., Plazzotta, F., Otero, C., González Bernaldo de Quirós, F., Baum, A., Benítez, S., Incorporación de tecnologías de la información y de las comunicaciones en el Hospital Italiano de Buenos Aires. CEPAL, 2012.
14. [14] Luna D, Waldhorn M, Ianicelli A, Risk M, González Bernaldo De Quirós F. Drug-drug interaction in a healthcare network of Argentina: frequency and characteristics. XIII Congr. Bras. em Informática em Saúde – CBIS 2012, 2012.
15. [15] Institute of Medicine (IOM), Health IT and patient safety: building safer systems for better care. 2012, The National Academies Press (US), Washington (DC), 10.1001/jama.308.21.2282-a.
16. [16] Patapovas, A., Dormann, H., Sedlmayr, B., Kirchner, M., Sonst, A., Müller, F., et al. Medication safety and knowledge-based functions: a stepwise approach against information overload. Br J Clin Pharmacol, 76, 2013, 14, 10.1111/bcp.12190.
17. [17] Paterno, M.D., Maviglia, S.M., Gorman, P.N., Seger, D.L., Yoshida, E., Seger, A.C., et al. Tiering drug-drug interaction alerts by severity increases compliance rates. J Am Med Informatics Assoc 16 (2009), 40–46, 10.1197/jamia.M2808.
18. [18] Feldstein, A., Simon, S.R., Schneider, J., Krall, M., Laferriere, D., Smith, D.H., et al. How to design computerized alerts to safe prescribing practices. Jt Comm J Qual Saf 30 (2004), 602–613.
19. [19] Duke, J.D., Li, X., Dexter, P., Adherence to drug–drug interaction alerts in high-risk patients: a trial of context-enhanced alerting. J Am Med Informatics Assoc 20 (2013), 494–498, 10.1136/amiajnl-2012-001073.
20. [20] Ammenwerth, E., Hackl, W.O., Riedmann, D., Jung, M., Contextualization of automatic alerts during electronic prescription: researchers’ and users' opinions on useful context factors. Stud Health Technol Inform 169 (2011), 920–924, 10.3233/978-1-60750-806-9-920.
21. [21] Bates, D.W., Kuperman, G.J., Wang, S., Gandhi, T., Kittler, A., Volk, L., et al. Ten commandments for effective clinical decision support: making the practice of evidence-based medicine a reality. J Am Med Informatics Assoc 10 (2003), 523–530, 10.1197/jamia.M1370.
22. [22] McEvoy, D.S., Sittig, D.F., Hickman, T.-T., Aaron, S., Ai, A., Amato, M., et al. Variation in high-priority drug-drug interaction alerts across institutions and electronic health records. J Am Med Informatics Assoc, 2016.
23. [23] Zucchero F, Hogan M, Sommer C. Evaluations of drug interactions. St. Louis, Missouri, USA; 2005.
24. [24] Osornio, A.L., Luna, D., Gambarte, M.L., Gomez, A., Reynoso, G., de Quirós, F.G.B., Creation of a local interface terminology to SNOMED CT. Stud Health Technol Inform 129 (2007), 765–769.
25. [25] Luna, D., Otero, V., Canosa, D., Montenegro, S., Otero, P., González Bernaldo de Quirós, F., Analysis and redesign of a knowledge database for a drug-drug interactions alert system. CIN 129 (2007), 885–889.
26. [26] Luna D, Baum AJ, Giunti G, Tarsetti E, Díaz S, Briatore A. Coincidencias de pares de interacción droga-droga en trabajos seleccionados y el Sistema Notificador De Interacciones Farmacológicas del Hospital Italiano de Buenos Aires. Congr. Argentino Med. Interna, SAM. Sociedad Argentina de Medicina; 2013.
27. [27] Scheife, R.T., Hines, L.E., Boyce, R.D., Chung, S.P., Momper, J.D., Sommer, C.D., et al. Consensus recommendations for systematic evaluation of drug-drug interaction evidence for clinical decision support. Drug Saf 38 (2015), 197–206, 10.1007/s40264-014-0262-8.
28. [28] Seidling, H.M., Phansalkar, S., Seger, D.L., Paterno, M.D., Shaykevich, S., Haefeli, W.E., et al. Factors influencing alert acceptance: a novel approach for predicting the success of clinical decision support. J Am Med Inf Assoc 18 (2011), 479–484, 10.1136/amiajnl-2010-000039.
29. [29] Luna, D., Quispe, M., Gonzalez, Z., Alemrares, A., Risk, M., Garcia Aurelio, M., et al. User-centered design to develop clinical applications. Literature review. Stud. Health Technol. Inform., 216, 2015, 967, 10.3233/978-1-61499-564-7-967.
30. [30] Russ, A.L., Chen, S., Melton, B.L., Johnson, E.G., Spina, J.R., Weiner, M., et al. A novel design for drug-drug interaction alerts improves prescribing efficiency. J Comm J Qual Patient Saf 41 (2015), 396–405.
31. [31] Sears, A., Jacko, J.A., Human-computer interaction: development process. 2009, CRC Press.
32. [32] Patel, V.L., Kannampallil, T.G., Cognitive informatics in biomedicine and healthcare. J Biomed Inform 53 (2015), 3–14, 10.1016/j.jbi.2014.12.007.
33. [33] Norman, D.A., Draper, S.W., User centered system design: new perspectives on human-computer interaction. 1st ed., 1986, CRC Press, Hillsdale, N.J.
34. [34] ISO 92410-210:2010: Part 210: Ergonomics of human–system interaction: Human-centred design for interactive systems. ISO; 2010.
35. [35] Kushniruk, A., Nøhr, C., Participatory design, user involvement and health IT evaluation. Stud Health Technol Inform 222 (2016), 139–151.
36. [36] Macaulay, C., Sloan, D., Jiang, X., Forbes, P., Loynton, S., Swedlow, J.R., et al. Usability and user-centered design in scientific software development. IEEE Softw 26 (2009), 96–102, 10.1109/MS.2009.27.
37. [37] Kitchenham, B., Sjoberg, D.I.K., Dyba, T., Brereton, O.P., Budgen, D., Host, M., et al. Trends in the quality of human-centric software engineering experiments – a quasi-experiment. IEEE Trans Softw Eng 39 (2013), 1002–1017, 10.1109/TSE.2012.76.
38. [38] Kitchenham B, Budgen D, Brereton P, Turner M. 2nd International workshop on realising evidence-based software engineering (REBSE-2): Overview and Introduction. Second Int. Work. Realis. Evidence-Based Softw. Eng. (REBSE ’07), IEEE; 2007, p. 1–1. doi: http://dx.doi.org/10.1109/REBSE.2007.1.
39. [39] Sjoeberg, D.I.K., Hannay, J.E., Hansen, O., Kampenes, V.B., Karahasanovic, A., Liborg, N.-K., et al. A survey of controlled experiments in software engineering. IEEE Trans Softw Eng 31 (2005), 733–753, 10.1109/TSE.2005.97.
40. [40] Marcilly, R., Peute, L., Beuscart-Zephir, M.C., From usability engineering to evidence-based usability in health IT. Evidence-Based Heal Inform Promot Saf Effic Sci Meth Ethical Policy 222 (2016), 126–138, 10.3233/978-1-61499-635-4-126.
41. [41] First hospital in Argentina awarded for digital maturity n.d. < https://www.eiseverywhere.com/file_uploads/b4ead7f8dcbcbe30ebebf601e62dfeaa_Argentina_HospitalItalianoPressRelease.pdf> [accessed September 27, 2016].
42. [42] Noticias Departamento de Informática en Salud del Hospital Italiano. Único Hospital en Argentina en acreditar calidad de Historia Clínica Electrónica 2016. < https://www1.hospitalitaliano.org.ar/infomed/index.php?contenido=ver_curso.php&id_curso=19752> [accessed September 27, 2016].
43. [43] Stanziola E, Uznayo MQ, Ortiz JM, Simón M, Otero C, Campos F, et al. User-centered design of health care software development: towards a cultural change. Stud Health Technol Inform, vol. 216. IOS Press; 2015. p. 368–71.
44. [44] Morae n.d. < https://www.techsmith.com/morae.html>.
45. [45] Simonsen, J., Robertson, T., Routledge international handbook of participatory design. 2013, Routledge.
46. [46] Luna, D., Otero, C., Almerares, A., Stanziola, E., Risk, M., Gonzalez Bernaldo de Quiros, F., Participatory design for drug-drug interaction alerts. Stud Health Technol Inform 210 (2015), 45–49, 10.3233/978-1-61499-512-8-45.
47. [47] Schoenberg, N.E., Ravdal, H., Using vignettes in awareness and attitudinal research. Int J Soc Res Methodol 3 (2000), 63–74, 10.1080/136455700294932.
48. [48] Evans, S.C., Roberts, M.C., Keeley, J.W., Blossom, J.B., Amaro, C.M., Garcia, A.M., et al. Vignette methodologies for studying clinicians’ decision-making: validity, utility, and application in ICD-11 field studies. Int J Clin Heal Psychol 15 (2015), 160–170, 10.1016/j.ijchp.2014.12.001.
49. [49] Soegaard M, Dam RF. The encyclopedia of human-computer interaction. Encycl Human-Computer Interact; 2012. < file:///books/hci.html>.
50. [50] Balsamiq n.d. < https://balsamiq.com/>.
51. [51] Jaspers, M.W.M., Steen, T., Van Den, Bos C., Geenen, M., The think aloud method: a guide to user interface design. Int J Med Inform 73 (2004), 781–795, 10.1016/j.ijmedinf.2004.08.003.
52. [52] Axure n.d. < http://www.axure.com/>.
53. [53] Jordan, P.W., Thomas, B., Weerdmeester, B.A., McClelland, A.L., SUS: a “quick and dirty” usability scale. 1986, Taylor and Francis, London.
54. [54] Brooke, J., SUS - a quick and dirty usability scale. Usability Eval Ind 189 (1996), 4–7, 10.1002/hbm.20701.
55. [55] Zheng, K., Fear, K., Chaffee, B.W., Zimmerman, C.R., Karls, E.M., Gatwood, J.D., et al. Development and validation of a survey instrument for assessing prescribers’ perception of computerized drug-drug interaction alerts. J Am Med Informatics Assoc 18 (2011), i51–i61, 10.1136/amiajnl-2010-000053.
56. [56] R: The R Project for Statistical Computing n.d. < https://www.r-project.org/>.
57. [57] Marcilly, R., Ammenwerth, E., Vasseur, F., Roehrer, E., Beuscart-Zéphir, M.C., Usability flaws of medication-related alerting functions: a systematic qualitative review. J Biomed Inform 55 (2015), 260–271, 10.1016/j.jbi.2015.03.006.
58. [58] McCoy, A.B., Thomas, E.J., Krousel-Wood, M., Sittig, D.F., Clinical decision support alert appropriateness: a review and proposal for improvement. Ochsner J 14 (2014), 195–202.
59. [59] Phansalkar, S., Zachariah, M., Seidling, H.M., Mendes, C., Volk, L., Bates, D.W., Evaluation of medication alerts in electronic health records for compliance with human factors principles. J Am Med Informatics Assoc 21 (2014), e332–e340, 10.1136/amiajnl-2013-002279.
60. [60] Russ, A.L., Zillich, A.J., Melton, B.L., Russell, S.a., Chen, S., Spina, J.R., et al. Applying human factors principles to alert design increases efficiency and reduces prescribing errors in a scenario-based simulation. J Am Med Inform Assoc, 2014, 1–10, 10.1136/amiajnl-2013-002045.
61. [61] Seidling, H.M., Klein, U., Schaier, M., Czock, D., Theile, D., Pruszydlo, M.G., et al. What, if all alerts were specific – estimating the potential impact on drug interaction alert burden. Int J Med Inform 83 (2014), 285–291, 10.1016/j.ijmedinf.2013.12.006.
62. [62] Rodrigues, A.T., Stahlschmidt, R., Granja, S., Falcão, A.L.E., Moriel, P., Mazzola, P.G., Clinical relevancy and risks of potential drug–drug interactions in intensive therapy. Saudi Pharm J 23 (2015), 366–370, 10.1016/j.jsps.2014.11.014.
63. [63] Smithburger, P.L., Kane-Gill, S.L., Seybert, A.L., Drug-drug interactions in the medical intensive care unit: an assessment of frequency, severity and the medications involved. Int J Pharm Pract 20 (2012), 402–408, 10.1111/j.2042-7174.2012.00221.x.
64. [64] Ismail, M., Khan, F., Noor, S., Haider, I., Haq, I.-U., Ali, Z., et al. Potential drug-drug interactions in medical intensive care unit of a tertiary care hospital in Pakistan. Int J Clin Pharm 38 (2016), 1052–1056, 10.1007/s11096-016-0340-3.
65. [65] Ahn, E.K., Kam, H.J., Park, D.K., Jung, E.Y., Lee, Y., Park, R.W., Differences among admitting departments in alerts and alert overrides for drug-drug interaction. Pharmacoepidemiol Drug Saf 23 (2014), 390–397, 10.1002/pds.3591.
66. ∗. Pediatr Crit Care Med, 17, 2016, 470, 10.1097/PCC.0000000000000694.
67. [67] Kushniruk, A.W., Patel, V.L., Cognitive and usability engineering methods for the evaluation of clinical information systems. J Biomed Inform 37 (2004), 56–76, 10.1016/j.jbi.2004.01.003.
68. [68] Green, L.A., Nease, D., Klinkman, M.S., Clinical reminders designed and implemented using cognitive and organizational science principles decrease reminder fatigue. J Am Board Fam Med 28 (2015), 351–359, 10.3122/jabfm.2015.03.140243.