Complementary methods of system usability evaluation: Surveys and observations during software design and development cycles

Journal of Biomedical Informatics

Volumn 43, Issue 5, 2010, Pages 782-790

  Horsky, Jan ^a,b,c   McColgan, Kerry ^a   Pang, Justine E ^a   Melnikas, Andrea J ^a   Linder, Jeffrey A ^a,b,c   Schnipper, Jeffrey L ^a,b,c   Middleton, Blackford ^a,b,c  

^a PARTNERS HEALTHCARE   (United States)

^b BRIGHAM AND WOMEN S HOSPITAL   (United States)

^c HARVARD MEDICAL SCHOOL   (United States)

Author keywords

Adoption of HIT; Clinical information systems; Design and development; Health information technology; Usability evaluations

Indexed keywords

ADOPTION OF HIT; CLINICAL DOCUMENTATION; CLINICAL INFORMATION SYSTEM; CLINICAL WORK; COMPLEMENTARY METHODS; CONTROL PROBLEMS; DESIGN AND DEVELOPMENT; ONLINE SURVEYS; QUALITY OF CARE; SOFTWARE DESIGN AND DEVELOPMENT; SYSTEM DESIGN PROCESS; SYSTEM USABILITY; USABILITY EVALUATION;

DESIGN; ELECTRONIC MAIL; INFORMATION SYSTEMS; SURVEYS; SYSTEMS ANALYSIS; USABILITY ENGINEERING;

SOFTWARE DESIGN;

ARTICLE; BIOMEDICINE; COMPUTER PROGRAM; HEALTH CARE QUALITY; HEALTH SURVEY; MEDICAL INFORMATION SYSTEM; PRIORITY JOURNAL;

DATA COLLECTION; DOCUMENTATION; ELECTRONIC HEALTH RECORDS; ELECTRONIC MAIL; FEMALE; HUMANS; INTERNET; MALE; MEDICAL INFORMATICS; MIDDLE AGED; PHYSICIANS; SOFTWARE DESIGN;

EID: 77956263242     PISSN: 15320464     EISSN: None     Source Type: Journal    
DOI: 10.1016/j.jbi.2010.05.010     Document Type: Article

References (49)

1. Ammenwerth E., Schnell-Inderst P., Machan C., Siebert U. The effect of electronic prescribing on medication errors and adverse drug events: a systematic review. J Am Med Inform Assoc 2008, 15:585-600.
2. Linder J.A., Ma J., Bates D.W., Middleton B., Stafford R.S. Electronic health record use and the quality of ambulatory care in the United States. Arch Intern Med 2007, 167:1400-1405.
3. Chaudhry B., Wang J., Wu S., Maglione M., Mojica W., Roth E., et al. Systematic review: impact of health information technology on quality, efficiency, and costs of medical care. Ann Intern Med 2006, 144:742-752. [see comment].
4. Kaushal R., Shojania K.G., Bates D.W. Effects of computerized physician order entry and clinical decision support systems on medication safety: a systematic review. Arch Intern Med 2003, 163:1409-1416.
5. Koppel R., Metlay J.P., Cohen A., Abaluck B., Localio A.R., Kimmel S.E., et al. Role of computerized physician order entry systems in facilitating medication errors. JAMA 2005, 293:1197-1203.
6. Horsky J., Kuperman G.J., Patel V.L. Comprehensive analysis of a medication dosing error related to CPOE. J Am Med Inform Assoc 2005, 12:377-382.
7. Ludwick D.A., Doucette J. Adopting electronic medical records in primary care: lessons learned from health information systems implementation experience in seven countries. Int J Med Inform 2009, 78:22-31.
8. Kaplan B., Harris-Salamone K.D. White paper: Health IT project success and failure: recommendations from literature and an AMIA workshop. J Am Med Inform Assoc 2009, 16:291-299.
9. DesRoches C.M., Campbell E.G., Rao S.R., Donelan K., Ferris T.G., Jha A.K., et al. Electronic health records in ambulatory care: a national survey of physicians. N Engl J Med 2008, 359:50-60.
10. Smelcer J.B., Miller-Jacobs H., Kantrovich L. Usability of electronic medical records. J Usability Stud 2009, 4:70-84.
11. Harrison M.I., Koppel R., Bar-Lev S. Unintended consequences of information technologies in health care: an interactive sociotechnical analysis. J Am Med Inform Assoc 2007, 14:542-549.
12. Pizziferri L., Kittler A.F., Volk L.A., Honour M.M., Gupta S., Wang S.J., et al. Primary care physician time utilization before and after implementation of an electronic health record: a time-motion study. J Biomed Inform 2005, 38:176-188.
13. Simon S.R., Soran C.S., Kaushal R., Jenter C.A., Volk L.A., Burdick E., et al. Physicians' use of key functions in electronic health records from 2005 to 2007: a statewide survey. J Am Med Inform Assoc 2009, 16:465-470.
14. Weir C.R., Nebeker J.J.R., Hicken B.L., Campo R., Drews F., LeBar B. A cognitive task analysis of information management strategies in a computerized provider order entry environment. J Am Med Inform Assoc 2007, 14:65-75.
15. Vicente K.J. Work domain analysis and task analysis: a difference that matters. Cognitive task analysis 2000, 101-118. Lawrence Erlbaum Associates, Inc., Mahwah, NJ. J.M. Schraagen, S.F. Chipman (Eds.).
16. Zhang J, Butler K. UFuRT: A work-centered framework and process for design and evaluation of information systems. HCI International Proceedings; 2007.
17. Schnipper J.L., Linder J.A., Palchuk M.B., Einbinder J.S., Li Q., Postilnik A., et al. " Smart Forms" in an electronic medical record: documentation-based clinical decision support to improve disease management. J Am Med Inform Assoc 2008, 15:513-523.
18. Sittig D.F., Stead W.W. Computer-based physician order entry: the state of the art. J Am Med Inform Assoc 1994, 1:108-123.
19. HIMSS EHR usability task force. Defining and testing EMR usability: principles and proposed methods of EMR usability evaluation and rating. HIMSS; 2009.
20. Ball M.J., Silva J.S., Bierstock S., Douglas J.V., Norcio A.F., Chakraborty J., et al. Failure to provide clinicians useful IT systems: opportunities to leapfrog current technologies. Methods Inf Med 2008, 47:4-7.
21. Zheng K., Padman R., Johnson M.P., Diamond H.S. An interface-driven analysis of user interactions with an electronic health records system. J Am Med Inform Assoc 2009, 16:228-237.
22. Schwarz N., Oyserman D. Asking questions about behavior: cognition, communication, and questionnaire construction. Am J Eval 2001, 22:127.
23. Jaspers M.W.M. A comparison of usability methods for testing interactive health technologies: methodological aspects and empirical evidence. Int J Med Inform 2009, 78:340-353.
24. Uldall-Espersen T., Frokjaer E., Hornbaek K. Tracing impact in a usability improvement process. Interact Comput 2008, 20:48-63.
25. Peleg M., Shachak A., Wang D., Karnieli E. Using multi-perspective methodologies to study users' interactions with the prototype front end of a guideline-based decision support system for diabetic foot care. Int J Med Inform 2009, 78:482-493.
26. Sittig D.F., Singh H. Eight rights of safe electronic health record use. JAMA 2009, 302:1111-1113.
27. Nielsen J. Iterative user interface design. IEEE Comput 1993, 26:32-41.
28. Gould J.D., Lewis C. Designing for usability: key principles and what designers think. Commun. ACM 1985, 28:300-311.
29. Walker J.M., Carayon P., Leveson N., Paulus R.A., Tooker J., Chin H., et al. EHR safety: the way forward to safe and effective systems. J Am Med Inform Assoc 2008, 15:272-277.
30. Leveson N.G. Intent specifications: an approach to building human-centered specifications. IEEE Trans Software Eng 2000, 26:15-35.
31. Wachter S.B., Agutter J., Syroid N., Drews F., Weinger M.B., Westenskow D. The employment of an iterative design process to develop a pulmonary graphical display. J Am Med Inform Assoc 2003, 10:363-372.
32. Morae. 3.1 ed., Okemos, MI: TechSmith Corporation; 2009.
33. Nielsen J., Mack R.L. Usability inspection methods 1994, John Wiley & Sons, New York.
34. Shneiderman B. Designing the user interface. Strategies for effective human-computer-interaction 2004, Addison Wesley Longman, Reading, MA. 4th ed.
35. Tognazzini B. Tog on interface 1992, Addison-Wesley, Reading, Mass.
36. Tufte E.R. The visual display of quantitative information 2001, Graphics Press, Cheshire, Conn. 2nd ed.
37. Atkinson B.F., Bennet T.O., Bahr G.S., Nelson M.M. Development of a multiple heuristics evaluation table (MHET) to support software development and usability analysis. Universal access in human-computer interaction: coping with diversity 2007, Springer, Berlin/Heidelberg.
38. Thyvalikakath T.P., Schleyer T.K., Monaco V. Heuristic evaluation of clinical functions in four practice management systems: a pilot study. J Am Dent Assoc 2007, 138:209-210.
39. Scandurra I., Hagglund M., Engstrom M., Koch S. Heuristic evaluation performed by usability-educated clinicians: education and attitudes. Stud Health Technol Inform 2007, 205-216.
40. Lai T.Y. Iterative refinement of a tailored system for self-care management of depressive symptoms in people living with HIV/AIDS through heuristic evaluation and end user testing. Int J Med Inform 2007, 76:S317-S324.
41. Tang Z., Johnson T.R., Tindall R.D., Zhang J. Applying heuristic evaluation to improve the usability of a telemedicine system. Telemed J E Health 2006, 12:24-34.
42. Choi J., Bakken S. Heuristic evaluation of a web-based educational resource for low literacy NICU parents. Stud Health Technol Inform 2006, 194-199.
43. Zhang J., Johnson T.R., Patel V.L., Paige D.L., Kubose T.K. Using usability heuristics to evaluate patient safety of medical devices. J Biomed Inform 2003, 36:23-30.
44. Graham M.J., Kubose T.K., Jordan D.A., Zhang J., Johnson T.R., Patel V.L. Heuristic evaluation of infusion pumps: implications for patient safety in intensive care units. Int J Med Inform 2004, 73:771-779.
45. Allen M., Currie L.M., Bakken S., Patel V.L., Cimino J.J. Heuristic evaluation of paper-based web pages: a simplified inspection usability methodology. J Biomed Inform 2006, 39:412-423.
46. Corbin J.M., Strauss A.L. Basics of qualitative research: techniques and procedures for developing grounded theory 2008, Sage Publications, Inc., Los Angeles, Calif. 3rd ed.
47. Hall J.G., Silva A. A conceptual model for the analysis of mishaps in human-operated safety-critical systems. Saf Sci 2008, 46:22-37.
48. Johnson C.M., Turley J.P. The significance of cognitive modeling in building healthcare interfaces. Int J Med Inform 2006, 75:163-172.
49. Ford E.W., McAlearney A.S., Phillips M.T., Menachemi N., Rudolph B. Predicting computerized physician order entry system adoption in US hospitals: can the federal mandate be met?. Int J Med Inform 2008, 77:539-545.