An evaluation of patients' experienced usability of a diabetes mHealth system using a multi-method approach

Journal of Biomedical Informatics

Volumn 59, Issue , 2016, Pages 115-129

  Georgsson, Mattias ^a,b   Staggers, Nancy ^a,c  

^a UNIVERSITY OF UTAH SCHOOL OF MEDICINE   (United States)

^b BLEKINGE INSTITUTE OF TECHNOLOGY   (Sweden)

^c UNIVERSITY OF UTAH   (United States)

Author keywords

Diabetes; Framework analysis; MHealth; Multi method evaluation; Usability; Usability Problem Taxonomy

Indexed keywords

BLOOD PRESSURE; DATA ACQUISITION; GLUCOSE; INFORMATION MANAGEMENT; MEDICAL PROBLEMS; RATING; SURVEYS; TAXONOMIES; USABILITY ENGINEERING;

FRAMEWORK ANALYSIS; MHEALTH; MULTI METHODS; USABILITY; USABILITY PROBLEMS;

PROBLEM SOLVING;

ADULT; AGED; ARTICLE; CLINICAL ARTICLE; CONTROLLED STUDY; DIASTOLIC BLOOD PRESSURE; FEASIBILITY STUDY; FEMALE; HEALTH CARE SYSTEM; HUMAN; MALE; MEDICAL INFORMATICS; NON INSULIN DEPENDENT DIABETES MELLITUS; PERSONAL EXPERIENCE; PRIORITY JOURNAL; QUESTIONNAIRE; RATING SCALE; SELF CARE; SEMI STRUCTURED INTERVIEW; SYSTOLIC BLOOD PRESSURE; DIABETES MELLITUS, TYPE 2; MIDDLE AGED; PATIENT SATISFACTION; STATISTICS AND NUMERICAL DATA; TELEMEDICINE; UTILIZATION;

ADULT; AGED; DIABETES MELLITUS, TYPE 2; FEMALE; HUMANS; MALE; MIDDLE AGED; PATIENT SATISFACTION; SURVEYS AND QUESTIONNAIRES; TELEMEDICINE;

EID: 84961575597     PISSN: 15320464     EISSN: None     Source Type: Journal    
DOI: 10.1016/j.jbi.2015.11.008     Document Type: Article

References (68)

1. Centers for Disease Control and Prevention, National Diabetes Statistics Report: Estimates of Diabetes and Its Burden in the United States, Department of Health and Human Services, Atlanta, GA, U.S., 2014.
2. Zhang P., et al. Global healthcare expenditure on diabetes for 2010 and 2030. Diab. Res. Clin. Pract. 2010, 87(3):293-301.
3. World Health Organization (WHO), Diabetes Fact Sheet N°312. 2015 January 2015; Available from: [cited 06 Sept 2015]. http://www.who.int/mediacentre/factsheets/fs312/en/.
4. The Diabetes Control and Complications Trial Research Group The effect of intensive treatment of diabetes on the development and progression of long-term complications in insulin-dependent diabetes mellitus. The diabetes control and complications trial research group. N. Engl. J. Med. 1993, 329(14):977-986.
5. Funnell M.M., et al. National standards for diabetes self-management education. Diab. Care 2011, 34(Suppl. 1):S89-S96.
6. Funnell M.M., Anderson R.M. The problem with compliance in diabetes. JAMA 2000, 284(13):1709.
7. Sutcliffe P., et al. Systematic review of communication technologies to promote access and engagement of young people with diabetes into healthcare. BMC Endocr. Disord. 2011, 11:1.
8. Chung Y.S., Kim Y., Lee C.H. Effectiveness of the smart care service for diabetes management. Healthc. Inform. Res. 2014, 20(4):288-294.
9. Liang X., et al. Effect of mobile phone intervention for diabetes on glycaemic control: a meta-analysis. Diab. Med. 2011, 28(4):455-463.
10. Holtz B., Lauckner C. Diabetes management via mobile phones: a systematic review. Telemed. J. E Health 2012, 18(3):175-184.
11. Waki K., et al. DialBetics: a novel smartphone-based self-management support system for type 2 diabetes patients. J. Diab. Sci. Technol. 2014, 8(2):209-215.
12. Mulvaney S.A., et al. A pilot test of a tailored mobile and web-based diabetes messaging system for adolescents. J. Telemed. Telecare 2012, 18(2):115-118.
13. Fioravanti A., et al. Automatic messaging for improving patients engagement in diabetes management: an exploratory study. Med. Biol. Eng. Comput. 2015, 53(12):1285-1294.
14. Arsand E., et al. Mobile phone-based self-management tools for type 2 diabetes: the few touch application. J. Diab. Sci. Technol. 2010, 4(2):328-336.
15. El-Gayar O., et al. Mobile applications for diabetes self-management: status and potential. J. Diab. Sci. Technol. 2013, 7(1):247-262.
16. Wolpin S., Stewart M. A deliberate and rigorous approach to development of patient-centered technologies. Semin. Oncol. Nurs. 2011, 27(3):183-191.
17. Lyles C.R., Sarkar U., Osborn C.Y. Getting a technology-based diabetes intervention ready for prime time: a review of usability testing studies. Curr. Diab. Rep. 2014, 14(10):534.
18. Zapata B.C., et al. Empirical studies on usability of mHealth apps: a systematic literature review. J. Med. Syst. 2015, 39(2):182.
19. Green C.A., et al. Approaches to mixed methods dissemination and implementation research: methods, strengths, caveats, and opportunities. Adm. Policy Ment. Health 2014.
20. Patton M.Q. Enhancing the quality and credibility of qualitative analysis. Health Serv. Res. 1999, 34(5 Pt 2):1189-1208.
21. Polit D.F. Nursing Research: Generating and Assessing Evidence for Nursing Practice 2012, Wolters Kluwer Health/Lippincott Williams & Wilkins, Philadelphia. 9th ed.
22. Patton M.Q. Qualitative Research & Evaluation Methods: Integrating Theory and Practice: the Definitive Text of Qualitative Inquiry Frameworks and Options 2015, SAGE Publications, Inc., Thousand Oaks, California.
23. Elo S., Kyngas H. The qualitative content analysis process. J. Adv. Nurs. 2008, 62(1):107-115.
24. A.W. Kushniruk, V.L. Patel, J.J. Cimino, Usability testing in medical informatics: cognitive approaches to evaluation of information systems and user interfaces, in: Proc AMIA Annu Fall Symp, 1997, pp. 218-222.
25. Cohen D.J., Crabtree B.F. Evaluative criteria for qualitative research in health care: controversies and recommendations. Ann. Fam. Med. 2008, 6(4):331-339.
26. J. Ritchie, L. Spencer, Qualitative data analysis for applied policy research, in: A. Bryman, R.G. Burgess (Eds.), Analyzing Qualitative Data, 1994, pp. 173-194.
27. Ritchie J., Lewis J.R. Qualitative Research Practice: A Guide for Social Science Students and Researchers 2003, SAGE, London.
28. Ward D.J., et al. Using framework analysis in nursing research: a worked example. J. Adv. Nurs. 2013, 69(11):2423-2431.
29. Gale N.K., et al. Using the framework method for the analysis of qualitative data in multi-disciplinary health research. BMC Med. Res. Methodol. 2013, 13:117.
30. Keenan S., et al. The usability problem taxonomy: a framework for classification and analysis. Empirical Softw. Eng. 1999, 4(1):71-104.
31. Jeffries R. Usability problem reports: helping evaluators communicate effectively with developers. Usability Inspection Methods 1994, 273-294. John Wiley & Sons Inc., New York. R.L. Mack, J. Nielsen (Eds.).
32. Dumas J.S., Redish J. A Practical Guide to Usability Testing 1999, vol. 2. Intellect Books, Exeter, England ; Portland, Or., 404p. Rev. ed.
33. Nielsen J., Molich R. Heuristic evaluation of user interfaces. Proceedings of the SIGCHI Conference on Human Factors in Computing Systems: Empowering People 1990, 249-256. ACM, Seattle, Washington, United States.
34. Nielsen J. Finding usability problems through heuristic evaluation. Proceedings of the SIGCHI Conference on Human Factors in Computing Systems 1992, 373-380. ACM, Monterey, California, United States.
35. Keenan S.L. Product Usability and Process Improvement Based on Usability Problem Classification 1996, Virginia Polytechnic Institute and State University.
36. Carroll J.M., Kellogg W.A., Rosson M.B. The Task-artifact Cycle. Designing Interaction: Psychology at the Human-Computer Interface 1991, 74-102. Cambridge University Press, New York. M. Carrol (Ed.).
37. Shneiderman B. Designing the User Interface: Strategies for Effective Human-Computer Interaction 1998, Addison Wesley Longman, Reading, Mass.
38. Capozza K., et al. Going mobile with diabetes support: a randomized study of a text message-based personalized behavioral intervention for type 2 diabetes self-care. Diab. Spectr. 2015, 28(2):83-91.
39. Nielsen J. Usability Engineering 1993, xiv. Academic Press, Boston, 358p.
40. Nielsen J. Heuristic evaluation. Usability Inspection Methods 1994, 25-62. John Wiley & Sons Inc., New York. R.L. Mack, J. Nielsen (Eds.).
41. Polson P.G., et al. Cognitive walkthroughs - a method for theory-based evaluation of user interfaces. Int. J. Man Mach. Stud. 1992, 36(5):741-773.
42. Wharton C., et al. The cognitive walkthrough method: a practitioner's guide. Usability Inspection Methods 1994, 105-140. John Wiley & Sons, Inc., New York. J. Nielsen, R.L. Mack (Eds.).
43. Ericsson K.A., Simon H.A. Protocol Analysis: Verbal Reports as Data 1984, 426. MIT Press, Cambridge, Mass.
44. Lewis C., Rieman J. Task-centered User Interface Design: A Practical Introduction 1993, University of Colorado, Boulder, Department of Computer Science.
45. Shah S.G., Robinson I. Benefits of and barriers to involving users in medical device technology development and evaluation. Int. J. Technol. Assess Health Care 2007, 23(1):131-137.
46. Yardley L., et al. Understanding reactions to an internet-delivered health-care intervention: accommodating user preferences for information provision. BMC Med. Inform. Decis. Mak. 2010, 10:52.
47. Dobson K.G., Hall P. A pilot study examining patient attitudes and intentions to adopt assistive technologies into type 2 diabetes self-management. J. Diab. Sci. Technol. 2014, 9(2):309-315.
48. Ericsson K.A., Simon H.A. Protocol Analysis: Verbal Reports as Data Revised Edition 1993, 'The' MIT Press.
49. Kushniruk A.W., Patel V.L. Cognitive and usability engineering methods for the evaluation of clinical information systems. J. Biomed. Inform. 2004, 37(1):56-76.
50. Nielsen J., Landauer T.K. A mathematical model of the finding of usability problems. Proceedings of the INTERACT '93 and CHI '93 Conference on Human Factors in Computing Systems 1993, 206-213. ACM, Amsterdam, The Netherlands.
51. Virzi R.A. Refining the test phase of usability evaluation: how many subjects is enough?. Hum. Factors 1992, 34(4):457-468.
52. Monk A., et al. Improving your human-computer interface: a practical technique. BCS Practitioner Series 1993, 99. Prentice-Hall, London.
53. Nielsen J., Mack R.L. Usability Inspection Methods 1994, xxiv. John Wiley & Sons, Inc., New York, 413p.
54. Nielsen Norman Group, Turn User Goals into Task Scenarios for Usability Testing, 2014 Jan 12; Available from: [cited 25 May 2015]. http://www.nngroup.com/articles/task-scenarios-usability-testing/.
55. Kushniruk A.W., et al. Emerging approaches to usability evaluation of health information systems: towards in-situ analysis of complex healthcare systems and environments. Stud. Health Technol. Inform. 2011, 169:915-919.
56. Haak M.J.V.D., Jong M.D.T.D., Schellens P.J. Retrospective vs. concurrent think-aloud protocols: testing the usability of an online library catalogue. Behav. Inform. Technol. 2003, 22(5):339-351.
57. Maguire M. Methods to support human-centred design. Int. J. Hum Comput Stud. 2001, 55(4):587-634.
58. Kvale S. InterViews : Learning the Craft of Qualitative Research Interviewing 2009, Sage Publications, Los Angeles. second ed. S. Brinkmann (Ed.).
59. Karsh B.T., et al. Health information technology: fallacies and sober realities. J. Am. Med. Inform. Assoc. 2010, 17(6):617-623.
60. TechSmith Corporation Morae Usability Testing and Market Research Software 2015, TechSmith Corporation, Okemos, MI, USA.
61. QSR International Pty Ltd NVivo 10 Research Software for Analysis and Insight 2015, QSR International Pty Ltd., Doncaster, Victoria, Australia.
62. D. Travis, How to Prioritise Usability Problems, 2009 October 15; Available from: [cited 2015 August 26]. http://www.userfocus.co.uk/articles/prioritise.html.
63. Protheroe J., et al. An intervention to promote patient participation and self-management in long term conditions: development and feasibility testing. BMC Health Serv. Res. 2010, 10:206.
64. Kirwan M., et al. Design, development, and formative evaluation of a smartphone application for recording and monitoring physical activity levels: the 10,000 Steps "iStepLog". Health Educ. Behav. 2013, 40(2):140-151.
65. Atack L., Luke R., Chien E. Evaluation of patient satisfaction with tailored online patient education information. Comput. Inform. Nurs. 2008, 26(5):258-264.
66. Horsky J., et al. Complementary methods of system usability evaluation: surveys and observations during software design and development cycles. J. Biomed. Inform. 2010, 43(5):782-790.
67. P.Y. Yen, S. Bakken, A comparison of usability evaluation methods: heuristic evaluation versus end-user think-aloud protocol - an example from a web-based communication tool for nurse scheduling, in: AMIA Annu Symp Proc, 2009, pp. 714-718.
68. Jeffries R., et al. User interface evaluation in the real world: a comparison of four techniques. Proceedings of the SIGCHI Conference on Human Factors in Computing Systems 1991, 119-124. ACM, New Orleans, Louisiana, USA.