The electronic patient reported outcome tool: Testing usability and feasibility of a mobile app and portal to support care for patients with complex chronic disease and disability in primary care settings

JMIR mHealth and uHealth

Volumn 4, Issue 2, 2016, Pages

  Gray, Carolyn Steele ^a,b   Gill, Ashlinder ^a   Khan, Anum Irfan ^a   Hans, Parminder Kaur ^b   Kuluski, Kerry ^a,b   Cott, Cheryl ^c  

^a UNIVERSITY OF TORONTO   (Canada)

^b MOUNT SINAI HOSPITAL   (Canada)

^c UNIVERSITY OF TORONTO   (Canada)

Author keywords

eHealth; Feasibility; mHealth; Multimorbidity; Pilot; Primary care; Usability

Indexed keywords

EID: 85019967398     PISSN: None     EISSN: 22915222     Source Type: Journal    
DOI: 10.2196/mhealth.5331     Document Type: Article

References (57)

1. Noël PH, Frueh BC, Larme AC, Pugh JA. Collaborative care needs and preferences of primary care patients with multimorbidity. Health Expect 2005 Mar;8(1):54-63. [doi: 10.1111/j.1369-7625.2004.00312.x] [Medline: 15713171]
2. Marengoni A, Angleman S, Melis R, Mangialasche F, Karp A, Garmen A, et al. Aging with multimorbidity: a systematic review of the literature. Ageing Res Rev 2011 Sep;10(4):430-439. [doi: 10.1016/j.arr.2011.03.003] [Medline: 21402176]
3. Bayliss EA, Bosworth HB, Noel PH, Wolff JL, Damush TM, Mciver L. Supporting self-management for patients with complex medical needs: recommendations of a working group. Chronic Illn 2007 Jun;3(2):167-175. [doi: 10.1177/1742395307081501] [Medline: 18083671]
4. Canadian Institute for Health Information. Primary Health Care. Seniors and the Health Care System: What Is the Impact of Multiple Chronic ConditionsURL:https://www.cihi.ca/en/info_phc_chronic_seniors_en.pdf [accessed 2015-11-06] [WebCite Cache ID 6cqDy149T]
5. Boyd CM, Darer J, Boult C, Fried LP, Boult L, Wu AW. Clinical practice guidelines and quality of care for older patients with multiple comorbid diseases: implications for pay for performance. JAMA 2005 Aug 10;294(6):716-724. [doi: 10.1001/jama.294.6.716] [Medline: 16091574]
6. Upshur Ross E G. Do clinical guidelines still make sense? No. Ann Fam Med 2014;12(3):202-203 [FREE Full text] [doi: 10.1370/afm.1654] [Medline: 24821890]
7. Institute of Medicine (US) Committee on Quality of Health Care in Amercia. Crossing the Quality Chasm: A New Health System for the 21st Century. National Academy of Sciences 2001 Mar:1-8. [Medline: 25057539]
8. Boyd CM, Fortin M. Future of Multimorbidity Research: How Should Understanding of Multimorbidity Inform Health System Design? Public Health Reviews. 2010. (2) p. 451-474 URL:http://www.publichealthreviews.eu/upload/pdf_files/ 8/PHR_32_2_Boyd.pdf[WebCite Cache ID 6hEgT4wT5]
9. Upshur C, Weinreb L. A survey of primary care provider attitudes and behaviors regarding treatment of adult depression: what changes after a collaborative care intervention? Prim Care Companion J Clin Psychiatry 2008;10(3):182-186 [FREE Full text] [Medline: 18615167]
10. Sinnott C, McHugh S, Browne J, Bradley C. GPs' perspectives on the management of patients with multimorbidity: systematic review and synthesis of qualitative research. BMJ Open 2013;3(9):e003610 [FREE Full text] [doi: 10.1136/bmjopen-2013-003610] [Medline: 24038011]
11. Reuben DB, Tinetti ME. Goal-oriented patient care - an alternative health outcomes paradigm. N Engl J Med 2012 Mar 1;366(9):777-779. [doi: 10.1056/NEJMp1113631] [Medline: 22375966]
12. Kuluski K, Gill A, Naganathan G, Upshur R, Jaakkimainen RL, Wodchis WP. A qualitative descriptive study on the alignment of care goals between older persons with multi-morbidities, their family physicians and informal caregivers. BMC Fam Pract 2013;14:133 [FREE Full text] [doi: 10.1186/1471-2296-14-133] [Medline: 24010523]
13. Ekdahl AW, Hellström I, Andersson L, Friedrichsen M. Too complex and time-consuming to fit in! Physicians' experiences of elderly patients and their participation in medical decision making: a grounded theory study. BMJ Open 2012;2(3):e001063 [FREE Full text] [doi: 10.1136/bmjopen-2012-001063]
14. McLean S, Sheikh A. Does telehealthcare offer a patient-centred way forward for the community-based management of long-term respiratory disease? Prim Care Respir J 2009 Sep;18(3):125-126 [FREE Full text] [doi: 10.3132/pcrj.2009.00006] [Medline: 19159046]
15. Appelboom G, Yang AH, Christophe BR, Bruce EM, Slomian J, Bruyère O, et al. The promise of wearable activity sensors to define patient recovery. J Clin Neurosci 2014 Jul;21(7):1089-1093. [doi: 10.1016/j.jocn.2013.12.003] [Medline: 24534628]
16. Zheng H, Nugent C, McCullagh P, Huang Y, Zhang S, Burns W, et al. Smart self management: assistive technology to support people with chronic disease. J Telemed Telecare 2010;16(4):224-227. [doi: 10.1258/jtt.2010.004017] [Medline: 20511581]
17. Verwey R, van der Weegen S, Spreeuwenberg M, Tange H, van der Weijden T, de WL. Technology combined with a counseling protocol to stimulate physical activity of chronically ill patients in primary care. Stud Health Technol Inform 2014;201:264-270. [Medline: 24943553]
18. Ricciardi L, Mostashari F, Murphy J, Daniel JG, Siminerio EP. A national action plan to support consumer engagement via e-health. Health Aff (Millwood) 2013 Feb;32(2):376-384. [doi: 10.1377/hlthaff.2012.1216] [Medline: 23381531]
19. Strecher V. Internet methods for delivering behavioral and health-related interventions (eHealth). Annu Rev Clin Psychol 2007;3:53-76. [doi: 10.1146/annurev.clinpsy.3.022806.091428] [Medline: 17716048]
20. van der Weegen Sanne, Verwey R, Spreeuwenberg M, Tange H, van der Weijden Trudy, de WL. The development of a mobile monitoring and feedback tool to stimulate physical activity of people with a chronic disease in primary care: a user-centered design. JMIR Mhealth Uhealth 2013;1(2):e8 [FREE Full text] [doi: 10.2196/mhealth.2526] [Medline: 25099556]
21. Becker S, Miron-Shatz T, Schumacher N, Krocza J, Diamantidis C, Albrecht U. mHealth 2.0: Experiences, Possibilities, and Perspectives. JMIR Mhealth Uhealth 2014;2(2):e24 [FREE Full text] [doi: 10.2196/mhealth.3328] [Medline: 25099752]
22. QoC Health. Who we areURL:http://qochealth.com/who-we-are/ [accessed 2016-01-19] [WebCite Cache ID 6eelxXIPw]
23. Devi K, Sen A, Hemachandran K. A working Framework for the User-Centered Design Approach and a Survey of the available Methods. International Journal of Scientific and Research Publications 2012 Apr;2(4):1-8.
24. Vermeulen J, Neyens JC, Spreeuwenberg MD, van RE, Sipers W, Habets H, de Witte Luc P. User-centered development and testing of a monitoring system that provides feedback regarding physical functioning to elderly people. Patient Prefer Adherence 2013 Aug;7:843-854 [FREE Full text] [doi: 10.2147/PPA.S45897] [Medline: 24039407]
25. Sánchez-Morillo D, Crespo M, León A, Crespo Foix Luis F. A novel multimodal tool for telemonitoring patients with COPD. Inform Health Soc Care 2015 Jan;40(1):1-22. [doi: 10.3109/17538157.2013.872114] [Medline: 24380372]
26. Tsiknakis M, Kouroubali A. Organizational factors affecting successful adoption of innovative eHealth services: a case study employing the FITT framework. Int J Med Inform 2009 Jan;78(1):39-52. [doi: 10.1016/j.ijmedinf.2008.07.001] [Medline: 18723389]
27. Sheehan B, Lee Y, Rodriguez M, Tiase V, Schnall R. A comparison of usability factors of four mobile devices for accessing healthcare information by adolescents. Appl Clin Inform 2012;3(4):356-366 [FREE Full text] [doi: 10.4338/ACI-2012-06-RA-0021] [Medline: 23227134]
28. Yang C, Linas B, Kirk G, Bollinger R, Chang L, Chander G, et al. Feasibility and Acceptability of Smartphone-Based Ecological Momentary Assessment of Alcohol Use Among African American Men Who Have Sex With Men in Baltimore. JMIR Mhealth Uhealth 2015;3(2):e67 [FREE Full text] [doi: 10.2196/mhealth.4344] [Medline: 26085078]
29. Brown W, Yen P, Rojas M, Schnall R. Assessment of the Health IT Usability Evaluation Model (Health-ITUEM) for evaluating mobile health (mHealth) technology. J Biomed Inform 2013 Dec;46(6):1080-1087 [FREE Full text] [doi: 10.1016/j.jbi.2013.08.001] [Medline: 23973872]
30. Al Ayubi Soleh U, Parmanto B, Branch R, Ding D. A Persuasive and Social mHealth Application for Physical Activity: A Usability and Feasibility Study. JMIR Mhealth Uhealth 2014;2(2):e25 [FREE Full text] [doi: 10.2196/mhealth.2902] [Medline: 25099928]
31. Bakhshi-Raiez F, de Keizer N F, Cornet R, Dorrepaal M, Dongelmans D, Jaspers M W M. A usability evaluation of a SNOMED CT based compositional interface terminology for intensive care. Int J Med Inform 2012 May;81(5):351-362. [doi: 10.1016/j.ijmedinf.2011.09.010] [Medline: 22030036]
32. Jia G, Zhou J, Yang P, Lin C, Cao X, Hu H, et al. Integration of user centered design in the development of health monitoring system for elderly. Conf Proc IEEE Eng Med Biol Soc 2013;2013:1748-1751. [doi: 10.1109/EMBC.2013.6609858] [Medline: 24110045]
33. Tjora A, Tran T, Faxvaag A. Privacy vs usability: a qualitative exploration of patients' experiences with secure Internet communication with their general practitioner. J Med Internet Res 2005;7(2):e15 [FREE Full text] [doi: 10.2196/jmir.7.2.e15] [Medline: 15998606]
34. Steele GC, Miller D, Kuluski K, Cott C. Tying eHealth Tools to Patient Needs: Exploring the Use of eHealth for Community-Dwelling Patients With Complex Chronic Disease and Disability. JMIR Res Protoc 2014 Nov;3(4):e67 [FREE Full text] [doi: 10.2196/resprot.3500] [Medline: 25428028]
35. Glazier R, Zagorski B, Rayner J. Institute for Clinical Evaluative Sciences. 2012. Comparison of Primary Care Models in Ontario by Demographics, Case Mix and Emergency Department Use, 2008/09 to 2009/10URL:http://www.ices.on.ca/ Publications/Atlases-and-Reports/2012/Comparison-of-Primary-Care-Models [accessed 2015-11-06] [WebCite Cache ID 6cqYaUd0G]
36. Steele GC, Khan A, Kuluski K, McKillop I, Sharpe S, Bierman A, et al. Improving Patient Experience and Primary Care Quality for Patients With Complex Chronic Disease Using the Electronic Patient-Reported Outcomes Tool: Adopting Qualitative Methods Into a User-Centered Design Approach. JMIR Res Protoc 2016;5(1):e28 [FREE Full text] [doi: 10.2196/resprot.5204] [Medline: 26892952]
37. Cella D, Riley W, Stone A, Rothrock N, Reeve B, Yount S, et al. The Patient-Reported Outcomes Measurement Information System (PROMIS) developed and tested its first wave of adult self-reported health outcome item banks: 2005-2008. J Clin Epidemiol 2010 Nov;63(11):1179-1194 [FREE Full text] [doi: 10.1016/j.jclinepi.2010.04.011] [Medline: 20685078]
38. Frei A, Svarin A, Steurer-Stey C, Puhan MA. Self-efficacy instruments for patients with chronic diseases suffer from methodological limitations - a systematic review. Health Qual Life Outcomes 2009;7:86 [FREE Full text] [doi: 10.1186/1477-7525-7-86] [Medline: 19781095]
39. Broderick JE, Schneider S, Junghaenel DU, Schwartz JE, Stone AA. Validity and reliability of patient-reported outcomes measurement information system instruments in osteoarthritis. Arthritis Care Res (Hoboken) 2013 Oct;65(10):1625-1633 [FREE Full text] [doi: 10.1002/acr.22025] [Medline: 23592494]
40. Green J, Thorogood N. Qualitative Methods for Health Research (Introducing Qualitative Methods series). 2nd edition. London, UK: SAGE Publications Ltd; 2009.
41. QSR International Ply Ltd. 2012. NVivo qualitative data analysis software, Version 10URL:http://www.qsrinternational.com/ product [accessed 2016-04-25] [WebCite Cache ID 6h1lX378S]
42. O'Kane MJ, Bunting B, Copeland M, Coates VE. Efficacy of self monitoring of blood glucose in patients with newly diagnosed type 2 diabetes (ESMON study): randomised controlled trial. BMJ 2008 May 24;336(7654):1174-1177 [FREE Full text] [doi: 10.1136/bmj.39534.571644.BE] [Medline: 18420662]
43. Bovend'Eerdt Thamar J H, Botell RE, Wade DT. Writing SMART rehabilitation goals and achieving goal attainment scaling: a practical guide. Clin Rehabil 2009 Apr;23(4):352-361. [doi: 10.1177/0269215508101741] [Medline: 19237435]
44. Miller D, Steele Gray C, Kuluski K, Cott C. Patient-Centered Care and Patient-Reported Measures: Let's Look Before We Leap. Patient 2015 Oct;8(4):293-299 [FREE Full text] [doi: 10.1007/s40271-014-0095-7] [Medline: 25354873]
45. Medina-Mora R, Winograd T, Flores R, Flores F. The action workflow approach to workflow management technology. 1992 Presented at: Proceedings of the 1992 ACM conference on Computer-supported cooperative work; November 01 - 04, 1992; Toronto, Ontario, Canada p. 281-288 URL:https://www.lri.fr/~mbl/ENS/CSCW/material/papers/ MedinaMora-CSCW92.pdf[WebCite Cache ID 6hcmksawY]
46. Schaink A, Kuluski K, Lyons R, Fortin M, Jadad A, Upshur R, et al. A scoping review and thematic classification of patient complexity: Offering a unifying framework. Journal of Comorbidity 2012;2(1):1-9.
47. Pappas C, Coscia E, Dodero G, Gianuzzi V, Earney M. A mobile e-health system based on workflow automation tools. 2002 Jun 07 Presented at: Proceedings of the 15th IEEE Symposium Computer-Based Medical Systems; 2002; Maribor, Slovenia p. 271-276 URL:http://doi.ieeecomputersociety.org/10.1109/CBMS.2002.1011388[WebCite Cache ID 6hcmuFxUM]
48. Tsasis P, Bains J. Management of complex chronic disease: facing the challenges in the Canadian health-care system. Health Serv Manage Res 2008 Nov;21(4):228-235. [doi: 10.1258/hsmr.2008.008001] [Medline: 18957400]
49. Yu P, Yu H. Lessons learned from the practice of mobile health application development. 2004 Sep 28 Presented at: Proceedings of the 28th IEEE Annual International Computer Software and Applications Conference ;2; 2004; Hong Kong p. 58-59 URL:http://www.computer.org/csdl/proceedings/compsac/2004/2209/02/220920058.pdf[WebCite Cache ID 6hcmYamR7] [doi: 10.1109/CMPSAC.2004.1342675]
50. Abras C, Maloney-Krichmar D, Preece J. User-centered design. In: Bainbridge W, editor. Encyclopedia of Human-Computer Interaction. Thousand Oaks, California: Sage Publications; 2004.
51. Marti P, Bannon L. Exploring user-centred design in practice: Some Caveats. Know Techn Pol 2009;22:7-15. [doi: 10.1007/x12130-009-9062-3]
52. Kaufman D, Starren J, Patel V, Morin P, Hilliman C, Pevzner J, et al. A cognitive framework for understanding barriers to the productive use of a diabetes home telemedicine system. AMIA Annu Symp Proc 2003:356-360 [FREE Full text] [Medline: 14728194]
53. Nielsen J, Landauer T. A mathematical model of the finding of usability problems. 1993 Apr 24-29 Presented at: Proceedings of the INTERACT'93 and CHI'93 conference on Human factors in computing systems ACM; 1993; Amsterdam, Netherlands p. 206-213 URL:http://people.cs.uct.ac.za/~dnunez/reading/papers/p206-nielsen.pdf[WebCite Cache ID 6hcmdA2tf]
54. Connelly K, Siek KA, Chaudry B, Jones J, Astroth K, Welch JL. An offline mobile nutrition monitoring intervention for varying-literacy patients receiving hemodialysis: a pilot study examining usage and usability. J Am Med Inform Assoc 2012;19(5):705-712 [FREE Full text] [doi: 10.1136/amiajnl-2011-000732] [Medline: 22582206]
55. Knight-Agarwal C, Davis DL, Williams L, Davey R, Cox R, Clarke A. Development and Pilot Testing of the Eating4two Mobile Phone App to Monitor Gestational Weight Gain. JMIR Mhealth Uhealth 2015;3(2):e44 [FREE Full text] [doi: 10.2196/mhealth.4071] [Medline: 26048313]
56. Zmily A, Mowafi Y, Mashal E. Study of the usability of spaced retrieval exercise using mobile devices for Alzheimer's disease rehabilitation. JMIR Mhealth Uhealth 2014;2(3):e31 [FREE Full text] [doi: 10.2196/mhealth.3136] [Medline: 25124077]
57. Mirkovic J, Kaufman DR, Ruland CM. Supporting cancer patients in illness management: usability evaluation of a mobile app. JMIR Mhealth Uhealth 2014;2(3):e33 [FREE Full text] [doi: 10.2196/mhealth.3359] [Medline: 25119490]