Exploring user learnability and learning performance in an app for depression: Usability study

JMIR Human Factors

Volumn 4, Issue 3, 2017, Pages

  Stiles Shields, Colleen ^a,b   Montague, Enid ^a,c   Lattie, Emily G ^a   Schueller, Stephen M ^a   Kwasny, Mary J ^a   Mohr, David C ^a  

^a NORTHWESTERN UNIVERSITY   (United States)

^b UNIVERSITY OF CHICAGO MEDICAL CENTER   (United States)

^c DEPAUL UNIVERSITY   (United States)

Author keywords

Apps; Cognitive therapy; Depression; Learning; Usability testing

Indexed keywords

EID: 85041090461     PISSN: None     EISSN: 22929495     Source Type: Journal    
DOI: 10.2196/humanfactors.7951     Document Type: Article

References (46)

1. Krebs P, Duncan DT. Health app use among US mobile phone owners: a national survey. JMIR Mhealth Uhealth 2015;3(4):e101 [FREE Full text] [doi: 10.2196/mhealth.4924] [Medline: 26537656]
2. Torous J, Friedman R, Keshavan M. Smartphone ownership and interest in mobile applications to monitor symptoms of mental health conditions. JMIR Mhealth Uhealth 2014;2(1):e2 [FREE Full text] [doi: 10.2196/mhealth.2994] [Medline: 25098314]
3. Smith A. Pew Research Center. 2015 Apr 01. U.S. smartphone use in 2015 URL:http://www.pewinternet.org/2015/04/01/ us-smartphone-use-in-2015/#[WebCite Cache ID 6q5fRT18J]
4. Gandhi M, Wang T. Rockhealth. 2015. Digital health consumer adoption URL:https://rockhealth.com/reports/ digital-health-consumer-adoption-2015 [accessed 2017-04-30] [WebCite Cache ID 6q7RFINOC]
5. Shen N, Levitan M, Johnson A, Bender JL, Hamilton-Page M, Jadad AA, et al. Finding a depression app: a review and content analysis of the depression app marketplace. JMIR Mhealth Uhealth 2015;3(1):e16 [FREE Full text] [doi: 10.2196/mhealth.3713] [Medline: 25689790]
6. Torous J, Powell AC. Current research and trends in the use of smartphone applications for mood disorders. Internet Interv 2015 May;2(2):169-173. [doi: 10.1016/j.invent.2015.03.002]
7. Mohr DC, Burns MN, Schueller SM, Clarke G, Klinkman M. Behavioral intervention technologies: evidence review and recommendations for future research in mental health. Gen Hosp Psychiatry 2013 Aug;35(4):332-338 [FREE Full text] [doi: 10.1016/j.genhosppsych.2013.03.008] [Medline: 23664503]
8. IMS Institute for Healthcare Informatics. Imshealth. 2015. Patient adoption of mhealth: use, evidence and remaining barriers to mainstream acceptance URL:http://www.imshealth.com/files/web/IMSH%20Institute/Reports/ Patient%20Adoption%20of%20mHealth/IIHI_Patient_Adoption_of_mHealth.pdf [accessed 2017-04-29] [WebCite Cache ID 6q5g42rbW]
9. CBITs. Google Play Store. Thought challenger URL:https://play.google.com/store/apps/details?id=edu.northwestern.cbits. intellicare.thoughtchallenger&hl=en[WebCite Cache ID 6q5gC5iwf]
10. Lattie EG, Schueller SM, Sargent E, Stiles-Shields C, Tomasino KN, Corden ME, et al. Uptake and usage of IntelliCare: a publicly available suite of mental health and well-being apps. Internet Interv 2016 May;4(2):152-158 [FREE Full text] [doi: 10.1016/j.invent.2016.06.003] [Medline: 27398319]
11. Mohr DC, Tomasino KN, Lattie EG, Palac HL, Kwasny MJ, Weingardt K, et al. IntelliCare: an eclectic, skills-based app suite for the treatment of depression and anxiety. J Med Internet Res 2017 Jan 05;19(1):e10 [FREE Full text] [doi: 10.2196/jmir.6645] [Medline: 28057609]
12. Beck J. Cognitive Behavior Therapy: Basics and Beyond, Second Edition. New York: The Guilford Press; 2011.
13. Barber JP, DeRubeis RJ. On second thought: where the action is in cognitive therapy for depression. Cogn Ther Res 1989 Oct;13(5):441-457. [doi: 10.1007/bf01173905]
14. Hundt NE, Mignogna J, Underhill C, Cully JA. The relationship between use of CBT skills and depression treatment outcome: a theoretical and methodological review of the literature. Behav Ther 2013 Mar;44(1):12-26. [doi: 10.1016/j.beth.2012.10.001] [Medline: 23312423]
15. Hekler EB, Klasnja P, Riley WT, Buman MP, Huberty J, Rivera DE, et al. Agile science: creating useful products for behavior change in the real world. Transl Behav Med 2016 Jun;6(2):317-328 [FREE Full text] [doi: 10.1007/s13142-016-0395-7] [Medline: 27357001]
16. Wyatt JC. Evidence-based health informatics and the scientific development of the field. In: Ammenwerth E, Rigby M, editors. Studies in Health Technology and Informatics. Amsterdam, Netherlands: IOS Press; 2016:14-24.
17. Mohr DC, Schueller SM, Riley WT, Brown CH, Cuijpers P, Duan N, et al. Trials of intervention principles: evaluation methods for evolving behavioral intervention technologies. J Med Internet Res 2015;17(7):e166 [FREE Full text] [doi: 10.2196/jmir.4391] [Medline: 26155878]
18. Usability. 2017. Usability testing URL:https://www.usability.gov/how-to-and-tools/methods/usability-testing.html [accessed 2017-04-30] [WebCite Cache ID 6q7Rc6dUs]
19. Tullis T, Albert B. Measuring the user experience: Collecting, analyzing, and presenting usability metrics, second edition. Burlington, MA: Morgan Kaufmann; 2008.
20. Ben-Zeev D, Kaiser SM, Brenner CJ, Begale M, Duffecy J, Mohr DC. Development and usability testing of FOCUS: a smartphone system for self-management of schizophrenia. Psychiatr Rehabil J 2013 Dec;36(4):289-296 [FREE Full text] [doi: 10.1037/prj0000019] [Medline: 24015913]
21. Mohr DC, Stiles-Shields C, Brenner C, Palac H, Montague E, Kaiser SM, et al. MedLink: a mobile intervention to address failure points in the treatment of depression in general medicine. Int Conf Pervasive Comput Technol Healthc 2015 May;2015:100-107 [FREE Full text] [Medline: 26640740]
22. Vilardaga R, Rizo J, Kientz JA, McDonell MG, Ries RK, Sobel K. User experience evaluation of a smoking cessation app in people with serious mental illness. Nicotine Tob Res 2016 May;18(5):1032-1038. [doi: 10.1093/ntr/ntv256] [Medline: 26581430]
23. Carroll JM, Rosson M. The paradox of the active user. In: Carroll JM, editor. Interfacing thought: Cognitive aspects of human-computer interaction. Cambridge, MA: MIT Press; 1987:80-111.
24. Koole ML. A model for framing mobile learning. In: Ally M, editor. Mobile learning: Transforming the delivery of education and training. Edmonton, AB: AU Press; 2009:25-47.
25. Harley A. NN Group. 2014. Instructional overlays and coach marks for mobile apps URL:https://www.nngroup.com/articles/ mobile-instructional-overlay/ [accessed 2017-04-30] [WebCite Cache ID 6q7RuSDFB]
26. Schueller SM, Washburn JJ, Price M. Exploring mental health providers' interest in using web and mobile-based tools in their practices. Internet Interv 2016 May;4(2):145-151. [doi: 10.1016/j.invent.2016.06.004] [Medline: 28090438]
27. Nielsen J. Usability engineering. Boston: Academic Press; 1993.
28. Harrison R, Flood D, Duce D. Usability of mobile applications: literature review and rationale for a new usability model. J Interact Sci 2013;1(1):1. [doi: 10.1186/2194-0827-1-1]
29. Flood D, Harrison R, Iacob C, Duce D. Evaluating mobile applications: a spreadsheet case study. Int J Mobile Hum Comput Int 2012;4:37-65. [doi: 10.4018/jmhci.2012100103]
30. Norman D. The Design of Everyday Things. New York: Basic Books; 2002.
31. Sauro J. Measuringu. 2012. Measuring errors in the user experience URL:https://measuringu.com/errors-ux/ [accessed 2017-04-30] [WebCite Cache ID 6q7S5kS6L]
32. Sauro J, Kindlund E. A method to standardize usability metrics into a single score. New York: ACM; 2005 Presented at: SIGCHI Conference on Human factors in Computing Systems; April 2-7, 2005; Portland, Oregon.
33. Rees CS, McEvoy P, Nathan PR. Relationship between homework completion and outcome in cognitive behaviour therapy. Cogn Behav Ther 2005;34(4):242-247. [doi: 10.1080/16506070510011548] [Medline: 16319035]
34. Gumport NB, Williams JJ, Harvey AG. Learning cognitive behavior therapy. J Behav Ther Exp Psychiatry 2015 Sep;48:164-169 [FREE Full text] [doi: 10.1016/j.jbtep.2015.03.015] [Medline: 25898288]
35. Luchini K, Quintana C, Soloway E. Pocket picomap: a case study in designing and assessing a handheld concept mapping tool for learners. New York: ACM; 2003 Presented at: IGCHI Conference on Human Factors in Computing Systems; Apr 5-10, 2003; Ft Lauderdale, FL p. 5-10.
36. Mueller TI, Leon AC, Keller MB, Solomon DA, Endicott J, Coryell W, et al. Recurrence after recovery from major depressive disorder during 15 years of observational follow-up. Am J Psychiatry 1999 Jul;156(7):1000-1006. [doi: 10.1176/ajp.156.7.1000] [Medline: 10401442]
37. Paykel ES. Partial remission, residual symptoms, and relapse in depression. Dialogues Clin Neurosci 2008;10(4):431-437 [FREE Full text] [Medline: 19170400]
38. Judd LL, Paulus MP, Zeller P. The role of residual subthreshold depressive symptoms in early episode relapse in unipolar major depressive disorder. Arch Gen Psychiatry 1999 Aug;56(8):764-765. [Medline: 10435613]
39. The MacArthur Foundation Initiative on Depression and Primary Care. Integration. 2009. Depression management toolkit URL:http://www.integration.samhsa.gov/clinical-practice/macarthur_depression_toolkit.pdf [accessed 2017-04-30] [WebCite Cache ID 6q7TbHXYC]
40. Kroenke K, Spitzer R. The PHQ-9: a new depression diagnostic and severity measure. Psychiatr Ann 2002 Sep 01;32(9):509-515. [doi: 10.3928/0048-5713-20020901-06]
41. Stiles-Shields C, Montague E, Lattie EG, Kwasny MJ, Mohr DC. What might get in the way: barriers to the use of apps for depression. Digit Health 2017 Jun 8;3. [doi: 10.1177/2055207617713827]
42. Zhang D, Adipat B. Challenges, methodologies, and issues in the usability testing of mobile applications. Int J Hum Comput Interact 2005 Jul 1;18(3):293-308. [doi: 10.1207/s15327590ijhc1803_3]
43. Harris PA, Taylor R, Thielke R, Payne J, Gonzalez N, Conde JG. Research electronic data capture (REDCap)-a metadata-driven methodology and workflow process for providing translational research informatics support. J Biomed Inform 2009 Apr;42(2):377-381 [FREE Full text] [doi: 10.1016/j.jbi.2008.08.010] [Medline: 18929686]
44. Wright JH, Wright AS, Salmon P, Beck AT, Kuykendall J, Goldsmith LJ, et al. Development and initial testing of a multimedia program for computer-assisted cognitive therapy. Am J Psychother 2002;56(1):76-86. [Medline: 11977785]
45. Owen JE, Jaworski BK, Kuhn E, Makin-Byrd KN, Ramsey KM, Hoffman JE. mHealth in the wild: using novel data to examine the reach, use, and impact of PTSD coach. JMIR Ment Health 2015;2(1):e7 [FREE Full text] [doi: 10.2196/mental.3935] [Medline: 26543913]
46. Sarkar U, Gourley GI, Lyles CR, Tieu L, Clarity C, Newmark L, et al. Usability of commercially available mobile applications for diverse patients. J Gen Intern Med 2016 Dec;31(12):1417-1426. [doi: 10.1007/s11606-016-3771-6] [Medline: 27418347]