Methods to evaluate the effects of internet-based digital health interventions for citizens: Systematic review of reviews

Journal of Medical Internet Research

Volumn 20, Issue 6, 2018, Pages

  Zanaboni, Paolo ^a   Ngangue, Patrice ^b   Mbemba, Gisele Irène Claudine ^c   Schopf, Thomas Roger ^a   Bergmo, Trine Strand ^a   Gagnon, Marie Pierre ^c,d  

^a UNIVERSITY HOSPITAL OF NORTH NORWAY   (Norway)

^b UNIVERSITÉ DU QUÉBEC EN ABITIBI TÉMISCAMINGUE   (Canada)

^c LAVAL UNIVERSITY   (Canada)

^d UNIVERSITÉ LAVAL   (Canada)

Author keywords

Electronic health records; Epidemiological methods; Evaluation studies as topic; Patient access to records; Patient portals; Review

Indexed keywords

HEALTH CARE DELIVERY; HUMAN; INTERNET; PATIENT SATISFACTION; QUALITATIVE RESEARCH; STANDARDS;

DELIVERY OF HEALTH CARE; HUMANS; INTERNET; PATIENT SATISFACTION; QUALITATIVE RESEARCH;

EID: 85048897554     PISSN: None     EISSN: 14388871     Source Type: Journal    
DOI: 10.2196/10202     Document Type: Article

References (51)

1. Murray E, Hekler EB, Andersson G, Collins LM, Doherty A, Hollis C, et al. Evaluating Digital Health Interventions: Key Questions and Approaches. Am J Prev Med 2016 Nov;51(5):843-851. [doi: 10.1016/j.amepre.2016.06.008] [Medline: 27745684]
2. Kostkova P. Grand challenges in digital health. Front Public Health 2015;3:134 [FREE Full _text] [doi: 10.3389/fpubh.2015.00134] [Medline: 26000272]
3. O'Connor S, Hanlon P, O'Donnell CA, Garcia S, Glanville J, Mair FS. Understanding factors affecting patient and public engagement and recruitment to digital health interventions: a systematic review of qualitative studies. BMC Med Inform Decis Mak 2016 Sep 15;16(1):120 [FREE Full _text] [doi: 10.1186/s12911-016-0359-3] [Medline: 27630020]
4. Irizarry T, DeVito DA, Curran CR. Patient Portals and Patient Engagement: A State of the Science Review. J Med Internet Res 2015;17(6):e148 [FREE Full _text] [doi: 10.2196/jmir.4255] [Medline: 26104044]
5. Free C, Phillips G, Galli L, Watson L, Felix L, Edwards P, et al. The effectiveness of mobile-health technology-based health behaviour change or disease management interventions for health care consumers: a systematic review. PLoS Med 2013;10(1):e1001362 [FREE Full _text] [doi: 10.1371/journal.pmed.1001362] [Medline: 23349621]
6. Mold F, Ellis B, de LS, Sheikh A, Wyatt JC, Cavill M, et al. The provision and impact of online patient access to their electronic health records (EHR) and transactional services on the quality and safety of health care: systematic review protocol. Inform Prim Care 2012;20(4):271-282 [FREE Full _text] [Medline: 23890339]
7. Ammenwerth E, Schnell-Inderst P, Hoerbst A. The impact of electronic patient portals on patient care: a systematic review of controlled trials. J Med Internet Res 2012;14(6):e162 [FREE Full _text] [doi: 10.2196/jmir.2238] [Medline: 23183044]
8. Frank SR. Digital health care--the convergence of health care and the Internet. J Ambul Care Manage 2000 Apr;23(2):8-17. [Medline: 10848396]
9. Agboola SO, Bates DW, Kvedar JC. Digital Health and Patient Safety. JAMA 2016 Apr 26;315(16):1697-1698. [doi: 10.1001/jama.2016.2402] [Medline: 27115372]
10. Garell C, Svedberg P, Nygren JM. A Legal Framework to Support Development and Assessment of Digital Health Services. JMIR Med Inform 2016 May 25;4(2):e17 [FREE Full _text] [doi: 10.2196/medinform.5401] [Medline: 27226391]
11. Georgsson M, Staggers N. Quantifying usability: an evaluation of a diabetes mHealth system on effectiveness, efficiency, and satisfaction metrics with associated user characteristics. J Am Med Inform Assoc 2016 Jan;23(1):5-11 [FREE Full _text] [doi: 10.1093/jamia/ocv099] [Medline: 26377990]
12. Lau F, Kuziemsky C. Handbook of eHealth Evaluation: An Evidence-based Approach. Victoria, British Columbia, Canada: University of Victoria; 2016.
13. Kaplan B. Evaluating informatics applications--some alternative approaches: theory, social interactionism, and call for methodological pluralism. Int J Med Inform 2001 Nov;64(1):39-56. [Medline: 11673101]
14. Mair FS, May C, O'Donnell C, Finch T, Sullivan F, Murray E. Factors that promote or inhibit the implementation of e-health systems: an explanatory systematic review. Bull World Health Organ 2012 May 1;90(5):357-364 [FREE Full _text] [doi: 10.2471/BLT.11.099424] [Medline: 22589569]
15. O'Connor S, Hanlon P, O'Donnell CA, Garcia S, Glanville J, Mair FS. Barriers and facilitators to patient and public engagement and recruitment to digital health interventions: protocol of a systematic review of qualitative studies. BMJ Open 2016 Dec 02;6(9):e010895 [FREE Full _text] [doi: 10.1136/bmjopen-2015-010895] [Medline: 27591017]
16. Monitoring and evaluating digital health interventions: a practical guide to conducting research and assessment. Geneva: World Health Organization; 2016.
17. Lau F, Hagens S, Muttitt S. A proposed benefits evaluation framework for health information systems in Canada. Healthc Q 2007;10(1):112-6, 118. [Medline: 17326376]
18. Smith V, Devane D, Begley CM, Clarke M. Methodology in conducting a systematic review of systematic reviews of healthcare interventions. BMC Med Res Methodol 2011;11(1):15 [FREE Full _text] [doi: 10.1186/1471-2288-11-15] [Medline: 21291558]
19. Moher D, Shamseer L, Clarke M, Ghersi D, Liberati A, Petticrew M, et al. Preferred reporting items for systematic review and meta-analysis protocols (PRISMA-P) 2015 statement. Syst Rev 2015 Jan;4:1 [FREE Full _text] [doi: 10.1186/2046-4053-4-1] [Medline: 25554246]
20. Shamseer L, Moher D, Clarke M, Ghersi D, Liberati A, Petticrew M, et al. Preferred reporting items for systematic review and meta-analysis protocols (PRISMA-P) 2015: elaboration and explanation. BMJ 2015 Jan 02;349:g7647 [FREE Full text] [Medline: 25555855]
21. Higgins J. Cochrane Handbook for Systematic Reviews of Interventions Version 5. Green S (editors) URL: http://handbook-5-1.cochrane.org/ [accessed 2018-02-15] [WebCite Cache ID _6xFPALEtJ]
22. Shea BJ, Grimshaw JM, Wells GA, Boers M, Andersson N, Hamel C, et al. Development of AMSTAR: a measurement tool to assess the methodological quality of systematic reviews. BMC Med Res Methodol 2007;7:10 [FREE Full _text] [doi: 10.1186/1471-2288-7-10] [Medline: 17302989]
23. Bouma G, Admiraal JM, de Vries EGE, Schröder CP, Walenkamp AME, Reyners AKL. Internet-based support programs to alleviate psychosocial and physical symptoms in cancer patients: a literature analysis. Crit Rev Oncol Hematol 2015 Jul;95(1):26-37. [doi: 10.1016/j.critrevonc.2015.01.011] [Medline: 25701515]
24. Mays N, Pope C, Popay J. Systematically reviewing qualitative and quantitative evidence to inform management and policy-making in the health field. J Health Serv Res Policy 2005 Jul;10 Suppl 1:6-20. [doi: 10.1258/1355819054308576] [Medline: 16053580]
25. de Lusignan S, Mold F, Sheikh A, Majeed A, Wyatt JC, Quinn T, et al. Patients' online access to their electronic health records and linked online services: a systematic interpretative review. BMJ Open 2014;4(9):e006021 [FREE Full _text] [doi: 10.1136/bmjopen-2014-006021] [Medline: 25200561]
26. Ammenwerth E, Schnell-Inderst P, Hoerbst A. Patient empowerment by electronic health records: first results of a systematic review on the benefit of patient portals. Stud Health Technol Inform 2011;165:63-67. [Medline: 21685587]
27. Kruse CS, Argueta DA, Lopez L, Nair A. Patient and provider attitudes toward the use of patient portals for the management of chronic disease: a systematic review. J Med Internet Res 2015;17(2):e40 [FREE Full _text] [doi: 10.2196/jmir.3703] [Medline: 25707035]
28. Mold F, de LS, Sheikh A, Majeed A, Wyatt JC, Quinn T, et al. Patients' online access to their electronic health records and linked online services: a systematic review in primary care. Br J Gen Pract 2015 Mar;65(632):e141-e151 [FREE Full _text] [doi: 10.3399/bjgp15X683941] [Medline: 25733435]
29. Kruse CS, Bolton K, Freriks G. The effect of patient portals on quality outcomes and its implications to meaningful use: a systematic review. J Med Internet Res 2015;17(2):e44 [FREE Full _text] [doi: 10.2196/jmir.3171] [Medline: 25669240]
30. Alkureishi MA, Lee WW, Lyons M, Press VG, Imam S, Nkansah-Amankra A, et al. Impact of Electronic Medical Record Use on the Patient-Doctor Relationship and Communication: A Systematic Review. J Gen Intern Med 2016 May;31(5):548-560 [FREE Full _text] [doi: 10.1007/s11606-015-3582-1] [Medline: 26786877]
31. Amante DJ, Hogan TP, Pagoto SL, English TM. A systematic review of electronic portal usage among patients with diabetes. Diabetes Technol Ther 2014 Nov;16(11):784-793. [doi: 10.1089/dia.2014.0078] [Medline: 24999599]
32. Bush RA, Connelly CD, Fuller M, Pérez A. Implementation of the Integrated Electronic Patient Portal in the Pediatric Population: A Systematic Review. Telemed J E Health 2015 Aug 10:1-16 [FREE Full _text] [doi: 10.1089/tmj.2015.0033] [Medline: 26258289]
33. Davis Giardina T, Menon S, Parrish DE, Sittig DF, Singh H. Patient access to medical records and healthcare outcomes: a systematic review. J Am Med Inform Assoc 2014;21(4):737-741. [doi: 10.1136/amiajnl-2013-002239] [Medline: 24154835]
34. Davis MM, Freeman M, Kaye J, Vuckovic N, Buckley DI. A systematic review of clinician and staff views on the acceptability of incorporating remote monitoring technology into primary care. Telemed J E Health 2014 May;20(5):428-438 [FREE Full _text] [doi: 10.1089/tmj.2013.0166] [Medline: 24731239]
35. Goldzweig CL, Orshansky G, Paige NM, Towfigh AA, Haggstrom DA, Miake-Lye I, et al. Electronic patient portals: evidence on health outcomes, satisfaction, efficiency, and attitudes: a systematic review. Ann Intern Med 2013 Nov 19;159(10):677-687. [doi: 10.7326/0003-4819-159-10-201311190-00006] [Medline: 24247673]
36. Liu J, Luo L, Zhang R, Huang T. Patient satisfaction with electronic medical/health record: a systematic review. Scand J Caring Sci 2013 Dec;27(4):785-791. [doi: 10.1111/scs.12015] [Medline: 23181421]
37. Osborn CY, Mayberry LS, Mulvaney SA, Hess R. Patient web portals to improve diabetes outcomes: a systematic review. Curr Diab Rep 2010 Dec;10(6):422-435 [FREE Full _text] [doi: 10.1007/s11892-010-0151-1] [Medline: 20890688]
38. Otte-Trojel T, de Bont A, Rundall TG, van de Klundert J. How outcomes are achieved through patient portals: a realist review. J Am Med Inform Assoc 2014 Jul;21(4):751-757. [doi: 10.1136/amiajnl-2013-002501] [Medline: 24503882]
39. Price M, Bellwood P, Kitson N, Davies I, Weber J, Lau F. Conditions potentially sensitive to a personal health record (PHR) intervention, a systematic review. BMC Med Inform Decis Mak 2015;15:32 [FREE Full _text] [doi: 10.1186/s12911-015-0159-1] [Medline: 25927384]
40. Stellefson M, Chaney B, Barry AE, Chavarria E, Tennant B, Walsh-Childers K, et al. Web 2.0 chronic disease self-management for older adults: a systematic review. J Med Internet Res 2013 Feb;15(2):e35 [FREE Full _text] [doi: 10.2196/jmir.2439] [Medline: 23410671]
41. Tao D, Or CK. Effects of self-management health information technology on glycaemic control for patients with diabetes: a meta-analysis of randomized controlled trials. J Telemed Telecare 2013 Apr 5:133-143. [doi: 10.1177/1357633X13479701] [Medline: 23563018]
42. Tulu B, Trudel J, Strong DM, Johnson SA, Sundaresan D, Garber L. Patient Portals: An Underutilized Resource for Improving Patient Engagement. Chest 2015 Jun 11:272-277. [doi: 10.1378/chest.14-2559] [Medline: 26066707]
43. Turner K, Klaman SL, Shea CM. Personal health records for people living with HIV: a review. AIDS Care 2016 Sep;28(9):1181-1187. [doi: 10.1080/09540121.2016.1153594] [Medline: 26917113]
44. Vimalananda VG, Gupte G, Seraj SM, Orlander J, Berlowitz D, Fincke BG, et al. Electronic consultations (e-consults) to improve access to specialty care: A systematic review and narrative synthesis. J Telemed Telecare 2015 May 20:323-330. [doi: 10.1177/1357633X15582108] [Medline: 25995331]
45. Grant MJ, Booth A. A typology of reviews: an analysis of 14 review types and associated methodologies. Health Info Libr J 2009 Jun;26(2):91-108 [FREE Full _text] [doi: 10.1111/j.1471-1842.2009.00848.x] [Medline: 19490148]
46. Rouleau G, Gagnon M, Côté J, Payne-Gagnon J, Hudson E, Dubois C. Impact of Information and Communication Technologies on Nursing Care: Results of an Overview of Systematic Reviews. J Med Internet Res 2017 Apr 25;19(4):e122 [FREE Full _text] [doi: 10.2196/jmir.6686] [Medline: 28442454]
47. Archer N, Fevrier-Thomas U, Lokker C, McKibbon KA, Straus SE. Personal health records: a scoping review. J Am Med Inform Assoc 2011;18(4):515-522 [FREE Full _text] [doi: 10.1136/amiajnl-2011-000105] [Medline: 21672914]
48. Murray E, Treweek S, Pope C, MacFarlane A, Ballini L, Dowrick C, et al. Normalisation process theory: a framework for developing, evaluating and implementing complex interventions. BMC Med 2010;8:63 [FREE Full _text] [doi: 10.1186/1741-7015-8-63] [Medline: 20961442]
49. Peters DH, Adam T, Alonge O, Agyepong IA, Tran N. Implementation research: what it is and how to do it. BMJ 2013 Nov 20;347:f6753. [Medline: 24259324]
50. Agarwal S, LeFevre AE, Lee J, L’Engle K, Mehl G, Sinha C, et al. Guidelines for reporting of health interventions using mobile phones: mobile health (mHealth) evidence reporting and assessment (mERA) checklist. BMJ 2016 Mar 17:i1174. [doi: 10.1136/bmj.i1174]
51. Eysenbach G, CONSORT-EHEALTH Group. CONSORT-EHEALTH: improving and standardizing evaluation reports of Web-based and mobile health interventions. J Med Internet Res 2011 Dec 31;13(4):e126 [FREE Full _text] [doi: 10.2196/jmir.1923] [Medline: 22209829]