The effectiveness of electronic differential diagnoses (DDX) generators: A systematic review and meta-analysis

PLoS ONE

Volumn 11, Issue 3, 2016, Pages

  Riches, Nicholas ^a   Panagioti, Maria ^b   Alam, Rahul ^a   Cheraghi Sohi, Sudeh ^a   Campbell, Stephen ^a   Esmail, Aneez ^a   Bower, Peter ^a  

^a UNIVERSITY OF MANCHESTER   (United Kingdom)

^b UNIVERSITY OF MANCHESTER   (United Kingdom)

Author keywords

[No Author keywords available]

Indexed keywords

CLINICAL PRACTICE; CONSULTATION; COST EFFECTIVENESS ANALYSIS; DATA BASE; DATA EXTRACTION; DIAGNOSIS; DIFFERENTIAL DIAGNOSIS; DOCTOR PATIENT RELATION; HUMAN; INFORMATION RETRIEVAL; PATIENT SAFETY; SATISFACTION; SCREENING; SYSTEMATIC REVIEW; UNCERTAINTY; COMPUTER ASSISTED DIAGNOSIS; META ANALYSIS; PROCEDURES; STANDARDS; TRENDS;

DIAGNOSIS, COMPUTER-ASSISTED; DIAGNOSIS, DIFFERENTIAL; HUMANS;

EID: 84962548067     PISSN: None     EISSN: 19326203     Source Type: Journal    
DOI: 10.1371/journal.pone.0148991     Document Type: Article

References (62)

1. Balla J, Heneghan C, Goyder C, Thompson M. Identifying early warning signs for diagnostic errors in primary care: a qualitative study. BMJ Open. 2012; 2:1-9. doi: 10.1136/bmjopen-2012-001539 PMID: 22983786.
2. Berner ES, Graber ML. Overconfidence as a cause of diagnostic error in medicine. The American journal of medicine. 2008; 121(5 Suppl):S2-23. doi: 10.1016/j.amjmed.2008.01.001 PMID: 18440350.
3. Kostopoulou O, Oudhoff J, Nath R, Delaney BC, Munro CW, Harries C, et al. Predictors of diagnostic accuracy and safe management in difficult diagnostic problems in family medicine. Med Decis Making. 2008; 28(5):668-80. doi: 10.1177/0272989X08319958 PMID: 18556634.
4. Schiff GD, Puopolo AL, Huben-Kearney A, Yu W, Keohane C, McDonough P, et al. Primary care closed claims experience of Massachusetts malpractice insurers. JAMA internal medicine. 2013; 173 (22):2063-8. Epub 2013/10/02. doi: 10.1001/jamainternmed.2013.11070 PMID: 24081145.
5. Graber ML, Kissam S, Payne VL, Meyer AN, Sorensen A, Lenfestey N, et al. Cognitive interventions to reduce diagnostic error: a narrative review. BMJ Qual Saf. 2012; 21(7):535-57. doi: 10.1136/bmjqs-2011-000149 PMID: 22543420.
6. Croskerry P. The importance of cognitive errors in diagnosis and strategies to minimize them. Acad Med. 2003; 78(8):775-80. doi: 10.1097/00001888-200308000-00003
7. Neurath PW, Enslein K, Mitchell GW Jr. Design of a computer system to assist in differential preoperative diagnosis for pelvic surgery. The New England journal of medicine. 1969; 280(14):745-9. Epub 1969/04/03. doi: 10.1056/NEJM196904032801403 PMID: 5773355.
8. Parodi S, Filiberti R, Marroni P, Libener R, Ivaldi G, Mussap M, et al. Differential diagnosis of pleural mesothelioma using Logic Learning Machine. BMC Bioinformatics. 2015; 16(Suppl 9):S3. doi: 10.1186/ 1471-2105-16-S9-S3 PMID: 26051106
9. Bond WF, Schwartz LM, Weaver KR, Levick D, Giuliano M, Graber ML. Differential diagnosis generators: An evaluation of currently available computer programs. Journal of General Internal Medicine. 2011; 27(2):213-9. doi: 10.1007/s11606-011-1804-8.
10. Garg A, Adhikari N, Beyene J, Sam J, Haynes R. Effects of computerized clinical decision support systems on practitioner performance. JAMA. 2005; 293(10):1223-38. PMID: 15755945
11. El-Kareh R, Hasan O, Schiff G. Use of health information technology to reduce diagnostic errors. BMJ Quality and Safety. 2013; 22(2):1-12.
12. Foot C, Naylor C, Imison C. The quality of GP diagnosis and referral. 2010.
13. Graber MA, VanScoy D. How well does decision support software perform in the emergency department? Emergency Medicine Journal. 2003; 20(5):426-8.
14. Liberati A, Altman DG, Tetzlaff J, Mulrow C, Gotzsche PC, Ioannidis JP, et al. The PRISMA statement for reporting systematic reviews and meta-analyses of studies that evaluate health care interventions: explanation and elaboration. Journal of clinical epidemiology. 2009; 62(10):e1-34. Epub 2009/07/28. doi: 10.1016/j.jclinepi.2009.06.006 PMID: 19631507.
15. Berner ES, Webster GD, Shugerman AA, Jackson JR, Algina J, Baker AL, et al. Performance of four computer-based diagnostic systems. New England Journal of Medicine. 1994; 330(25):1792-6. doi: 10.1056/NEJM199406233302506
16. Cohen J. A coefficient of agreement for nominal scales. Educ Psychol Meas. 1960; 20:37-46.
17. Whiting PF, Rutjes AW, Westwood ME, Mallett S, Deeks JJ, Reitsma JB, et al. QUADAS-2: a revised tool for the quality assessment of diagnostic accuracy studies. Ann Intern Med. 2011; 155(8):529-36. doi: 10.7326/0003-4819-155-8-201110180-00009 PMID: 22007046
18. Borenstein M. Comprehensive meta-analysis. Englewood, NJ: Biostat. 2005.
19. Friedman CP, Elstein AS, Wolf FM, Murphy GC, Franz TM, Heckerling PS, et al. Enhancement of clinicians' diagnostic reasoning by computer-based consultation: a multisite study of 2 systems. Jama. 1999; 282(19):1851-6. Epub 1999/11/26. PMID: 10573277.
20. Heckerling PS, Elstein AS, Terzian CG, Kushner MS. The effect of incomplete knowledge on the diagnoses of a computer consultant system. Medical Informatics. 1991; 16(4):363-70.
21. Murphy GC, Friedman CP, Elstein AS, Wolf FM, Miller T, Miller JG. The influence of a decision support system on the differential diagnosis of medical practitioners at three levels of training. Proceedings: a conference of the American Medical Informatics Association AMIA Annual Fall Symposium AMIA Fall Symposium. 1996:219-23. PMID: 8947660.
22. Ramnarayan P, Roberts GC, Coren M, Nanduri V, Tomlinson A, Taylor PM, et al. Assessment of the potential impact of a reminder system on the reduction of diagnostic errors: a quasi-experimental study. BMC Med Inform Decis Mak. 2006; 6:22. Epub 2006/05/02. doi: 10.1186/1472-6947-6-22 PMID: 16646956; PubMed Central PMCID: PMCPmc1513379.
23. Ramnarayan P, Winrow A, Coren M, Nanduri V, Buchdahl R, Jacobs B, et al. Diagnostic omission errors in acute paediatric practice: impact of a reminder system on decision-making. BMC Medical Informatics and Decision Making. 2006; 6(1):37. doi: 10.1186/1472-6947-6-37
24. Sedgwick P, Marston L. Meta-analyses: standardised mean differences2013 2013-12-06 11:36:16.
25. Durlak J. How to Select, Calculate, and Interpret Effect Sizes. Journal of Pediatric Psychology. 2009; 34(9):917-28. doi: 10.1093/jpepsy/jsp004 PMID: 19223279
26. Higgins JPT, Thompson SG, Deeks JJ, Altman DG. Measuring inconsistency in meta-analyses. BMJ: British Medical Journal. 2003; 327(7414):557-60. PMC192859. PMID: 12958120
27. Egger M, Davey-Smith G, Schneider M, Minder C. Bias in meta-analysis detected by a simple, graphical test. BMJ. 1997; 315(7109):629-34. PMID: 9310563
28. Panic N, Leoncini E, de Belvis G, Ricciardi W, Boccia S. Evaluation of the endorsement of the preferred reporting items for systematic reviews and meta-analysis (PRISMA) statement on the quality of published systematic review and meta-analyses. PloS one. 2013; 8(12):e83138. Epub 2014/01/05. doi: 10.1371/journal.pone.0083138 PMID: 24386151; PubMed Central PMCID: PMC3873291.
29. Apkon M, Mattera JA, Lin Z, Herrin J, Bradley EH, Carbone M, et al. A randomized outpatient trial of a decision-support information technology tool. Archives of internal medicine. 2005; 165(20):2388-94. CN-00531948 UPDATE. PMID: 16287768
30. Arene I, AhmedW, Fox M, Barr CE, Fisher K. Evaluation of quick medical reference (QMR) as a teaching tool. MD Comput: computers in medical practice. 1998; 15(5):323-6. PMID: 9753979.
31. Bacchus CM, Quinton C, O'Rourke K, Detsky AS. A randomized crossover trial of quick medical reference (QMR) as a teaching tool for medical interns. Journal of General Internal Medicine. 1994; 9 (11):616-21.
32. Bankowitz RA, McNeil MA, Challinor SM, Parker RC, Kapoor WN, Miller RA. A computer-assisted medical diagnostic consultation service. Implementation and prospective evaluation of a prototype. Annals of Internal Medicine. 1989; 110(10):824-32.
33. Berner ES, Maisiak RS, Cobbs CG, Taunton OD. Effects of a decision support system on physicians' diagnostic performance. Journal of the American Medical Informatics Association. 1999; 6(5):420-7.
34. Carlson J, Abel M, Bridges D, Tomkowiak J. The impact of a diagnostic reminder system on student clinical reasoning during simulated case studies. Simulation in healthcare: Journal of the Society for Simulation in Healthcare. 2011; 6(1):11-7. PMID: 21330845.
35. Elkin PL, Liebow M, Bauer BA, Chaliki S, Wahner-Roedler D, Bundrick J, et al. The introduction of a diagnostic decision support system (DXplain) into the workflow of a teaching hospital service can decrease the cost of service for diagnostically challenging Diagnostic Related Groups (DRGs). International Journal of Medical Informatics. 2010; 79(11):772-7. doi: 10.1016/j.ijmedinf.2010.09.004
36. Elstein AS, Friedman CP, Wolf FM, Murphy G, Miller J, Fine P, et al. Effects of a decision support system on the diagnostic accuracy of users: a preliminary report. Journal of the American Medical Informatics Association: JAMIA. 1996; 3(6):422-8. PMID: 8930858.
37. Graber ML. Taking steps towards a safer future: measures to promote timely and accurate medical diagnosis. The American journal of medicine. 2008; 121(5 Suppl):S43-6. doi: 10.1016/j.amjmed.2008. 02.006 PMID: 18440355.
38. Graber ML, Tompkins D, Holland JJ. Resources medical students use to derive a differential diagnosis. Med Teach. 2009; 31(6):522-7. Epub 2009/10/09. PMID: 19811168.
39. Hammersley JR, Cooney K. Evaluating the Utility of Available Differential Diagnosis Systems. Proceedings of the Annual Symposium on Computer Application in Medical Care. 1988:229-31.
40. Lange LL, Haak SW, Lincoln MJ, Thompson CB, Turner CW, Weir C, et al. Use of Iliad to improve diagnostic performance of nurse practitioner students. The Journal of nursing education. 1997; 36(1):36-45. PMID: 8986960.
41. Lemaire JB, Schaefer JP, Martin LA, Faris P, Ainslie MD, Hull RD. Effectiveness of the quick medical reference as a diagnostic tool. Cmaj. 1999; 161(6):725-8.
42. Li YC, Haug PJ, Lincoln MJ, Turner CW, Pryor TA, Warner HH. Assessing the behavioral impact of a diagnostic decision support system. Annual Symposium on Computer Application [sic] in Medical Care Symposium on Computer Applications in Medical Care. 1995:805-9. PMID: 8563402.
43. Miller RA, McNeil MA, Challinor SM. The INTERNIST-1/QUICK MEDICAL REFERENCE Project - Status report. Western Journal of Medicine. 1986; 145(6):816-22.
44. Nelson SJ, Blois MS, Tuttle MS, Erlbaum M, Harrison P, Kim H, et al. Evaluating RECONSIDER. A computer program for diagnostic prompting. J Med Syst. 1985; 9(5-6):379-88. Epub 1985/12/01. PMID: 3912452.
45. Ramnarayan P, Cronje N, Brown R, Negus R, Coode B, Moss P, et al. Validation of a diagnostic reminder system in emergency medicine: A multi-centre study. Emergency Medicine Journal. 2007; 24 (9):619-24. doi: 10.1136/emj.2006.044107
46. Waxman HS, Worley WE. Computer-assisted adult medical diagnosis: subject review and evaluation of a new microcomputer-based system. Medicine. 1990; 69(3):125-36. PMID: 2189054.
47. Wexler JR, Swender PT, TunnessenWWJr, Oski FA. Impact of a system of computer-assisted diagnosis. Initial evaluation of the hospitalized patient. American Journal of Diseases of Children. 1975; 129 (2):203-5. CN-00217040. PMID: 1091140
48. Wolf FM, Friedman CP, Elstein AS, Miller JG, Murphy GC, Heckerling P, et al. Changes in diagnostic decision-making after a computerized decision support consultation based on perceptions of need and helpfulness: a preliminary report. Proceedings:a conference of the American Medical Informatics Association AMIA Annual Fall Symposium AMIA Fall Symposium. 1997:263-7. PMID: 9357629.
49. Bavdekar SB, Pawar M. Evaluation of an Internet delivered pediatric diagnosis support system (ISABEL) in a tertiary care center in India. Indian pediatrics. 2005; 42(11):1086-91. PMID: 16340049.
50. Feldman MJ, Barnett GO. An approach to evaluating the accuracy of DXplain. Computer Methods & Programs in Biomedicine. 1991; 35(4):261-6. PMID: 1752121.
51. Gozum ME. Emulating cognitive diagnostic skills without clinical experience: a report of medical students using Quick Medical Reference and Iliad in the diagnosis of difficult clinical cases. Annual Symposium on Computer Application [sic] in Medical Care Symposium on Computer Applications in Medical Care. 1994:991. PMID: 7950096.
52. Lau LM, Warner HR, Poulsen AL. Research review: a computer-based diagnostic model for individual case review. Topics in health information management. 1995; 15(3):67-79. PMID: 10140306.
53. Lincoln MJ, Turner CW, Haug PJ, Warner HR, Williamson JW, Bouhaddou O, et al. Iliad training enhances medical students' diagnostic skills. Journal of Medical Systems. 1991; 15(1):93-110. doi: 10.1007/BF00993883
54. Miller RA, Pople HE Jr, Myers JD. Internist-1, an experimental computer-based diagnostic consultant for general internal medicine. The New England journal of medicine. 1982; 307(8):468-76. Epub 1982/ 08/19. doi: 10.1056/nejm198208193070803 PMID: 7048091.
55. Ramnarayan P, Tomlinson A, Rao A, Coren M, Winrow A, Britto J. ISABEL: A web-based differential diagnostic aid for paediatrics: Results from an initial performance evaluation. Archives of Disease in Childhood. 2003; 88(5):408-13. doi: 10.1136/adc.88.5.408
56. Rodriguez-Gonzalez A, Torres-Nino J, Mayer MA, Alor-Hernandez G, Wilkinson MD. Analysis of a multilevel diagnosis decision support system and its implications: A case study. Computational and Mathematical Methods in Medicine. 2012; 2012(367345). doi: 10.1155/2012/367345
57. Graber ML, Mathew A. Performance of a web-based clinical diagnosis support system for internists. J Gen Intern Med. 2008; 23 Suppl 1:37-40. doi: 10.1007/s11606-007-0271-8 PMID: 18095042; PubMed Central PMCID: PMC2150633.
58. Bankowitz R, Blumenfeld B, Guise Bettinsoli N. User variability in abstracting and entering printed case histories with Quick Medical Reference (QMR). Proceedings of the Eleventh Annual Symposium on Computer Applications in Medical Care. 1987:68-73.
59. Bond W, Schwartz L, Weaver K, Levick D, Giuliano M, Graber M. A qualitative review of differential diagnosis generators. Poster presented at: The 32nd Annual Meeting of the Society for Medical Decision Making, Ontario, Canada. 2010.
60. Campbell M, Fitzpatrick R, Haines A, Kinmonth AL, Sandercock P, Spiegelhalter D, et al. Framework for design and evaluation of complex interventions to improve health. BMJ. 2000; 321(7262):694-6. doi: 10.1136/bmj.321.7262.694 PMID: 10987780
61. McAuley L, Pham B, Tugwell P, Moher D. Does the inclusion of grey literature influence estimates of intervention effectiveness reported in meta-analyses? Lancet 2000; 356(9237):1228-31. PMID: 11072941
62. Kontopantelis E, Springate DA, Reeves D. A Re-Analysis of the Cochrane Library Data: The Dangers of Unobserved Heterogeneity in Meta-Analyses. PloS one. 2013; 8(7):e69930. doi: 10.1371/journal. pone.0069930 PMID: 23922860