Artificial intelligence, bias and clinical safety

BMJ Quality and Safety

Volumn 28, Issue 3, 2019, Pages 231-237

  Challen, Robert ^a,b   Denny, Joshua ^c   Pitt, Martin ^d   Gompels, Luke ^b   Edwards, Tom ^b   Tsaneva Atanasova, Krasimira ^a  

^a UNIVERSITY OF EXETER   (United Kingdom)

^b TAUNTON AND SOMERSET HOSPITAL   (United Kingdom)

^c VANDERBILT UNIVERSITY MEDICAL CENTER   (United States)

^d UNIVERSITY OF EXETER MEDICAL SCHOOL   (United Kingdom)

Author keywords

artificial intelligence; clinical decision support systems; clinical safety; machine learning

Indexed keywords

ALGORITHM; ARTIFICIAL INTELLIGENCE; AUTOMATION; CLINICAL DECISION SUPPORT SYSTEM; CLINICAL SAFETY; DERMATOLOGIST; MACHINE LEARNING; MEDICAL RESEARCH; QUALITY CONTROL; REVIEW; SAFETY; STATISTICAL BIAS; HEALTH CARE QUALITY; HUMAN; PATIENT SAFETY;

ALGORITHMS; ARTIFICIAL INTELLIGENCE; BIAS; HUMANS; PATIENT SAFETY; QUALITY OF HEALTH CARE;

EID: 85059884477     PISSN: 20445415     EISSN: None     Source Type: Journal    
DOI: 10.1136/bmjqs-2018-008370     Document Type: Review

References (53)

1. Dreyfus HL, Dreyfus SE. What artificial experts can and cannot do. AI Soc 1992;6:18-26.
2. Rao A, Verweij G, Cameron E. Sizing the prize: what's the real value of AI for your business and how can you capitalise? PwC. 2017. Available: https://www. pwc. com/ gx/ en/ issues/ analytics/ assets/ pwc-ai-analysis-sizing-the-prize-report. pdf
3. Hall W, Pesenti J. Growing the artificial intelligence industry in the UK-GOV.UK. Department for digital, culture, media & sport and department for business, energy & industrial strategy. 2017. Available: https://www. gov. uk/ government/ uploads/ system/ uploads/ attachment-data/ file/ 652097/ Growing-the-artificial-intelligence-industry-in-the-UK. pdf
4. Duggal R, Brindle I, Bagenal J. Digital healthcare: regulating the revolution. BMJ 2018;360:k6.
5. McCoy A, Das R. Reducing patient mortality, length of stay and readmissions through machine learning-based sepsis prediction in the emergency department, intensive care unit and hospital floor units. BMJ Open Qual 2017;6:e000158.
6. Esteva A, Kuprel B, Novoa RA, et al. Dermatologist-level classification of skin cancer with deep neural networks. Nature 2017;542:115-8.
7. De Fauw J, Ledsam JR, Romera-Paredes B, et al. Clinically applicable deep learning for diagnosis and referral in retinal disease. Nat Med 2018;24:1342-50.
8. Walsh CG, Ribeiro JD, Franklin JC. Predicting risk of suicide attempts over time through machine learning. Clin Psychol Sci 2017:1-13.
9. Rajpurkar P, Irvin J, Zhu K. CheXNet: radiologist-level pneumonia detection on chest x-rays with deep learning. arXiv [cs.CV]. 2017. Available: http:// arxiv. org/ abs/ 1711. 05225
10. Gulshan V, Peng L, Coram M, et al. Development and validation of a deep learning algorithm for detection of diabetic retinopathy in retinal fundus photographs. JAMA 2016;316:2402.
11. Char DS, Shah NH, Magnus D. Implementing machine learning in health care-addressing ethical challenges. N Engl J Med 2018;378:981-3.
12. Kaushal R, Shojania KG, Bates DW. Effects of computerized physician order entry and clinical decision support systems on medication safety: a systematic review. Arch Intern Med 2003;163:1409-16.
13. Hippisley-Cox J, Coupland C, Vinogradova Y, et al. Predicting cardiovascular risk in England and Wales: prospective derivation and validation of QRISK2. BMJ 2008;336:1475-82.
14. Bouch DC, Thompson JP. Severity scoring systems in the critically ill. Cont Edu Anaesth Crit Care Pain 2008;8:181-5.
15. Koppel Ret al. Role of computerized physician order entry systems in facilitating medication errors. JAMA 2005;293:1197-203.
16. Han YYet al. Unexpected increased mortality after implementation of a commercially sold computerized physician order entry system. Pediatrics 2005;116:1506-12.
17. Miller RA. Medical diagnostic decision support systems-past, present, and future: a threaded bibliography and brief commentary. J Am Med Inform Assoc 1994;1:8-27.
18. Nurek M, Kostopoulou O, Delaney BC, et al. Reducing diagnostic errors in primary care. a systematic meta-review of computerized diagnostic decision support systems by the LINNEAUS collaboration on patient safety in primary care. Eur J Gen Pract 2015;21(sup1):8-13.
19. Bond WF, Schwartz LM, Weaver KR, et al. Differential diagnosis generators: an evaluation of currently available computer programs. J Gen Intern Med 2012;27:213-9.
20. Calvert JS, Price DA, Chettipally UK, et al. A computational approach to early sepsis detection. Comput Biol Med 2016;74:69-73.
21. Desautels T, Calvert J, Hoffman J, et al. Prediction of sepsis in the intensive care unit with minimal electronic health record data: a machine learning approach. JMIR Med Inform 2016;4:e28.
22. Thompson RF, Valdes G, Fuller CD, et al. Artificial intelligence in radiation oncology: a specialty-wide disruptive transformation? Radiother Oncol 2018;129:421-6.
23. Ghassemi MM, Clifford GD. Optimal medication dosing from suboptimal clinical examples: A deep reinforcement learning approach 23. In:2016 38th annual international conference of the IEEE Engineering in Medicine and Biology Society (EMBC), 2016: 2978-81.
24. Weng W-H, Gao M, He Z. Representation and reinforcement learning for personalized glycemic control in septic patients. arXiv [cs.LG]. 2017. Available: http:// arxiv. org/ abs/ 1712. 00654
25. K-H Y, Kohane IS. Framing the challenges of artificial intelligence in medicine. BMJ Qual Saf 2019;28:238-41.
26. Faliagka E, Tsakalidis A, Tzimas G. An integrated e-recruitment system for automated personality mining and applicant ranking. Internet Research 2012;22:551-68.
27. Silver D, Huang A, Maddison CJ, et al. Mastering the game of Go with deep neural networks and tree search. Nature 2016;529:484-9.
28. Nuti G, Mirghaemi M, Treleaven P, et al. Algorithmic trading. Computer 2011;44:61-9.
29. Evans R (deepmind), Gao J (deepmind). DeepMind AI reduces google data centre cooling bill by 40%. Available: https:// deepmind. com/ blog/ deepmind-ai-reduces-google-data-centrecooling-bill-40/
30. Office of the Assistant Secretary for Research and Technology. Automated driving systems 2.0 A vision for safety. national highway traffic safety administration. 2017. Available: https:// www. nhtsa. gov/ document/ automated-driving-systems-20-voluntary-guidance
31. IIHS Status Report newsletter. 2018. Available: https://www. iihs. org/ externaldata/ srdata/ docs/ sr5304. pdf
32. Amodei D, Olah C, Steinhardt J. Concrete problems in AI safety. arXiv [cs.AI]. 06565, 2016.
33. Bothe MK, Dickens L, Reichel K, et al. The use of reinforcement learning algorithms to meet the challenges of an artificial pancreas. Expert Rev Med Devices 2013;10:661-73.
34. Prasad N, Cheng L-F, Chivers C. A reinforcement learning approach to weaning of mechanical ventilation in intensive care units. arXiv [cs.AI]. 2017. Available: http:// arxiv. org/ abs/ 1704. 06300
35. Forman G, Scholz M. Apples-to-apples in cross-validation studies: pitfalls in classifier performance measurement. ACM SIGKDD Explorations Newsletter Published Online First. 2010. Available: https:// dl. acm. org/ citation. cfm? id= 1882479
36. Lobo JM, Jiménez-Valverde A, Real R. AUC: a misleading measure of the performance of predictive distribution models. Glob Ecol Biogeogr 2008;17:145-51.
37. Haixiang G, Yijing L, Shang J, et al. Learning from classimbalanced data: review of methods and applications. Expert Syst Appl 2017;73:220-39.
38. Storkey AJ. When Training and Test Sets are Different: Characterising Learning Transfer. In: Lawrence CSS, ed. Dataset shift in machine learning. MIT Press, 2013: 3-28.
39. Bae S-H, Yoon K-J. Polyp detection via imbalanced learning and discriminative feature learning. IEEE Trans Med Imaging 2015;34:2379-93.
40. Davis SE, Lasko TA, Chen G, et al. Calibration drift in regression and machine learning models for acute kidney injury. J Am Med Inform Assoc 2017;24:1052-61.
41. Sculley D, Phillips T, Ebner D. Machine learning: the highinterest credit card of technical debt. 2018. Available: https:// research. google. com/ pubs/ pub43146. htmlhttp:// citeseerx. ist. psu. edu/ viewdoc/ summary? doi= 10. 1. 1. 675. 9675 [Accessed 5 Mar 2018].
42. Mao Q, Jay M, Hoffman JL, et al. Multicentre validation of a sepsis prediction algorithm using only vital sign data in the emergency department, general ward and ICU. BMJ Open 2018;8:e017833.
43. Megler V, Gregoire S. Training models with unequal economic error costs using Amazon sagemaker. AWS machine learning blog. 2018. Available: https:// aws. amazon. com/ blogs/ machine-learning/ training-models-with-unequal-economicerror-costs-using-amazon-sagemaker/ [Accessed 19 Oct 2018].
44. Adler P, Falk C, Friedler SA, et al. Auditing black-box models for indirect influence. arXiv [stat.ML], 2016.
45. Caruana R, Lou Y, Gehrke J. Intelligible models for healthcare: predicting pneumonia risk and hospital 30-day readmission. In:Proceedings of the 21th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining. NY, USA: ACM, 2015: 1721-30.
46. Varshney KR. Engineering safety in machine learning. In:2016 Information Theory and Applications Workshop. ITA, 2016: 1-5.
47. Dawson NV, Arkes HR. Systematic errors in medical decision making: judgment limitations. J Gen Intern Med 1987;2:183-7.
48. Parasuraman R, Manzey DH. Complacency and bias in human use of automation: an attentional Integration. Hum Factors 2010;52:381-410.
49. Tsai TL, Fridsma DB, Gatti G. Computer decision support as a source of interpretation error: the case of electrocardiograms. J Am Med Inform Assoc 2003;10:478-83.
50. Garcia J, Fernandez F, Fern F. Safe exploration of state and action spaces in reinforcement learning. J Artif Intell Res 2012;45:515-64.
51. Leike J, Martic M, Krakovna V. AI safety gridworlds. arXiv [cs. LG]. 2017. Available: http:// arxiv. org/ abs/ 1711. 09883
52. Bossuyt PM, Reitsma JB, Bruns DE, et al. Towards complete and accurate reporting of studies of diagnostic accuracy: the STARD initiative. Standards for Reporting of Diagnostic Accuracy. Clin Chem 2003;49:1-6.
53. Shimabukuro DW, Barton CW, Feldman MD, et al. Effect of a machine learning-based severe sepsis prediction algorithm on patient survival and hospital length of stay: a randomised clinical trial. BMJ Open Respir Res 2017;4:e000234.