Automation bias and verification complexity: A systematic review

Journal of the American Medical Informatics Association

Volumn 24, Issue 2, 2017, Pages 423-431

  Lyell, David ^a   Coiera, Enrico ^a  

^a MACQUARIE UNIVERSITY   (Australia)

Author keywords

Clinical cognitive biases; Complexity; Decision support systems

Indexed keywords

AUTOMATION; COGNITIVE BIAS; CONTROLLED STUDY; DECISION SUPPORT SYSTEM; DIAGNOSIS; EMBASE; ERROR; EVALUATION STUDY; HUMAN; MEDLINE; MONITORING; PSYCINFO; SCOPUS; STATISTICAL SIGNIFICANCE; SYSTEMATIC REVIEW; WEB OF SCIENCE; ATTITUDE TO COMPUTERS; CLINICAL DECISION SUPPORT SYSTEM; STATISTICAL BIAS;

ATTITUDE TO COMPUTERS; AUTOMATION; BIAS; DECISION SUPPORT SYSTEMS, CLINICAL; HUMANS;

EID: 85016430011     PISSN: 10675027     EISSN: 1527974X     Source Type: Journal    
DOI: 10.1093/jamia/ocw105     Document Type: Review

References (60)

1. Wolfstadt JI, Gurwitz JH, Field TS, et al. The effect of computerized physician order entry with clinical decision support on the rates of adverse drug events: a systematic review. J General Int Med 2008;23(4):451-458.
2. Garg AX, Adhikari NJ, McDonald H, et al. Effects of computerized clinical decision support systems on practitioner performance and patient outcomes: a systematic review. JAMA 2005;293(10):1223-1238.
3. Coiera E, Westbrook J, Wyatt J. The safety and quality of decision support systems. Methods Inf Med 2006;45(Suppl 1):20-25.
4. Coiera E. Technology, cognition and error. BMJ Qual Saf 2015;24(7): 417-422.
5. Mosier KL, Skitka LJ. Human decision makers and automated decision aids: made for each other. In: R Parasuraman, M Mouloua, eds. Automation and Human Performance: Theory and Applications. Hillsdale, NJ, England: Lawrence Erlbaum Associates; 1996:201-220.
6. Parasuraman R, Molloy R, Singh IL. Performance consequences of automation-induced "complacency." Int JAviation Psychol. 1993;3(1):1-23.
7. Billings C, Lauber J, Funkhouser H, Lyman G, HuffE. NASA Aviation Safety Reporting System. (Technical Report TM-X-3445). Moffett Field, Calif.: NASA Ames Research Center; 1976.
8. Parasuraman R, Manzey DH. Complacency and bias in human use of automation: an attentional integration. Human Factors 2010;52(3):381-410.
9. Molloy R, Parasuraman R. Monitoring automation failures: effects of automation reliability and task complexity. Proceedings of the Human Factors and Ergonomics Society Annual Meeting. October 12-16, 1992. Atlanta, Ga.: Human Factors and Ergonomics Society; 1992:1518-1521.
10. Goddard K, Roudsari A, Wyatt JC. Automation bias: a systematic review of frequency, effect mediators, and mitigators. J Am Med Inform Assoc 2012;19(1):121-127.
11. Hillson SD, Connelly DP, Yuanli Liu. The effects of computer-assisted electrocardiographic interpretation on physicians' diagnostic decisions. Med Decis Mak 1995;15(2):107-112.
12. Southern WN, Arnsten JH. The Effect of Erroneous Computer Interpretation of ECGs on Resident Decision Making. Med Decis Mak 2009;29(3): 372-376.
13. Tsai TL, Fridsma DB, Gatti G. Computer decision support as a source of interpretation error: the case of electrocardiograms. J Am Med Inform Assoc 2003;10(5):478-483.
14. Alberdi E, Povykalo A, Strigini L, Ayton P. Effects of incorrect computeraided detection (CAD) output on human decision-making in mammography. Acad Radiol 2004;11(8):909-918.
15. Povyakalo AA, Alberdi E, Strigini L, Ayton P. How to discriminate between computer-aided and computer-hindered decisions: a case study in mammography. Med Decis Mak 2013;33(1):98-107.
16. Bogun F, Anh D, Kalahasty G, et al. Misdiagnosis of atrial fibrillation and its clinical consequences. Am J Med 2004;117(9):636-642.
17. Bagheri N, Jamieson GA. The impact of context-related reliability on automation failure detection and scanning behaviour. Conf Proc IEEE Int Conf Syst Man Cybern, Vol. 1. October 10-13, 2004. 2004:212-217.
18. Bailey NR, Scerbo MW. Automation-induced complacency for monitoring highly reliable systems: the role of task complexity, system experience, and operator trust. Theoretical Issues Ergonomics Sci 2007;8(4):321-348.
19. Bailey NR, Scerbo MW, Freeman FG, Mikulka PJ, Scott LA. Comparison of a brain-based adaptive system and a manual adaptable system for invoking automation. Human Factors 2006;48(4):693-709.
20. Prinzel LJ III, Freeman FG, Prinzel HD. Individual differences in complacency and monitoring for automation failures. Individual Differences Res 2005;3(1):27-49.
21. Rovira E, McGarry K, Parasuraman R. Effects of imperfect automation on decision making in a simulated command and control task. Hum Factors 2007;49(1):76-87.
22. Singh AL, Tiwari T, Singh IL. Effects of automation reliability and training on automation-induced complacency and perceived mental workload. J Indian Acad Appl Psychol 2009;35(Spec iss):9-22.
23. Singh IL, Sharma HO, Parasuraman R. Effects of Training and Automation Reliability on Monitoring Performance in a Flight Simulation Task. Proceedings of the Human Factors and Ergonomics Society Annual Meeting 2000;44(13):53-56.
24. Singh IL, Singh AL, Saha PK. Monitoring performance and mental workload in an automated system. Proceedings of the International Conference on Engineering Psychology and Cognitive Ergonomics, July 22-27, 2007. Beijing, China: Springer Verlag; 2007: 426-435.
25. Bagheri N, Jamieson GA. Considering Subjective Trust and Monitoring Behavior in Assessing Automation-induced "Complacency." Mahwah, N.J.: Lawrence Erlbaum Assoc; 2004.
26. Moher D, Liberati A, TetzlaffJ, Altman DG. Preferred Reporting Items for Systematic Reviews and Meta-Analyses: The PRISMA Statement. Ann Int Med 2009;151(4):264-269.
27. Higgins JPT, AltmanDG, Gøtzsche PC, et al. The Cochrane Collaboration's tool for assessing risk of bias in randomised trials. BMJ 2011;343:55:48.
28. Parasuraman R, de Visser E, Lin M-K, Greenwood PM. Dopamine beta hydroxylase genotype identifies individuals less susceptible to bias in computer-assisted decision making. PLoS ONE 2012;7(6):e39675.
29. Marx C, Malich A, Facius M, et al. Are unnecessary follow-up procedures induced by computer-aided diagnosis (CAD) in mammography? Comparison of mammographic diagnosis with and without use of CAD. Eur J Radiol 2004;51(1):66-72.
30. Goddard K, Roudsari A, Wyatt JC. Automation bias: empirical results assessing influencing factors. Int J Med Inform 2014;83(5):368-375.
31. Golchin K, Roudsari A. Study of the effects of clinical decision support system's incorrect advice and clinical case difficulty on users' decision making accuracy. Stud Health Technol Inform 2011;164:13-16.
32. Harris WC, Goernert PN. The effect of levels of automation on supervisory performance in a multi-task environment. Proceedings of the Human Factors and Ergonomics Society Annual Meeting, October 5-9, 1998. Santa Monica, Calif.: Human Factors and Ergonomics Society; 1998: 128-132.
33. Singh AL, Tiwari T, Singh IL. Performance feedback, mental workload and monitoring efficiency. J Indian Acad Appl Psychol 2010;36(1):151-158.
34. Singh IL, Sharma HO, Parasuraman R. Effects of manual training and automation reliability on automation induced complacency in flight simulation task. Psychol Stud 2001;46(1-2):21-27.
35. Skitka LJ, Mosier K, Burdick MD. Accountability and automation bias. International J Hum Comp Stud 2000;52(4):701-717.
36. Skitka LJ, Mosier KL, Burdick M. Does automation bias decision-making? Int J Hum Comp Stud 1999;51(5):991-1006.
37. Skitka LJ, Mosier KL, BurdickM, Rosenblatt B. Automation bias and errors: are crews better than individuals? Int J Aviat Psychol 2000;10(1):85-97.
38. Mosier KL, Skitka LJ, Dunbar M, McDonnell L. Aircrews and automation bias: The advantages of teamwork? Int J Aviat Psychol 2001;11(1):1-14.
39. Mosier KL, Skitka LJ, Heers S, Burdick M. Automation bias: decisionmaking and performance in high-tech cockpits. Int JAviat Psychol 1998;8(1):47-63.
40. McGuirl JM, Sarter NB. Supporting trust calibration and the effective use of decision aids by presenting dynamic system confidence information. Hum Factors 2006;48(4):656-665.
41. Sarter NB, Schroeder B. Supporting decision making and action selection under time pressure and uncertainty: the case of in-flight icing. Hum Factors 2001;43(4):573-583.
42. Olson WA, Sarter NB. Management by consent in human-machine systems: when and why it breaks down. Human Factors 2001;43(2):255-266.
43. de Boer RJ, Heems W, Hurts K. The duration of automation bias in a realistic setting. Int J Aviat Psychol 2014;24(4):287-299.
44. Metzger U, Parasuraman R. Automation in future air traffic management: effects of decision aid reliability on controller performance and mental workload. Human Factors 2005;47(1):35-49.
45. Xu X, Wickens CD, Rantanen EM. Effects of conflict alerting system reliability and task difficulty on pilots' conflict detection with cockpit display of traffic information. Ergonomics 2007;50(1):112-130.
46. Bahner J, Huper A-D, Manzey D. Misuse of automated decision aids: complacency, automation bias and the impact of training experience. Int J Hum Comp Stud 2008;66(9):688-699.
47. Bahner JE, Elepfandt MF, Manzey D. Misuse of diagnostic aids in process control: The effects of automation misses on complacency and automation bias. Proceedings of the Human Factors and Ergonomics Society Annual Meeting, September 22-26, 2008. New York: Human Factors and Ergonomics Society; 2008: 1330-1334.
48. Manzey D, Reichenbach J, Onnasch L. Human performance consequences of automated decision aids: the impact of degree of automation and system experience. J Cogn Eng Decis Mak 2012;6(1):57-87.
49. Reichenbach J, Onnasch L, Manzey D. Misuse of automation: the impact of system experience on complacency and automation bias in interaction with automated aids. Proceedings of the Human Factors and Ergonomics Society Annual Meeting, September 27 to October 1, 2010. San Francisco, Calif.: Human Factors and Ergonomics Society; 2010: 374-378.
50. Reichenbach J, Onnasch L, Manzey D. Human performance consequences of automated decision aids in states of sleep loss. Human Factors 2011;53(6):717-728.
51. Rice S, McCarley JS. Effects of response bias and judgment framing on operator use of an automated aid in a target detection task. J Exp Psychol 2011;17(4):320-331.
52. Bertovic M, Fahlbruch B, Mueller C. Human factors perspective on the reliability of NDT in nuclear applications. Mater Test 2013;55(4):243-253.
53. Huiyang L, Wickens CD, SarterN, Sebok A. Stages and levels of automation in support of space teleoperations. HumFactors 2014;56(6):1050-1061.
54. Wickens CD, Dixon SR. The benefits of imperfect diagnostic automation: a synthesis of the literature. Theoretical Issues Ergonomics Sci 2007;8(3):201-212.
55. Dixon SR, Wickens CD. Automation reliability in unmanned aerial vehicle control: a reliance-compliance model of automation dependence in high workload. Hum Factors 2006;48(3):474-486.
56. Amalberti R, Wioland L. Human error in aviation. In: Soekkha HM, editor. Aviation Safety. The Netherlands: VSP; 1997. p. 91-108.
57. Sweller J, Ayres P, Kalyuga S. Cognitive Load Theory. New York: Springer; 2011.
58. Sintchenko V, Coiera EW. Which clinical decisions benefit fromautomation? A task complexity approach. Int JMed Inform2003;70(2-3):309-316.
59. Coiera E. The science of interruption. BMJ Qual Saf 2012;21(5):357-360.
60. Goddard K. Automation Bias and Prescribing Decision Support-rates, Mediators and Mitigators. Unpublished doctoral thesis, City University London; 2012.