Improving the radiologist-CAD interaction: Designing for appropriate trust

Clinical Radiology

Volumn 70, Issue 2, 2015, Pages 115-122

  Jorritsma, W ^a   Cnossen, F ^b   Van Ooijen, P M A ^a,b  

^a UNIVERSITY MEDICAL CENTER GRONINGEN   (Netherlands)

^b UNIVERSITY OF GRONINGEN   (Netherlands)

Author keywords

[No Author keywords available]

Indexed keywords

ALGORITHM; AUTOMATION; CALIBRATION; COMPREHENSION; COMPUTER AIDED DESIGN; COMPUTER AIDED DIAGNOSIS; DECISION MAKING; DIAGNOSTIC ERROR; HUMAN; PHYSICIAN ATTITUDE; PRIORITY JOURNAL; PROFESSIONAL COMPETENCE; RADIOLOGIST; REVIEW; TASK PERFORMANCE; TRUST; ATTITUDE TO COMPUTERS; COMPUTER ASSISTED DIAGNOSIS; PROCEDURES; RADIOLOGY; REPRODUCIBILITY;

ATTITUDE TO COMPUTERS; HUMANS; RADIOGRAPHIC IMAGE INTERPRETATION, COMPUTER-ASSISTED; RADIOLOGY; REPRODUCIBILITY OF RESULTS; TRUST;

EID: 84925232476     PISSN: 00099260     EISSN: 1365229X     Source Type: Journal    
DOI: 10.1016/j.crad.2014.09.017     Document Type: Review

References (82)

1. Beam CA, Layde PM, Sullivan DC. Variability in the interpretation of screening mammograms by US radiologists: findings from a national sample. Arch Intern Med 1996;156:209-13.
2. Berg WA, Campassi C, Langenberg P, et al. Breast imaging reporting and data system: inter- and intraobserver variability in feature analysis and final assessment. Am J Roentgenol 2000;174:1769-77.
3. Kundel HL, Nodine CF, Carmody D. Visual scanning, pattern recognition and decision-making in pulmonary nodule detection. Invest Radiol 1978;13:175-81.
4. Austin JH, Romney BM, Goldsmith LS. Missed bronchogenic carcinoma: radiographic findings in 27 patients with a potentially resectable lesion evident in retrospect. Radiology 1992;182:115-22.
5. Pinto A, Brunese L. Spectrum of diagnostic errors in radiology. World J Radiol 2010;2:377-83.
6. Fitzgerald R. Error in radiology. Clin Radiol 2001;56:938-46.
7. Robinson PJA. Radiology's Achilles' heel: error and variation in the interpretation of the Röntgen image. Br J Radiol 1997;70:1085-98.
8. Renfrew DL, Franken EA, Berbaum KS, et al. Error in radiology: classification and lessons in 182 cases presented at a problem case conference. Radiology 1992;183:145-50.
9. Martin JE, Moskowitz M, Milbrath JR. Breast cancer missed by mammography. Am J Roentgenol 1979;132:737-9.
10. Chan H, Doi K, Vyborny CJ, et al. Improvement in radiologists' detection of clustered microcalcifications on mammograms: the potential of computer-aided diagnosis. Invest Radiol 1990;25:1102-10.
11. Nawano S, Murakami K, Moriyama N, et al. Computer-aided diagnosis in full digital mammography. Invest Radiol 1999;34:310-6.
12. Awai K, Murao K, Ozawa A, et al. Pulmonary nodules at chest CT: effect of computer-aided diagnosis on radiologists' detection performance. Radiology 2004;230:347-52.
13. Li F, Aoyama M, Shiraishi J, et al. Radiologists' performance for differentiating benign from malignant lung nodules on high-resolution CT using computer-estimated likelihood of malignancy. Am J Roentgenol 2004;183:1209-15.
14. Kasai S, Li F, Shiraishi J, et al. Usefulness of computer-aided diagnosis schemes for vertebral fractures and lung nodules on chest radiographs. Am J Roentgenol 2008;191:260-5.
15. White CS, Pugatch R, Koonce T, et al. Lung nodule CAD software as a second reader: a multicenter study. Acad Radiol 2008;15:326-33.
16. Drew T, Cunningham C, Wolfe JM. When and why might a computeraided detection (CAD) system interfere with visual search? An eyetracking study. Acad Radiol 2012;19:1260-7.
17. Nishikawa RM, Schmidt RA, Linver MN, et al. Clinically missed cancer: how effectively can radiologists use computer-aided detection? Am J Roentgenol 2012;198:708-16.
18. Brem RF, Schoonjans JM. Radiologist detection of microcalcifications with and without computer-aided detection: a comparative study. Clin Radiol 2001;56:150-4.
19. Taylor PM, Champness J, Given-Wilson RM, et al. An evaluation of the impact of computer-based prompts on screen readers' interpretation of mammograms. Br J Radiol 2004;77:21-7.
20. Taylor P, Given-Wilson R, Champness J, et al. Assessing the impact of CAD on the sensitivity and specificity of film readers. Clin Radiol 2004;59:1099-105.
21. Zheng B, Swensson RG, Golla S, et al. Detection and classification performance levels of mammographic masses under different computeraided detection cueing environments. Acad Radiol 2004;11:398-406.
22. Gur D, Sumkin JH, Rockette HE, et al. Changes in breast cancer detection and mammography recall rates after the introduction of a computeraided detection system. J Natl Cancer Inst 2004;96:185-90.
23. De Hoop B, De Boo DW, Gietema HA, et al. Computer-aided detection of lung cancer on chest radiographs: effect on observer performance. Radiology 2010;257:532-40.
24. Brown MS, Goldin JG, Rogers S, et al. Computer-aided lung nodule detection in CT: results of large-scale observer test. Acad Radiol 2005;12:681-6.
25. Petrick N, Haider M, Summers RM, et al. CT colonography with computer-aided detection as a second reader: observer performance study. Radiology 2008;246:148-56.
26. Povyakalo AA, Alberdi E, Strigini L, et al. How to discriminate between computer-aided and computer-hindered decisions: a case study in mammography. Med Decis Mak 2013;33:98-107.
27. Lee JD, Moray N. Trust, self-confidence, and operators' adaptation to automation. Int J Hum Comput Stud 1994;40:153-84.
28. Lee J, Moray N. Trust, control strategies and allocation of function in human-machine systems. Ergonomics 1992;35:1243-70.
29. Muir BM. Trust in automation: part I. Theoretical issues in the study of trust and human intervention in automated systems. Ergonomics 1994;37:1905-22.
30. Muir BM. Trust between humans and machines, and the design of decision aids. IntJ Man Mach Stud 1987;27:527-39.
31. Wickens CD, Hollands JG. Engineering psychology and human performance. 3rd ed. Upper Saddle River, NJ: Prentice Hall; 1999.
32. Lee JD, See KA. Trust in automation: designing for appropriate reliance. Hum Factors 2004;46:50-80.
33. Dzindolet MT, Peterson SA, Pomranky RA, et al. The role of trust in automation reliance. IntJ Hum Comput Stud 2003;58:697-718.
34. Parasuraman R, Riley V. Humans and automation: use, misuse, disuse, abuse. Hum Factors 1997;39:230-53.
35. Dzindolet MT, Pierce LG, Beck HP, et al. The perceived utility of human and automated aids in a visual detection task. Hum Factors 2002;44:79-94.
36. Beck HP, Dzindolet MT, Pierce LG. Automation usage decisions: controlling intent and appraisal errors in a target detection task. Hum Factors 2007;49:429-37.
37. Wang L, Jamieson GA, Hollands JG. Trust and reliance on an automated combat identification system. Hum Factors 2009;51:281-91.
38. Nishikawa RM, Edwards A, Schmidt RA, et al. Can radiologists recognize that a computer has identified cancers that they have overlooked?. In: Proceedings of SPIE 6146. San Diego, CA: SPIE; 2006. p. 1-8.
39. De Boo DW, Uffmann M, Weber M, et al. Computer-aided detection of small pulmonary nodules in chest radiographs: an observer study. Acad Radiol 2011;18:1507-14.
40. Lee KH, Goo JM, Park CM, et al. Computer-aided detection of malignant lung nodules on chest radiographs: effect on observers' performance. Korean J Radiol 2012;13:564-71.
41. Halligan S, Altman DG, Mallett S, et al. Computed tomographic colonography: assessment of radiologist performance with and without computer-aided detection. Gastroenterology 2006;131:1690-9.
42. Taylor SA, Robinson C, Boone D, et al. Polyp characteristics correctly annotated by computer-aided detection software but ignored by reporting radiologists during CT colonography. Radiology 2009;253:715-23.
43. Shiraishi J, Abe H, Engelmann R, et al. Computer-aided diagnosis to distinguish benign from malignant solitary pulmonary nodules on radiographs: ROC analysis of radiologists' performancedinitial experience. Radiology 2003;227:469-74.
44. Skitka LJ, Mosier KL, Burdick M. Does automation bias decision-making? Int J Hum Comput Stud 1999;51:991-1006.
45. Itoh M. Toward over-trust-free advanced driver assistance systems. Cogn Technol Work 2012;14:51-60.
46. Alberdi E, Povyakalo A, Strigini L, et al. Effects of incorrect computeraided detection (CAD) output on human decision-making in mammography. Acad Radiol 2004;11:909-18.
47. Povyakalo AA, Alberdi E, Strigini L, et al. Evaluating "human + advisory computer" systems: a case study. In: Proceedings of the 18th British HCI Group Annual Conference. Leeds: British HCI Group; 2004. p. 93-6.
48. Zheng B, Ganott MA, Britton CA, et al. Breast imaging soft-copy mammographic readings with different computer-assisted detection cuing environments: preliminary findings. Radiology 2001;221:633-40.
49. Beyer F, Zierott L, Fallenberg EM, et al. Comparison of sensitivity and reading time for the use of computer-aided detection (CAD) of pulmonary nodules at MDCT as concurrent or second reader. Eur Radiol 2007;17:2941-7.
50. Gilbert FJ, Astley SM, McGee MA, et al. Single reading with computeraided detection and double reading of screening mammograms in the United Kingdom national breast screening program. Radiology 2006;241:47-53.
51. Shiraishi J, Abe H, Li F, et al. Computer-aided diagnosis for the detection and classification of lung cancers on chest radiographs: ROC analysis of radiologists' performance. Acad Radiol 2006;13:995-1003.
52. Samulski M, Hupse R, Boetes C, et al. Using computer-aided detection in mammography as a decision support. Eur Radiol 2010;20:2323-30.
53. Hupse R, Samulski M, Lobbes M, et al. Standalone computer-aided detection compared to radiologists' performance for the detection of mammographic masses. Eur Radiol 2013;23:93-100.
54. Chan H, Sahiner B, Helvie MA, et al. Improvement of radiologists' characterization of mammographic masses by using computer-aided diagnosis: an ROC study. Radiology 1999;212:817-27.
55. Hadjiiski L, Chan H, Sahiner B, et al. Improvement in radiologists' characterization of malignant and benign breast masses on serial mammograms with computer-aided diagnosis: an ROC study. Radiology 2004;233:255-65.
56. Horsch K, Giger ML, Vyborny CJ, et al. Classification of breast lesions with multimodality computer-aided diagnosis: observer study results on an independent clinical data set. Radiology 2006;240:357-68.
57. Jiang Y, Nishikawa RM, Schmidt RA, et al. Improving breast cancer diagnosis with computer-aided diagnosis. Acad Radiol 1999;6:22-33.
58. Kashikura Y, Nakayama R, Hizukuri A, et al. Improved differential diagnosis of breast masses on ultrasonographic images with a computer-aided diagnosis scheme for determining histological classifications. Acad Radiol 2013;20:471-7.
59. Awai K, Murao K, Ozawa A, et al. Pulmonary nodules: estimation of malignancy at thin-section helical CTdeffect of computer-aided diagnosis on performance of radiologists. Radiology 2006;239:276-84.
60. Matsuki Y, Nakamura K, Watanabe H, et al. Usefulness of an artificial neural network for differentiating benign from malignant pulmonary nodules on high-resolution CT. Am J Roentgenol 2002;178:657-63.
61. Andre AD, Cutler HA. Displaying uncertainty in advanced navigation systems. In: Proceedings of the Human Factors and Ergonomics Society 42nd Annual Meeting. Santa Monica, CA: Human Factors and Ergonomics Society; 1998. p. 31-5.
62. McGuirl JM, Sarter NB. Supporting trust calibration and the effective use of decision aids by presenting dynamic system confidence information. Hum Factors 2006;48:656-65.
63. Beller J, Heesen M, Vollrath M. Improving the driver-automation interaction: an approach using automation uncertainty. Hum Factors 2013;55:1130-41.
64. Madhavan P, Wiegmann DA, Lacson FC. Automation failures on tasks easily performed by operators undermine trust in automated aids. Hum Factors 2006;48:241-56.
65. Hartswood M, Procter R, Rouncefield M, et al. "Repairing" the machine: a case study of the evaluation of computer-aided detection tools in breast screening. In: Proceedings of the 8th European Conference on Computer-Supported Cooperative Work. Helsinki: Kluwer Academic Publishers; 2003. p. 375-94.
66. Malich A, Sauner D, Marx C, et al. Influence of breast lesion size and histologic findings on tumor detection rate of a computer-aided detection system. Radiology 2003;228:851-6.
67. Bahner JE, Huper A, Manzey D. Misuse of automated decision aids: complacency, automation bias and the impact of training experience. Int J Hum Comput Stud 2008;66:688-99.
68. Shortliffe EH, Davis R, Axline SG, et al. Computer-based consultations in clinical therapeutics: explanation and rule acquisition capabilities of the MYCIN system. Comput Biomed Res 1975;8:303-20.
69. Ye LR, Johnson PE. The impact of explanation facilities on user acceptance of expert systems advice. MIS Q 1995;19:157-72.
70. Eining MM, Jones DR, Loebbecke JK. Reliance on decision aids: an examination of auditors' assessment of management fraud. Audit A J Pract Theory 1997;16:1-19.
71. Herlocker JL, Konstan JA, Riedl J. Explaining collaborative filtering recommendations. In: Proceedings of the 2000 ACM Conference on Computer Supported Cooperative Work. Philadelphia: ACM Press; 2000. p. 241-50.
72. Cramer H, Evers V, Ramlal S, et al. The effects of transparency on trust in and acceptance of a content-based art recommender. User Model User-Adapt Interact 2008;18:455-96.
73. Wang W, Benbasat I. Recommendation agents for electronic commerce: effects of explanation facilities on trusting beliefs. J Manag Inf Syst 2007;23:217-46.
74. Hologic, ImageChecker CAD [software]. 2007. http://hologic.com/products/imaging/mammography/image-analytics. [Accessed April 1, 2014].
75. Gurung J, Maataoui A, Khan M, et al. Automated detection of lung nodules in multidetector CT: influence of different reconstruction protocols on performance of a software prototype. Fortschr Röntgenstr 2006;178:71-7.
76. Marten K, Grillhösl A, Seyfarth T, et al. Computer-assisted detection of pulmonary nodules: evaluation of diagnostic performance using an expert knowledge-based detection system with variable reconstruction slice thickness settings. Eur Radiol 2005;15:203-12.
77. Petrick N, Sahiner B, Armato SG, et al. Evaluation of computer-aided detection and diagnosis systems. Med Phys 2013;40:087001.
78. Roy MC, Dewit O, Aubert BA. The impact of interface usability on trust in web retailers. Internet Res 2001;11:388-98.
79. Huo Z, Summers RM, Paquerault S, et al. Quality assurance and training procedures for computer-aided detection and diagnosis systems in clinical use. Med Phys 2013;40:077001.
80. Wang Y, Jiang S, Wang H, et al. CAD algorithms for solid breast masses discrimination: evaluation of the accuracy and interobserver variability. Ultrasound Med Biol 2010;36:1273-81.
81. Balleyguier C, Kinkel K, Fermanian J, et al. Computer-aided detection (CAD) in mammography: does it help the junior or the senior radiologist? Eur J Radiol 2005;54:90-6.
82. Kobayashi T, Xu XW, MacMahon H, et al. Effect of a computer-aided diagnosis scheme on radiologists' performance in detection of lung nodules on radiographs. Radiology 1996;199:843-8.