Assessing reader performance in radiology, an imperfect science: Lessons from breast screening

Clinical Radiology

Volumn 67, Issue 7, 2012, Pages 623-628

  Soh, B P ^a   Lee, W ^a   Kench, P L ^a   Reed, W M ^a   McEntee, M F ^a   Poulos, A ^a   Brennan, P C ^a  

^a UNIVERSITY OF SYDNEY   (Australia)

Author keywords

[No Author keywords available]

Indexed keywords

ACCURACY; BREAST CANCER; CANCER SCREENING; COMPUTER PROGRAM; EXTERNAL VALIDITY; HUMAN; MAMMOGRAPHY; MEDICAL AUDIT; PRIORITY JOURNAL; RADIOLOGY; READING; RECEIVER OPERATING CHARACTERISTIC; REVIEW; SCREENING TEST; SENSITIVITY AND SPECIFICITY; STANDARDIZATION;

BREAST NEOPLASMS; CLINICAL COMPETENCE; FEMALE; HUMANS; MAMMOGRAPHY; OBSERVER VARIATION;

EID: 84861602704     PISSN: 00099260     EISSN: 1365229X     Source Type: Journal    
DOI: 10.1016/j.crad.2012.02.007     Document Type: Review

References (53)

1. WHO international agency for research on cancer World cancer report Available at 2003 http://www.iarc.fr/en/publications/pdfs-online/wcr/2003/index. php [accessed 10.06.11]
2. Australian institute of health and welfare (AIHW), Cancer Australia & Australasian association of cancer registries 2008 Cancer survival and prevalence in Australia: cancers diagnosed from 1982 to 2004 Cancer Series no. 42. Cat. no. CAN 38 2008 AIHW Canberra
3. K. Kerlikowske, D. Grady, and S.M. Rubin Efficacy of screening mammography - a meta-analysis J Am Med Assoc 273 1995 149 154
4. BreastScreen Australia program About the program - background Available at http://www.cancerscreening.gov.au/internet/screening/publishing.nsf/Content/ breastscreen-about [accessed 24.04.09]
5. C.A. Beam, P.M. Layde, and D.C. Sullivan Variability in the interpretation of screening mammograms by US radiologists. Findings from a national sample Arch Intern Med 156 1996 209 213
6. J.G. Elmore, C.K. Wells, and C.H. Lee Variability in radiologists' interpretations of mammograms New Engl J Med 331 1994 1493 1499
7. L.G. Kessler, M.R. Andersen, and R. Etzioni Much ado about mammography variability J Natl Cancer Inst 94 2002 1346 1347
8. J.G. Elmore, D.L. Miglioretti, and L.M. Reisch Screening mammograms by community radiologists: variability in false-positive rates J Natl Cancer Inst 94 2002 1373 1380
9. R.E. Bird, T.W. Wallace, and B.C. Yankaskas Analysis of cancers missed at screening mammography Radiology 184 1992 613 617
10. BreastScreen Australia evaluation Evaluation final report Available at http://www.health.gov.au/internet/screening/publishing.nsf/Content/ 8463830B90E5BDF5CA25762A000193C6/$File/Breastscreen%20Aust-REPORT.pdf [accessed 11.11.11]
11. J. Patnick Annual review 2005: One vision: NHS Breast screening programme 2005 Fulwood House Sheffield
12. M. Kossoff, L. Brothers, and J. Cawson BreastScreen Australia: how we handle variability in interpretative skills Semin Breast Dis 6 2003 123 127
13. R.G. Blanks, R.L. Bennett, and M.G. Wallis Does individual programme size affect screening performance? Results from the United Kingdom NHS breast screening programme J Med Screen 9 2002 11 14
14. P.C. Brennan, and B.P. Soh The challenge of assessing reader performance in mammography Clin Radiol 67 2012 192
15. R.L. Bennett, and R.G. Blanks Should a standard be defined for the positive predictive value (PPV) of recall in the UK NHS breast screening programme? Breast 16 2007 55 59
16. C. Balleyguier, S. Ayadi, and K.V. Nguyen BIRADS classification in mammography Eur J Radiol 61 2007 192 194
17. A. Gale PERFORMS: a self-assessment scheme for radiologists in breast screening Semin Breast Dis 6 2003 148 152
18. H.J. Scott, and A.G. Gale Breast screening: PERFORMS identifies key mammographic training needs Br J Radiol 79 2006 S127 S133
19. BREAST BreastScreen reader assessment strategy Available at http://www.breastaustralia.com/ [accessed 20.10.11]
20. S.D. Levitt, and J.A. List What do laboratory experiments measuring social preferences reveal about the real world? J Econ Perspect 21 2007 153 174
21. S.D. Levitt, J.A. List, and N. Bardsley What do laboratory experiments tell us about the real world? JEL: C9 (Design of Experiments), C93 (Field Experiments) J Econ Perspect 21 2006 153 174
22. L.D. Introna, and E.A. Whitley About experiments and style: a critique of laboratory research in information system Inform Technol People 13 2000 161 173
23. D.G. Mook In defence of external invalidity Am Psychol 38 1983 379 387
24. C. Cheng Assessing coping flexibility in real-life and laboratory settings: a multimethod approach J Pers Soc Psychol 80 2001 814 833
25. J.K. Esser, and R.G. Marriott Mediation tactics: a comparison of field and laboratory research J Appl Soc Psychol 25 1995 1530 1546
26. A. Sunderland, J.E. Harris, and A.D. Baddeley Do laboratory tests predict everyday memory? A Neuropsychological study J Verb Learn Verb Beh 22 1983 341 357
27. R.R. Burke, B.A. Harlam, and B.E. Kahn Comparing dynamic consumer choice in real and computer-simulated environments J Consum Res 19 1992 71 82
28. C.M. Rutter, and S. Taplin Assessing mammographers' accuracy: a comparison of clinical and test performance J Clin Epidemiol 53 2000 443 450
29. H.J. Scott, A. Evan, and A.G. Gale The relationship between real life breast screening and an annual self-assessment scheme Proc SPIE 7263 2009 72631E-1-72631E-7
30. D. Gur, A.L. Bandos, and C.S. Cohen The "laboratory" effect: comparing radiologists' performance and variability during prospective clinical and laboratory mammography interpretations Radiology 249 2008 47 53
31. L. Berkowitz, and E. Donnerstein External validity is more than skin deep - some answers to criticisms of laboratory experiments Am Psychol 37 1982 245 257
32. G. Gigerenzer External validity of laboratory experiments: the frequency-validity relationship Am J Psychol 97 1984 185 195
33. M. Bateson, D. Nettle, and G. Roberts Cues of being watched enhance cooperation in a real-world setting Biology letter 2 2006 412 414
34. M. Benz, and S. Meier Do people behave in experiments as in real Life? Evidence from donations Exper Econ 11 2008 268 281
35. E.R. Babbie The practice of social research 1975 Wadsworth Belmont, CA 254
36. M. Gurven, and J. Winking Collective action in action: prosocial behavior in and out of the laboratory Am Anthropol 110 2008 179 190
37. T.K. Egglin, and A.R. Feinstein Context bias. A problem in diagnostic radiology J Am Med Assoc 276 1996 1752 1755
38. Quality standards and certification requirements for mammography facilities - FDA Interim rule with request for comments Fed Regist 58 1993 67565 67572
39. W. Reed, A. Poulos, and M. Rickard Reader practice in mammography screen reporting in Australia JMIRO 53 2009 530 537
40. P.C. Brennan, J. Ryan, and M. Evanoff The impact of acoustic noise found within clinical departments on radiology performance Acad Radiol 15 2008 472 476
41. P.C. Brennan, M. McEntee, and M. Evanoff Ambient lighting: effect of illumination on soft-copy viewing of radiographs of the wrist AJR Am J Roentgenol 188 2007 177 180
42. E. McCarthy, and P.C. Brennan Viewing conditions for diagnostic images in three major Dublin hospitals: a comparison with WHO and CEC recommendations Br J Radiol 76 2003 94 97
43. C. Mello-Thoms Perception of breast cancer: eye-position analysis of mammogram interpretation Acad Radiol 10 2003 4 12
44. D.J. Manning, S. Ethell, and T. Donovan How do radiologists do it? The influence of experience and training on searching for chest nodules Radiography 12 2006 134 142
45. C.F. Nodine, H.L. Kundel, and S.C. Lauver Nature of expertise in searching mammograms for breast masses Acad Radiol 3 1996 1000 1006
46. E.A. Krupinski Visual scanning patterns of radiologists searching mammograms Acad Radiol 3 1996 137 144
47. C. Mello-Thoms, L. Hardesty, and J. Sumkin Effects of lesion conspicuity on visual search in mammogram reading Acad Radiol 12 2005 830 840
48. D. Gur, A.L. Bandos, and C.R. Fuhman The prevalence effect in a laboratory environment: changing the confidence ratings Acad Radiol 14 2007 49 53
49. W.M. Reed, J.T. Ryan, and M.F. McEntee The effect of abnormality - prevalence expectation on expect observer performance and visual search Radiology 258 2011 938 943
50. D.B. Larson Changing radiologists' expectations: false information versus years of experience Radiology 261 2011 327
51. J.M. Wolfe Explicit expectations and the effects of prevalence Radiology 261 2011 328
52. S.A. Nicolau, X. Pennec, and L. Soler An augmented reality system for liver thermal ablation: design and evaluation on clinical cases Med Image Anal 13 2009 494 506
53. S. Ciatto, D. Ambrogetti, and D. Morrone Analysis of the results of a proficiency test in screening mammography at the CSPO of Florence: review of 705 tests Radiol med 111 2006 797 803