Establishing a Gold Standard for Test Sets. Variation in Interpretive Agreement of Expert Mammographers

Academic Radiology

Volumn 20, Issue 6, 2013, Pages 731-739

  Onega, Tracy ^a,b   Anderson, Melissa L ^c   Miglioretti, Diana L ^c,d,e,o   Buist, Diana S M ^c   Geller, Berta ^f,g   Bogart, Andy ^c   Smith, Robert A ^f   Sickles, Edward A ^h   Monsees, Barbara ⁱ   Bassett, Lawrence ^j   Carney, Patricia A ^k,l   Kerlikowske, Karla ^g,h,m   Yankaskas, Bonnie C ⁿ  

^a NORRIS COTTON CANCER CENTER   (United States)

^b DARTMOUTH HITCHCOCK MEDICAL CENTER   (United States)

^c GROUP HEALTH RESEARCH INSTITUTE   (United States)

^d UNIVERSITY OF WASHINGTON   (United States)

^e UNIVERSITY OF CALIFORNIA   (United States)

^f UNIVERSITY OF VERMONT COLLEGE OF MEDICINE   (United States)

^g UNIVERSITY OF VERMONT   (United States)

^h UNIVERSITY OF CALIFORNIA   (United States)

ⁱ WASHINGTON UNIVERSITY   (United States)

^j UNIVERSITY OF CALIFORNIA   (United States)

^k OREGON HEALTH AND SCIENCE UNIVERSITY   (United States)

^l OREGON HEALTH AND SCIENCE UNIVERSITY   (United States)

^m UNIVERSITY OF CALIFORNIA   (United States)

ⁿ UNIVERSITY OF NORTH CAROLINA SCHOOL OF MEDICINE   (United States)

^o AMERICAN CANCER SOCIETY   (United States)

more..

Author keywords

Expert opinion; Gold standard; Mammography; Test set; Variation

Indexed keywords

ADULT; AGED; ARTICLE; CALCIFICATION; CANCER SIZE; CLINICAL ASSESSMENT; CLINICAL EVALUATION; CLINICAL PRACTICE; FEMALE; GOLD STANDARD; HUMAN; MAJOR CLINICAL STUDY; MAMMOGRAPHY; PRIORITY JOURNAL; RADIOLOGIST; RECALL; SCREENING TEST;

ADULT; AGED; BREAST NEOPLASMS; FEMALE; HUMANS; MAMMOGRAPHY; MIDDLE AGED; OBSERVER VARIATION; PREVALENCE; PROFESSIONAL COMPETENCE; REFERENCE STANDARDS; REGISTRIES; REPRODUCIBILITY OF RESULTS; SENSITIVITY AND SPECIFICITY; UNITED STATES;

EID: 84877340589     PISSN: 10766332     EISSN: 18784046     Source Type: Journal    
DOI: 10.1016/j.acra.2013.01.012     Document Type: Article

References (24)

1. Elmore J.G., Miglioretti D.L., Carney P.A. Does practice make perfect when interpreting mammography? Part II. JNCI 2003, 95:250-252.
2. Nodine C.F., Kundel H.L., Mello-Thomas C., et al. How experience and training influence mammography expertise. Acad Radiol 1999, 6:575-585.
3. Elmore J.G., Wells C.K., Lee C.H., et al. Variability in radiologists' interpretations of mammograms. New Engl J Med 1994, 331:1493-1499.
4. Elmore J.G., Jackson S.L., Abraham L., et al. Variability in interpretive performance at screening mammography and radiologists' characteristics associated with accuracy. Radiology 2009, 253:641-651.
5. Baker J.A., Kornguth P.J., Floyd C.E. Breast imaging reporting and data system standardized mammography lexicon: observer variability in lesion description. Am J Roentgenol 1996, 166:773-778.
6. Berg W.A., Campassi C., Langenberg P., et al. Breast Imaging Reporting and Data System: inter- and intraobserver variability in feature analysis and final assessment. Am J Roentgenol 2000, 174:1769-1777.
7. Kerlikowske K., Grady D., Barclay J., et al. Variability and accuracy in mammographic interpretation using the American College of Radiology Breast Imaging Reporting and Data System. J Nat Cancer Inst 1998, 90:1801-1809.
8. Kundel H.L., Polansky M. Measurement of observer agreement. Radiology 2003, 228:303-308.
9. Elmore J.G., Wells C.K., Lee C.H., et al. Variability in radiologists' interpretations of mammograms. N Engl J Med 1994, 331:1493-1499.
10. ® assessment categories (the Florence experience). Breast 2006, 15:44-51.
11. Revesz G., Kundel H.L., Bonitatibus M. The effect of verification on the assessment of imaging techniques. Invest Radiol 1990, 25:461-464.
12. Ballard-Barbash R., Taplin S.H., Yankaskas B.C., et al. Breast Cancer Surveillance Consortium: a national mammography screening and outcomes database. AJR Am J Roentgenol 1997, 169:1001-1008.
13. National Cancer Institute. Breast Cancer Surveillance Consortium Homepage. Accessed April 14, 2009. http://breastscreening.cancer.gov/.
14. Carney P.A., Geller B.M., Moffett H., et al. Current medico-legal and confidentiality issues in large multi-center research programs. Am J Epidemiol 2000, 152:371-378.
15. Carney P.A., Bogart T.A., Geller B.M., et al. Association between time spent interpreting, level of confidence, and accuracy of screening mammography. AJR Am J Roentgenol 2012, 198:970-978.
16. American College of Radiology (ACR) ACR BI-RADS - Mammography. ACR Breast Imaging and Reporting and Data System, Breast Imaging Atlas 2003, American College of Radiology, Reston, VA. 4th ed.
17. Hukkinen K., Kivisaari L., Vehmas T. Impact of the number of readers on mammography interpretation. Acta Radiol 2006, 47:655-659.
18. Shaw C.M., Flanagan F.L., Fenlon H.M., et al. Consensus review of discordant findings maximizes cancer detection rate in double-reader screening mammography: Irish National Breast Screening Program experience. Radiology 2009, 250:354-362.
19. Duijm L.E., Louwman M.W., Groenewoud J.H., et al. Inter-observer variability in mammography screening and effect of type and number of readers on screening outcome. Br J Cancer 2009, 100:901-907.
20. Robinson P.J.A., Wilson D., Coral A., et al. Variation between experienced observers in the interpretation of accident and emergency radiographs. Br J Radiol 1999, 72:323-330.
21. Elsheikh T.M., Asa S.L., Chan J.K.C., et al. Interobserver and intraobserver variation among experts in the diagnosis of thyroid follicular lesions with borderline nuclear features of papillary carcinoma. Am J Clin Pathol 2008, 130:736-744.
22. Brealey S., Scally A.J. Bias in plain film reading performance studies. Br J Radiol 2001, 74:307-316.
23. Brealey S.D., Scally A.J., Hahn S., et al. Evidence of reference standard related bias in studies of plain radiograph reading performance: a meta-regression. Br J Radiol 2007, 80:406-413.
24. Sickles E.A. The American College of Radiology's mammography interpretive skills assessment (MISA) examination. Semin Breast Dis 2003, 6:133-139.