Assessment of Radiologist Performance in the Detection of Lung Nodules. Dependence on the Definition of "Truth"

Academic Radiology

Volumn 16, Issue 1, 2009, Pages 28-38

  Armato III, Samuel G ^a   Roberts, Rachael Y ^a   Kocherginsky, Masha ^a   Aberle, Denise R ^b   Kazerooni, Ella A ^c   MacMahon, Heber ^a   van Beek, Edwin J R ^d   Yankelevitz, David ^f   McLennan, Geoffrey ^e   McNitt Gray, Michael F ^b   Meyer, Charles R ^c   Reeves, Anthony P ^g   Caligiuri, Philip ^a   Quint, Leslie E ^c   Sundaram, Baskaran ^c   Croft, Barbara Y ^h   Clarke, Laurence P ^h  

^a UNIVERSITY OF CHICAGO   (United States)

^b UNIVERSITY OF CALIFORNIA   (United States)

^c UNIVERSITY OF MICHIGAN MEDICAL SCHOOL   (United States)

^d UNIVERSITY OF IOWA   (United States)

^e UNIVERSITY OF IOWA   (United States)

^f WEILL CORNELL MEDICAL COLLEGE   (United States)

^g Cornell University   (United States)

^h NATIONAL CANCER INSTITUTE   (United States)

Author keywords

computed tomography (CT); computer aided diagnosis (CAD); interobserver variability; Lung nodule; thoracic imaging

Indexed keywords

ADULT; AGED; ARTICLE; CLINICAL ARTICLE; COMPUTER ASSISTED RADIOGRAPHY; COMPUTER ASSISTED TOMOGRAPHY; FALSE POSITIVE RESULT; FEMALE; HUMAN; IMAGE ANALYSIS; IMAGE PROCESSING; LUNG NODULE; MALE; PANEL STUDY; PRIORITY JOURNAL; RADIOLOGIST; TASK PERFORMANCE; TUMOR VOLUME;

ADULT; AGED; ARTIFACTS; FEMALE; HUMANS; LUNG NEOPLASMS; MALE; MIDDLE AGED; OBSERVER VARIATION; PROFESSIONAL COMPETENCE; RADIOGRAPHIC IMAGE INTERPRETATION, COMPUTER-ASSISTED; REPRODUCIBILITY OF RESULTS; SENSITIVITY AND SPECIFICITY; SOLITARY PULMONARY NODULE; TOMOGRAPHY, X-RAY COMPUTED;

EID: 57049184118     PISSN: 10766332     EISSN: None     Source Type: Journal    
DOI: 10.1016/j.acra.2008.05.022     Document Type: Article

References (20)

1. Wormanns D., Ludwig K., Beyer F., et al. Detection of pulmonary nodules at multirow-detector CT: effectiveness of double reading to improve sensitivity at standard-dose and low-dose chest CT. Eur Radiol 15 (2005) 14-22
2. Leader J.K., Warfel T.E., Fuhrman C.R., et al. Pulmonary nodule detection with low-dose CT of the lung: agreement among radiologists. Am J Roentgenol 185 (2005) 973-978
3. Novak C.L., Qian J., Fan L., et al. Inter-observer variations on interpretation of multi-slice CT lung cancer screening studies, and the implications for computer-aided diagnosis. SPIE Proc 4684 (2002) 68-79
4. Ochs R., Kim H.J., Angel E., et al. Forming a reference standard from LIDC data: impact of reader agreement on reported CAD performance. SPIE Proc 6514 (2007)
5. Paquerault S., Petrick N., Myers K.J., et al. Impact of a computer-aided detection (CAD) system on reader performance: assessment based on a truthing panel compared to the true gold standard. Radiology 245 (2007) 546-547
6. Petrick N., Gallas B.D., Samuelson F.W., et al. Influence of panel size and expert skill on truth panel performance when combining expert ratings. SPIE Proc 5749 (2005) 49-57
7. Armato III S.G., McLennan G., McNitt-Gray M.F., et al. Lung Image Database Consortium: developing a resource for the medical imaging research community. Radiology 232 (2004) 739-748
8. Armato III S.G., McNitt-Gray M.F., Reeves A.P., et al. The Lung Image Database Consortium (LIDC): an evaluation of radiologist variability in the identification of lung nodules on CT scans. Acad Radiol 14 (2007) 1409-1421
9. Dodd L.E., Wagner R.F., Armato III S.G., et al. Assessment methodologies and statistical issues for computer-aided diagnosis of lung nodules in computed tomography: contemporary research topics relevant to the Lung Image Database Consortium. Acad Radiol 11 (2004) 462-475
10. McNitt-Gray M.F., Armato III S.G., Meyer C.R., et al. The Lung Image Database Consortium (LIDC) data collection process for nodule detection and annotation. Acad Radiol 14 (2007) 1464-1474
11. Miller D.P., O'Shaughnessy K.F., Wood S.A., et al. Gold standards and expert panels: a pulmonary nodule case study with challenges and solutions. SPIE Proc 5372 (2004) 173-184
12. Revesz G., Kundel H.L., and Bonitatibus M. The effect of verification on the assessment of imaging techniques. Invest Radiol 18 (1983) 194-198
13. Jiang Y. A Monte Carlo simulation method to understand expert-panel consensus truth and double readings. Medical Image Perception Conference XII. 2007 (2007), The University of Iowa, Iowa City, IA
14. Clarke L.P., Croft B.Y., Staab E., et al. National Cancer Institute initiative: lung image database resource for imaging research. Acad Radiol 8 (2001) 447-450
15. Department of Health and Human Services. Standards for privacy of individually identifiable health information: final rules. Fed Register 67 (2002) 53182-53272
16. Kundel H.L., Nodine C.F., and Carmody D. Visual scanning, pattern recognition and decision-making in pulmonary nodule detection. Invest Radiol 13 (1978) 175-181
17. Meyer C.R., Johnson T.D., McLennan G., et al. Evaluation of lung MDCT nodule annotation across radiologists and methods. Acad Radiol 13 (2006) 1254-1265
18. Reeves A.P., Biancardi A.M., Apanasovich T.V., et al. The Lung Image Database Consortium (LIDC): a comparison of different size metrics for pulmonary nodule measurements. Acad Radiol 14 (2007) 1475-1485
19. Schwartz L.H., Ginsberg M.S., DeCorato D., et al. Evaluation of tumor measurements in oncology: Use of film-based and electronic techniques. J Clin Oncol 18 (2000) 2179-2184
20. Ross J.C., Miller J.V., Turner W.D., et al. An analysis of early studies released by the Lung Imaging Database Consortium (LIDC). Acad Radiol 14 (2007) 1382-1388