Teaching Receiver Operating Characteristic Analysis. An Interactive Laboratory Exercise

Academic Radiology

Volumn 19, Issue 12, 2012, Pages 1452-1456

  Eng, John ^a  

^a JOHNS HOPKINS UNIVERSITY SCHOOL OF MEDICINE   (United States)

Author keywords

Diagnostic accuracy; Education; Observer study; Receiver operating characteristic analysis; ROC analysis

Indexed keywords

AREA UNDER THE CURVE; ARTICLE; CALCULATION; CONTROLLED STUDY; FRACTURE; HUMAN; LEARNING; MEDICAL STUDENT; PRIORITY JOURNAL; RADIOLOGY; RECEIVER OPERATING CHARACTERISTIC;

AREA UNDER CURVE; HUMANS; INTERNSHIP AND RESIDENCY; RADIOGRAPHIC IMAGE INTERPRETATION, COMPUTER-ASSISTED; RADIOLOGY; ROC CURVE; SENSITIVITY AND SPECIFICITY; TEACHING;

EID: 84868300152     PISSN: 10766332     EISSN: 18784046     Source Type: Journal    
DOI: 10.1016/j.acra.2012.09.003     Document Type: Article

References (21)

1. Lusted L.B. Signal detectability and medical decision-making. Science 1971, 171:1217-1219.
2. Metz C.E. Basic principles of ROC analysis. Semin Nucl Med 1978, 8:283-298.
3. Metz C.E. ROC methodology in radiologic imaging. Invest Radiol 1986, 21:720-733.
4. Hanley J.A., McNeil B.J. The meaning and use of the area under a receiver operating characteristic (ROC) curve. Radiology 1982, 143:29-36.
5. Obuchowski N.A. ROC analysis. Am J Roentgenol 2005, 184:364-372.
6. Eng J. Receiver operating characteristic analysis: a primer. Acad Radiol 2005, 12:909-916.
7. Metz C.E. ROC analysis in medical imaging: a tutorial review of the literature. Radiol Phys Technol 2008, 1:2-12.
8. Scott W.W., Bluemke D.A., Mysko W.K., et al. Interpretation of emergency department radiographs by radiologists and emergency medicine physicians: teleradiology workstation versus radiograph readings. Radiology 1995, 195:223-229.
9. Eng J., Mysko W.K., Weller G.E., et al. Interpretation of emergency department radiographs: a comparison of emergency medicine physicians with radiologists, residents with faculty, and film with digital display. Am J Roentgenol 2000, 175:1233-1238.
10. Eng J. ROC analysis: web-based calculator for ROC curves. Accessed August 31, 2012. http://www.jrocfit.org.
11. Dorfman D.D., Alf E. Maximum likelihood estimation of parameters of signal detection theory and determination of confidence intervals: rating method data. J Math Psychol 1969, 6:487-496.
12. Metz C.E. Some practical issues of experimental design and data analysis in radiological ROC studies. Invest Radiol 1989, 24:234-245.
13. Berbaum K.S., El-Khoury G.Y., Franken E.A., et al. Impact of clinical history on fracture detection with radiography. Radiology 1988, 168:507-511.
14. Berbaum K.S., Franken E.A., El-Khoury G.Y. Impact of clinical history on radiographic detection of fractures: a comparison of radiologists and orthopedists. Am J Roentgenol 1989, 153:1221-1224.
15. Loy C.T., Irwig L. Accuracy of diagnostic tests read with and without clinical information: a systematic review. JAMA 2004, 292:1602-1609.
16. Gur D., Bandos A.I., Rockette H.E., et al. Is an ROC-type response truly always better than a binary response in observer performance studies?. Acad Radiol 2010, 17:639-645.
17. Egglin T.K., Feinstein A.R. Context bias: a problem in diagnostic radiology. JAMA 1996, 276:1752-1755.
18. Yoon L.S., Haims A.H., Brink J.A., et al. Evaluation of an emergency radiology quality assurance program at a level I trauma center: abdominal and pelvic CT studies. Radiology 2002, 224:42-46.
19. Hofvind S., Geller B.M., Rosenberg R.D., et al. Screening-detected breast cancers: discordant independent double reading in a population-based screening program. Radiology 2009, 253:652-660.
20. Dorfman D.D., Berbaum K.S., Metz C.E. Receiver operating characteristic analysis: generalization to the population of readers and patients with the jackknife method. Invest Radiol 1992, 27:723-731.
21. Hillis S.L., Berbaum K.S., Metz C.E. Recent developments in the Dorfman-Berbaum-Metz procedure for mulitreader ROC study analysis. Acad Radiol 2008, 15:647-661.