Comparing areas under receiver operating characteristic curves: Potential impact of the "last" experimentally measured operating point

Radiology

Volumn 247, Issue 1, 2008, Pages 12-15

  Gur, David ^a   Bandos, Andriy I ^b   Rockette, Howard E ^b  

^a UNIVERSITY OF PITTSBURGH SCHOOL OF MEDICINE   (United States)

^b UNIVERSITY OF PITTSBURGH GRADUATE SCHOOL OF PUBLIC HEALTH   (United States)

Author keywords

[No Author keywords available]

Indexed keywords

AREA UNDER THE CURVE; DIAGNOSTIC ACCURACY; FALSE POSITIVE RESULT; INTERMETHOD COMPARISON; LABORATORY DIAGNOSIS; MAMMOGRAPHY; NOTE; PRIORITY JOURNAL; RECEIVER OPERATING CHARACTERISTIC; SCREENING TEST; ARTICLE; COMPARATIVE STUDY; DIAGNOSTIC IMAGING; HUMAN; ROC CURVE; STATISTICAL ANALYSIS;

AREA UNDER CURVE; DATA INTERPRETATION, STATISTICAL; DIAGNOSTIC IMAGING; HUMANS; ROC CURVE;

EID: 42449111250     PISSN: 00338419     EISSN: 15271315     Source Type: Journal    
DOI: 10.1148/radiol.2471071321     Document Type: Note

References (25)

1. Pisano ED, Gatsonis C, Hendrick E, et al. Diagnostic performance of digital versus film mammography for breast-cancer screening. N Engl J Med 2005;353(17):1773-1783.
2. Fenton JJ, Taplin SH, Carney PA, et al. Influence of computer-aided detection on performance of screening mammography. N Engl J Med 2007;356(14):1399-1409.
3. Gur D, Rockette HE, Armfield DR, et al. Prevalence effect in a laboratory environment. Radiology 2003;228(1):10-14.
4. Skaane P, Balleyguier C, Diekmann F, et al. Breast lesion detection and classification: comparison of screen-film mammography and full-field digital mammography with softcopy reading - observer performance study. Radiology 2005;237(1):37-44.
5. Shiraishi J, Abe H, Li F, Engelmann R, MacMahon H, Doi K. Computer-aided diagnosis for the detection and classification of lung cancers on chest radiographs ROC analysis of radiologists' performance. Acad Radiol 2006;13(8):995-1003.
6. McClish DK. Analyzing a portion of the ROC curve. Med Decis Making 1989;9(3):190-195.
7. Jiang Y, Metz CE, Nishikawa RM. A receiver operating characteristic partial area index for highly sensitive diagnostic tests. Radiology 1996;201(3):745-750.
8. Metz CE. Some practical issues of experimental design and data analysis in radiological ROC studies. Invest Radiol 1989;24(3):234-245.
9. Obuchowski NA, Rockette HE. Hypothesis testing of the diagnostic accuracy for multiple diagnostic tests: ANOVA approach with dependent observations. Communications in Statistics - Simulation and Computation 1995;24:285-308.
10. Dorfman DD, Berbaum KS, Metz CE. Receiver operating characteristic rating analysis: generalization to the population of readers and patients with the jackknife method. Invest Radiol 1992;27(9):723-731.
11. Toledano AY. Three methods for analysing correlated ROC curves: a comparison in real data sets from multi-reader, multi-case studies with a factorial design. Stat Med 2003;22(18):2919-2933.
12. Obuchowski NA, Beiden SV, Berbaum KS, et al. Multireader, multicase receiver operating characteristic analysis: an empirical comparison of five methods. Acad Radiol 2004;11(9):980-995.
13. Beiden SV, Wagner RF, Campbell G. Components-of-variance models and multiple-bootstrap experiments: an alternative method for random-effects, receiver operating characteristic analysis. Acad Radiol 2000;7(5):341-349.
14. Bandos AI, Rockette HE, Gur D. A permutation test sensitive to differences in areas for comparing ROC curves from a paired design. Stat Med 2005;24(18):2873-2893.
15. DeLong ER, DeLong DM, Clarke-Pearson DL. Comparing the area under two or more correlated receiver operating characteristic curves: a nonparametric approach. Biometrics 1988;44(3):837-845.
16. Campbell G, Douglas MA, Bailey JJ. Nonparametric comparison of two tests of cardiac function on the same patient population using the entire ROC curve. In: Computers in Cardiology 1988 Conference proceedings. New York, NY: IEEE, 1988; 267-270.
17. Wieand HS, Gail MM, Hanley JA. A nonparametric procedure for comparing diagnostic tests with paired or unpaired data. Institute of Mathematical Statistics Bulletin 1983;12:213-214.
18. Pesce LL, Metz CE. Reliable and computationally efficient maximum-likelihood estimation of "proper" binormal ROC curves. Acad Radiol 2007;14(7):814-829.
19. Dorfman DD, Berbaum KS, Brandser EA. A contaminated binormal model for ROC data. I. Some interesting examples of binormal degeneracy. Acad Radiol 2000;7(6):420-426.
20. Glueck DH, Lamb MM, Lewin JM, Pisano ED. Two-modality mammography may confer an advantage over either full-field digital mammography or screen-film mammography. Acad Radiol 2007;14(6):670-676.
21. Wagner RF, Metz CE, Campbell G. Assessment of medical imaging systems and computer aids: a tutorial review. Acad Radiol 2007;14(6):723-748.
22. Wei J, Hadjiiski LM, Sahiner B, et al. Computer-aided detection systems for breast masses: comparison of performances on fullfield digital mammograms and digitized screen-film mammograms. Acad Radiol 2007;14(6):659-669.
23. Rana RS, Jiang Y, Schmidt RA, Nishikawa RM, Liu B. Independent evaluation of computer classification of malignant and benign calcifications in full-field digital mammograms. Acad Radiol 2007;14(3):363-370.
24. Walsh SJ. Limitations to the robustness of binormal ROC curves: effects of model misspecification and location of decision threshold on bias, precision, size and power. Stat Med 1997;16(6):669-679.
25. Gur D, Rockette HE, Bandos AI. "Binary" and "non-binary" detection tasks: are current performance measures optimal? Acad Radiol 2007;14(7):871-876.