A permutation test for comparing ROC curves in multireader studies: A multi-reader ROC, permutation test

Academic Radiology

Volumn 13, Issue 4, 2006, Pages 414-420

  Bandos, Andriy I ^a   Rockette, Howard E ^a   Gur, David ^b  

^a UNIVERSITY OF PITTSBURGH GRADUATE SCHOOL OF PUBLIC HEALTH   (United States)

^b UNIVERSITY OF PITTSBURGH   (United States)

Author keywords

Diagnostic test; Multireader; Permutations; Receiver operating characteristic (ROC) curves

Indexed keywords

ANALYTIC METHOD; ANALYTICAL ERROR; ARTICLE; COMPARATIVE STUDY; COMPUTER SIMULATION; DIAGNOSTIC ACCURACY; DIAGNOSTIC IMAGING; DIAGNOSTIC TEST; MATHEMATICAL MODEL; MATHEMATICAL PARAMETERS; OBSERVER VARIATION; PARAMETRIC TEST; PERMUTATION TEST; PRIORITY JOURNAL; PROCESS DESIGN; RECEIVER OPERATING CHARACTERISTIC; SAMPLE; STATISTICAL ANALYSIS; VALIDATION PROCESS; VARIANCE;

ANALYSIS OF VARIANCE; COMPUTER SIMULATION; DATA INTERPRETATION, STATISTICAL; IMAGE ENHANCEMENT; IMAGE INTERPRETATION, COMPUTER-ASSISTED; MODELS, STATISTICAL; OBSERVER VARIATION; REPRODUCIBILITY OF RESULTS; ROC CURVE; SENSITIVITY AND SPECIFICITY;

EID: 33645081450     PISSN: 10766332     EISSN: None     Source Type: Journal    
DOI: 10.1016/j.acra.2005.12.012     Document Type: Article

References (26)

1. C.E. Metz ROC methodology in radiologic imaging Invest Radiol 21 1986 720 733
2. D. Bamber The area above the ordinal dominance graph and the area below the receiver operating characteristic graph J Math Psychol 12 1975 387 415
3. J.A. Hanley, B.J. McNeil The meaning and use of the area under a receiver operating characteristic (ROC) curve Radiology 143 1982 29 36
4. H.S. Wieand, M. Gail, B. James, K. James A family of nonparametric statistics for comparing diagnostic markers with paired or unpaired data Biometrika 76 1989 585 592
5. J.A. Hanley, B.J. McNeil A method of comparing the area under two ROC curves derived from the same cases Radiology 148 1983 839 843
6. E.R. DeLong, D.M. DeLong, D.L. Clarke-Pearson Comparing the area under two or more correlated receiver operating characteristic curves a nonparametric approach Biometrics 44 1988 837 845
7. D.D. Dorfman, E. Alf Maximum likelihood estimation of parameters of signal detection theory-a direct solution Psychometrika 33 1968 117 124
8. D.D. Dorfman, E. Alf Jr Maximum likelihood estimation of parameters of signal detection theory and determination of confidence intervals - rating-method data J Math Psychol 6 1969 487 496
9. C.E. Metz, P.-L. Wang, H.B. Kronman A new approach for testing the significance of differences between ROC curves measured from correlated data F. Deconick Information Processing in Medical Imaging VIII 1984 Nijhof The Haugue 432 445
10. C.E. Metz, B.A. Herman, J. Shen Maximum likelihood estimation of receiver operating characteristic (ROC) curves from continuously distributed data Stat Med 17 1998 1033 1053
11. E.S. Venkatraman, C.B. Begg A distribution-free procedure for comparing receiver operating characteristic curves from a paired experiment Biometrika 83 1996 835 848
12. A.I. Bandos, H.E. Rockette, D. Gur A permutation test sensitive to differences in areas for comparing ROC curves from a paired design Stat Med 24 2005 2873 2893
13. D.D. Dorfman, K.S. Berbaum, C.E. Metz Receiver operating characteristic rating analysis generalization to the population of readers and patients with the jackknife method Invest Radiol 27 1992 723 731
14. D.D. Dorfman, K.S. Berbaum, R.V. Lenth, Y.F. Chen, B.A. Donaghy Monte Carlo validation of a multireader method for receiver operating characteristic discrete rating data factorial experimental design Acad Radiol 5 1998 591 602
15. N.A. Obuchowski, H.E. Rockette Hypothesis testing of diagnostic accuracy for multiple readers and multiple tests an ANOVA approach with dependent observations Commun Stat Simulation Computation 24 1995 285 308
16. N.A. Obuchowski Multi-reader multi-modality ROC studies hypothesis testing and sample size estimation using an ANOVA approach with dependent observations with rejoinder Acad Radiol 2 suppl 1995 S22 S29
17. H.H. Song Analysis of correlated ROC areas in diagnostic testing Biometrics 53 1997 370 382
18. S.V. Beiden, R.F. Wagner, G. Campbell Components-of-variance models and multiple-bootstrap experiments an alternative method for random-effects receiver operating characteristic analysis Acad Radiol 7 2000 341 349
19. S.V. Beiden, R.F. Wagner, G. Campbell, C.E. Metz, Y. Jiang Components-of-variance models for random-effects ROC analysis the case of unequal variance structure across modalities Acad Radiol 8 2001 605 615
20. S.V. Beiden, R.F. Wagner, G. Campbell, H.P. Chan Analysis of uncertainties in estimates of components of variance in multivariate ROC analysis Acad Radiol 8 2001 616 622
21. H. Ishwaran, C.A. Gatsonis A general class of hierarchical ordinal regression models with applications to correlated ROC analysis Can J Stat 28 2000 731 750
22. M.-L.T. Lee, B.A. Rosner The average area under correlated receiver operating characteristic curves a nonparametric approach based on generalized two-sample Wilcoxon statistics Appl Stat 50 2001 337 344
23. N.A. Obuchowski, S.V. Beiden, K.S. Berbaum Multireader, multicase receiver operating characteristic analysis an empirical comparison of five methods Acad Radiol 11 2004 980 995
24. H.H. Zhou, N.A. Obuchowski, D.K. McClish Statistical Methods in Diagnostic Medicine 2002 Wiley New York
25. J.A. Hanley The robustness of the 'binormal' assumption used in fitting ROC curves Med Decision Making 8 1988 197 203
26. A.Y. Toledano, C. Gatsonis Ordinal regression methodology for ROC curves derived from correlated data Stat Med 15 1996 1807 1826