A comparison of denominator degrees of freedom methods for multiple observer ROC analysis

Statistics in Medicine

Volumn 26, Issue 3, 2007, Pages 596-619

  Hillis, Stephen L ^a  

^a VETERANS AFFAIRS MEDICAL CENTER   (United States)

Author keywords

Corrected F; Degrees of freedom; Diagnostic radiology; Receiver operating characteristic (ROC) curve

Indexed keywords

AORTA DISSECTION; AREA UNDER THE CURVE; ARTICLE; CONFIDENCE INTERVAL; CONTROLLED STUDY; DATA ANALYSIS; DIAGNOSTIC ACCURACY; HUMAN; INTERMETHOD COMPARISON; JACKKNIFE TEST; MAJOR CLINICAL STUDY; NUCLEAR MAGNETIC RESONANCE IMAGING; NULL HYPOTHESIS; RECEIVER OPERATING CHARACTERISTIC; SAMPLE SIZE; STATISTICAL ANALYSIS; STATISTICAL MODEL; STATISTICAL SIGNIFICANCE; THORACIC AORTA;

AORTIC ANEURYSM, THORACIC; AREA UNDER CURVE; COMPUTER SIMULATION; CONFIDENCE INTERVALS; DATA INTERPRETATION, STATISTICAL; HUMANS; MAGNETIC RESONANCE IMAGING; OBSERVER VARIATION; ROC CURVE;

EID: 33846416961     PISSN: 02776715     EISSN: 10970258     Source Type: Journal    
DOI: 10.1002/sim.2532     Document Type: Article

References (29)

1. Swets JA, Pickett RM. Evaluation of Diagnostic Systems: Methods from Signal Detection Theory. Academic Press: New York, 1982.
2. Metz CE. ROC methodology in radiologic imaging. Investigative Radiology 1986; 21:720-733.
3. Swets JA. Measuring the accuracy of diagnostic systems. Science 1988; 240:1285-1293.
4. Hanley JA. Receiver operating characteristic (ROC) methodology: the state of the art. Critical Reviews in Diagnostic Imaging 1989; 29:307-335.
5. Hanley JA, McNeil BJ. The meaning and use of the area under a receiver operating characteristic (ROC) curve. Radiology 1982; 143:29-36.
6. Obuchowski NA, Beiden SV, Berbaum KS, Hillis SL, Ishwaran H, Song HH, Wagner RF. Multicase ROC analyses: an empirical comparison of five methods. Academic Radiology 2004; 11:980-995 DOI:10.1016/ j.acra.2004.04.014.
7. Dorfman DD, Berbaum KS, Metz CE. Receiver operating characteristic rating analysis: generalization to the population of readers and patients with the jackknife method. Investigative Radiology 1992; 27:723-731.
8. Dorfman DD, Berbaum KS, Lenth RV, Chen YF, Donaghy BA. Monte Carlo validation of a multireader method for receiver operating characteristic discrete rating data: factorial experimental design. Academic Radiology 1998; 5:591-602.
9. Obuchowski NA, Rockette HE. Hypothesis testing of the diagnostic accuracy for multiple diagnostic tests: an ANOVA approach with dependent observations. Communications in Statistics: Simulation and Computation 1995; 24:285-308.
10. Obuchowski NA. Multi-reader multi-modality ROC studies: hypothesis testing and sample size estimation using an ANOVA approach with dependent observations. With rejoinder. Academic Radiology 1995; 2(Suppl 1): S22-S29.
11. Hillis SL, Obuchowski NA, Schartz KM, Berbaum KS. A comparison of the Dorfman-Berbaum-Metz and Obuchowski-Rockette Methods for receiver operating characteristic (ROC) data. Statistics in Medicine 2005; 24:1579-1607 DOI:10.1002/sim.2024.
12. DeLong ER, DeLong DM, Clarke-Pearson DL. Comparing the areas under two or more correlated receiver operating characteristic curves: a non-parametric approach. Biometrics 1988; 44:837-844.
13. Satterthwaite FE. Synthesis of variance. Psychometrika 1941; 6:309-316.
14. Satterthwaite FE. An approximate distribution of estimates of variance components. Biometric Bulletin 1946; 2:110-114.
15. Pavur R, Nath R. Exact F tests in an ANOVA procedure for dependent observations. Multivariate Behavioral Research 1984; 19:408-420.
16. Quenoille MH. Approximate tests of correlation in time series. Journal of the Royal Statistical Society, Series B 1949; 11:68-84.
17. Quenoille MH. Notes on bias in estimation. Biometrika 1956; 43:353-360.
18. Tukey JW. Bias and confidence in not quite large samples (abstract). Annals of Mathematical Statistics 1958; 29:614.
19. Hillis SL, Berbaum KS. Power estimation for the Dorfman-Berbaum-Metz method. Academic Radiology 2004; 11:1260-1273 DOI:10.1016/ j.acra.2004.08.009.
20. Hillis SL, Berbaum KS. Monte Carlo validation of the Dorfman-Berbaum-Metz method using normalized pseudovalues and less data-based model simplification. Academic Radiology 2005; 12:1534-1541 DOI:10.1016/j.acra.2005.07.012.
21. Gaylor DW, Hopper FN. Estimating the degrees of freedom for linear combinations of mean squares by Satterthwaite's formula. Technometrics 1969; 11:691-706.
22. Roe CA, Metz CE. Dorfman-Berbaum-Metz method for statistical analysis of multireader, multimodality receiver operating characteristic data: validation with computer simulation. Academic Radiology 1997; 4: 298-303.
23. Dorfman DD, All Jr E. Maximum likelihood estimation of parameters of signal-detection theory and determination of confidence intervals-rating method data. Journal of Mathematical Psychology 1969; 6: 487-496.
24. Dorfman DD. RSCORE II. In Evaluation of Diagnostic Systems: Methods from Signal Detection Theory, Swets JA, Pickett RM (eds). Academic Press: San Diego, CA, 1982; 212-232.
25. SAS for Windows. Version 9.1, copyright (c) 2002-2003 by SAS Institute Inc.: Cary, NC, U.S.A.
26. Van Dyke CW, White RD, Obuchowski NA, Geisinger MA, Lorig RJ, Meziane MA. Cine MRI in the diagnosis of thoracic aortic dissection. 79th RSNA Meetings, Chicago, IL, 28 November-3 December, 1993.
27. Smith JH, Lewis TO. Determining the effects of intra-class correlation on factorial-experiments. Communications in Statistics, Part A - Theory and Methods 1980; 9:1353-1364.
28. Searle SR. Linear Models. Wiley: New York, 1971; 55-59.
29. Harville DA. Matrix Algebra From a Statistician's Perspective. Springer: New York; 335-338.