Hierarchical models for ROC curve summary measures: Design and analysis of multi-reader, multi-modality studies of medical tests

Statistics in Medicine

Volumn 27, Issue 2, 2008, Pages 243-256

  Wang, Fei ^a   Gatsonis, Constantine A ^b  

^a BOSTON UNIVERSITY   (United States)

^b BROWN UNIVERSITY   (United States)

Author keywords

Hierarchical modelling; Meta analysis; Multi reader multi modality ROC analysis; Sample size determination

Indexed keywords

AREA UNDER THE CURVE; ARTICLE; BAYES THEOREM; BREAST DISEASE; COMPUTER AIDED DESIGN; CORRELATION ANALYSIS; DATA ANALYSIS; DIAGNOSTIC ACCURACY; DIGITAL MAMMOGRAPHY; HUMAN; INTERMETHOD COMPARISON; MATHEMATICAL COMPUTING; METHODOLOGY; ROC CURVE; SAMPLE SIZE; SCREENING TEST; SIMULATION;

DATA INTERPRETATION, STATISTICAL; DIAGNOSIS, COMPUTER-ASSISTED; DIAGNOSTIC TESTS, ROUTINE; FEMALE; HUMANS; MAMMOGRAPHY; META-ANALYSIS AS TOPIC; MIDDLE AGED; MODELS, STATISTICAL; ROC CURVE; UNITED STATES;

EID: 38849110678     PISSN: 02776715     EISSN: 10970258     Source Type: Journal    
DOI: 10.1002/sim.2828     Document Type: Article

References (40)

1. Dorfman DD, Berbaum KS, Metz CE. Receiver operating characteristic rating analysis: generalization to the population of readers and patients with Jackknife method. Investigative Radiology 1992; 27(9):723-731.
2. Dorfman DD, Berbaum KS, Lenth RV. Multireader, multicase receiver operating characteristic methodology: a bootstrap analysis. Academic Radiology 1995; 2:626-633.
3. Beiden SV, Wagner RF, Campbell G. Components-of-variance models and multiple-bootstrap experiments: an alternative method for random-effects, receiver operating characteristic analysis. Academic Radiology 2000; 7(5):341-349.
4. Obuchowski NA. Multi-reader multi-modality ROC studies: hypothesis testing and sample size determination using an ANOVA approach with dependent observations. Academic Radiology 1995; 2:S22-S29.
5. Obuchowski NA. Hypothesis testing of diagnostic accuracy for multiple readers and multiple tests: an ANOVA approach with dependent observations. Communications in Statistics, Part B: Simulation and Computation 1995; 24(2):285-308.
6. Hanley JA. Receiver operating characteristic curves. Encyclopedia of Biostatistics 1998.
7. Zhou X-H, Obuchowski NA, McClish D. Statistical Methods in Diagnostic Medicine. Wiley: New York, 2002.
8. Pepe M. Receiver operating characteristic methodology. Journal of the American Statistical Association 2000; 95:308-311.
9. Pepe M. The Statistical Evaluation of Medical Tests for Misclassification and Prediction. Oxford: New York, 2003.
10. Dorfman DD, Alf E. Maximum likelihood estimation of parameters of signal detection theory and determination of confidence intervals - rating method data. Journal of Mathematical Psychology 1969; 6:487-496.
11. Toledano A, Gatsonis CA. Ordinal regression methodology for ROC curves derived from correlated data. Statistics in Medicine 1996; 15:1807-1826.
12. Tosteson A, Begg C. A general regression methodology for ROC curve estimation. Medical Decision Making 1988; 8:204-215.
13. Metz C, Herman B, Shen J. Maximum likelihood estimation of receiver operating characteristic (ROC) curves from continuously-distributed data. Statistics in Medicine 1998; 17(9):1033-1053.
14. Delong ER, Delong DM, Clarke-Pearson DL. Comparing the areas under two or more correlated receiver operating characteristic curves: a nonparametric approach. Biometrics 1988; 44(3):837-845.
15. Obuchowski NA, McClish D. Sample size determination for diagnostic accuracy studies involving binormal ROC curve indices. Statistics in Medicine 1997; 16:1529-1542.
16. Toledano A. Three methods for analyzing correlated ROC curves: a comparison in real data sets. Statistics in Medicine 2003; 22:2919-2933.
17. Ishwaran H, Gatsonis CA. A general class of hierarchical ordinal regression models with applications to correlated ROC analysis. Canadian Journal of Statistics 2000; 28:731-750.
18. Dorfman DD, Berbaum KS, Lenth RV, Chen YF, Donaghy BA. Monte Carlo validation of a multireader method for receiver operating characteristic discrete rating data: factorial experimental design. Academic Radiology 1998; 5(9):591-602.
19. Obuchowski NA, Beiden SV, Berbaum KS, Hillis SL, Ishwaran H, Song HH, Wagner RF. Multireader, multicase receiver operating characteristic analysis. Academic Radiology 2004; 11:980-995.
20. Thompson ML, Zucchini W. On the statistical analysis of ROC curves. Statistics in Medicine 1989; 8:1277-1290.
21. Zhou X-H. Empirical Bayes combination of estimated areas under ROC curves using estimating equations. Medical Decision Making 1996; 16:24-28.
22. Hellmich M, Abrams KR, Sutton AJ. Bayesian approaches to meta-analysis of ROC curves. Medical Decision Making 1999; 19(3):252-264.
23. Rockette HE, Campbell WL, Britton CA, Holbert JM, King JL, Gur D. Empiric assessment of parameters that affect the design of multireader receiver operating characteristic studies. Academic Radiology 1999; 6(12):723-729.
24. Browne WJ, Draper D, Goldstein H, Rasbash J. Bayesian and likelihood methods for fitting multilevel models with complex level-1 variation. Computational Statistics and Data Analysis 2002; 39:203-225.
25. Lindley DV. The choice of sample size. The Statistician 1997; 46(2):129-138.
26. Pham-Gia T. On Bayesian analysis, Bayesian decision theory and the sample size problem. The Statistician 1997; 46:139-144.
27. Adcock CJ. Sample size determination: a review. The Statistician 1997; 46(2):261-283.
28. Joseph L, Wolfson DB. Interval-based versus decision theoretic criteria for the choice of sample size. The Statistician 1997; 46(2):145-149.
29. Joseph L, Belisle P. Bayesian sample size determination for normal means and differences between normal means. The Statistician 1997; 46:209-226.
30. Rahme E, Joseph L, Gyorkos TW. Bayesian sample size determination for estimating binomial parameters from data subject to misclassification. Applied Statistics 2000; 49(1): 119-128.
31. Inoue L, Berry DA, Parmigiani G. A Bayesian view of the classical sample size determination. Technical Report, Department of Biostatistics, MD Anderson Cancer Center: Houston, TX, 2000.
32. Zou KH, Normand SLT. On determination of sample size in hierarchical binomial models. Statistics in Medicine 2001; 20:2163-2182.
33. Normand SLT, Zou KH. Sample size considerations in observational health care quality studies. Statistics in Medicine 2002; 21:331-345.
34. Wang F, Gelfand AE. A simulation-based approach to Bayesian sample size determination for performance under a given model and for separating models. Statistical Science 2002; 17(2):193-208.
35. Obuchowski NA. Computing sample size for receiver operating characteristic studies, Investigative Radiology 1994; 29(2):238-243.
36. Hanley JA, McNeil BJ. The meaning and use of the area under a receiver operating characteristic (ROC) curve. Radiology 1982; 143:29-36.
37. Nishikawa RM, Gatsonis CA, Schnall MD, Giger ML, Sajda P, Chen M, Kundel H. Large scale observer study to measure the benefits of computer-aided detection to screening mammography. Radiology 1999; 213:150.
38. Gelfand AE, Sahu S, Carlin BP. Efficient Parametrization for normal linear mixed effects models. Biometrika 1995; 82:479-488.
39. Hillis SL, Obuchowski NA, Schartz KM, Berbaum KS. A comparison of the Dorfman-Berbaum-Metz and Obuchowski-Rockette methods for receiver operating characteristic (ROC) data. Statistics in Medicine 2005; 24:1579-1607.
40. Hillis SL. A comparison of denominator degrees of freedom methods for multiple observer ROC analysis, 15 March 2006 (published online).