Monte Carlo validation of a multireader method for receiver operating characteristic discrete rating data: Factorial experimental design

Academic Radiology

Volumn 5, Issue 9, 1998, Pages 591-602

  Dorfman, D D ^a   Berbaum, K S ^a   Lenth, R V ^a   Chen, Y F ^a   Donaghy, B A ^a  

^a E11 SSH   (United States)

Author keywords

Decision theory; Diagnostic radiology; Receiver operating characteristic curve (ROC)

Indexed keywords

ARTICLE; COMPUTER SIMULATION; FACTORIAL ANALYSIS; HYPOTHESIS; PRIORITY JOURNAL; RADIODIAGNOSIS; RECEIVER OPERATING CHARACTERISTIC; REGRESSION ANALYSIS; SYSTEM ANALYSIS; VALIDATION PROCESS;

EID: 0031752721     PISSN: 10766332     EISSN: None     Source Type: Journal    
DOI: 10.1016/S1076-6332(98)80294-8     Document Type: Article

References (34)

1. Rockette HE Gur D Kurs-Lasky M King JL On the generalization of receiver operating characteristic analysis to the population of readers and cases with the jackknife method: an assessment Acad Radiol 2 1995 66 69
2. Dorfman DD Berbaum KS Metz CE Receiver operating characteristic rating analysis: generalization to the population of readers and patients with the jackknife method Invest Radiol 27 1992 723 731
3. Swets JA Pickett RM Evaluation of diagnostic systems: methods from signal detection theory 1982 Academic Press
4. Hanley JA McNeil BJ A method of comparing the areas under receiver operating characteristic curves derived from the same cases Radiology 148 1983 839 843
5. McNeil BJ Hanley JA Statistical approaches to the analysis of receiver operating characteristic (ROC) curves Med Decis Making 4 1984 137 150
6. Winer BJ Statistical principles in experimental design 2nd ed. 1971 McGraw-Hill New York, NY
7. Advances in statistical methods for diagnostic radiology: a symposium Acad Radiol 2 suppl 1 1995 S1 S84
8. Obuchowski NA Zepp RC Simple steps for improving multiple-reader studies in radiology AJR 166 1996 517 521
9. Roe CA Metz CE The Dorfman-Berbaum-Metz method for statistical analysis of multi-reader, multi-modality ROC data: validation by computer simulation Acad Radiol 4 1997 298 303
10. Obuchowski NA Rockette HE Hypothesis testing of diagnostic accuracy for multiple readers and multiple tests: an ANOVA approach with dependent observations Commun Stat Simulations 24 1995 285 308
11. Pearson ES Please NW Relation between the shape of population distribution and the robustness of four simple test statistics Biometrika 62 1975 223 241
12. Donaldson TS Robustness of the F test to errors of both kinds and the correlation between the numerator and denominator of the F-ratio J Am Stat Assoc 3 1968 660 676
13. Snedecor GW Cochran WG Statistical methods 8th ed. 1989 Iowa State University New York, NY
14. Song HH Analysis of correlated ROC areas in diagnostic testing Biometrics 53 1997 370 382
15. Neter J Wasserman W Kutner MH Applied linear statistical models: regression, analysis of variance, and experimental designs 3rd ed 1990 Irwin Ames, Iowa 300 301
16. Neter J Wasserman W Kutner MH Applied linear statistical models: regression, analysis of variance, and experimental designs 3rd ed 1990 Irwin Homewood, Ill 623 624
17. Gayen AK The distribution of the variance ratio in random samples of any size drawn from non-normal universes Biometrika 37 1950 236 255
18. Bevan MF Denton JQ Myers JL The robustness of the F test to violations of continuity and form of treatment population Br J Math Stat Psychol 27 1974 199 204
19. Dorfman DD Berbaum KS Lenth RV Multi-reader multi-case ROC methodology: a bootstrap analysis Acad Radiol 2 1995 626 633
20. Swets JA Pickett RM Evaluation of diagnostic systems: methods from signal detection theory 1982 Academic Press Homewood, Ill
21. Dorfman DD Alf E Jr Maximum likelihood estimation of parameters of signal detection theory: a direct solution Psychometrika 33 1968 117 124
22. Dorfman DD Alf E Jr. Maximum likelihood estimation of parameters of signal detection theory and determination of confidence intervals: rating method data J Math Psych 6 1969 487 496
23. Dorfman DD Berbaum KS Abu-Dagga HI RSCORE 100: computer program for estimation of parameters of the binormal ROC model (1994 version) 1994 Departments of Radiology and Psychology, the University of Iowa New York, NY
24. Rockette HE Gur D Metz CE The use of continuous and discrete confidence judgments in receiver operating characteristic studies of diagnostic imaging techniques Invest Radiol 27 1992 169 172
25. King JL Britton CA Gur D Rockette HE Davis PL On the validity of the continuous and discrete confidence rating scales in receiver operating characteristic studies Invest Radiol 28 1993 962 963
26. Dorfman DD Berbaum KS Degeneracy and discrete ROC rating data Acad Radiol 2 1995 907 915
27. Dorfman DD Berbaum KS Metz CE Lenth RV Hanley JA Abu-Dagga H Proper receiver operating characteristic analysis: the bigamma model Acad Radiol 4 1997 138 149
28. Pan X Metz CE The “proper” binormal model: parametric receiver operating characteristic curve estimation with degenerate data Acad Radiol 4 1997 380 389
29. American College of Radiology Breast imaging reporting and data system (BI-RADS) 1993 American College of Radiology Iowa City, Iowa
30. Yoshida H Doi K Nishikawa RM Giger ML Schmidt RA An improved computer-assisted diagnostic scheme using wavelet transform for detecting clustered microcalcifications in digital mammograms Acad Radiol 3 1996 621 627
31. Metz CE Shen JH Herman BA New methods for estimating a binormal ROC curve from continuously distributed test results Presented at the 1990 Joint Statistical Meetings of the American Statistical Association and the Biometric Society Anaheim, Calif August 1990
32. Vittitoe NF Baker JA Floyd CE Jr. Fractal texture analysis in computer-aided diagnosis of solitary pulmonary nodules Acad Radiol 4 1997 96 101
33. Evans M Hastings N Peacock B Statistical distributions 2nd ed 1993 Wiley Reston, VA 1 127
34. Cacioppo JT Dorfman DD Waveform moment analysis in psychological research Psychol Bull 102 1987 421 438