Statistical power considerations for a utility endpoint in observer performance studies

Academic Radiology

Volumn 20, Issue 7, 2013, Pages 798-806

  Abbey, Craig K ^a,b   Samuelson, Frank W ^c   Gallas, Brandon D ^c  

^a UNIVERSITY OF CALIFORNIA   (United States)

^b UNIVERSITY OF CALIFORNIA   (United States)

^c CENTER FOR DEVICES AND RADIOLOGICAL HEALTH   (United States)

Author keywords

Observer performance endpoints; Statistical power; Utility

Indexed keywords

ARTICLE; DIAGNOSTIC IMAGING; EXPERIMENTAL DESIGN; HUMAN; POWER ANALYSIS; PRIORITY JOURNAL; RECEIVER OPERATING CHARACTERISTIC; SIMULATION;

COMPUTER SIMULATION; MODELS, STATISTICAL; RADIOLOGY; ROC CURVE; STATISTICS AS TOPIC;

EID: 84878898210     PISSN: 10766332     EISSN: 18784046     Source Type: Journal    
DOI: 10.1016/j.acra.2013.02.008     Document Type: Article

References (36)

1. Gallas B.D., Chan H.P., D'Orsi C.J., et al. Evaluating imaging and computer-aided detection and diagnosis devices at the FDA. Acad Radiol 2012, 19:463-477.
2. Metz C.E. Basic principles of ROC analysis. Semin Nucl Med 1978, 8:283-298.
3. Metz C.E. ROC analysis in medical imaging: a tutorial review of the literature. Radiol Phys Technol 2008, 1:2-12.
4. Obuchowski N.A. ROC analysis. AJR Am J Roentgenol 2005, 184:364-372.
5. Swets J.A., Pickett R.M. Evaluation of diagnostic systems: methods from signal detection theory 1982, Academic Press, New York.
6. Bamber D. The area above the ordinal dominance graph and the area below the receiver operating characteristic graph. J Math Psychol 1975, 12:387-415.
7. Hanley J.A., McNeil B.J. The meaning and use of the area under a receiver operating characteristic (ROC) curve. Radiology 1982, 143:29-36.
8. Metz C.E. ROC methodology in radiologic imaging. Invest Radiol 1986, 21:720-733.
9. Hanley J.A. Receiver operating characteristic (ROC) methodology: the state of the art. Crit Rev Diagnostic Imaging 1989, 29:307.
10. Green D.M., Swets J.A. Signal detection theory and psychophysics 1966, Wiley, New York.
11. Peterson W., Birdsall T., Fox W. The theory of signal detectability. IREProfessional Group on Information Theory 1954, 4:171-212.
12. Tanner W.P., Swets J.A. A decision-making theory of visual detection. Psychol Rev 1954, 61:401-409.
13. Pauker S.G., Kassirer J.P. Therapeutic decision making: a cost-benefit analysis. N Engl J Med 1975, 293:229-234.
14. Hilden J. The area under the ROC curve and its competitors. Med Decis Making 1991, 11:95-101.
15. Moons K.G., Stijnen T., Michel B.C., et al. Application of treatment thresholds to diagnostic-test evaluation: an alternative to the comparison of areas under receiver operating characteristic curves. Med Decis Making 1997, 17:447-454.
16. Schisterman E.F., Perkins N.J., Liu A., et al. Optimal cut-point and its corresponding Youden Index to discriminate individuals using pooled blood samples. Epidemiology 2005, 16:73-81.
17. Schwartz W.B., Gorry G.A., Kassirer J.P., et al. Decision analysis and clinical judgment. Am J Med 1973, 55:459-472.
18. Lusted L.B. Introduction to medical decision making 1968, CThomas, Springfield, IL.
19. Weinstein M.C., Fineberg H.V., Elstein A.S., et al. Clinical decision analysis 1980, Saunders, Philadelphia, PA.
20. Sunshine J. Contributed comment. Acad Radiol 1995, 2(Suppl):S72-S74.
21. Halpern E.J., Albert M., Krieger A.M., et al. Comparison of receiver operating characteristic curves on the basis of optimal operating points. Acad Radiol 1996, 3:245-253.
22. Wagner R.F., Beam C.A., Beiden S.V. Reader variability in mammography and its implications for expected utility over the population of readers and cases. Med Decis Making 2004, 24:561-572.
23. Abbey C.K., Eckstein M.P., Boone J.M. An equivalent relative utility metric for evaluating screening mammography. Med Decis Making 2010, 30:113-122.
24. Abbey CK, Eckstein MP, Boone JM. Estimating the relative utility of screening mammography. In Press: Medical Decision Making.
25. Roe C.A., Metz C.E. Dorfman-Berbaum-Metz method for statistical analysisof multireader, multimodality receiver operating characteristic data: validation with computer simulation. Acad Radiol 1997, 4:298-303.
26. Dorfman D.D., Berbaum K.S., Metz C.E. Receiver operating characteristic rating analysis. Generalization to the population of readers and patients with the jackknife method. Invest Radiol 1992, 27:723-731.
27. Roe C.A., Metz C.E. Variance-component modeling in the analysis of receiver operating characteristic index estimates. Acad Radiol 1997, 4:587-600.
28. Beiden S.V., Wagner R.F., Campbell G., et al. Components-of-variance models for random-effects ROC analysis: the case of unequal variance structures across modalities. Acad Radiol 2001, 8:605-615.
29. Metz C.E., Kronman H.B. Statistical significance tests for binormal ROC curves. J Math Psychol 1980, 22:218-243.
30. Swets J.A. Form of empirical ROCs in discrimination and diagnostic tasks: implications for theory and measurement of performance. Psychol Bull 1986, 99:181-198.
31. Hillis S.L. Simulation of unequal-variance binormal multireader ROC decision data: an extension of the Roe and Metz simulation model. Acad Radiol 2012, 19:1518-1528.
32. Hillis S.L., Berbaum K.S. Using the mean-to-sigma ratio as a measure of theimproperness of binormal ROC curves. Acad Radiol 2011, 18:143-154.
33. Dorfman D.D., Berbaum K.S. A contaminated binormal model for ROC data: part III. Initial evaluation with detection ROC data. Acad Radiol 2000, 7:438-447.
34. Dorfman D.D., Berbaum K.S. A contaminated binormal model for ROC data: Part II. A formal model. Acad Radiol 2000, 7:427-437.
35. Dorfman D.D., Berbaum K.S., Brandser E.A. A contaminated binormal model for ROC data: part I. Some interesting examples of binormal degeneracy. Acad Radiol 2000, 7:420-426.
36. Hillis S.L., Berbaum K.S., Metz C.E. Recent developments in the Dorfman-Berbaum-Metz procedure for multireader ROC study analysis. Acad Radiol 2008, 15:647-661.