The meaning and use of the volume under a three-class ROC surface (VUS)

IEEE Transactions on Medical Imaging

Volumn 27, Issue 5, 2008, Pages 577-588

  He, Xin ^a,b   Frey, Eric C ^a,b  

^a IEEE

^b JOHNS HOPKINS UNIVERSITY SCHOOL OF MEDICINE   (United States)

Author keywords

Ideal observer; Receiver operating characteristic (ROC) analysis; Three class classification

Indexed keywords

COMPUTATIONAL METHODS; DECISION THEORY; DISCRIMINANT ANALYSIS; ERROR ANALYSIS; OPTIMIZATION; SIGNAL TO NOISE RATIO; STATISTICS;

IDEAL OBSERVERS; LINEAR DISCRIMINANT ANALYSIS; RECEIVER OPERATING CHARACTERISTIC (ROC) ANALYSIS; STANDARD ERROR; THREE-CLASS CLASSIFICATION;

SIGNAL RECEIVERS;

ALGORITHM; ARTICLE; COMPUTER ASSISTED DIAGNOSIS; EVALUATION; IMAGE ENHANCEMENT; METHODOLOGY; REPRODUCIBILITY; ROC CURVE; SENSITIVITY AND SPECIFICITY; THREE DIMENSIONAL IMAGING;

ALGORITHMS; IMAGE ENHANCEMENT; IMAGE INTERPRETATION, COMPUTER-ASSISTED; IMAGING, THREE-DIMENSIONAL; REPRODUCIBILITY OF RESULTS; ROC CURVE; SENSITIVITY AND SPECIFICITY;

EID: 43049173512     PISSN: 02780062     EISSN: None     Source Type: Journal    
DOI: 10.1109/TMI.2007.908687     Document Type: Article

References (33)

1. X. He, C. E. Metz, J. M. Links, B. M. Tsui, and E. C. Frey, "Three-class ROC analysis - A decision theoretic approach under the ideal observer framework," IEEE Trans. Med. Imag., vol 25, no. 5, pp. 571-581, May 2006.
2. X. He and E. C. Frey, 'Three-class ROC analysis-The equal error assumption and the optimality of three-class ROC surface/hypersurface with the ideal observer," IEEE Trans. Med. Imag., vol. 25, no. 5, pp. 979-986, May 2006.
3. X. He and E. C. Frey, "An optimal three-class linear observer derived from decision theory," IEEE Trans. Med. Imag., vol. 26, no. 1, pp. 77-83, Jan. 2007.
4. J. A. Hanley and B. J. McNeil, "The meaning and use of the area under a receiver operating characteristic (ROC) curve," Radiology, vol. 143, pp. 29-36, 1982.
5. D. M. Green and J. A. Swets, Signal Detection Theory and Psychophysics. Huntington, NY: Krieger, 1973.
6. D. Bamber, "The area above the ordinal dominance graph and the area below the receiver operating characteristic graph," J. Math. Psych., vol. 12, pp. 387-415, 1975.
7. H. H. Barrett, C. K. Abbey, and E. Clarkson, "Objective assessment of image quality. III. ROC metrics, ideal observers, and the likelihood-generating functions," J. Opt. Soc. Amer., vol. 15, pp. 1520-1535, 1998.
8. H. H. Barrett and K. J. Myers, Foundations of Image Science. New York: Wiley, 2003.
9. C. E. Metz, "Basic principles of ROC analysis," Sem. Nucl. Med., vol. 8, pp. 283-298, 1978.
10. C. E. Metz, "ROC methodology in radiologic imaging," Invest. Radiol., vol. 21, pp. 720-733, 1986.
11. M. L. Thompson and W. Zucchini, "On the statistical analysis of ROC curves," Statist. Med., vol. 8, pp. 1277-1290, 1989.
12. B. K. Scurfield, "Generalization of the theory of signal detectability to n-event m-dimensional forced-choice tasks," J. Math. Psych., vol. 42, pp. 5-31, 1998.
13. S. Dreiseitl, L. Ohno-Machado, and M. Binder, "Comparing three-class diagnostic tests by three-way ROC analysis," Med. Decis. Making, vol. 20, pp. 323-331, 2000.
14. E. L. Lehmann, Nonparametrics: Statistical Methods Based on Ranks. New York: McGraw-Hill, 1975.
15. C. Pierre, "Confidence interval estimation for distribution systems power consumption by using the bootstrap method," M.S. thesis, Dept. Elect. Eng., Virginia Polytechnic Inst. State Univ., Blacksburg, 1997.
16. B. Efron and R. J. Tsibshirani, An Introduction to the Bootstrap. London, U.K.: Chapman & Hall, 1993.
17. X. Song, E. C. Frey, X. He, W. P. Segars, and B. M. W. Tsui, "A mathematical observer study for evaluation of a model-based compensation method for crosstalk in simultaneous dual isotope SPECT," presented at the IEEE Med. Imag. Conf., Portaland, OR, 2003.
18. X. He, E. C. Frey, J. M. Links, and B. M. W. Tsui, "Three-class ROC analysis and three-class hotelling observer for myocardial perfusion SPECT optimization and evaluation." J. Nucl. Med., vol. 45, pp. 42P-42p, 2004.
19. J. Swets and R. Pickett, Evaluation of Diagnostic Systems: Methods from Signal Detection Theory. New York: Academic, 1982.
20. J. A. Swets and T. G. Birdsall, "The human use of information. 3. decision-making in signal-detection and recognition situations involving multiple alternatives," IRE Trans. Inf. Theory, vol. 2. pp. 138-165, 1956.
21. B. K. Scurfield, "Multiple-event forced-choice tasks in the theory of signal detectability," J. Math. Psychol., vol. 40, pp. 253-269, 1996.
22. D. C. Edwards, L. Lan, C. E. Metz, M. L. Giger, and R. M. Nishikawa, "Estimating three-class ideal observer decision variables for computerized detection and classification of mammographic mass lesions," Med. Phys., vol. 31, pp. 81-90, 2004.
23. D. C. Edwards and C. E. Metz, "Restrictions on the three-class ideal observer's decision boundary lines," IEEE Trans. Med. Imag., vol. 24, no. 12, pp. 1566-1573, Dec. 2005.
24. D. C. Edwards and C. E. Metz, "Analysis of proposed three-class classification decision rules in terms of the ideal observer decision rule." J. Math. Psychol., vol. 50, pp. 478-487, 2006.
25. D. C. Edwards, C. E. Metz, and M. A. Kupinski, "Ideal observers and optimal ROC hypersurfaces in n-class classification," IEEE Trans. Med. Imag., vol. 23, no. 7, pp. 891-895, Jul. 2004.
26. D. C. Edwards, C. E. Metz, and R. M. Nishikawa, "The hypervolume under the ROC hypersurface of "near-guessing" and "near- perfect" ob-servers in n-class classification tasks," IEEE Trans. Med. Imag., vol. 24, no. 3, pp. 293-299, Mar. 2005.
27. H.-P. Chan, B. Sahiner, L. M. Hadjiiski, N. Petrick, and C. Zhou, "Design of three-class classifiers in computer-aided diagnosis: Monte carlo simulation study," Proc. SPIE, vol. 5032, pp. 567-578, 2003.
28. D. Mossman, "Three-way ROCs," Med. Decision Making, vol. 19, pp. 78-89, 1999.
29. C. T. Nakas and C. T. Yiannoutsos, "Ordered multiple-class ROC analysis with continuous measurements," Statist. Med., vol. 23, pp. 3437-3449, 2004.
30. C. Xiong, G. van Belle, J. P. Miller, and J. C. Morris, "Measuring and estimating diagnostic accuracy when there are three ordinal diagnostic groups," Statist. Med., vol. 25, no. 7, pp. 1251-1273, 2006.
31. B. Sahiner, H.-P. Chan, and L. M. Hadjiiski, "Performance analysis of 3-class classifiers: Properties of the 3-D ROC surface and the normalized volume under the surface." Proc. SPIE, vol. 6146, 2006.
32. B. Sahiner and H. P. ChanL, M. Hadjilski, "Performance analysis of three-class classifiers: Properties of a 3-D ROC surface and the normalized volume under the surface for the ideal observer," IEEE Trans. Med. Imag., vol. 27, no. 2, pp. 215-227, Feb. 2008.
33. X. He and E. C. Frey, "Describing three-class task performance: Three-Class volume under ROC surface (VUS) and three-class hotelling trace (3-HT) as figures of merit." Proc. SPIE: Image Perception, Observer Performance, Technol. Assessment, vol. 6515, 2007.