A scaling transformation for classifier output based on likelihood ratio: Applications to a CAD workstation for diagnosis of breast cancer

Medical Physics

Volumn 39, Issue 5, 2012, Pages 2787-2804

  Horsch, Karla ^a   Pesce, Lorenzo L ^a   Giger, Maryellen L ^a   Metz, Charles E ^a   Jiang, Yulei ^a  

^a UNIVERSITY OF CHICAGO   (United States)

Author keywords

CAD user interface; classifier output scale; computer aided diagnosis; likelihood ratio; mammography; ROC analysis; sonography

Indexed keywords

COMPUTER AIDED ANALYSIS; DISTRIBUTION FUNCTIONS; MAMMOGRAPHY; MEDICAL IMAGING; METADATA; ULTRASONIC IMAGING; ULTRASONOGRAPHY; USER INTERFACES;

CAD WORKSTATIONS; COMPUTER CLASSIFIERS; CUMULATIVE DISTRIBUTION FUNCTION; FALSE POSITIVE FRACTIONS; LIKELIHOOD RATIOS; ROC ANALYSIS; SCALING TRANSFORMATION; TRUE POSITIVE FRACTIONS;

COMPUTER AIDED DIAGNOSIS;

EID: 84861623756     PISSN: 00942405     EISSN: None     Source Type: Journal    
DOI: 10.1118/1.3700168     Document Type: Article

References (31)

1. H. E. Rockette, D. Gur, L. A. Cooperstein, N. A. Obuchowski, J. L. King, C. R. Fuhrman, E. K. Tabor, and C. E. Metz, Effect of two rating formats in multi-disease ROC study of chest images., Invest. Radiol. 25, 225-229 (1990). 10.1097/00004424-199003000-00002 (Pubitemid 20085768)
2. H. P. Chan, B. Sahiner, M. A. Helvie, N. Petrick, M. A. Roubidoux, T. E. Wilson, D. D. Adler, C. Paramagul, J. S. Newman, and S. Sanjay-Gopal, Improvement of radiologists' characterization of mammographic masses by using computer-aided diagnosis: An ROC study., Radiology 212, 817-827 (1999). (Pubitemid 29410988)
3. Y. Jiang, R. M. Nishikawa, R. A. Schmidt, C. E. Metz, M. L. Giger, and K. Doi, Improving breast cancer diagnosis with computer-aided diagnosis., Acad. Radiol. 6 (1), 22-33 (1999). 10.1016/S1076-6332(99)80058-0 (Pubitemid 29061477)
4. R. F. Wagner, S. V. Beiden, and C. E. Metz, Continuous vs. categorical data for ROC analysis: some quantitative considerations., Acad. Radiol. 8, 328-334 (2001). 10.1016/S1076-6332(03)80502-0 (Pubitemid 32666455)
5. Z. Hou, M. L. Giger, C. J. Vyborny, and C. E. Metz, Breast cancer: Effectiveness of computer-aided diagnosis-Observer study with independent database of mammograms., Radiology 224, 560-588 (2002). 10.1148/radiol. 2242010703
6. L. Hadjiiski, H. P. Chan, B. Sahiner, M. A. Helvie, M. A. Roubidoux, C. Blane, C. Paramagul, N. Retrick, J. Bailey, K. Klein, M. Foster, S. Patterson, D. Adler, A. Nees, and J. Shen, Improvement in radiologists' characterization of malignant and benign masses on serial mammograms with computer-aided diagnosis: An ROC study., Radiology 233, 255-265 (2004). 10.1148/radiol.2331030432 (Pubitemid 39280982)
7. K. Horsch, M. L. Giger, C. J. Vyborny, L. Lan, E. B. Mendelson, and R. E. Hendrick, Multi-modality computer-aided diagnosis for the classification of breast lesions: Observer study results on an independent clinical dataset., Radiology 240, 357-368 (2006). 10.1148/radiol.2401050208 (Pubitemid 44098631)
8. J. G. Dolan, D. R. Bordley, and A. I. Mushlin, An evaluation of clinicians' subjective prior probability estimates., Med. Decis. Making 6, 216-223 (1986). 10.1177/0272989X8600600406 (Pubitemid 17199377)
9. A. G. Levy and J. C. Hershey, Value-induced bias in medical decision making., Med. Decis. Making 28, 269-276 (2008). 10.1177/0272989X07311754 (Pubitemid 351441945)
10. C. E. Metz, ROC methodology in radiologic imaging., Invest. Radiol. 21, 720-733 (1986). 10.1097/00004424-198609000-00009 (Pubitemid 16015664)
11. M. L. Giger, Z. Huo, L. Lan, and C. J. Vyborny, Intelligent search workstation for computer aided diagnosis., in Proceedings of Computer Assisted Radiology and Surgery (CARS) 2000, edited by, H. U. Lemke, K. Inamura, D. Kunio, M. W. Vannier, and, A. G. Farman, (Elsevier, Philadelphia, PA, 2000), pp. 822-827.
12. M. L. Giger, Z. Huo, C. J. Vyborny, L. Lan, R. M. Nishikawa, and I. Rosenbourgh, Results of an observer study with an intelligent mammographic workstation for CAD., in Digital Mammography: IWDM 2002, edited by, Peitgen H.-O., (Springer, Berlin, 2002), pp. 297-303.
13. Here, a normalized histogram is the standard histogram divided by the product of the bin size and the total number of values binned. The normalized histogram estimates the density function.
14. The Spearman rank correlation coefficient Ref. (Spear) is one way to measure case-by-case variation in the rank order of the output of two classifiers. In particular, Spearman's coefficient can be used to quantify how well an unspecified monotonic function can describe the relationship between two variables.
15. M. G. Kendall, Rank and product-moment correlation., Biometrika 36, 177-193 (1949). 10.2307/2332540
16. C. E. Metz and J.-H. Shen, Gains in accuracy from replicated readings of diagnostic images: Prediction and assessment in terms of ROC analysis., Med. Decis. Making 12, 60-75 (1992). 10.1177/0272989X9201200110
17. K. Horsch, M. L. Giger, and C. E. Metz, Prevalence scaling: Applications to an intelligent workstation for the diagnosis of breast cancer., Acad. Radiol. 15, 1446-1457 (2008). 10.1016/j.acra.2008.04.022
18. C. E. Metz, B. A. Herman, and J.-H. Shen, Maximum-likelihood estimation of ROC curves from continuously-distributed data., Stat. Med. 17, 1033 (1998). 10.1002/(SICI)1097-0258(19980515)17:9<1033::AID-SIM784>3.0.CO;2-Z (Pubitemid 28203832)
19. C. E. Metz and X. Pan, 'Proper' binormal ROC curves: Theory and maximum-likelihood estimation., J. Math. Psychol. 43, 1-33 (1999). 10.1006/jmps.1998.1218
20. L. L. Pesce and C. E. Metz, Reliable and computationally efficient maximum-likelihood estimation of proper' binormal ROC curves., Acad. Radiol. 14, 814-829 (2007). 10.1016/j.acra.2007.03.012 (Pubitemid 46898850)
21. C. E. Metz, P-L Wang, and H. B. Kronman, A new approach for testing the significance of differences between ROC curves measured from correlated data. in Information Processing in Medical Imaging, edited by, F. Deconinck, (Nihoff, The Hague, 1984), pp. 432-445.
22. See supplementary material at http://dx.doi.org/10.1118/1.3700168 E-MPHYA6-39-004205 for Appendices 1-3.
23. On December 29, 2009, the Breast Cancer Surveillance Consortium reported on its website that 11,200 out of 273,601 diagnostic mammography examinations performed between 1996 and 2007 received a tissue diagnosis of ductal carcinoma in situ or invasive cancer within 30 days prior to or 1 year following the examination.
24. Web site of the Breast Cancer Surveillance Consortium. http://breastscreening.cancer.gov/data/performance/diagnostic. Accessed December 29, 2009.
25. pop depends on the application of interest. Possible choices include the prevalence in a relevant clinical population, the prevalence in the database used to train the computer classifier, or the prevalence in the database used in a particular observer study.
26. H. L. Van Trees, Detection, Estimation and Modulation Theory: Part I (Wiley, New York, 1968).
27. In the utility function referenced here, the individual utilities of correctly and incorrectly performing a biopsy are linearly related to the total utility.
28. C. E. Metz, Basic principles of ROC analysis., Semin. Nucl. Med. 8, 283-298 (1978). 10.1016/S0001-2998(78)80014-2 (Pubitemid 9075026)
29. D. M. Green and J. A. Swets, Signal Detection Theory and Psychophysics (Kriger, Huntington, NY, 1974).
30. PROPROC. Website of Kurt Rossman Laboratories, Department of Radiology, University of Chicago: http://rocweb.bsd.uchicago.edu/MetzROC.
31. L. L. Pesce, K. Horsch, K. Drucker, and C. E. Metz, Semi-parametric estimation of the relationship between ROC operating points and the test-result scale: Application to the proper binormal model., Acad. Radiol. 18, 1537-1548 (2011). 10.1016/j.acra.2011.08.003