Spatial Localization Accuracy of Radiologists in Free-Response Studies: Inferring Perceptual FROC Curves from Mark-Rating Data

Academic Radiology

Volumn 14, Issue 1, 2007, Pages 4-18

  Chakraborty, Dev ^a   Yoon, Hong Jun ^a   Mello Thoms, Claudia ^a  

^a UNIVERSITY OF PITTSBURGH SCHOOL OF MEDICINE   (United States)

Author keywords

acceptance radius; CAD evaluation; FROC curves; imaging systems assessment; Localization accuracy; observer performance; perceptual analysis

Indexed keywords

ALGORITHM; ARTICLE; COMPARATIVE STUDY; COMPUTER ASSISTED RADIOGRAPHY; CONTROLLED STUDY; DIAGNOSTIC ACCURACY; FREE RESPONSE RECEIVER OPERATING CHARACTERISTIC; HUMAN; IMAGING SYSTEM; MAMMOGRAPHY; OBSERVATIONAL METHOD; OBSERVER VARIATION; PERCEPTION; PRIORITY JOURNAL; RADIOLOGIST; RECEIVER OPERATING CHARACTERISTIC; RESPONSE SURFACE METHOD; SCORING SYSTEM; STATISTICAL MODEL;

IMAGE INTERPRETATION, COMPUTER-ASSISTED; MATHEMATICS; MODELS, THEORETICAL; OBSERVER VARIATION;

EID: 33845389587     PISSN: 10766332     EISSN: None     Source Type: Journal    
DOI: 10.1016/j.acra.2006.10.015     Document Type: Article

References (38)

1. Egan J.P., Greenburg G.Z., and Schulman A.I. Operating characteristics, signal detectability and the method of free response. J Acoust Soc Am 33 (1961) 993-1007
2. Bunch P.C., Hamilton J.F., Sanderson G.K., et al. A free-response approach to the measurement and characterization of radiographic-observer performance. J Appl Photogr Eng 4 (1978) 166-171
3. Chakraborty D.P. Maximum likelihood analysis of free-response receiver operating characteristic (FROC) data. Med Phys 16 (1989) 561-568
4. Chakraborty D.P., Breatnach E.S., Yester M.V., et al. Digital and conventional chest imaging: a modified ROC study of observer performance using simulated nodules. Radiology 158 (1986) 35-39
5. Chakraborty D.P., and Winter L.H.L. Free-response methodology: alternate analysis and a new observer-performance experiment. Radiology 174 (1990) 873-881
6. Chakraborty D.P., and Berbaum K.S. Observer studies involving detection and localization: modeling, analysis and validation. Med Phys 31 (2004) 2313-2330
7. Zheng B., Chakraborty D.P., Rockette H.E., et al. A comparison of two data analyses from two observer performance studies using jackknife ROC and JAFROC. Med Phys 32 (2005) 1031-1034
8. Penedo M., Souto M., Tahoces P.G., et al. Free-response receiver operating characteristic evaluation of lossy JPEG2000 and object-based set partitioning in hierarchical trees compression of digitized mammograms. Radiology 237 (2005) 450-457
9. Bornefalk H., and Hermansson A.B. On the comparison of FROC curves in mammography CAD systems. Med Phys 32 (2005) 412-417
10. Bornefalk H. Estimation and comparison of CAD system performance in clinical settings. Acad Radiol 12 (2005) 687-694
11. Edwards D.C., Kupinski M.A., Metz C.E., et al. Maximum likelihood fitting of FROC curves under an initial-detection-and-candidate-analysis model. Med Phys 29 (2002) 2861-2870
12. Metz C.E. Evaluation of digital mammography by ROC analysis. In: Doi K. (Ed). Digital mammography '96 (1996), Elsevier Science, Amsterdam, the Netherlands 61-68
13. Metz C.E. Some practical issues of experimental design and data analysis in radiological ROC studies. Investig Radiol 24 (1989) 234-245
14. Metz C.E. ROC methodology in radiologic imaging. Investig Radiol 21 (1986) 720-733
15. Zheng B., Shah R., Wallace L., et al. Computer-aided detection in mammography: an assessment of performance on current and prior images. Acad Radiol 9 (2002) 1245-1250
16. Gur D., Stalder J.S., Hardesty L.A., et al. Computer-aided detection performance in mammographic examination of masses: assessment. Radiology 233 (2004) 418-423
17. Kallergi M., Carney G.M., and Gaviria J. Evaluating the performance of detection algorithms in digital mammography. Med Phys 26 (1999) 267-275
18. In: Giger M.L., Doi K., Giger M.L., et al. (Eds). Digital mammography '96: current issues in CAD for mammography (1996), Elsevier Science B.V
19. Nishikawa R.M., and Yarusso L.M. Variations in measured performance of CAD schemes due to database composition and scoring protocol. Proc SPIE 3338 (1998) 840-844
20. Reiser I., Nishikawa R.M., Giger M.L., et al. Computerized mass detection for digital breast tomosynthesis directly from the projection images. Med Phys 33 (2006) 482-491
21. Sahiner B., Chan H.P., Hadjiiski L.M., et al. Joint two-view information for computerized detection of microcalcifications on mammograms. Med Phys 33 (2006) 2574-2585
22. Zheng B., Leader J.K., Abrams G.S., et al. A multi view based computer aided detection scheme for breast masses. Med Phys 33 (2006) 3135-3143
23. Duchowski A.T. Eye tracking methodology: theory and practice (2002), Clemson University, Clemson, SC
24. Green D.M., and Swets J.A. Signal detection theory and psychophysics (1966), John Wiley & Sons, New York
25. Metz C.E. Basic principles of ROC analysis. Semin Nucl Med VIII (1978) 283-298
26. Chakraborty D.P. ROC curves predicted by a model of visual search. Phys Med Biol 51 (2006) 3463-3482
27. Chakraborty D.P. A search model and figure of merit for observer data acquired according to the free-response paradigm. Phys Med Biol 51 (2006) 3449-3462
28. Kundel H.L., and Nodine C.F. A visual concept shapes image perception. Radiology 146 (1983) 363-368
29. Nodine C.F., and Kundel H.L. Using eye movements to study visual search and to improve tumor detection. RadioGraphics 7 (1987) 1241-1250
30. Kundel H.L., and Nodine C.F. Modeling visual search during mammogram viewing. Proc SPIE 5372 (2004) 110-115
31. Galassi M., Davies J., Theiler J., et al. GNU scientific library reference manual (2005), Network Theory Limited, Bristol, UK
32. Stuart A., Ord K., and Arnold S. Kendall's advance theory of statistics: classical inference and the linear model vol 2A (2004), Oxford University Press, New York
33. Larsen R.J., and Marx M.L. An introduction to mathematical statistics and its applications (2001), Prentice-Hall Inc, Upper Saddle River, NJ
34. Press W.H., Flannery B.P., Teukolsky S.A., et al. Numerical recipes in C: the art of scientific computing (1988), Cambridge University Press, Cambridge, UK
35. Dorfman D.D., and Alf E. Maximum likelihood estimation of parameters of signal-detection theory and determination of confidence intervals-rating method data. J Math Psychol 6 (1969) 487-496
36. Mello-Thoms C., Dunn S., Nodine C.F., et al. The perception of breast cancers: a spatial frequency analysis of what differentiates missed from reported cancers. IEEE Trans Med Imaging 22 (2003) 1297-1306
37. Dorfman D.D., Berbaum K.S., and Metz C.E. ROC characteristic rating analysis: generalization to the population of readers and patients with the jackknife method. Investig Radiol 27 (1992) 723-731
38. Arora R., Kundel H.L., and Beam C.A. Perceptually based FROC analysis. Acad Radiol 12 (2005) 1567-1574