Observer studies involving detection and localization: Modeling, analysis, and validation

Medical Physics

Volumn 31, Issue 8, 2004, Pages 2313-2330

  Chakraborty, Dev P ^a   Berbaum, Kevin S ^a  

^a NONE

Author keywords

FROC analysis; Localization; Observer performance; ROC analysis; Statistical power

Indexed keywords

EYE MOVEMENTS;

FROC ANALYSIS; LOCALIZATION; OBSERVER PERFORMANCE; ROC ANALYSIS; STATISTICAL POWER;

STATISTICS;

ALGORITHM; ALTERNATIVE FREE RESPONSE RECEIVER OPERATING CHARACTERISTIC; ANALYTIC METHOD; CALCULATION; CONFERENCE PAPER; DIAGNOSTIC IMAGING; DIAGNOSTIC VALUE; DORFMAN BERBAUM METZ METHOD; EYE MOVEMENT; IMAGE ANALYSIS; IMAGING SYSTEM; INTERMETHOD COMPARISON; JACKKNIFE FREE RESPONSE RECEIVER OPERATING CHARACTERISTIC; MATHEMATICAL ANALYSIS; MEASUREMENT; MODEL; NOISE; PERFORMANCE; PRIORITY JOURNAL; RECEIVER OPERATING CHARACTERISTIC; SAMPLE SIZE; SCORING SYSTEM; SIGNAL DETECTION; SIMULATION; STATISTICAL ANALYSIS; VALIDATION PROCESS;

EID: 4043148805     PISSN: 00942405     EISSN: None     Source Type: Journal    
DOI: 10.1118/1.1769352     Document Type: Conference Paper

References (40)

1. R. Wagner, S. V. Beiden, G. Campbell, C. E. Metz, and W. M. Sacks, "Assessment of medical imaging and computer-assist systems: Lessons from recent experience," Acad. Radiol. 9, 1264-1277 (2002).
2. D. D. Dorfman, K. S. Berbaum, and C. E. Metz, "ROC characteristic rating analysis: Generalization to the population of readers and patients with the jackknife method," Invest. Radiol. 27, 723-731 (1992).
3. D. P. Chakraborty, "Statistical power in observer performance studies: A comparison of the ROC and free-response methods in tasks involving localization," Acad. Radiol. 9, 147-156 (2002).
4. S. J. Starr, C. E. Metz, L. B. Lusted, and D. J. Goodenough, "Visual detection and localization of radiographic images," Radiology 116, 533-538 (1975).
5. R. G. Swensson, "Unified measurement of observer performance in detecting and localizing target objects on images," Med. Phys. 23, 1709-1725 (1996).
6. N. A. Obuchowski, M. L. Lieber, and K. A. Powell, "Data analysis for detection and localization of multiple abnormalities with application to mammography," Acad. Radiol. 7, 516-525 (2000).
7. K. O. Hajian-Tilaki, J. A. Hanley, L. Joseph, and J. P. Collet, "Extension of receiver operating characteristic analysis to data concerning multiple signal detection tasks," Acad. Radiol. 4, 222-229 (1997).
8. C. M. Rutter, "Bootstrap estimation of diagnostic accuracy with patient-clustered data," Acad. Radiol. 7, 413-419 (2000).
9. P. C. Bunch, J. F. Hamilton, G. K. Sanderson, and A. H. Simmons, "A free-response approach to the measurement and characterization of radiographic-observer performance," J. Appl. Photogr. Eng. 4, 166-171 (1978).
10. D. P. Chakraborty, "Maximum likelihood analysis of free-response receiver operating characteristic (FROC) data," Med. Phys. 16, 561-568 (1989).
11. D. P. Chakraborty and L. Winter, "Free-response methodology: Alternate analysis and a new observer-performance experiment," Radiology 174, 873-881 (1990).
12. C. E. Metz, "Evaluation of digital mammography by ROC analysis," in Digital Mammography '96, edited by K. Doi et al. (Elsevier Science, Amsterdam, 1996).
13. D. P. Chakraborty, "Data analysis for detection and localization of multiple abnormalities with application to mammography. [letter; comment]," Acad. Radiol. 7, 553-554 (2000); discussion 554-556.
14. E. A. Krupinski, "Visual scanning patterns of radiologists searching mammograms," Acad. Radiol. 3, 137-144 (1996).
15. H. L. Kundel and P. S. J. La Follette, "Visual search patterns and experience with radiological images," Radiology 103, 523-528 (1972).
16. A. Hillstrom, "Repetition effects in visual search," Percept. Psychophys. 2, 800-817 (2000).
17. C. E. Metz and H. Kronman, "Statistical significance tests for binormal ROC curves," J. Math. Psychol. 22, 218-242 (1980).
18. J. A. Hanley, "The robustness of the 'binormal' assumptions used in fitting ROC curves," Med. Decis. Making 8, 197-203 (1988).
19. C. Roe and C. E. Metz, "Dorfman-Berbaum-Metz method for statistical analysis of multireader, multimodality receiver operating characteristic data: Validation with computer simulation," Acad. Radiol. 4, 298-303 (1997).
20. D. D. Dorfman, K. S. Berbaum, R. V. Lenth, Y.-F. Chen, and B. A. Donaghy, "Monte Carlo validation of a multireader method for receiver operating characteristic discrete rating data: Factorial experimental design," Acad. Radiol. 5, 591-602 (1998).
21. D. D. Dorfman, K. S. Berbaum, R. V. Lenth, and Y.-F. Chen, "Monte Carlo validation of a multireader method for receiver operating characteristic discrete rating data: Split plot experimental design," in Proceedings of SPIE, Medical Imaging, Image Perception and Performance, 1999.
22. C. H. Hu, H. L. Kundel, C. F. Nodine, E. A. Krupinski, and L. C. Toto, "Searching for bone fractures: A comparison with pulmonary nodule search," Acad. Radiol. 1, 25-32 (1994).
23. H. L. Kundel, C. F. Nodine, and D. Carmody, "Visual scanning, pattern recognition and decision-making in pulmonary nodule detection," Invest. Radiol. 13, 175-181 (1978).
24. C. F. Nodine and H. L. Kundel, "Using eye movements to study visual search and to improve tumor detection," Radiographics 7, 1241-1250 (1987).
25. R. E. Hendrick et al., Mammography Quality Control Manual, 4th ed. (American College of Radiology, Committee on Quality Assurance in Mammography, 1999).
26. D. P. Chakraborty, "Proposed solution to the FROC problem and an invitation to collaborate," Proc. SPIE 5034, 204-212 (2003).
27. D. Dorfman and E. J. Alf, "Maximum likelihood estimation of parameters of signal-detection theory and determination of confidence intervals-rating method data," J. Math. Psychol. 6, 487-496 (1969).
28. J. A. Hanley and B. J. McNeil, "A method of comparing the areas under receiver operating characteristic curves derived from the same cases," Radiology 148, 839-843 (1983).
29. C. Roe and C. E. Metz, "Variance-component modeling in the analysis of receiver operating characteristic index estimates," Acad. Radiol. 4, 587-600 (1997).
30. A. E. Burgess, "Comparison of receiver operating characteristic and forced choice observer performance measurement methods," Med. Phys. 22, 643-655 (1995).
31. W. H. Press, B. P. Flannery, S. A. Teulosky, and W. T. Vetterling, Numerical Recipes in C: The Art of Scientific Computing (Cambridge University Press, Cambridge, 1988), p. 735.
32. ICRU, Medical Imaging: The Assessment of Image Quality (International Commission on Radiation Units and Measurements, Bethesda, MD, 1996), Vol. 54.
33. K. S. Berbaum, E. A. Franken, D. D. Dorfman, S. A. Rooholamini, M. H. Kathol, T. J. Barloon, F. M. Behlke, Y. Sato, C. H. Lu, G. Y. El-Khoury, F. W. Flickinger, and W. J. Montgomery, "Satisfaction of search in diagnostic radiology," Invest. Radiol. 25, 133-140 (1990).
34. K. S. Berbaum, G. Y. El-Khoury, E. A. Franken, D. M. Kuehn, D. M. Meis, D. D. Dorfman, N. G. Warnock, B. H. Thompson, S. Kao, and M. H. Kathol, "Missed fractures resulting from satisfaction of search effect," Emerg. Radiol. 1, 242-249 (1994).
35. K. S. Berbaum, E. A. Franken, D. D. Dorfman, E. M. Miller, R. T. Caldwell, D. M. Kuehn, and M. L. Berbaum, "Role of faulty visual search in the satisfaction of search effect in chest radiography," Acad. Radiol. 5, 9-19 (1998).
36. S. Samuel, H. L. Kundel, C. F. Nodine, and L. C. Toto, "Mechanism of satisfaction of search: Eye position recordings in the reading of chest radiographs," Radiology 194, 895-902 (1995).
37. N. A. Obuchowski, M. L. Lieber, and K. A. Powell, "Data analysis for detection and localization of multiple abnormalities with application to mammography [see comments]," Acad. Radiol. 7, 516-525 (2000).
38. X.-H. Zhou, D. K. McClish, and N. A. Obuchowski, Statistical Methods in Diagnostic Medicine (Wiley-Interscience, New York, 2002).
39. R. G. Swensson, G. Maitz, J. L. King, and D. Gur, "Using incomplete and imprecise localization data on images to improve estimates of detection accuracy," Proc. SPIE 3663, 74-81 (1999).
40. R. G. Swensson, J. L. King, and D. Gur, "A constrained formulation for the receiver operating characteristic (ROC) curve based on probability summation," Med. Phys. 28, 1597-1609 (2001).