Investigation of methods for analyzing location specific observer performance data

Progress in Biomedical Optics and Imaging - Proceedings of SPIE

Volumn 6917, Issue , 2008, Pages

  Chakraborty, D P ^a   Yoon, Hong Jun ^a  

^a UNIVERSITY OF PITTSBURGH SCHOOL OF MEDICINE   (United States)

Author keywords

CAD evaluation; FROC analysis; IDCA; JAFROC; Observer performance; ROC; Search model

Indexed keywords

BEHAVIORAL RESEARCH; CMOS INTEGRATED CIRCUITS; COMPUTER AIDED DESIGN; COMPUTER NETWORKS; CORRELATION METHODS; CURVES (ROAD); DESIGN; IMAGING TECHNIQUES; MATHEMATICAL MODELS; MEDICAL IMAGING; MULTITASKING; NUMERICAL METHODS; PARAMETER ESTIMATION; RANGE FINDING; STATISTICS; TECHNOLOGICAL FORECASTING; TECHNOLOGY;

(SPM) CLASSES; AVERAGE NUMBERS; CAD DATA; DATA CORRELATIONS; FALSE POSITIVE (FP); FIGURE-OF-MERIT (FOM); FIGURES OF MERIT (FOM); HUMAN OBSERVERS; IMAGE PERCEPTION; MODEL BASED (OPC); MODEL PARAMETERS; OBSERVER PERFORMANCES; RECEIVER OPERATOR CHARACTERISTIC (ROC) CURVES; ROC METHOD; STATISTICAL POWER; TECHNOLOGY ASSESSMENT (TA);

STATISTICAL METHODS;

EID: 44949113405     PISSN: 16057422     EISSN: None     Source Type: Conference Proceeding    
DOI: 10.1117/12.770531     Document Type: Conference Paper

References (43)

1. Lusted, L. B., "Signal detectability and medical decision making", Science 171, 1217-1219. (1971).
2. Metz, C. E., "Basic principles of ROC analysis", Seminars in Nuclear Medicine 8 (4), 283-298 (1978).
3. Metz, C. E., "ROC Methodology in Radiologic Imaging", Inv Radiol 21 (9), 720-733 (1986).
4. Metz, C. E., "Receiver Operating Characteristic Analysis: A Tool for the Quantitative Evaluation of Observer Performance and Imaging Systems", J Am Coll Radiol 3,413-422 (2006).
5. Swensson, R. G., "Unified measurement of observer performance in detecting and localizing target objects on images", Med. Phys. 23 (10), 1709-1725 (1996).
6. Starr, S. J., Metz, C. E., Lusted, L. B. and Goodenough, D. J., "Visual detection and localization of radiographic images", Radiology 116, 533-538 (1975).
7. Starr, S. J., Metz, C. E. and Lusted, L. B., "Comments on generalization of Receiver Operating Characteristic analysis to detection and localization tasks", Phys. Med. Biol. 22, 376-379 (1977).
8. Egan, J. P., Greenburg, G. Z. and Schulman, A. I., "Operating characteristics, signal detectability and the method of free response." J Acoust Soc. Am. 33, 993-1007 (1961).
9. Bunch, P. C., Hamilton, J. F., Sanderson, G. K. and Simmons, A. H., "A Free-Response Approach to the Measurement and Characterization of Radiographic-Observer Performance", J of Appl Photogr. Eng. 4 (4), 166-171 (1978).
10. Chakraborty, D. P., Breatnach, E. S., Yester, M. V., Soto, B., Barnes, G. T. and Fraser, R. G., "Digital and Conventional Chest Imaging: A Modified ROC Study of Observer Performance Using Simulated Nodules", Radiology 158, 35-39 (1986).
11. Bunch, P. C., Hamilton, J. F., Sanderson, G. K. and Simmons, A. H., "A Free-Response Approach to the Measurement and Characterization of Radiographic-Observer Performance", Proc. SPIE 127,124-135 (1977).
12. Edwards, D. C., Kupinski, M. A., Metz, C. E. and Nishikawa, R. M., "Maximum likelihood fitting of FROC curves under an initial-detection-and- candidate-analysis model", Med Phys 29 (12), 2861-2870 (2002).
13. Yoon, H. J., Zheng, B., Sahiner, B. and Chakraborty, D. P., "Evaluating computer-aided detection algorithms", Med Phys 34 (6), 2024-2038 (2007).
14. Chakraborty, D. P. and Berbaum, K. S., "Observer studies involving detection and localization: Modeling, analysis and validation", Med Phys 31 (8), 2313-2330 (2004).
15. Penedo, M., Souto, M., Tahoces, P. G., Carreira, J. M., Villalon, J., Porto, G., Seoane, C., Vidal, J. J., Berbaum, K. S., Chakraborty, D. P. and Fajardo, L. L., "Free-Response Receiver Operating Characteristic Evaluation of Lossy JPEG2000 and Object-based Set Partitioning in Hierarchical Trees Compression of Digitized Mammograms", Radiology 237 (2), 450-457 (2005).
16. Zheng, B., Chakraborty, D. P., Rockette, H. E., Maitz, G. S. and Gur, D., "A comparison of two data analyses from two observer performance studies using Jackknife ROC and JAFROC", Med Phys 32 (4), 1031-1034 (2005).
17. Ruschin, M., Timberg, P., Bath, M., Hemdal, B., Svahn, T., Saunders, R., Samei, E., Andersson, I., Mattsson, S., Chakraborty, D. P. and Tingberg, A., "Dose dependence of mass and microcalcification detection in digital mammography: free response human observer studies", Med. Phys. 34, 400-407 (2007).
18. Timberg, P., Ruschin, M., Båth, M., Hemdal, B., Andersson, I., Mattsson, S., Chakraborty, D., Saunders, R., Samei, E. and Tingberg, A., "Potential for lower absorbed dose in digital mammography: a JAFROC experiment using clinical hybrid images with simulated dose reduction" Proc. SPIE Medical Imaging, 6146, (2006).
19. Volokh, L., Liu, C. and Tsui, B. M. W., "Exploring FROC paradigm: initial experience with clinical applications" Proc. SPIE Medical Imaging, 6146, (2006).
20. Chakraborty, D. P., "A search model and figure of merit for observer data acquired according to the free-response paradigm", Phys. Med. Biol. 51,3449-3462 (2006).
21. Chakraborty, D, P., "ROC Curves predicted by a model of visual search", Phys. Med. Biol. 51, 3463-3482 (2006).
22. Chakraborty, D. P. and Yoon, H. J., "Operating characteristics predicted by a search model for diagnostic tasks involving lesion localization (accepted Med. Phys.)", Med. Phys. (2008).
23. Chakraborty, D. P., "FROC curves using amodel of visual search" Proc. SPIE Medical Imaging, 6515, (2007).
24. Dorfman, D. D. and Alf, E., "Maximum-Likelihood Estimation of Parameters of Signal-Detection Theory and Determination of Confidence Intervals - Rating-Method Data." Journal of Mathematical Psychology 6,487-496 (1969).
25. Metz, C. E., "ROC software", http://xray.bsd.uchicago.edu/krl/ rocsoft6.htm (2006).
26. Dorfman, D. D. and Berbaum, K. S., "A contaminated binomial model for ROC data: Part II. A formal model", Acad Radiol. 7 (6), 427-437 (2000).
27. Dorfman, D. D. and Berbaum, K. S., "A contaminated binormal model for ROC data: Part III. Initial evaluation with detection", Acad Radiol. 7 (6), 438-447 (2000).
28. Dorfman, D. D., Berbaum, K. S. and Brandser, E. A., "A contaminated binormal model for ROC data: Part I. Some interesting examples of binormal degeneracy", Acad Radiol. 7 (6), 420-426 (2000).
29. Dorfman, D. D., Berbaum, K. S., Metz, C. E., Lenth, R. V., Hanley, J. A. and Abu Dagga, H., "Proper Receiving Operating Characteristic Analysis: The Bigamma model", Acad. Radiol. 4 (2), 138-149 (1997).
30. Pan, X. and Metz, C. E., "The proper binormal model: parametric receiver operating characteristic curve estimation with degenerate data", Acad. Radiol. 4 (5), 380-389 (1997).
31. Pesce, L. L. and Metz, C. E., "Reliable and Computationally Efficient Maximum-Likelihood Estimation of Proper Binormal ROC Curves", Acad Radiol 14 (7), 814-829 (2007).
32. Metz, C. E., Herman, B. and Shen, J., "Maximum-Likelihood Estimation of Receiver Operating Characteristic (ROC) Curves from Continuously-Distributed Data", Statistics in Medicine 17,1033-1053 (1998).
33. Metz, C. E. and Pan, X., "Proper Binormal ROC Curves: Theory and Maximum-Likelihood Estimation", J Math Psychol 43 (1), 1-33 (1999).
34. Samuelson, F. W. and Petrick, N., "Comparing image detection algorithms using resampling", 2006 IEEE International Symposium on Biomedical Imaging: From Nano to Micro, 1312-1315 (2006).
35. Efron, B., [The Jackknife, the Bootstrap and Other Resampling Plans]. Capital City Press, Montpelier, (1982).
36. Matsumoto, M. and Nishimura, T., "Mersenne Twister - a 623-dimensionally equidistributed uniform pseudorandom number generator", ACM Transactions on Modeling and Computer Simulation 8 (1), 3-30 (1998).
37. Galassi, M., Davies, J., Theiler, J., Gough, B., Jungman, G., Booth, M. and Rossi, F., [GNU Scientific Library Reference Manual], 1.6 ed. Network Theory Limited, Bristol, UK, (2005).
38. C Mello-Thoms, L. H., J Sumkin, M Ganott, C Hakim, C Britton, J Stalder, G Maitz., "Effects of lesion conspicuiry on visual search in mammogram reading", Acad Radiol 12, 830-840 (2005).
39. Steen, F. H., [Elements of mathematical statistics]. Duxbury press, Boston, (1982).
40. Hillis, S. L. and Berbaum, K. S., "Power Estimation for the Dorfman-Berbaum-Metz Method", Acad. Radiol. 11(11), 1260-1273(2004).
41. Swensson, R. G. and Judy, P. F., "Detection of noisy visual targets: Models for the effects of spatial uncertainty and signal-to-noise ratio", Perception & Psychophyics 29 (6), 521-534 (1981).
42. Obuchowski, N. A., Lieber, M. L. and Powell, K. A., "Data analysis for detection and localization of multiple abnormalities with application to mammography. [see comments]", Acad. Radiol. 7 (7), 516-525 (2000).
43. Chakraborty, D. P., "Data analysis for detection and localization of multiple abnormalities with application to mammography", Acad. Radiol. 7 (7), 553-554 (2000).