JAFROC analysis revisited: Figure-of-merit considerations for human observer studies

Progress in Biomedical Optics and Imaging - Proceedings of SPIE

Volumn 7263, Issue , 2009, Pages

  Chakraborty, D P ^a   Yoona, Hong Jun ^a  

^a UNIVERSITY OF PITTSBURGH SCHOOL OF MEDICINE   (United States)

Author keywords

Free response; Imaging system assessment; JAFROC; JAFROC1; Lesion localization; Lesioncentric analysis; Simulators; Weighted analysis

Indexed keywords

FREE-RESPONSE; IMAGING SYSTEM ASSESSMENT; JAFROC; JAFROC1; LESION LOCALIZATION; LESIONCENTRIC ANALYSIS; WEIGHTED ANALYSIS;

IMAGING SYSTEMS; OPTOELECTRONIC DEVICES; SIMULATORS; TECHNOLOGICAL FORECASTING;

MEDICAL IMAGING;

EID: 67249156410     PISSN: 16057422     EISSN: None     Source Type: Conference Proceeding    
DOI: 10.1117/12.810859     Document Type: Conference Paper

References (36)

1. Egan, J. P., Greenburg, G. Z. and Schulman, A. I., "Operating characteristics, signal detectability and the method of free response." J Acoust Soc. Am. 33, 993-1007 (1961).
2. Bunch, P. C., Hamilton, J. F., Sanderson, G. K. and Simmons, A. H., "A Free-Response Approach to the Measurement and Characterization of Radiographic-Observer Performance", J of Appl Photogr. Eng. 4 (4), 166-171 (1978). (Pubitemid 9408892)
3. Chakraborty, D. P., Breatnach, E. S., Yester, M. V., Soto, B., Barnes, G. T. and Fraser, R. G., "Digital and Conventional Chest Imaging: A Modified ROC Study of Observer Performance Using Simulated Nodules", Radiology 158, 35-39 (1986). (Pubitemid 16169420)
4. Bunch, P. C., Hamilton, J. F., Sanderson, G. K. and Simmons, A. H., "A Free-Response Approach to the Measurement and Characterization of Radiographic-Observer Performance", Proc. SPIE 127, 124-135 (1977).
5. Chakraborty, D. P., "Maximum Likelihood analysis of free-response receiver operating characteristic (FROC) data", Med. Phys. 16 (4), 561-568 (1989). (Pubitemid 19224306)
6. Chakraborty, D. P. and Berbaum, K. S., "Observer studies involving detection and localization: Modeling, analysis and validation", Medical Physics 31 (8), 2313-2330 (2004). (Pubitemid 39079724)
7. Penedo, M., Souto, M., Tahoces, P. G., Carreira, J. M., Villalon, J., Porto, G., Seoane, C., Vidal, J. J., Berbaum, K. S., Chakraborty, D. P. and Fajardo, L. L., "Free-Response Receiver Operating Characteristic Evaluation of Lossy JPEG2000 and Object-based Set Partitioning in Hierarchical Trees Compression of Digitized Mammograms", Radiology 237 (2), 450-457 (2005). (Pubitemid 41507879)
8. Ruschin, M., Timberg, P., Bath, M., Hemdal, B., Svahn, T., Saunders, R., Samei, E., Andersson, I., Mattsson, S., Chakraborty, D. P. and Tingberg, A., "Dose dependence of mass and microcalcification detection in digital mammography: free response human observer studies", Med. Phys. 34, 400 - 407 (2007). (Pubitemid 46185118)
9. Svahn, T., Hemdal, B., Ruschin, M., Chakraborty, D. P., Andersson, I., Tingberg, A. and Mattson, S., "Dose reduction and its influence on diagnostic accuracy and radiation risk in digital mammography: an observer performance study using an anthropomorphic breast phantom", British Journal of Radiology 80, 557-562 (2007). (Pubitemid 47343083)
10. Vikgren, J., Zachrisson, S., Svalkvist, A., Johnsson, A. A., Boijsen, M., Flinck, A., Kheddache, S. and Bath, M., "Comparison of Chest Tomosynthesis and Chest Radiography for Detection of Pulmonary Nodules: Human Observer Study of Clinical Cases", Radiology 249 (3), 1034-1041 (2008).
11. Zanca, F., Chakraborty, D. P., Van Ongeval, C., Jacobs, J., Claus, F., Marchal, G. and Bosmans, H., "An improved method for simulating microcalcifications in digital mammograms", Medical Physics 35 (9), 4012- 4018 (2008).
12. Sahiner, B., Hadjiiski, L. M., Chan, H.-P., Shi, J., Cascade, P. N., Kazerooni, E. A., Zhou, C., Wei, J., Chughtai, A. R., Poopat, C., Song, T., Nojkova, J. S., Frank, L. and Attili, A., "Effect of CAD on radiologists' detection of lung nodules on thoracic CT scans: observer performance study", SPIE Medical Imaging 2007: Perception, Observer Performance, and Technology Assessment 6515 (2007).
13. McEntee, M. F., Ryan, J., Evanoff, M. G., Keeling, A., Chakraborty, D., Manning, D. and Brennan, P. C., "Ambient lighting: setting international standards for the viewing of softcopy chest images", Proc. SPIE Medical Imaging 2007: Image Perception, Observer Performance, and Technology Assessment 6515 (2007).
14. Brennan, P. C., McEntee, M., Evanoff, M., Phillips, P., O'Connor, W. T. and Manning, D. J., "Ambient lighting: effect of illumination on soft-copy viewing of radiographs of the wrist", Am J Roentgenol 188 (2), W177-180 (2007).
15. Volokh, L., Liu, C. and Tsui, B. M. W., "Exploring FROC paradigm - initial experience with clinical applications" Medical Imaging 2006, Image Perception, Observer Performance and Technology Assessment, 6146, (2006).
16. Timberg, P., Ruschin, M., Båth, M., Hemdal, B., Andersson, I., Mattsson, S., Chakraborty, D., Saunders, R., Samei, E. and Tingberg, A., "Potential for lower absorbed dose in digital mammography: a JAFROC experiment using clinical hybrid images with simulated dose reduction", SPIE Medical Imaging 2006: Perception, Observer Performance, and Technology Assessment 6146 (2006).
17. Chakraborty, D. P., "Validation and Statistical Power Comparison of Methods for Analyzing Free- response Observer Performance Studies", Academic Radiology 15 (12), 1554-1566 (2008).
18. Edwards, D. C., Kupinski, M. A., Metz, C. E. and Nishikawa, R. M., "Maximum likelihood fitting of FROC curves under an initial-detection-and- candidate-analysis model", Med Phys 29 (12), 2861-2870 (2002). (Pubitemid 36008482)
19. Bornefalk, H., "Estimation and Comparison of CAD System Performance in Clinical Settings", Acad Radiol 12, 687-694 (2005). (Pubitemid 40791085)
20. Bornefalk, H. and Hermansson, A. B., "On the comparison of FROC curves in mammography CAD systems", Med. Phys. 32 (2), 412-417 (2005). (Pubitemid 40322322)
21. Popescu, L. M. and Lewitt, R. M., "Small nodule detectability evaluation using a generalized scan statistic model", Phys. Med. Biol. 51 (23), 6225-6244 (2006). (Pubitemid 46217661)
22. Song, T., Bandos, A. I., Rockette, H. E. and Gur, D., "On comparing methods for discriminating between actually negative and actually positive subjects with FROC type data", Medical Physics 35 (4), 1547-1558 (2008).
23. Bandos, A. I., Rockette, H. E., Song, T. and Gur, D., "Area under the Free-Response ROC Curve (FROC) and a Related Summary Index", Biometrics EPUB, xx (2008).
24. Chakraborty, D. P., "ROC Curves predicted by a model of visual search", Phys. Med. Biol. 51, 3463- 3482 (2006). (Pubitemid 44026412)
25. Chakraborty, D. P., "A search model and figure of merit for observer data acquired according to the freeresponse paradigm", Phys. Med. Biol. 51, 3449-3462 (2006). (Pubitemid 44026411)
26. Roe, C. A. and Metz, C. E., "Dorfman-Berbaum-Metz Method for Statistical Analysis of Multireader, Multimodality Receiver Operating Characteristic Data: Validation with Computer Simulation", Acad. Radiol. 4, 298-303 (1997). (Pubitemid 127469854)
27. Dorfman, D. D., Berbaum, K. S., Lenth, R. V., Chen, Y.-F. and Donaghy, B. A., "Monte Carlo validation of a multireader method for receiver operating characteristic discrete rating data: Factorial experimental design", Acad. Radiol. 5, 591-602 (1998). (Pubitemid 28405446)
28. Hillis, S. L. and Berbaum, K. S., "Monte Carlo validation of the dorfman-berbaum-metz method using normalized pseudovalues and less data-based model simplification", Academic Radiology 12 (12), 1534- 1541 (2005). (Pubitemid 41729034)
29. Roe, C. A. and Metz, C. E., "Variance-Component Modeling in the Analysis of Receiver Operating Characteristic Index Estimates", Acad. Radiol. 4 (8), 587-600 (1997). (Pubitemid 127470980)
30. Obuchowski, N. A. and Rockette, H. E., "Hypothesis Testing of the Diagnostic Accuracy for Multiple Diagnostic Tests: An ANOVA Approach with Dependent Observations", Communications in Statistics: Simulation and Computation 24, 285-308. (1995).
31. Yoon, H. J., Zheng, B., Sahiner, B. and Chakraborty, D. P., "Evaluating computer-aided detection algorithms", Medical Physics 34 (6), 2024-2038 (2007). (Pubitemid 47055738)
32. Chakraborty, D. P., "Independent Versus Sequential Reading in ROC Studies of Computer-Assist Modalities: Analysis of Components of Variance", Acad Radiol. 10 (2), 212 (2003).
33. Dorfman, D. D., Berbaum, K. S. and Metz, C. E., "ROC characteristic rating analysis: Generalization to the Population of Readers and Patients with the Jackknife method", Invest. Radiol. 27 (9), 723-731 (1992).
34. Hillis, S. L., Berbaum, K. S. and Metz, C. E., "Recent developments in the Dorfman-Berbaum-Metz procedure for multireader ROC study analysis", Acad Radiol 15 (5), 647-661 (2008).
35. Chakraborty, D. P. and Yoon, H. J., "Operating characteristics predicted by models for diagnostic tasks involving lesion localization", Med. Phys. 35 (2), 435-445 (2008). (Pubitemid 351190863)
36. Berbaum, K. S., Franken, E. A., Dorfman, D. D., Rooholamini, S. A., Kathol, M. H., Barloon, T. J., Behlke, F. M., Sato, Y., Lu, C. H., El-Khoury, G. Y., Flickinger, F. W. and Montgomery, W. J., "Satisfaction of Search in Diagnostic Radiology", Invest. Radiol. 25 (2), 133-140 (1990). (Pubitemid 20072671)