Analysing data from observer studies in medical imaging research: An introductory guide to free-response techniques

Radiography

Volumn 20, Issue 4, 2014, Pages 295-299

  Thompson, J D ^a,b   Manning, D J ^a,c   Hogg, P ^a  

^a UNIVERSITY OF SALFORD   (United Kingdom)

^b UNIVERSITY HOSPITALS OF MORECAMBE BAY NHS FOUNDATION TRUST   (United Kingdom)

^c LANCASTER UNIVERSITY   (United Kingdom)

Author keywords

Free response receiver operating characteristic analysis; JAFROC; Observer performance

Indexed keywords

AREA UNDER THE CURVE; ARTICLE; DATA ANALYSIS; DIAGNOSTIC ACCURACY; DIAGNOSTIC IMAGING; DIAGNOSTIC VALUE; EYE TRACKING; FALSE NEGATIVE RESULT; FALSE POSITIVE RESULT; FREE RESPONSE RECEIVER OPERATING CHARACTERISTIC ANALYSIS; HUMAN; RATING SCALE; RECEIVER OPERATING CHARACTERISTIC; STANDARDIZATION; STATISTICAL ANALYSIS; STATISTICAL SIGNIFICANCE; VISION; VISUAL ACUITY;

EID: 84908554217     PISSN: 10788174     EISSN: 15322831     Source Type: Journal    
DOI: 10.1016/j.radi.2014.04.005     Document Type: Article

References (29)

1. Burgess A. Signal detection theory - a brief history. The handbook of medical image perception and techniques 2010, 26-46. Cambridge University Press, New York. E. Samei, E. Krupinski (Eds.).
2. Burgess A.E. The Rose model, revisited. JOpt Soc Am A Opt Image Sci Vis 1999, 16:633-664.
3. Manning D.J. Evaluation of diagnostic performance in radiography. Radiography 1998, 4:49-60.
4. Tingberg A., Båth M., Håkansson M., Medin J., Besjakov J., Sandborg M., et al. Evaluation of image quality of lumbar spine images: a comparison between FFE and VGA. Radiat Prot Dosimetry 2005, 114:53-61.
5. Krupinski E. Perceptual factors in reading medical images. The handbook of medical image perception and techniques 2010, 81-90. Cambridge University Press, New York. E. Samei, E. Krupinski (Eds.).
6. Manning D. Cognitive factors in reading medical images. The handbook of medical image perception and techniques 2010, 91-106. Cambridge University Press, New York. E. Samei, E. Krupinski (Eds.).
7. Chakraborty D. Statistical power in observer-performance studies: comparison of the receiver operating characteristic and free-response methods in tasks involving localization. Acad Radiol 2002, 9:147-156.
8. Zou K.H., Liu A., Bandos A.I., Ohno-Machado L., Rockette H.E. Statistical evaluation of diagnostic performance: topics in ROC analysis 2012, Chapman and Hall, Boca Raton.
9. Robinson P.J.A. Radiology's Achilles' heel: error and variation in the interpretation of the Röntgen image. Br J Radiol 1997, 70:1085-1098.
10. Jiang Y., Metz C.E. BI-RADS data should not be used to estimate ROC curves. Radiology 2010, 256:29-31.
11. Metz C.E. Receiver operating characteristic analysis: a tool for the quantitative evaluation of observer performance and imaging systems. JAm Coll Radiol 2006, 3:413-422.
12. Hanley J.A., McNeil B.J. The meaning and use of area under a receiver operating characteristic curve. Radiology 1982, 143:29-36.
13. Zou K.H., O'Malley A.J., Mauri L. Receiver-operating characteristic analysis for evaluating diagnostic tests and predictive models. Circulation 2007, 115:654-657.
14. Chakraborty D.P. Recent developments in FROC methodology. The handbook of medical image perception and techniques 2010, 216-239. Cambridge University Press, New York. E. Samei, E. Krupinski (Eds.).
15. Chakraborty D., Yoon H.J., Mello-Thoms C. Spatial localization accuracy of radiologists in free-response studies: inferring perceptual FROC curves from mark-rating data. Acad Radiol 2007, 14:4-18.
16. Chakraborty D.P. Validation and statistical power comparison of methods for analyzing free-response observer performance studies. Acad Radiol 2008, 15:1554-1566.
17. Bandos A.I., Rockette H.E., Song T., Gur D. Area under the free-response ROC curve (FROC) and a related summary index. Biometrics 2009, 65:247-256.
18. Haygood T.M., Ryan J., Brennan P.C., Li S., Marom E.M., McEntee M.F., et al. On the choice of acceptance radius in free-response observer performance studies. Br J Radiol 2013, 86:42313554.
19. Chakraborty D.P. New developments in observer performance methodology in medical imaging. Semin Nucl Med 2011, 41:401-418.
20. Chakraborty D.P. Astatus report on free-response analysis. Radiat Prot Dosimetry 2010, 139:20-25.
21. Hillis S.L., Berbaum K.S. Monte Carlo validation of the Dorfman-Berbaum-Metz method using normalized pseudovalues and less data-based model simplification. Acad Radiol 2005, 12:1534-1541.
22. Chakraborty D.P., Berbaum K.S. Observer studies involving detection and localization: modeling, analysis, and validation. Med Phys 2004, 31:2313-2330.
23. Chakraborty D.P. Clinical relevance of the ROC and free-response paradigms for comparing imaging system efficacies. Radiat Prot Dosimetry 2010, 139:37-41.
24. Chakraborty D.P. Prediction accuracy of a sample-size estimation method for ROC studies. Acad Radiol 2010, 17:628-638.
25. JAFROC analysis software [computer program]. Version 4.2 2014, Pittsburgh. http://www.devchakraborty.com/downloads.
26. Obuchowski N.A. Sample size tables for receiver operating characteristic studies. AJR Am J Roentgenol 2000, 175:603-608.
27. Obuchowski N.A., Hillis S.L. Sample size tables for computer-aided detection studies. AJR Am J Roentgenol 2011, 197:W821-W828.
28. Håkansson M., Svensson S., Zachrisson S., Svalkvist A., Båth M., Månsson L.G. VIEWDEX: an efficient and easy-to-use software for observer performance studies. Radiat Prot Dosimetry 2010, 139:42-51.
29. Thompson J., Hogg P., Thompson S., Manning D., Szczepura K. ROCView: prototype software for data collection in jackknife alternative free-response receiver operating characteristic analysis. Br J Radiol 2012, 85:1320-1326.