Probability densities and confidence intervals for target recognition performance metrics

Proceedings of SPIE - The International Society for Optical Engineering

Volumn 5808, Issue , 2005, Pages 373-382

  Parker, David R ^a   Gustafson, Steven C ^a   Ross, Timothy D ^b  

^a AIR FORCE INSTITUTE OF TECHNOLOGY   (United States)

^b AIR FORCE RESEARCH LABORATORY   (United States)

Author keywords

Automatic target recognition; Performance metrics; Receiver operating characteristic

Indexed keywords

ERROR ANALYSIS; PROBABILITY DENSITY FUNCTION; PROCESS CONTROL;

PERFORMANCE METRICS; PROBABILITY DENSITY; SYSTEMS UNDER TESTS (SUT);

TARGETS;

EID: 27544491745     PISSN: 0277786X     EISSN: None     Source Type: Conference Proceeding    
DOI: 10.1117/12.602322     Document Type: Conference Paper

References (28)

1. A. R. Wise, D. Fitzgerald, and T. D. Ross, "The Adaptive SAR ATR problem set (AdaptSAPS)," in Algorithms for Synthetic Aperture Radar Imagery XI, E. G. Zelnio, ed., 5427, pp. 366-375, SPIE, 2004.
2. T. D. Ross and M. E. Minardi, "Discrimination and Confidence Error in Detector Reported Scores," in Algorithms for Synthetic Aperture Radar Imagery XI, 5427, pp. 342-353, SPIE, 1 2004.
3. S. G. Alsing, The Evaluation of Competing Classifiers. PhD thesis, Air Force Institute of Technology, 2000.
4. J. Hanley, "Receiver operating characteristic (ROC) curves," in Encyclopedia of Biostatistics, P. Armitage and T. Colton, eds., pp. 3738-3745, Wiley, 1999.
5. H. L. VanTrees, Detection, Estimation, and Modulation Theory, Part I Detection, Estimation, and Linear Modulation Theory, Wiley, 1968.
6. L. L. Scharf, Statistical Signal Processing - Detection, Estimation, and Time Series Analysis. Addison Wesley, July 1991.
7. M. Barkat, Signal Detection and Estimation, Artech House, 1991.
8. K. Shanmugan and A. Breipohl, Random Signals, Detection, Estimation and Data Analysis, Wiley, 1988.
9. T. D. Wilkens, Elementary Signal Detection Theory, Oxford University Press, 2002.
10. D. M. Green and J. A. Swetz, Signal Detection Theory and Psychophysics, Peninsula Publishing, 1988.
11. D. R. Parker, S. C. Gustafson, and T. D. Ross, "Integrated receiver operating characteristic and confidence error metrics for assessing the performance of automatic target recognition systems," in Algorithms for Synthetic Aperture Radar Imagery XI, E. G. Zelnio, ed., pp. 354-365, SPIE, 1988.
12. C. E. Metz, "Statistical analysis of ROC data in evaluating diagnostic performance," in Multiple Regression Analysis: Applications in the Health Sciences, D. Herbert and R. Myers, eds., pp. 365-384, 1984.
13. A. Sorribas, J. March, and J. Trujillano, "A new parametric method based on S-distributions for computing receiver operating characteristic curves for continuous diagnostic tests," Statistics in Medicine 21, pp. 1213-1235, May 2002.
14. G. Claeskens, B.-Y. Jing, L. Peng, and W. Zhou, "Empirical likelihood confidence regions for comparison distributions and ROC curves," Canadian Journal of Statistics 31, pp. 173-190, 2003.
15. J. A. Swetz and R. M. Pickett, Evaluation of Diagnostic Systems: Methods from Signal Detection Theory, Academic Press, 1982.
16. C. Metz and X. Pan, ""Proper" binormal ROC curves: Theory and maximum-likelihood estimation," Journal of Mathematical Psychology 43, pp. 1-33, March 1999.
17. D. D. Dorfman, K. S. Berbaum, C. E. Metz, R. V. Lenth, J. A. Hanley, and H. A. Dagga, "Proper receiver operating characteristic analysis: The bigamma model," Academic Radiology 4, pp. 138-149, February 1997.
18. K. Jensen, H. H. Muller, and H. Schafer, "Regional confidence bands for ROC curves," Statistics in Medicine 19, pp. 493-509, 2000.
19. G. Ma and W. Hall, "Confidence Bands for Receiver Operating Characteristic Curves," Medical Decision Making 13, pp. 191-197, 1993.
20. R. Hilgers, "Distribution-Free Confidence-Bounds for ROC Curves," Methods of Information in Medicine 30, pp. 96-101, April 1991.
21. H. Schafer, "Efficient Confidence Bounds for ROC Curves," Statistics in Medicine 13(15), pp. 1551-1561, 1994.
22. W. Mendenhall, D. D. Wackerly, and R. L. Scheaffer, Mathematical Statistics with Applications, PWS-Kent, 1990.
23. A. E. Raftery, F. Balabdaoui, T. Gneiting, and M. Polakowski, "Using Bayesian Model Averaging to Calibrate Forecast Ensembles," Tech. Rep. 440, University of Washington, December 2003.
24. J. A. Hoeting, D. Madigan, A. E. Raftery, and C. T. Volin, "Bayesian model averaging: a tutorial," Statistical Science 14(4), pp. 382-417, 1999. Corrected version available at http://www.stat.washington.edu/www/ research/online/hoeting1999.pdf.
25. D. Madigan and A. E. Raftery, "Model Selection and Accounting for Model Uncertainty in Graphical Models Using Occam's Window," Journal of the American Statistical Association 89(428), pp. 1535-1546, 1994.
26. M. M. Barbieri and J. O. Berger, "Optimal predictive model selection," The Annals of Statistics 32, pp. 870-897, 2004.
27. T. D. Ross, "Accurate Confidence Intervals for Binomial Proportion and Poisson Rate Estimation," Computers in Biology and Medicine 33, pp. 509-531, 2003.
28. N. A. Obuchowski and M. L. Lieber, "Confidence intervals for the receiver operating characteristic area in studies with small samples," Academic Radiology 5, pp. 561-571, 1998.