Classifier performance prediction for computer-aided diagnosis using a limited dataset

Medical Physics

Volumn 35, Issue 4, 2008, Pages 1559-1570

  Sahiner, Berkman ^a   Chan, Heang Ping ^a   Hadjiiski, Lubomir ^a  

^a UNIVERSITY OF MICHIGAN MEDICAL SCHOOL   (United States)

Author keywords

Bootstrap; Classifier performance; Finite sample size; Resampling

Indexed keywords

COMPUTER AIDED DIAGNOSIS; COVARIANCE MATRIX; DISCRIMINANT ANALYSIS; FORECASTING; MEAN SQUARE ERROR; NORMAL DISTRIBUTION; SAMPLING; STATISTICAL TESTS;

BOOTSTRAP; CLASSIFIER PERFORMANCE; FINITE SAMPLE SIZES; FISHER'S LINEAR DISCRIMINANT ANALYSIS; MULTI-VARIATE NORMAL DISTRIBUTIONS; RECEIVER OPERATING CHARACTERISTIC CURVES; RESAMPLING; ROOT MEAN SQUARED ERRORS;

MONTE CARLO METHODS;

ACCURACY; AREA UNDER THE CURVE; ARTICLE; BOOTSTRAPPING; CALCULATION; CLASSIFIER; COMPUTER ASSISTED DIAGNOSIS; DATA ANALYSIS; DISCRIMINANT ANALYSIS; MATHEMATICAL ANALYSIS; MONTE CARLO METHOD; PERFORMANCE; PREDICTION; PRIORITY JOURNAL; SAMPLE SIZE;

EID: 41449091519     PISSN: 00942405     EISSN: None     Source Type: Journal    
DOI: 10.1118/1.2868757     Document Type: Article

References (22)

1. H. P. Chan, B. Sahiner, R. F. Wagner, and N. Petrick, " Classifier design for computer-aided diagnosis: Effects of finite sample size on the mean performance of classical and neural network classifiers., " Med. Phys. 26, 2654-2668 (1999).
2. G. T. Toussaint, " Bibliography on estimation of misclassification., " IEEE Trans. Inf. Theory IT20, 472-479 (1974).
3. B. Efron, " Estimating the error rate of a prediction rule: Improvement on cross-validation., " J. Am. Stat. Assoc. 78, 316-331 (1983).
4. B. Efron and R. Tibshirani, " Improvements on cross-validation: The 0.632+ bootstrap method., " J. Am. Stat. Assoc. 92, 548-560 (1997).
5. D. J. Hand, " Recent advances in error rate estimation., " Pattern Recogn. Lett. 4, 335-346 (1986).
6. R. A. Schiavo and D. J. Hand, " Ten more years of error rate research., " Int. Statist. Rev. 68, 295-310 (2000).
7. G. D. Tourassi and C. E. Floyd, " The effect of data sampling on the performance evaluation of artificial neural networks in medical diagnosis., " Med. Decis Making 17, 186-192 (1997).
8. E. Arana, P. Delicado, and L. Marti-Bonmati, " Validation procedures in radiologic diagnostic models: Neural network and logistic regression., " Invest. Radiol. 34, 636-642 (1999).
9. E. W. Steyerberg, F. E. Harrell, G. Borsboom, M. J. C. Eijkemans, Y. Vergouwe, and J. D. F. Habbema, " Internal validation of predictive models: Efficiency of some procedures for logistic regression analysis., " J. Clin. Epidemiol. 54, 774-781 (2001).
10. W. A. Yousef, R. F. Wagner, and M. H. Loew, " Comparison of nonparametric methods for assessing classifier performance in terms of ROC parameters., " in Proceedings of the 33rd Applied Imagery Pattern Recognition Workshop (IEEE, 2004), pp. 190-195.
11. B. Sahiner, H. P. Chan, N. Petrick, L. M. Hadjiiski, S. Paquerault, and M. N. Gurcan, " Resampling schemes for estimating the accuracy of a classifier designed with a limited data set., " Presented at the Medical Image Perception Conference IX, Airlie Conference Center, Warrenton, VA,, September 20-23, 2001.
12. D. D. Boos, " Introduction to the bootstrap world., " Stat. Sci. 18, 168-174 (2003).
13. K. Fukunaga and R. R. Hayes, " Effects of sample size on classifier design., " IEEE Trans. Pattern Anal. Mach. Intell. 11, 873-885 (1989).
14. P. A. Lachenbruch, " An almost unbiased method of obtaining confidence intervals for the probability of misclassification in discriminant analysis., " Biometrics 23, 639-645 (1967).
15. P. A. Lachenbruch, Discriminant Analysis (Hafner Press, New York, 1975).
16. C. E. Metz, B. A. Herman, and J. H. Shen, " Maximum-likelihood estimation of receiver operating characteristic (ROC) curves from continuously-distributed data., " Stat. Med. 17, 1033-1053 (1998).
17. K. Fukunaga, Introduction to Statistical Pattern Recognition, 2nd ed. (Academic Press, New York, 1990).
18. A. P. Bradley, " The use of the area under the ROC curve in the evaluation of machine learning algorithms., " Pattern Recogn. 30, 1145-1159 (1997).
19. G. W. Brier, " Verification of forecasts expressed in terms of probability., " Mon. Weather Rev. 75, 1-3 (1950).
20. S. Wu and P. Flach, " A scored AUC metric for classifier evaluation and selection., " Proceedings of the ICML 2005 Workshop on ROC Analysis in Machine Learning (International Machine Learning Society, Bonn, Germany, 2005).
21. K. O. Hajian-Tilaki, J. A. Hanley, L. Joseph, and J. P. Collet, " A comparison of parametric and nonparametric approaches to ROC analysis of quantitative diagnostic tests., " Med. Decis Making 17, 94-102 (1997).
22. W. A. Yousef, R. F. Wagner, and M. H. Loew, " Estimating the uncertainty in the estimated mean area under the ROC curve of a classifier., " Pattern Recogn. Lett. 26, 2600-2610 (2005).