Effect of finite sample size on feature selection and classification: A simulation study

Medical Physics

Volumn 37, Issue 2, 2010, Pages 907-920

  Way, Ted W ^a   Sahiner, Berkman ^a   Hadjiiski, Lubomir M ^a   Chan, Heang Ping ^a  

^a UNIVERSITY OF MICHIGAN MEDICAL SCHOOL   (United States)

Author keywords

Feature selection; Linear discriminant analysis; Sample size effect; Support vector machines

Indexed keywords

COMPUTER AIDED DIAGNOSIS; COVARIANCE MATRIX; DISCRIMINANT ANALYSIS; SAMPLING; SUPPORT VECTOR MACHINES;

COMPUTER AIDED DIAGNOSIS(CAD); FEATURE SELECTION AND CLASSIFICATION; FEATURE SELECTION METHODS; FISHER'S LINEAR DISCRIMINANT ANALYSIS; LINEAR DISCRIMINANT ANALYSIS; MULTIVARIATE GAUSSIAN DISTRIBUTIONS; RECEIVER OPERATING CHARACTERISTIC CURVES; SAMPLE SIZES;

FEATURE EXTRACTION;

SANDFLY FEVER SICILIAN VIRUS;

EID: 75749099885     PISSN: 00942405     EISSN: None     Source Type: Journal    
DOI: 10.1118/1.3284974     Document Type: Article

References (39)

1. H. P. Chan, B. Sahiner, R. F. Wagner, and N. Petrick, " Classifier design for computer-aided diagnosis: Effects of finite sample size on the mean performance of classical and neural network classifiers.," Med. Phys. MPHYA6 0094-2405 26, 2654-2668 (1999). 10.1118/1.598805 (Pubitemid 30007711)
2. B. Sahiner, H. P. Chan, N. Petrick, R. F. Wagner, and L. M. Hadjiiski, " Feature selection and classifier performance in computer-aided diagnosis: The effect of finite sample size.," Med. Phys. MPHYA6 0094-2405 27, 1509-1522 (2000). 10.1118/1.599017 (Pubitemid 30624246)
3. B. Sahiner, H. P. Chan, and L. Hadjiiski, " Classifier performance prediction for computer-aided diagnosis using a limited data set.," Med. Phys. MPHYA6 0094-2405 35, 1559-1570 (2008). 10.1118/1.2868757
4. B. Sahiner, H. P. Chan, and L. M. Hadjiiski, " Classifier performance estimation under the constraint of a finite sample size: Resampling schemes applied to neural network classifiers.," Neural Networks NNETEB 0893-6080 21, 476-483 (2008). 10.1016/j.neunet.2007.12.012
5. Q. Li and K. Doi, " Analysis and minimization of overtraining effect in rule-based classifiers for computer-aided diagnosis.," Med. Phys. MPHYA6 0094-2405 33, 320-328 (2006). 10.1118/1.1999126 (Pubitemid 43183973)
6. Q. Li and K. Doi, " Comparison of typical evaluation methods for computer-aided diagnostic schemes: Monte Carlo simulation study.," Med. Phys. MPHYA6 0094-2405 34, 871-876 (2007). 10.1118/1.2437130 (Pubitemid 46355650)
7. S. V. Beiden, M. A. Maloof, and R. F. Wagner, " A general model for finite-sample effects in training and testing of competing classifiers.," IEEE Trans. Pattern Anal. Mach. Intell. ITPIDJ 0162-8828 25, 1561-1569 (2003). 10.1109/TPAMI.2003.1251149
8. A. Jain and D. Zongker, " Feature selection: Evaluation, application, and small sample size performance.," IEEE Trans. Pattern Anal. Mach. Intell. ITPIDJ 0162-8828 19, 153-158 (1997). 10.1109/34.574797
9. P. Pudil, J. Novovicová, and J. Kittler, " Floating search methods in feature selection.," Pattern Recogn. Lett. PRLEDG 0167-8655 15, 1119-1125 (1994). 10.1016/0167-8655(94)90127-9
10. M. Kudo and J. Sklansky, " Comparison of algorithms that select features for pattern classifiers.," Pattern Recogn. PTNRA8 0031-3203 33, 25-41 (2000). 10.1016/S0031-3203(99)00041-2 (Pubitemid 32081876)
11. C. Sima and E. R. Dougherty, " What should be expected from feature selection in small-sample settings.," Bioinformatics BOINFP 1367-4803 22, 2430-2436 (2006). 10.1093/bioinformatics/btl407 (Pubitemid 44574149)
12. J. P. Hua, W. D. Tembe, and E. R. Dougherty, " Performance of feature-selection methods in the classification of high-dimension data.," Pattern Recogn. PTNRA8 0031-3203 42, 409-424 (2009). 10.1016/j.patcog.2008.08. 001
13. J. W. Lee, J. B. Lee, M. Park, and S. H. Song, " An extensive comparison of recent classification tools applied to microarray data.," Comput. Stat. Data Anal. CSDADW 0167-9473 48, 869-885 (2005). 10.1016/j.csda.2004.03.017
14. X. G. Zhang, X. Lu, Q. Shi, X. Q. Xu, H. C. E. Leung, L. N. Harris, J. D. Iglehart, A. Miron, J. S. Liu, and W. H. Wong, " Recursive SVM feature selection and sample classification for mass-spectrometry and microarray data.," BMC Bioinf. BBMIC4 1471-2105 7, 13 (2006). 10.1186/1471-2105-7-13 (Pubitemid 43193492)
15. J. G. Dy and C. E. Brodley, " Feature selection for unsupervised learning.," J. Mach. Learn. Res. ZZZZZZ 1532-4435 5, 845-889 (2004).
16. S. Krishnan, K. Samudravijaya, and P. V. S. Rao, " Feature selection for pattern classification with Gaussian mixture models: A new objective criterion.," Pattern Recogn. Lett. PRLEDG 0167-8655 17, 803-809 (1996). 10.1016/0167-8655(96)00047-5 (Pubitemid 126384184)
17. R. Kohavi and G. H. John, " Wrappers for feature subset selection.," Artif. Intell. AINTBB 0004-3702 97, 273-324 (1997). 10.1016/S0004-3702(97)00043-X
18. L. Yu and H. Liu, " Efficient feature selection via analysis of relevance and redundancy.," J. Mach. Learn. Res. ZZZZZZ 1532-4435 5, 1205-1224 (2004).
19. K. Fukunaga, Introduction to Statistical Pattern Recognition, 2nd ed. (Academic, New York, 1990).
20. R. O. Duda and P. E. Hart, Pattern Classification and Scene Analysis (Wiley, New York, 1973).
21. D. J. Hand, Discrimination and Classification (Wiley, New York, 1981).
22. H. H. Barrett, C. K. Abbey, and E. Clarkson, " Objective assessment of image quality. III. ROC metrics, ideal observers, and likelihood-generating functions.," J. Opt. Soc. Am. A Opt. Image Sci. Vis. 15, 1520-1535 (1998). 10.1364/JOSAA.15.001520 1084-7529
23. T. W. Way, L. M. Hadjiiski, B. Sahiner, H. -P. Chan, P. N. Cascade, E. A. Kazerooni, N. Bogot, and C. Zhou, " Computer-aided diagnosis of pulmonary nodules on CT scans: Segmentation and classification using 3D active contours.," Med. Phys. MPHYA6 0094-2405 33, 2323-2337 (2006). 10.1118/1.2207129 (Pubitemid 43967219)
24. M. M. Galloway, " Texture classification using gray level run lengths.," Comput. Graph. Image Process. CGIPBG 0146-664X 4, 172-179 (1975). 10.1016/S0146-664X(75)80008-6
25. B. R. Dasarathy and E. B. Holder, " Image characterizations based on joint gray-level run-length distributions.," Pattern Recogn. Lett. PRLEDG 0167-8655 12, 497-502 (1991). 10.1016/0167-8655(91)80014-2
26. T. Marill and D. Green, " On the effectiveness of receptors in recognition systems.," IEEE Trans. Inf. Theory IETTAW 0018-9448 9, 11-17 (1963). 10.1109/TIT.1963.1057810
27. A. W. Whitney, " A direct method of nonparametric measurement selection.," IEEE Trans. Comput. ITCOB4 0018-9340 C-20, 1100-1103 (1971). 10.1109/T-C.1971.223410
28. N. R. Draper, Applied Regression Analysis (Wiley, New York, 1998).
29. M. M. Tatsuoka, Multivariate Analysis, Techniques for Educational and Psychological Research, 2nd ed. (Macmillan, New York, 1988).
30. M. J. Norusis, SPSS for Windows Release 6 Professional Statistics (SPSS, Chicago, 1993).
31. S. D. Stearns, " On selecting features for pattern classifiers.," Third International Conference on Pattern Recognition, Coronado, CA, 1976, pp. 71-75.
32. A. K. Jain, R. P. W. Duin, and J. Mao, " Statistical pattern recognition: A review.," IEEE Trans. Pattern Anal. Mach. Intell. ITPIDJ 0162-8828 22, 4-37 (2000). 10.1109/34.824819 (Pubitemid 32132953)
33. P. A. Lachenbruch, Discriminant Analysis (Hafner, New York, 1975).
34. C. J. C. Burges, " A tutorial on support vector machines for pattern recognition.," Data Min. Knowl. Discov. DMKDFD 1384-5810 2, 121-167 (1998). 10.1023/A:1009715923555 (Pubitemid 128126769)
35. Y. Arzhaeva, M. Prokop, D. M. J. Tax, P. A. De Jong, C. M. Schaefer-Prokop, and B. van Ginneken, " Computer-aided detection of interstitial abnormalities in chest radiographs using a reference standard based on computed tomography.," Med. Phys. MPHYA6 0094-2405 34, 4798-4809 (2007). 10.1118/1.2795672 (Pubitemid 350197719)
36. P. Campadelli, E. Casiraghi, and D. Artioli, " A fully automated method for lung nodule detection from postero-anterior chest radiographs.," IEEE Trans. Med. Imaging ITMID4 0278-0062 25, 1588-1603 (2006). 10.1109/TMI.2006.884198 (Pubitemid 44936708)
37. A. K. Jerebko, J. D. Malley, M. Franaszek, and R. M. Summers, " Support vector machines committee classification method for computer-aided polyp detection in CT colonography.," Acad. Radiol. ZZZZZZ 1076-6332 12, 479-486 (2005). 10.1016/j.acra.2004.04.024
38. S. Ruping, " Incremental learning with support vector machines.," Proceedings of the IEEE International Conference on Data Mining, 2001, pp. 641-642.
39. O. Chapelle, P. Haffner, and V. N. Vapnik, " Support vector machines for histogram-based image classification.," IEEE Trans. Neural Netw. ITNNEP 1045-9227 10, 1055-1064 (1999). 10.1109/72.788646