Training neural network classifiers for medical decision making: The effects of imbalanced datasets on classification performance

Neural Networks

Volumn 21, Issue 2-3, 2008, Pages 427-436

  Mazurowski, Maciej A ^a   Habas, Piotr A ^a   Zurada, Jacek M ^a   Lo, Joseph Y ^b   Baker, Jay A ^b   Tourassi, Georgia D ^b  

^a University of Louisville   (United States)

^b DUKE UNIVERSITY MEDICAL CENTER   (United States)

Author keywords

Class imbalance; Classification; Computer aided diagnosis; Feed forward neural networks

Indexed keywords

CLASSIFICATION (OF INFORMATION); COMPUTER AIDED DIAGNOSIS; CORRELATION METHODS; FEEDFORWARD NEURAL NETWORKS; MEDICAL APPLICATIONS; PARTICLE SWARM OPTIMIZATION (PSO);

CLASS IMBALANCE; COMPENSATION; OVERSAMPLING;

DECISION MAKING;

ALGORITHM; ARTICLE; ARTIFICIAL NEURAL NETWORK; BREAST CANCER; CANCER DIAGNOSIS; COMPUTER ASSISTED DIAGNOSIS; CORRELATION ANALYSIS; EXPERIMENTAL STUDY; HUMAN; HUMAN TISSUE; MEDICAL DECISION MAKING; MEDICAL INFORMATICS; POSITIVE FEEDBACK; PRIORITY JOURNAL; RECEIVER OPERATING CHARACTERISTIC; SAMPLE SIZE; SIMULATION; TRAINING; VALIDATION PROCESS;

ALGORITHMS; ARTIFICIAL INTELLIGENCE; AUTOMATIC DATA PROCESSING; BREAST NEOPLASMS; COMPUTER SIMULATION; DECISION MAKING; DIAGNOSIS, COMPUTER-ASSISTED; FEEDBACK; HUMANS; NEURAL NETWORKS (COMPUTER); ROC CURVE;

EID: 40649126091     PISSN: 08936080     EISSN: None     Source Type: Journal    
DOI: 10.1016/j.neunet.2007.12.031     Document Type: Article

References (40)

1. Aha D.W., Kibler D., and Albert M.K. Instance-based learning algorithms. Machine Learning 6 (1991) 37-66
2. Barnard E., and Botha E.C. Back-propagation uses prior information efficiently. IEEE Transactions on Neural Networks 4 (1993) 794-802
3. Beiden S.V., Maloof M.A., and Wagner R.F. A general model for finite-sample effects in training and testing of competing classifiers. IEEE Transactions on Pattern Analysis and Machine Intelligence 23 (2003) 1561-1569
4. Boroczky L., Zhao L., and Lee K.P. Feature subset selection for improving the performance of false positive reduction in lung nodule CAD. IEEE Transactions Information Technology in Biomedicine 10 (2006) 504-511
5. Bradley A.P. The use of the area under the roc curve in the evaluation of machine learning algorithms. Pattern Recognition 30 (1997) 1145-1159
6. Chan, H.-P., Sahiner, B., & Hadjiiski, L. (2004). Sample size and validation issues on the development of CAD systems. In Proceedings of the 18th international congress and exhibition on computer assisted radiology and surgery
7. Chawla N.V., Bowyer K.W., Hall L.O., and Kegelmeyer W.P. Smote: synthetic minority over-sampling technique. Journal of Artificial Intelligence Research 16 (2002) 321-357
8. Clerc M., and Kennedy J. The particle swarm-explosion, stability, and convergence in a multidimensional complex space. IEEE Transactions on Evolutionary Computation 6 (2002) 58-73
9. Duda R.O., Hart P.E., and Stork D.G. Pattern classification (2000), Wiley-Interscience
10. Elazmeh W., Japkowicz N., and Matwin S. Evaluating misclassifications in imbalanced data. Lecture Notes in Computer Science 4212 (2006) 126-137
11. Fukunaga K., and Hayes R.R. Effect of sample size in classifier design. IEEE Transactions on Pattern Analysis and Machine Intelligence 11 (1989) 873-885
12. Gur D., Wagner R.F., and Chan H.-P. On the repeated use of databases for testing incremental improvement of computer-aided detection schemes. Academic Radiology 11 (2004) 103-105
13. Habas, P. A., Zurada, J. M., Elmaghraby, A. S., & Tourassi, G. D. (2007). Particle swarm optimization of neural network CAD systems with clinically relevant objectives. In Proceedings of medical imaging 2007: Computer-aided diagnosis (pp. 65140M)
14. Hamamoto Y., Uchimura S., and Tomita S. On the behavior of artificial neural network classifiers in high-dimensional spaces. IEEE Transactions on Pattern Analysis and Machine Intelligence 18 (1996) 571-574
15. Jain A., and Zongker D. Feature selection: evaluation, application, and small sample performance. IEEE Transactions on Pattern Analysis and Machine Intelligence 19 (1997) 153-158
16. Japkowicz, N. (2000). Learning from imbalanced data sets: a comparison of various strategies. In AAAI workshop on learning from imbalanced data sets (pp. 00-05)
17. Japkowicz N., and Stephen S. The class imbalance problem: a systematic study. Intelligent Data Analysis Journal 6 (2002) 429-450
18. Jesneck J.L., Lo J.Y., and Baker J.A. Breast mass lesions: computer-aided diagnosis models with mammographic and sonographic descriptors. Radiology 244 (2007) 390-398
19. Jiang Y., Metz C.E., and Nishikawa R.M. A receiver operating characteristic partial area index for highly sensitive diagnostic tests. Radiology 201 (1996) 745-750
20. Kawamoto K., Houlihan C.A., Balas E.A., and Lobach D.F. Improving clinical practice using clinical decision support systems: a systematic review of trials to identify features critical to success. British Medical Journal 330 (2005) 765-772
21. Kennedy, J., & Eberhart, R. (1995). Particle swarm optimization. In Proceedings of IEEE international conference on neural networks (pp. 1942-1948)
22. Li Q., and Doi K. Reduction of bias and variance for evaluation of computer-aided diagnostic schemes. Medical Physics 33 (2006) 868-875
23. Li Q., and Doi K. Comparison of typical evaluation methods for computer-aided diagnostic schemes: Monte Carlo simulation study. Medical Physics 34 (2007) 871-876
24. Lisboa P.J. A review of evidence of health benefit from artificial neural networks in medical intervention. Neural Networks 15 (2002) 11-39
25. Lisboa P.J., and Taktak A.F.G. The use of artificial neural networks in decision support in cancer: a systematic review. Neural Networks 19 (2006) 408-415
26. Maloof, M. A. (2003). Learning when data sets are imbalanced and when costs are unequal and unknown. In Proceedings of workshop on learning from imbalanced data sets
27. Mazurowski, M. A., Habas, P. A., Zurada, J. M., & Tourassi, G. D. (2007). Impact of low class prevalence on the performance evaluation of neural network based classifiers: Experimental study in the context of computer-assisted medical diagnosis. In Proceedings of international joint conference on neural networks (pp. 2005-2009)
28. Metz C.E., Herman B.A., and Shen J.-H. Maximum likelihood estimation of receiver operating characteristic (ROC) curves from continuously-distributed data. Statistics in Medicine 17 (1998) 1033-1053
29. Mitchell T. Machine learning (1997), McGraw Hill
30. Obuchowski N.A. Receiver operating characteristic curves and their use in radiology. Radiology 229 (2003) 3-8
31. Raudys S. On dimensionality, sample size, and classification error of nonparametric linear classification algorithms. IEEE Transactions on Pattern Analysis and Machine Intelligence 19 (1997) 667-671
32. Raudys S.J., and Jain A.K. Small sample size effects in statistical pattern recognition: recommendations for practitioners. IEEE Transactions on Pattern Analysis and Machine Intelligence 13 (1991) 252-264
33. Rumelhart D.E., Hinton G.E., and Williams R.J. Learning internal representations by error propagation. Parallel distributed processing: Explorations in the microstructure of cognition volume 1 (1986), MIT Press 318-362
34. Sahiner B., Chan H.P., Petrick N., Wagner R.F., and Hadjiiski L. Feature selection and classifier performance in computer-aided diagnosis: the effect of finite sample size. Medical Physics 27 (2000) 1509-1522
35. Sampat M.P., Markey M.K., and Bovik A.C. Computer-aided detection and diagnosis in mammography. Handbook of image and video processing (2005), Academic Press 1195-1217
36. van den Bergh F., and Engelbrecht A.P. A cooperative approach to particle swarm optimization. IEEE Transactions on Evolutionary Computation 8 (2004) 225-239
37. Wagner R.F., Chan H.-P., Sahiner B., and Petrick N. Finite-sample effects and resampling plans: applications to linear classifiers in computer-aided diagnosis. Proceedings of medical imaging 1997: Image processing (1997) 467-477
38. Weiss, G. M., & Provost, F. (2001). The effect of class distribution on classifier learning: an empirical study. Technical report, Department of Computer Science, Rutgers University
39. Zhang G.P. Neural networks for classification: a survey. IEEE Transactions on Systems, Man, and Cybernetics-Part C: Applications and Reviews 30 (2000) 451-462
40. Zhou Z.-H., and Liu X.-Y. Training cost-sensitive neural networks with methods addressing the class imbalance problem. IEEE Transactions on Knowledge and Data Engineering 18 (2006) 63-77