A kernel-based two-class classifier for imbalanced data sets

IEEE Transactions on Neural Networks

Volumn 18, Issue 1, 2007, Pages 28-41

  Hong, Xia ^a   Chen, Sheng ^b   Harris, Chris J ^b  

^a UNIVERSITY OF READING   (United Kingdom)

^b UNIVERSITY OF SOUTHAMPTON   (United Kingdom)

Author keywords

Forward selection; Imbalanced data sets; Kernel classifier; Leave one out (LOO) cross validation; Receiver operating characteristics (ROCs)

Indexed keywords

LEAST SQUARES APPROXIMATIONS; MATHEMATICAL MODELS; OPTIMIZATION; PARAMETER ESTIMATION;

CROSS VALIDATION; FORWARD SELECTION; IMBALANCED DATA SETS; KERNEL CLASSIFIER; RECEIVER OPERATING CHARACTERISTICS;

ALGORITHMS;

ALGORITHM; ARTICLE; ARTIFICIAL INTELLIGENCE; AUTOMATED PATTERN RECOGNITION; CLUSTER ANALYSIS; COMPUTER ANALYSIS; COMPUTER SIMULATION; FACTUAL DATABASE; INFORMATION RETRIEVAL; METHODOLOGY;

ALGORITHMS; ARTIFICIAL INTELLIGENCE; CLUSTER ANALYSIS; COMPUTER SIMULATION; COMPUTING METHODOLOGIES; DATABASES, FACTUAL; INFORMATION STORAGE AND RETRIEVAL; PATTERN RECOGNITION, AUTOMATED;

EID: 33846046907     PISSN: 10459227     EISSN: None     Source Type: Journal    
DOI: 10.1109/TNN.2006.882812     Document Type: Article

References (44)

1. M. Stone, "Cross validatory choice and assessment of statistical predictions," J. Roy. Stat. Soc., Ser. B, vol. 36, pp. 117-147, 1974.
2. R. O. Duda, P. E. Hart, and D. G. Stork, Pattern Classification. New York: Wiley, 2001.
3. L. Breiman, "Bagging predictors," Mach. Learn., vol. 24, no. 2, pp. 123-140, 1996.
4. L. Breiman, "Arcing classifiers," Ann. Stat., vol. 26, no. 3, pp. 801-849, 1996.
5. L. Breiman, "Random forests," Mach. Learn., vol. 45, no. 1, pp. 5-12, 2001.
6. R. A. Jacobs, M. I. Jordan, S. J. Nowlan, and G. E. Hint, "Adaptive mixtures of local experts," Neural Comput., vol. 3, pp. 79-87, 1991.
7. C. Ji and S. Ma, "Combinations of weak classifiers," IEEE Trans. Neural Netw., vol. 8, no. 1, pp. 32-42, Jan. 1997.
8. E. M. Kleinberg, "Stochastic discrimination," Ann. Math. Artif. Intell., vol. 1, no. 1-4, pp. 207-239, 1990.
9. F. Provost and T. Fawcett, "Robust classification for imprecise environments," Mach. Learn., vol. 42, pp. 203-231, 2001.
10. R. Xu and D. Wunsch, II, "Survey of clustering algorithms," IEEE Trans. Neural Netw., vol. 16, no. 3, pp. 645-678, May 2005.
11. V. Vapnik, The Nature of Statistical Learning Theory. New York: Springer-Verlag, 1995.
12. M. E. Tipping, "Sparse Bayesian learning and the relevance vector machine," J. Mach. Learn. Res., vol. 1, pp. 211-244, 2001.
13. B. Scholkopf and A. J. Smola, Learning With Kernels: Support Vector Machine, Regularization, Optimization and Beyond. Cambridge, MA: MIT Press, 2002.
14. X. Hong and C. J. Harris, "Nonlinear model structure design and construction using orthogonal least squares and D-optimality design," IEEE Trans. Neural Netw., vol. 13, no. 5, pp. 1245-1250, Sep. 2001.
15. S. Chen, S. A. Billings, and W. Luo, "Orthogonal least squares methods and their applications to non-linear system identification," Int. J. Contr., vol. 50, pp. 1873-1896, 1989.
16. S. Chen, Y. Wu, and B. L. Luk, "Combined genetic algorithm optimization and regularized orthogonal least squares learning for radial basis function networks," IEEE Trans. Neural Netw., vol. 10, no. 5, pp. 1239-1243, Sep. 1999.
17. S. Chen, X. Hong, and C. J. Harris, "Sparse kernel regression modelling using combined locally regularized orthogonal least squares and D-optimality experimental design," IEEE Trans. Autom. Control, vol. 48, no. 6, pp. 1029-1036, Jun. 2003.
18. X. Hong, P. M. Sharkey, and K. Warwick, "A robust nonlinear identification algorithm using press statistic and forward regression," IEEE Trans. Neural Netw., vol. 14, no. 2, pp. 454-458, Mar. 2003.
19. S. Chen, X. Hong, C. J. Harris, and P. M. Sharkey, "Sparse modeling using orthogonal forward regression with press statistic and regularization," IEEE Trans. Syst., Man, Cybern., B: Cybern., vol. 34, no. 2, pp. 898-911, Apr. 2004.
20. K. Z. Mao, "RBF neural network center selection based on fisher ratio class separability measure," IEEE Trans. Neural Netw., vol. 13, no. 5, pp. 1211-1217, Sep. 2002.
21. S. Chen, X. X. Wang, X. Hong, and C. J. Harris, "Kernel classifier construction using orthogonal forward selection and boosting with Fisher ratio class separability," IEEE Trans. Neural Netw., 2006, to be published.
22. K. Z. Mao and G. B. Huang, "Neuron selection for RBF neural network classifier based on data structure preserving criterion," IEEE Trans. Neural Netw., vol. 16, no. 6, pp. 1531-1540, Nov. 2005.
23. X. Hong, S. Chen, and C. J. Harris, "Fast kernel classifier construction algorithm using orthogonal forward selection to minimize leave-one-out misclassification rate," in Proc. 2nd Int. Conf. Intell. Comp. Pt. I, D. S. Huang, K. Li, and G. W. Irwin, Eds., Kuming, China, 2006, pp. 106-114.
24. F. Provost, T. Fawcett, and R. Kohavi, "The case against accuracy estimation for comparing induction algorithms," in Proc. 15th Int. Conf. Mach. Learn. (ICML), San Francisco, CA, 1998, pp. 443-453.
25. J. Leskovec and J. Shawe-Taylor, "Linear programming boost for uneven datasets," in Proc. Int. Conf. Mach. Learn. (ICML), Washington, DC, 2003, pp. 456-463.
26. G. Karakoulas and J. Shawe-Taylor, "Optimizing classifiers for imbalanced training sets," in Proc. Neural Inf. Process. Workshop (NIPS'98), Denver, CO, 1998, pp. 253-259.
27. P. D. Turney, "Cost-sensitive classification: Empirical evaluation of a hybrid genetic decision tree induction algorithm," J. Artif. Intell. Res., no. 2, pp. 369-409, 1995.
28. K. Veropoulos, C. Campbell, and N. Cristianini, "Controlling the sensitivity of support vector machine," in Proc. Int. Joint Conf. Artif. Intell. (IJCAI99), Stockholm, Sweden, 1999, vol. Workshop ML3, pp. 55-60.
29. G. Wu and E. Y. Chang, "Aligning boundary in kernel space for learning imbalanced dataset," in Proc. 4th IEEE Int. Conf. Data Mining (ICDM 2004), Brighton, U.K., 2004, pp. 265-272.
30. G. M. Fung and O. L. Mangasarian, "Multicategory proximal support vector machine classifier," Mach. Learn., vol. 59, no. 1-2, pp. 77-97, 2005.
31. N. Japkowicz, "Supervised versus unsupervised binary-learning by feedforward neural networks," Mach. Learn., vol. 42, no. 1-2, pp. 97-122, 2001.
32. S. Tan, "Neighbor-weighted k-nearest neighbor for unbalanced text corpus," Expert Syst. Appl., vol. 28, pp. 667-671, 2005.
33. M. Kubat, R. Holte, and S. Matwin, "Learning when negative examples abound," in Proc. Eur. Conf. Mach. Learn. (ECML 97), Prague, Czech Republic, 1997, pp. 1467-153.
34. N. V. Chawla, K. W. Bowyer, L. O. Hall, and W. P. Kegelmeyer, "Smote: Synthetic minority oversampling technique," J. Artif. Intell. Res., vol. 16, pp. 321-357, 2002.
35. C. Chen, A. Liaw, and L. Breiman, "Using random forest to learn imbalanced data" Dept. Statistics, Univ. California Berkeley, Tech. Rep. 666, 2004.
36. J. P. Egan, Signal Detection Theory and ROC Analysis. New York: Academic, 1975.
37. J. A. Swets, Signal Detection Theory and ROC Analysis in Psychology and Diagnostics: Collected Papers. Mahwah, NJ: Lawrence Erlbaum Associates, 1996.
38. R. H. Myers, Classical and Modern Regression With Applications, 2nd ed. Boston, MA: PWS-Kent, 1990.
39. D. J. Mackay, "Bayesian interpolation," Neural Comput., vol. 4, no. 3, pp. 415-447, 1992.
40. D. W. Marquardt, "Generalised inverse, ridge regression, biased linear estimation and nonlinear estimation," Technometrics, vol. 12, no. 3, pp. 591-612, 1970.
41. C. van Rijsbergen, Information Retrieval. London, U.K.: Butterworths, 1979.
42. S. Hettich, C. L. Blake, and C. J. Merz, Repository of Machine Learning Databases Univ. California Irvine, 1998 [Online]. Available: http://www.ics.uci.edu/~mlearn/MLRepository.html
43. K. K. Lee, C. J. Harris, S. R. Gunn, and P. A. S. Reed, "Classification of imbalanced data with transparent kernel," in Proc. INNS/IEEE Int. Joint Conf. Neural Netw. (IJCNN), Washington, DC, 2001, pp. 2410-2415.
44. UCL Machine Learning Group, Elena Database 2004 [Online]. Available: http://www.dice.ucl.ac.be/mlg/?page=elena