SVM-FuzCoC: A novel SVM-based feature selection method using a fuzzy complementary criterion

Pattern Recognition

Volumn 43, Issue 11, 2010, Pages 3712-3729

  Moustakidis, S P ^a   Theocharis, J B ^a  

^a ARISTOTLE UNIVERSITY OF THESSALONIKI   (Greece)

Author keywords

Feature redundancy; Feature selection; Fuzzy complementary criterion; Fuzzy sets; Support vector machines

Indexed keywords

ECONOMIC AND SOCIAL EFFECTS; FUZZY SETS; ITERATIVE METHODS; REDUNDANCY; SUPPORT VECTOR MACHINES;

CLASSIFICATION ACCURACY; COMPLEMENTARY FEATURES; COMPUTATIONAL REQUIREMENTS; DIMENSIONALITY REDUCTION; FEATURE REDUNDANCY; FEATURE SELECTION METHODS; FUZZY COMPLEMENTARY CRITERION; HIGH-DIMENSIONAL FEATURE SPACE;

FEATURE EXTRACTION;

EID: 78049340183     PISSN: 00313203     EISSN: None     Source Type: Journal    
DOI: 10.1016/j.patcog.2010.05.007     Document Type: Article

References (48)

1. A.L. Blum, P. Langley, Selection of relevant features and examples in machine learning, Artif. Intell.97(1997)245-271.
2. M. Ben-Bassat, Pattern recognition and reduction of dimensionality, in: P.R. Krishnaiah, L.N. Kanal (Eds.), Handbook of Statistics-II, North Holland, 1982, pp. 773-791.
3. A. Jain, D. Zongker, Feature selection: evaluation, application, and small sample performance, IEEE Trans. Pattern Anal. Mach. Intell. 19 (2) (1997).
4. M. Dash, H. Liu, Feature selection for classification, Intell. Data Anal. 1 (3) (1997) 131-156.
5. Y. Kim, W. Street, F. Menczer, Feature selection for unsupervised learning via evolutionary search, in: Proceedings of the Sixth ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, 2000, pp. 365-369.
6. R. Kohavi, G.H. John, Wrappers for feature subset selection, Artif. Intell. 97 (1-2)(1997) 273-324.
7. J. Bi, K.P. Bennett, M. Embrechts, C.M. Breneman, M. Song, Dimensionality reduction via sparse support vector machines, J. Mach. Learn. Res. (2003) 1229-1243.
8. H. Liu, H. Motoda, in: Feature Selection for Knowledge Discovery and Data Mining, Kluwer Academic Publisher, Dordrecht, 1998.
9. I. Guyon, A. Elisseeff, An introduction to variable and feature selection, J. Mach. Learn. Res. (2003) 1157-1182.
10. R. Battiti, Using mutual information for selecting features in supervised neural net learning, IEEE Trans. Neural Netw. (1994) 537-550.
11. H. Peng, F. Long, C. Ding, Feature selection based on mutual information criteria of max-dependency, max-relevance, and min-redundancy, IEEE Trans. Pattern Anal. Mach. Intell. (2005) 1226-1238.
12. C.H. Ooi, M. Chetty, S.W. Teng, Differential prioritization in feature selection and classifier aggregation for multiclass microarray datasets, Data Min. Knowl. Discovery 114 (2007) 329-366.
13. Y. Li, Z.F. Wu, Fuzzy feature selection based on min-max learning rule and extension matrix, Pattern Recognition 41 (2008) 217-226.
14. K.Z. Mao, Orthogonal forward selection and backward elimination algorithms for feature subset selection, IEEE Trans. Syst. Man Cybern. B (2004) 629-634.
15. X. Fu, L. Wang, Data dimensionality reduction with application to simplifying RBF network structure and improving classification performance, IEEE Trans. Syst. Man Cybern. B (2003) 399-409.
16. I. Kononenko, Estimating attributes: analysis and extensions of relief, in: Machine Learning: ECML-94, Lecture Notes in Computer Science, vol. 784, Springer, Berlin, Heidelberg, 1994, pp. 171-182.
17. D Zhang, S Chen, Z.-H Zhou, Constraint score: a new filter method for feature selection with pairwise constraints, Pattern Recognition 41 (5) (2008) 1440-1451.
18. P. Mitra, C.A. Murthy, S.K. Pal, Unsupervised feature selection using feature similarity, IEEE Trans. Pattern Anal. Mach. Intell. 24 (3) (2002) 301-312.
19. S. Das, Filters, wrappers and a boosting-based hybrid for feature selection, in: Proceedings of the 18th International Conference on Machine Learning, 2001, pp. 74-81.
20. A.Y. Ng, On feature selection: learning with exponentially many irrelevant features as training examples, in: Proceedings of the 15th International Conference on Machine Learning, 1998, pp. 404-412.
21. P.M. Narendra, K. Fukunaga, A branch and bound algorithm for feature subset selection, IEEE Trans. Comput. 26 (9) (1977) 917-922.
22. A. Jain, D. Zongker, Feature selection: evaluation, application, and small sample performance, IEEE Trans. Pattern Anal. Mach. Intell. 19 (1997) 153-158.
23. P. Pudil, J. Novovicova, J. Kittler, Floating search methods in feature selection, Pattern Recognition Lett. 15 (1994) 1119-1125.
24. S. D. Stearns, On selecting features for pattern classifiers, in: Proceedings of the Third International Conference on Pattern Recognition, Coronado, CA, 1976, pp. 71-75.
25. C.J.C. Burges, A tutorial on support vector machines for pattern recognition, Data Min. Knowl. Discovery 2 (2) (1998) 121-167.
26. K.R. Muller, S. Mika, G. Ratsch, K. Tsuda, B. Scholkopf, An introduction to kernel-based learning algorithms, IEEE Trans. Neural Netw. 12 (2) (2001) 181-201.
27. Chih-Wei Hsu, Chih-Jen Lin, A comparison of methods for multiclass support vector machines, IEEE Trans. Neural Netw. 13 (2) (2002) 415-425.
28. D Dubois, H. Prade, in: Fuzzy Sets and Systems: Theory and Applications, Academic Press, 1980.
29. A.H. Gunatilaka, B.A. Baertlein, Feature-level and decision-level fusion of non coincidently sampled sensors for land mine detection, IEEE Trans. Pattern Anal. Mach. Intell. 23 (6) (2001) 577-589.
30. Ludmilla Kuncheva, in: Combining Pattern Classifiers: Methods and Algorithms, John Wiley and Sons Inc., New Jersey, 2004.
31. S.-B. Cho, J.H. Kim, Combining multiple neural networks by fuzzy integral for robust classification, IEEE Trans. Syst. Man Cybern. 25 (2) (1995) 380-385.
32. L.I. Kuncheva, J.C. Bezdek, R. Duin, Decision templates for multiple classifier fusion: an experimental comparison, Pattern Recognition 34 (2001) 299-314.
33. Y. Freund, R.E. Shapire, A decision-theoretic generalization of on-line learning and an application to boosting, J. Comput. Syst. Sci. 55 (1) (1997) 119-139.
34. F. Hoffmann, Combining boosting and evolutionary algorithms for learning of fuzzy classification rules, Fuzzy Sets Syst. 14 (2004) 47-58.
35. UCI machine learning repository,
36. J.C. Bezdek, in: Pattern Recognition with Fuzzy Objective Function Algorithms, Plenum, New York, 1981.
37. T. Golub, D. Slonim, P. Tamayo, C. Huard, M. Gaasenbeek, J. Mesirov, H. Coller, M. Loh, J. Downing, M. Caligiuri, et al., Molecular classification of cancer: class discovery and class prediction by gene expression monitoring, Science 286 (1999) 531-537.
38. M. Shipp, K. Ross, P. Tamayo, A. Weng, J. Kutok, R. Aguiar, M. Gaasenbeeck, M. Angelo, M. Reich, G. Pinkus, et al., Diffuse large B-cell lymphoma outcome prediction by geneexpression profiling and supervised machine learning, Nat. Med. 8 (2002) 68-74.
39. J. Khan, J. Wei, M. Ringner, L. Saal, M. Ladanyi, F. Westermann, F. Berthold, M. Schwab, C.R Antonescu, C. Peterson, P. Meltzer, Classification and diagnostic prediction of cancers using expression profiling and artificial neural networks, Nat. Med. 7 (2001) 673-679.
40. D. Li, W. Pedrycz, N.J. Pizzi, Fuzzy wavelet packet based feature extraction method and its application to biomedical signal classification, IEEE Trans. Biomed. Eng. vol. 52 (2005) 1132-1139.
41. J. Platt, Probabilistic outputs for support vector machines and comparison to regularized likelihood methods, in: A.J. Smola, P. Bartlett, B. Schoelkopf, D. Schurmans (Eds.), Advances in Large Margin Classifiers, pp. 61-74, 2000.
42. A. Madevska-Bogdanova, D. Nikolik, L. Curfs, Probabilistic SVM outputs for pattern recognition using analytical geometry, Neurocomputing 62 (1-4) (2004) 293-303.
43. B. Zadrozny, C. Elkan, Obtaining calibrated probability estimates from decision trees and naive Bayesian classifiers, in: Proceedings of the 18th international Conference on Machine Learning, iCML'01, 2001, pp. 609-616.
44. J. Drish, in: Obtaining calibrated estimates from Support Vector Machines, Technical Report, University of California, San Diego, 2001.
45. D. Bouchaffra, V. Govindaraju, S. Srihari, A methodology for mapping scores to probabilities, IEEE Trans. Pattern Anal. Mach. intell. 21 (9) (1999) 923-927.
46. B. Zadrozny, C. Elkan, Transforming classifier scores into accurate multiclass probability estimates, in: Proceedings of the ACM SiGKDD international Conference on Knowledge Discovery and Data Mining, 2002, pp. 694-699.
47. A. Niculescu-Mizil, R. Caruana, Predicting good probabilities with supervised learning, in: Proceedings of the 22nd international Conference on Machine Learning, iCML 2005, pp. 625-632.
48. N. Kwak, C-H Choi, improved mutual information feature selector for neural networks in supervised learning, in: Proceedings of the Joint Conference on Neural Networks, Washington, DC, July 1999.