Improving feature selection process resistance to failures caused by curse-of-dimensionality effects

Kybernetika

Volumn 47, Issue 3, 2011, Pages 401-425

  Somol, Petr ^a   Grim, Jiří ^a   Novovičová, Jana ^a   Pudil, Pavel ^b  

^a INSTITUTE OF INFORMATION THEORY AND AUTOMATION   (Czech Republic)

^b UNIVERSITY OF ECONOMICS   (Czech Republic)

Author keywords

Curse of dimensionality; Dimensionality reduction; Feature selection; Machine learning; Over fitting; Stability

Indexed keywords

EID: 83455221244     PISSN: 00235954     EISSN: None     Source Type: Journal    
DOI: None     Document Type: Article

References (52)

1. G. Brown: A new perspective for information theoretic feature selection. In: Proc. AISTATS '09, JMLR: W&CP 5 (2009), pp. 49-56.
2. Ch.-Ch. Chang and Ch.-J. Lin: LIBSVM: A Library for SVM, 2001. http://www. csie.ntu.edu.tw/-cjlin/libsvm.
3. S. Das: Filters, wrappers and a boosting-based hybrid for feature selection. In: Proc. 18th Int. Conf. on Machine Learning (ICML '01), Morgan Kaufmann Publishers Inc. 2001, pp. 74-81.
4. M. Dash, K. Choi, P. Scheuermann, and H. Liu: Feature selection for clustering-a filter solution. In: Proc. 2002 IEEE Int. Conf. on Data Mining (ICDM '02), Vol. 00, IEEE Comp. Soc. 2002, p. 115.
5. P. A. Devijver and J. Kittler: Pattern Recognition: A Statistical Approach. Prentice Hall 1982.
6. D. Dutta, R. Guha, D. Wild, and T. Chen: Ensemble feature selection: Consistent descriptor subsets for multiple qsar models. J. Chem. Inf. Model. 43 (2007), 3, pp. 989-997. (Pubitemid 46973714)
7. C. Emmanouilidis et al.: Multiple-criteria genetic algorithms for feature selection in neuro-fuzzy modeling. In: Internat. Conf. on Neural Networks, Vol. 6, 1999, pp. 4387-4392. (Pubitemid 30593589)
8. A. Frank and A. Asuncion: UCI Machine Learning Repository, 2010.
9. I. A. Gheyas and L. S. Smith: Feature subset seleciton in large dimensionality domains. Pattern Recognition 43 (2010), 1, 5-13.
10. F. W. Glover and G. A. Kochenberger, eds.: Handbook of Metaheuristics. Internat. Ser. Operat. Research & Management Science 5, Springer 2003.
11. S. Günter and H. Bunke: An evaluation of ensemble methods in handwritten word recog. based on feature selection. In: Proc. ICPR '04, IEEE Comp. Soc. 2004, pp. 388-392.
12. I. Guyon and A. Elisseeff: An introduction to variable and feature selection. J. Mach. Learn. Res. 3 (2003), 1157-1182.
13. I. Guyon, S. Gunn, M. Nikravesh, and L.A. Zadeh, eds.: Feature Extraction-Foundations and Applications. Studies in Fuzziness and Soft Comp. 207 Physica, Springer 2006.
14. Tin Kam Ho: The random subspace method for constructing decision forests. IEEE Trans. PAMI 20 (1998), 8, 832-844. (Pubitemid 128741345)
15. F. Hussein, R. Ward, and N. Kharma: Genetic algorithms for feature selection and weighting, a review and study. In: Proc. 6th ICDAR, Vol. 00, IEEE Comp. Soc. 2001, pp. 1240-1244.
16. R. Jensen: Performing feature selection with ACO. Studies Comput. Intelligence 34, Springer 2006, pp. 45-73.
17. Special issue on variable and feature selection. J. Machine Learning Research. http://www. jmlr.org/papers/special/feature.html, 2003.
18. A. Kalousis, J. Prados, and M. Hilario: Stability of feature selection algorithms: A study on high-dimensional spaces. Knowledge Inform. Systems 12 (2007), 1, 95-116.
19. J. Kittler, M. Hatef, R.P. W. Duin, and J. Matas: On combining classifiers. IEEE Trans. PAMI 20 (1998), 3, 226-239. (Pubitemid 128741312)
20. R. Kohavi and G.H. John: Wrappers for feature subset selection. Artificial Intelligence 97 (1997), 1-2, 273-324. (Pubitemid 127401107)
21. I. Kononenko: Estimating attributes: Analysis and extensions of RELIEF. In: Proc. ECML-94, Springer 1994, pp. 171-182.
22. L. I. Kuncheva: A stability index for feature selection. In: Proc. 25th IASTED Internat. Mul.-Conf. AIAP'07, ACTA Pr. 2007, pp. 390-395.
23. C. Lai, M. J. T. Reinders, and L. Wessels: Random subspace method for multivariate feature selection. Pattern Recognition Lett. 27 (2006), 10, 1067-1076.
24. H. Liu and H. Motoda: Feature Selection for Knowledge Discovery and Data Mining. Kluwer Academic Publishers 1998.
25. H. Liu and L. Yu: Toward integrating feature selection algorithms for classification and clustering. IEEE Trans. KDE 17 (2005), 4, 491-502. (Pubitemid 40495592)
26. S. Nakariyakul and D.P. Casasent: Adaptive branch and bound algorithm for selecting optimal features. Pattern Recognition Lett. 28 (2007), 12, 1415-1427. (Pubitemid 46990579)
27. S. Nakariyakul and D.P. Casasent: An improvement on floating search algorithms for feature subset selection. Pattern Recognition 42 (2009), 9, 1932-1940.
28. J. Novovičová, P. Pudil, and J. Kittler: Divergence based feature selection for multimodal class densities. IEEE Trans. PAMI 18 (1996), 2, 218-223. (Pubitemid 126772809)
29. P. Pudil, J. Novovičová, N. Choakjarernwanit, and J. Kittler: Feature selection based on the approximation of class densities by finite mixtures of special type. Pattern Recognition 28 (1995), 9, 1389-1398.
30. P. Pudil, J. Novovičová, and J. Kittler: Floating search methods in feature selection. Pattern Recognition Lett. 15 (1994), 11, 1119-1125.
31. S. J. Raudys: Feature over-selection. In: Proc. S+SSPR, Lecture Notes in Comput. Sci. 4109, Springer 2006, pp. 622-631.
32. V. C. Raykar et al.: Bayesian multiple instance learning: Automatic feature selection and inductive transfer. In: Proc. ICML '08, ACM 2008, pp. 808-815.
33. J. Reunanen: A pitfall in determining the optimal feature subset size. In: Proc. 4th Internat. Workshop on Pat. Rec. in Inf. Systs (PRIS 2004), pp. 176-185.
34. J. Reunanen: Less biased measurement of feature selection benefits. In: Stat. and Optimiz. Perspectives Workshop, SLSFS, Lecture Notes in Comput. Sci. 3940, Springer 2006, pp. 198-208. (Pubitemid 44029890)
35. Y. Saeys, I. Inza, and P. Larranaga: A review of feature selection techniques in bioinformatics. Bioinformatics 23 (2007), 19, 2507-2517. (Pubitemid 350048351)
36. A. Salappa, M. Doumpos, and C. Zopounidis: Feature selection algorithms in classification problems: An experimental evaluation. Optimiz. Methods Software 22 (2007), 1, 199-212. (Pubitemid 44878967)
37. F. Sebastiani: Machine learning in automated text categorization. ACM Comput. Surveys 34 (2002), 1, 1-47.
38. M. Sebban and R. Nock: A hybrid filter/wrapper approach of feature selection using information theory. Pattern Recognition 35 (2002), 835-846. (Pubitemid 34128810)
39. P. Somol, J. Grim, and P. Pudil: Criteria ensembles in feature selection. In: Proc. MCS, Lecture Notes in Comput. Sci. 5519, Springer 2009, pp. 304-313.
40. P. Somol, J. Grim, and P. Pudil: The problem of fragile feature subset preference in feature selection methods and a proposal of algorithmic workaround. In: ICPR 2010. IEEE Comp. Soc. 2010.
41. P. Somol, J. Novovičová, and P. Pudil: Flexible-hybrid sequential floating search in statistical feature selection. In: Proc. S+SSPR, Lecture Notes in Comput. Sci. 4109, Springer 2006, pp. 632-639. (Pubitemid 44543154)
42. P. Somol and J. Novovičová: Evaluating the stability of feature selectors that optimize feature subset cardinality. In: Proc. S+SSPR, Lecture Notes in Comput. Sci. 5342 Springer 2008, pp. 956-966.
43. P. Somol, J. Novovičová, J. Grim, and P. Pudil: Dynamic oscillating search algorithms for feature selection. In: ICPR 2008. IEEE Comp. Soc. 2008.
44. P. Somol, J. Novovičová, and P. Pudil: Improving sequential feature selection methods performance by means of hybridization. In: Proc. 6th IASTED Int. Conf. on Advances in Computer Science and Engrg. ACTA Press 2010.
45. P. Somol and P. Pudil: Oscillating search algorithms for feature selection. In: ICPR 2000, IEEE Comp. Soc. 02 (2000), 406-409.
46. P. Somol, P. Pudil, and J. Kittler: Fast branch & bound algorithms for optimal feature selection. IEEE Trans. on PAMI 26 (2004), 7, 900-912.
47. Y. Sun: Iterative RELIEF for feature weighting: Algorithms, theories, and applications. IEEE Trans. PAMI 29 (2007), 6, 1035-1051. (Pubitemid 46667414)
48. M.-A. Tahir et al: Simultaneous feature selection and feature weighting using hybrid tabu search/k-nearest neighbor classifier. Patt. Recognition Lett. 28 (2007), 4, 438-446.
49. A. W. Whitney: A direct method of nonparametric measurement selection. IEEE Trans. Comput. 20 (1971), 9, 1100-1103.
50. Y. Yang and J. O. Pedersen: A comparative study on feature selection in text categorization. In: Proc. 14th Internat. Conf. on Machine Learning (ICML '97), Morgan Kaufmann 1997, pp. 412-420.
51. L. Yu and H. Liu: Feature selection for high-dimensional data: A fast correlationbased filter solution. In: Proc. 20th Internat. Conf. on Machine Learning (ICML-03), Vol. 20, Morgan Kaufmann 2003, pp. 856-863.
52. Z. Zhu, Y. S. Ong, and M. Dash: Wrapper-filter feature selection algorithm using a memetic framework. IEEE Trans. Systems Man Cybernet., Part B 37 (2007), 1, 70.