Fast dependency-aware feature selection in very-high-dimensional pattern recognition

Conference Proceedings - IEEE International Conference on Systems, Man and Cybernetics

Volumn , Issue , 2011, Pages 502-509

  Somol, Petr ^a   Grim, Jiří ^a   Pudil, Pavel ^b  

^a INSTITUTE OF INFORMATION THEORY AND AUTOMATION   (Czech Republic)

^b UNIVERSITY OF ECONOMICS   (Czech Republic)

Author keywords

classification; feature selection; generalization; high dimensionality; machine learning; over fitting; pattern recognition; ranking; stability

Indexed keywords

GENERALIZATION; HIGH DIMENSIONALITY; MACHINE-LEARNING; OVERFITTING; RANKING;

CLASSIFICATION (OF INFORMATION); COMPUTATIONAL COMPLEXITY; CONVERGENCE OF NUMERICAL METHODS; CYBERNETICS; PATTERN RECOGNITION; PATTERN RECOGNITION SYSTEMS; QUALITY CONTROL;

FEATURE EXTRACTION;

EID: 83755184214     PISSN: 1062922X     EISSN: None     Source Type: Conference Proceeding    
DOI: 10.1109/ICSMC.2011.6083733     Document Type: Conference Paper

References (48)

1. I. Guyon, S. Gunn, M. Nikravesh, and L. Zadeh, Eds., Feature Extraction - Foundations and Applications, ser. Studies in Fuzziness and Soft Computing. Physica-Verlag, Springer, 2006, vol. 207.
2. F. J. Ferri, P. Pudil, M. Hatef, and J. Kittler, "Comparative study of techniques for large-scale feature selection," Machine Intelligence and Pattern Recognition, vol. 16, 1994.
3. R. Kohavi and G. H. John, "Wrappers for feature subset selection," Artif. Intell., vol. 97, no. 1-2, pp. 273-324, 1997.
4. H. Liu and H. Motoda, Feature Selection for Knowledge Discovery and Data Mining. Norwell, MA, USA: Kluwer Academic Publishers, 1998.
5. "Special issue on variable and feature selection," J. of Machine Learning Research. http://www. jmlr.org/papers/special/feature.html, 2003.
6. H. Liu and L. Yu, "Toward integrating feature selection algorithms for classification and clustering," IEEE Transactions on Knowledge and Data Engineering, vol. 17, no. 4, pp. 491-502, 2005. (Pubitemid 40495592)
7. Y. Sun, "Iterative relief for feature weighting: Algorithms, theories, and applications," IEEE Trans. Pattern Anal. Mach. Intell., vol. 29, no. 6, pp. 1035-1051, 2007. (Pubitemid 46667414)
8. P. A. Devijver and J. Kittler, Pattern Recognition: A Statistical Approach. Englewood Cliffs, London, UK: Prentice Hall, 1982.
9. A. Salappa, M. Doumpos, and C. Zopounidis, "Feature selection algorithms in classification problems: An experimental evaluation," Optimization Methods and Software, vol. 22, no. 1, pp. 199-212, 2007. (Pubitemid 44878967)
10. S. B. Serpico and L. Bruzzone, "A new search algorithm for feature selection in hyper-spectral remote sensing images," IEEE Trans. on Geoscience and Remote Sensing, vol. 39, no. 7, pp. 1360-1367, 2001. (Pubitemid 32732652)
11. S. Yu, "Feature selection and classifier ensembles: A study on hyperspectral remote sensing data," Ph.D. dissertation, University of Antwerp, Antwerp, Netherlands, 2003.
12. F. Sebastiani, "Machine learning in automated text categorization," ACM Computing Surveys, vol. 34, no. 1, pp. 1-47, March 2002.
13. J. G. Dy and C. E. Brodley, "Feature selection for unsupervised learning,"J. Mach. Learn. Res., vol. 5, pp. 845-889, 2004.
14. E. P. Xing, Feature Selection in Microarray Analysis. Springer, 2003, pp. 110-129.
15. Y. Saeys, I. Inza, and P. Larrañaga, "A review of feature selection techniques in bioinformatics," Bioinformatics, vol. 23, no. 19, pp. 2507-2517, 2007. (Pubitemid 350048351)
16. M. A. Tahir, A. Bouridane, F. Kurugollu, and A. Amira, "Feature selection using tabu search for improving the classification rate of prostate needle biopsies," in Proc. ICPR '04, vol. 2. Washington, DC, USA: IEEE Computer Society, 2004, pp. 335-338.
17. G. Brown, "A New Perspective for Information Theoretic Feature Selection," in Proc. AISTATS '09, vol. 5 of JMLR: W&CP 5, 2009, pp. 49-56.
18. P. Somol, P. Pudil, and J. Kittler, "Fast branch & bound algorithms for optimal feature selection," IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 26, no. 7, pp. 900-912, 2004.
19. J. Fan and R. Li, "Statistical challenges with high dimensionality: Feature selection in knowledge discovery," 2006. [Online]. Available: http://www.citebase.org/abstract?id=oai:arXiv.org:math/0602133
20. Y. Yang and J. O. Pedersen, "A comparative study on feature selection in text categorization," in ICML '97: Proc. 14th Int. Conf. on Machine Learning. San Francisco, CA, USA: Morgan Kaufmann Publishers Inc., 1997, pp. 412-420.
21. D. Mladenić and M. Grobelnik, "Feature selection for unbalanced class distribution and naive bayes," in ICML '99: Proc. 16th Int. Conf. on Machine Learning. San Francisco, CA, USA: Morgan Kaufmann Publishers Inc., 1999, pp. 258-267.
22. T. Liu, S. Liu, Z. Chen, and W.-Y. Ma, "An evaluation on feature selection for text clustering," in Proc. 20th Int. Conf. on Machine Learning (ICML-2003), Washington, DC, USA, 2003, pp. 488-495.
23. N. Vasconcelos, "Feature selection by maximum marginal diversity: optimality and implications for visual recognition," in Proc. IEEE Conf. On Computer Vision And Pattern Recognition, vol. 1. Washington, DC, USA: IEEE Computer Society, June 2003, pp. 762-769.
24. M. E. Farmer, S. Bapna, and A. K. Jain, "Large scale feature selection using modified random mutation hill climbing," in ICPR '04: Proc. 17th Int. Conf. on Pattern Recognition, vol. 2. Washington, DC, USA: IEEE Computer Society, 2004, pp. 287-290.
25. J. Reunanen, "Overfitting in making comparisons between variable selection methods," J. Mach. Learn. Res., vol. 3, pp. 1371-1382, 2003.
26. -, "A pitfall in determining the optimal feature subset size," in Proc. 4th Int. Workshop on Pattern Recognition in Information Systems (PRIS 2004), Porto, Portugal, 2004, pp. 176-185.
27. -, "Less biased measurement of feature selection benefits," in Statistical and Optimization Perspectives Workshop, SLSFS 2005, Bohinj, Slovenia, February 23-25, 2005, Revised Selected Papers, C. Saunders, M. Grobelnik, S. Gunn, and J. Shawe-Taylor, Eds., vol. LNCS 3940. Springer Berlin / Heidelberg, 2006, pp. 198-208.
28. Š. J. Raudys, R. Baumgartner, and R. Somorjai, "On understanding and assessing feature selection bias," in Artificial Intelligence in Medicine, vol. LNAI 3581. Springer-Verlag, 2005, pp. 468-472.
29. Š. J. Raudys, "Feature over-selection," in Structural, Syntactic, and Statistical Pattern Recognition, vol. LNCS 4109. Berlin / Heidelberg, Germany: Springer-Verlag, 2006, pp. 622-631.
30. L. I. Kuncheva, "A stability index for feature selection," in Proc. 25th IASTED International Multi-Conference AIAP'07. Anaheim, CA, USA: ACTA Press, 2007, pp. 390-395.
31. J. T. Souza, S. Matwin, and N. Japkowicz, "Parallelizing feature selection,"Algorithmica, vol. 45, no. 3, pp. 433-456, 2006.
32. L. Yu and H. Liu, "Feature selection for high-dimensional data: A fast correlation-based filter solution," in ICML-03: Proc. 20th Int. Conf. on Machine Learning, vol. 20. Washington DC, USA: Morgan Kaufmann Publishers Inc., 2003, pp. 856-863.
33. F. Fleuret, "Fast binary feature selection with conditional mutual information,"J. Mach. Learn. Res., vol. 5, pp. 1531-1555, 2004.
34. H. Peng, F. Long, and C. Ding, "Feature selection based on mutual information: Criteria of max-dependency, max-relevance, and min-redundancy, "IEEE Trans. Pattern Anal. Mach. Intell., vol. 27, no. 8, pp. 1226-1238, 2005. (Pubitemid 41245053)
35. K. Kira and L. A. Rendell, "The feature selection problem: Traditional methods and a new algoirthm," in AAAI-92: Proc. 10th Nat. Conf. on Artifical Intelligence, W. Swartout, Ed. AAAI Press/The MIT Press, August 1992, pp. 129-134.
36. I. Kononenko, "Estimating attributes: Analysis and extensions of relief,"in ECML-94: Proc. European Conf. on Machine Learning. Secaucus, NJ, USA: Springer-Verlag New York, Inc., 1994, pp. 171-182.
37. J. Yang and Y.-P. Li, "Orthogonal relief algorithm for feature selection,"in Intelligent Computing, vol. LNCS 4113. Berlin / Heidelberg, Germany: Springer-Verlag, 2006, pp. 227-234.
38. F. Hussein, R. Ward, and N. Kharma, "Genetic algorithms for feat. select. and weighting, a review and study," icdar, vol. 00, p. 1240, 2001.
39. F. W. Glover and G. A. Kochenberger, Eds., Handbook of Metaheuristics, ser. Int. Series in Operations Research & Management Science. Springer, 2003, vol. 57.
40. M. A. Tahir, A. Bouridane, and F. Kurugollu, "Simultaneous feature selection and feature weighting using hybrid tabu search/k-nearest neighbor classifier," Pattern Recogn. Lett., vol. 28, no. 4, pp. 438-446, 2007. (Pubitemid 44918133)
41. J. C. W. Debuse and V. J. Rayward-Smith, "Feature subset selection within a simulated annealing data mining algorithm," J. Intell. Inf. Syst., vol. 9, no. 1, pp. 57-81, 1997.
42. J. Novovičová, P. Somol, and P. Pudil, "Oscillating feature subset search algorithm for text categorization," in Structural, Syntactic, and Statistical Pattern Recognition, vol. LNCS 4109. Berlin / Heidelberg, Germany: Springer-Verlag, 2006, pp. 578-587.
43. I. A. Gheyas and L. S. Smith, "Feature subset selection in large dimensionality domains," Pattern Recognition, vol. 43, no. 1, pp. 5-13, 2010.
44. C. Lai, M. J. T. Reinders, and L. Wessels, "Random subspace method for multivariate feature selection," Pattern Recogn. Lett., vol. 27, no. 10, pp. 1067-1076, 2006.
45. C.-C. Chang and C.-J. Lin, LIBSVM: a library for SVM, 2001, http://www.csie.ntu.edu.tw/ cjlin/libsvm.
46. A. Asuncion and D. Newman, "UCI machine learning repository, http://www.ics.uci.edu/~mlearn/ mlrepository.html," 2007. [Online]. Available: http://www.ics.uci.edu/~mlearn/MLRepository.html
47. P. Pudil, J. Novovičová, and J. Kittler, "Floating search methods in feature selection," Pattern Rec. Lett., vol. 15, no. 11, pp. 1119-1125, 1994.
48. P. Somol and J. Novovičová, "Evaluating stability and comparing output of feature selectors that optimize feature subset cardinality," IEEE Trans. Pattern Anal. Mach. Intell., vol. 32, no. 11, pp. 1921-1939, 2010.