Empirical evaluation of feature selection methods in classification

Intelligent Data Analysis

Volumn 14, Issue 3, 2010, Pages 265-281

  Čehovin, Luka ^a   Bosnić, Zoran ^a  

^a UNIVERSITY OF LJUBLJANA   (Slovenia)

Author keywords

Feature selection; Gini index; Random forest feature selector; ReliefF; Sequential backward selection; Sequential forward selection

Indexed keywords

EID: 77953517919     PISSN: 1088467X     EISSN: 15714128     Source Type: Journal    
DOI: 10.3233/IDA-2010-0421     Document Type: Article

References (32)

1. D.W. Aha and R.L. Bankert, Feature selection for case-based classification of cloud types: An empirical comparison, Technical report, Menlo Park, CA, USA, 1994.
2. D.W. Aha and R.L. Bankert, in: A Comparative Evaluation of Sequential Feature Selection Algorithms, D. Fisher and H. Lenz, eds, Proceedings of the Fifth International Workshop on Artificial Intelligence and Statistics (Ft. Lauderdale, FL, USA) 1995, pp. 1-7.
3. E. Alpaydin, Introduction to Machine Learning, TheMIT Press: Cambridge,Massachussetts, 2004.
4. A. Asuncion and D.J. Newman, UCI machine learning repository, http://archive.ics.uci.edu/ml/, 2007.
5. A.L. Blum and P. Langley, Selection of relevant features and examples in machine learning, Artificial Intelligence 97(1-2) (1997), 245-271.
6. L. Breiman, Random forests, Machine Learning 45(1) (2001), 5-32.
7. L. Breiman, J.H. Friedman, R.A. Olshen and C.J. Stone, Classification and Regression Trees, Wadsworth International Group, Belmont, CA, USA, 1984.
8. C. Chang and C. Lin, LIBSVM: A Library for Support Vector Machines, 2001, Software available at http://www.csie. ntu.edu.tw/ cjlin/libsvm.
9. M. Dash and H. Liu, Feature selection for classification, IntelligentData Analysis 1 (1997), 131-156.
10. J. Dem?sar, Statistical comparisons of classifiers over multiple data sets, J Mach Learn Res 7 (2006), 1-30. (Pubitemid 43022939)
11. F. Ferri, P. Pudil, M. Hatef and J. Kittler, Comparative Study of Techniques for Large-Scale Feature Selection, in: Pattern Recognition in Practice IV, Multiple Paradigms, Comparative Studies and Hybrid Systems, 1994, pp. 403-413.
12. M. Grimaldi, P. Cunningham and A. Kokaram, An Evaluation of Alternative Feature Selection Strategies and Ensemble Techniques for Classifying Music, Workshop in Multimedia Discovery and Mining (Dubrovnik, Croatia), 2003.
13. I. Guyon and A. Elisseeff, An introduction to variable and feature selection, Journal of Machine Learning Research (2003), 1157-1182.
14. I.M. Guyon, S.R. Gunn, A. Ben-Hur and G. Dror, Result Analysis of the Nips 2003 Feature Selection Challenge, Advances in Neural Information Processing Systems, 2004.
15. M.A. Hall and G. Holmes, Benchmarking attribute selection techniques for discrete class data mining, IEEE Transactions on Knowledge and Data Engineering 15(6) (2003), 1437-1447.
16. R. Ihaka and R. Gentleman, R: A language for data analysis and graphics, Journal of Computational and Graphical Statistics 5(3) (1996), 299-314.
17. A. Jain and D. Zongker, Feature selection: Evaluation, application, and small sample performance, IEEE Transactions on Pattern Analysis and Machine Intelligence 19(2) (1997), 153-158.
18. I. Jeffery, D. Higgins and A. Culhane, Comparison and evaluation of methods for generating differentially expressed gene lists from microarray data, BMC Bioinformatics 7(1) (2006), 359.
19. G.H. John, R. Kohavi and K. Pfleger, Irrelevant Features and the Subset Selection Problem, Proceedings of the Eleventh International Conference on Machine Learning, 1994, 121-129.
20. R. Kohavi and G.H. John, Wrappers for feature subset selection, Artificial Intelligence 97(1-2) (1997), 273-324.
21. I. Kononenko, Estimating Attributes: Analysis and Extensions of RELIEF, European Conference on Machine Learning, 1994, 171-182.
22. I. Kononenko and M. Kukar, Machine learning and data mining, Horwood Publishing, Chichester, UK, 2007.
23. M.Kudo and J. Sklansky, AComparative Evaluation of Medium-and Large-Scale Feature Selectors forPattern Classifiers, First InternationalWorkshop on Statistical Techniques in Pattern Recognition (Prague, Czech Republic), 1997, 91-96.
24. M. Kudo and J. Sklansky, Classifier-independent feature selection for two-stage feature selection, Syntactical and Structural Pattern Recognition/Statistical Pattern Recognition (1998), 548-554.
25. H. Liu, J. Li and L. Wong, A comparative study on feature selection and classification methods using gene expression profiles and proteomic patterns, Genome Inform 13 (2002), 51-60.
26. H. Liu and H. Motoda, Computational Methods of Feature Selection, Chapman & Hall/CRC, 2007.
27. H. Liu and L. Yu, Toward integrating feature selection algorithms for classification and clustering, IEEE Transactions on Knowledge and Data Engineering 17(4) (2005), 491-502.
28. W.S. McCulloch and W. Pitts, A logical calculus of the ideas imminent in nervous activity, Bulletin of Mathematical Biophysics 5 (1943), 115-133.
29. M. Robnik?Sikonja and I. Kononenko, Theoretical and empirical analysis of relieff and rrelieff, Machine Learning 53(1-2) (2003), 23-69.
30. Y. Saeys, I. Inza and P. Larra?naga, A review of feature selection techniques in bioinformatics, Bioinformatics 23(19) (2007), 2507-2517.
31. V.N. Vapnik, The Nature of Statistical Learning Theory, Springer, New York, NY, USA, November 1999.
32. Y. Yang and J.O. Pedersen, A Comparative Study on Feature Selection in Text Categorization, Proceedings of the Fourteenth International Conference on Machine Learning (San Francisco, CA, USA), Morgan Kaufmann Publishers Inc., 1997, 412-420.